text:
# Examining the Higgs boson potential at lepton and hadron colliders: a comparative analysis

## I Introduction

The CERN Large Hadron Collider (LHC) is widely regarded as capable of directly observing the agent responsible for electroweak symmetry breaking and fermion mass generation. This is generally believed to be a light Higgs boson with mass $`m_H<200`$ GeV . The LHC will easily find a light Standard Model (SM) Higgs boson with very moderate luminosity . Moreover, the LHC will have significant capability to determine many of its properties , such as its decay modes and couplings , including invisible decays and possibly even rare decays to light fermions . An $`e^+e^{}`$ linear collider with a center of mass energy of 350 GeV or more can significantly improve these preliminary measurements, in some cases by an order of magnitude in precision, if an integrated luminosity of 500 fb-1 can be achieved .

Starting from the requirement that the Higgs boson has to restore unitarity of weak boson scattering at high energies in the SM , perhaps the most important measurement after the Higgs boson discovery is of the Higgs potential itself, which requires measurement of the trilinear and quartic Higgs boson self-couplings, $`\lambda `$ and $`\stackrel{~}{\lambda }`$, respectively. These can be probed directly only by multiple Higgs boson production (at any future collider). While $`\lambda `$ can be measured in Higgs boson pair production, triple Higgs boson production is needed to probe $`\stackrel{~}{\lambda }`$. Since the cross sections for three Higgs boson production processes are more than a factor $`10^3`$ smaller than those for Higgs pair production at linear colliders , and about an order of magnitude smaller at hadron colliders , the quartic Higgs boson coupling will likely remain elusive even at the highest collider energies and luminosities considered so far.

Several studies of Higgs boson pair production in $`e^+e^{}`$ collisions have been conducted over the past few years , deriving quantitative sensitivity limits for the trilinear Higgs self-coupling $`\lambda `$ for several proposed linear colliders with center of mass energies spanning the range from 500 GeV to 3 TeV. For example, a study employing neural net techniques found that $`\lambda `$ could be measured for $`m_H=120`$ GeV with a precision of about $`20\%`$ at a 500 GeV linear collider with an integrated luminosity of 1 ab-1 . In contrast, the potential of the LHC, a luminosity-upgraded LHC (SLHC) which would gather 10 times the amount of data expected in the first run, and a Very Large Hadron Collider (VLHC), has been examined only recently . These studies investigated Higgs pair production via gluon fusion and subsequent decay to same-sign dileptons and three leptons via $`W`$ bosons. They established that future hadron machines can probe the Higgs potential for $`m_H>150`$ GeV. At the LHC, with an integrated luminosity of 300 fb-1, a vanishing of $`\lambda `$ can be excluded at the $`95\%`$ confidence level or better over the entire range $`150\mathrm{GeV}<m_H<200`$ GeV. At a VLHC, the Higgs boson self-coupling can be determined with a precision of a few percent with the same integrated luminosity for $`m_H=180`$ GeV, which is similar or better than the limits achievable at a 3 TeV $`e^+e^{}`$ collider with 5 ab-1 .

In this paper we present an analysis of the converse: we look at Higgs boson pair production for $`m_H140`$ GeV at future hadron colliders, and estimate the prospects for probing $`\lambda `$ if $`m_H150`$ GeV at a future linear collider with a center of mass energy of $`0.51`$ TeV. To fully compare the capabilities of $`e^+e^{}`$ linear colliders and hadron colliders, we also extrapolate the results of Ref.  to $`m_H>120`$ GeV and center of mass energies larger than 500 GeV. In Sec. II we recall the definition of the Higgs boson self-couplings and briefly discuss SM and non-SM predictions for these parameters. In Sec. III we analyze Higgs boson pair production via gluon fusion with subsequent decay into four $`b`$ jets and $`b\overline{b}\tau \tau `$ final states at the LHC, SLHC and a VLHC, which we assume to be a $`pp`$ collider operating at 200 TeV with a luminosity of $`=2\times 10^{34}\mathrm{cm}^2\mathrm{s}^1`$ . We also briefly comment on the prospects for observing a pair of MSSM Higgs bosons in the $`b\overline{b}\tau \tau `$ decay channel. We discuss Higgs boson pair production in $`e^+e^{}`$ collisions in Sec. IV. In Sec. V we determine how well the Higgs potential could be reconstructed at future lepton and hadron colliders. We draw conclusions in Sec. VI.

## II Higgs boson self-couplings

The trilinear and quartic Higgs boson couplings $`\lambda `$ and $`\stackrel{~}{\lambda }`$ are defined through the potential

$$V(\eta _H)=\frac{1}{2}m_H^2\eta _H^2+\lambda v\eta _H^3+\frac{1}{4}\stackrel{~}{\lambda }\eta _H^4,$$
(1)
where $`\eta _H`$ is the physical Higgs field, $`v=(\sqrt{2}G_F)^{1/2}`$ is the vacuum expectation value, and $`G_F`$ is the Fermi constant. In the SM,

$$\stackrel{~}{\lambda }=\lambda =\lambda _{SM}=\frac{m_H^2}{2v^2}.$$
(2)
Regarding the SM as an effective theory, the Higgs boson self-couplings $`\lambda `$ and $`\stackrel{~}{\lambda }`$ are per se free parameters. $`S`$-matrix unitarity constrains $`\stackrel{~}{\lambda }`$ to $`\stackrel{~}{\lambda }8\pi /3`$ . Since future collider experiments likely cannot probe $`\stackrel{~}{\lambda }`$, we concentrate on the trilinear coupling $`\lambda `$ in the following. The quartic Higgs coupling does not affect the Higgs pair production processes discussed in this paper. Our results, with the exception of the constraints on $`V(\eta _H)`$ discussed in Sec. V (where we assume $`\stackrel{~}{\lambda }=\lambda _{SM}`$) are therefore independent of the value assumed for $`\stackrel{~}{\lambda }`$.

In the SM, radiative corrections decrease $`\lambda `$ by $`411\%`$ for $`120\mathrm{GeV}<m_H<200`$ GeV . Larger deviations are possible in scenarios beyond the SM. For example, in two Higgs doublet models where the lightest Higgs boson is forced to have SM like couplings to vector bosons, quantum corrections may increase the trilinear Higgs boson coupling by up to $`100\%`$ . In the Minimal Supersymmetric Standard Model (MSSM), loop corrections modify the self-coupling of the lightest Higgs boson, which has SM-like couplings, by up to $`8\%`$ for light stop squarks . Anomalous Higgs boson self-couplings also appear in various other scenarios beyond the SM, such as models with a composite Higgs boson , or in “little Higgs” models . In many cases, the anomalous Higgs boson self-couplings can be parameterized in terms of higher dimensional operators which are induced by integrating out heavy degrees of freedom. A systematic analysis of Higgs boson self-couplings in a higher dimensional operator approach can be found in Ref. .

## III A low mass Higgs boson at hadron colliders

At LHC energies, inclusive Higgs boson pair production is dominated by gluon fusion. Other processes, such as weak boson fusion, $`qqqqHH`$ , associated production with heavy gauge bosons, $`q\overline{q}VHH`$ ($`V=W,Z`$, or associated production with top quark pairs, $`gg,q\overline{q}t\overline{t}HH`$ , yield cross sections which are factors of 10 – 30 smaller than that for $`ggHH`$ . Since Higgs boson pair production at the LHC is already rate limited, we concentrate on the gluon fusion process in the following. For $`m_H<140`$ GeV, the dominant decay mode of the SM Higgs boson is $`Hb\overline{b}`$. In the following, we examine the largest overall branching ratio production, which yields four $`b`$-quark final states, and decays where one Higgs boson decays into $`b\overline{b}`$ and the other into a $`\tau `$ pair, $`ggHHb\overline{b}\tau ^+\tau ^{}`$.

For all our calculations we assume an integrated luminosity of 300 fb-1 for LHC and VLHC , and 3000 fb-1 for the SLHC. We choose $`\alpha _s(M_Z)=0.1185`$ , and assume a $`b`$-tagging efficiency of $`50\%`$ for all hadron colliders. Signal and background cross sections are consistently calculated using CTEQ4L  parton distribution functions. We include minimal detector effects by Gaussian smearing of the parton momenta according to ATLAS expectations , and take into account energy loss in the $`b`$-jets via a parameterized function. In addition, we include an efficiency of $`68\%`$ for capturing each $`Hb\overline{b}`$ or $`H\tau ^+\tau ^{}`$ decay in the signal in its relevant mass bin. All tree level processes are calculated using madgraph  and retain a finite $`b`$-quark mass of 4.6 GeV.

### A $`𝐩p4b`$

We perform the calculation of the signal, $`ggHH4b`$, as in Ref. , including the effects of next-to-leading order (NLO) QCD corrections via a multiplicative factor $`K=1.65`$ at LHC and $`K=1.35`$ at VLHC energies . The largest background to consider is QCD continuum four $`b`$-quark production. The factorization and renormalization scale choices are taken to be $`m_H`$. There is large uncertainty due to scale variation in the QCD backgrounds, but this is irrelevant given our findings which follow. The choice of scales for the signal rate can have a large impact as well, for example varying the scale between the Higgs boson mass and the invariant mass of the final state Higgs boson pair, as chosen as a default in the public version of the NLO matrix elements . However, after including the approximate NLO corrections as a $`K`$-factor we find that the scale dependence of the cross section as a measure of the theoretical uncertainty is strongly reduced, which is the main reason to compute and include these higher order corrections. We require that all four $`b`$-quarks in the event are tagged.

The kinematic acceptance cuts for events at the LHC are:

$`p_T(b)>75,\mathrm{\hspace{0.17em}65},\mathrm{\hspace{0.17em}40},\mathrm{\hspace{0.17em}20}\mathrm{GeV},|\eta (b)|<2.5,`$ (3)
$`\mathrm{\Delta }R(b,b)>0.7,m_H30\mathrm{GeV}<m_{b\overline{b}}<m_H+10\mathrm{GeV}`$ (4)
which are motivated first by requirements that these all-hadronic events can pass the ATLAS and CMS triggers with reasonable efficiency , and that two $`b`$-quark pairs each reconstruct to a window around the known Higgs boson mass, asymmetric due to energy loss in the $`b`$-jets. This invariant mass constraint on the six possible bottom pairs defines the candidates to reconstruct the Higgs bosons. We also use the cuts of Eq. (3) for the SLHC and VLHC. Preliminary studies concluded  that cuts similar to those listed in Eq. (3) should be sufficient, although increased background from event pile-up is expected to degrade detector performance at the SLHC.

Comparing the signal and the backgrounds, there is first of all an important difference between the $`4b`$ final state and the $`4W`$ final state investigated previously : the background of the bottom final state does not involve any massive particles, like top quarks or $`W`$ bosons. All four bottom jets in the QCD background process are either produced without a strong azimuthal correlation between each other or come from (mostly collinear) gluon splitting. The latter will to a large degree be removed by the $`m_{b\overline{b}}`$ cut together with the $`\mathrm{\Delta }R(b,b)`$ cut. This on the one hand requires a hard central gluon to split into two bottom quarks, which on the other hand cannot be boosted together. Even though the Higgs bosons are produced close to rest, they are still very massive states with a non-negligible transverse momentum, which decay to effectively massless bottom quarks. Translated into the geometry of the four bottom jets, this means that we can require a sizable transverse momentum of the bottom pairs which should reconstruct the Higgs bosons, and also require that these bottom jets lie close to each other in the azimuthal plane:

$`p_T(bb)^{\mathrm{min}}>105\mathrm{GeV},p_T(bb)^{\mathrm{max}}>115\mathrm{GeV},`$ (5)
$`\mathrm{}\varphi (b,b)^{\mathrm{min}}<0.5\pi ,\mathrm{}\varphi (b,b)^{\mathrm{max}}<0.7\pi .`$ (6)
As in the $`4W`$ signal case , we will later try to determine the Higgs boson self-coupling from the shape of the invariant mass of the final state. For that reason we do not apply any cuts which make use of the fact that the signal involves two heavy massive particles produced in a fairly narrow range of the $`4b`$ invariant mass. However, for a fixed invariant mass of the $`4b`$ final state we expect more forward jets for the QCD background which in turn do not need to have as large a transverse momentum. We therefore require the scalar sum of the transverse momentum to obey:

$$p_T>270\mathrm{GeV}.$$
(7)
Extracting the Higgs boson self-coupling follows the same path as for the $`4W`$ final state . To discriminate signal and background, we can use the visible invariant mass, $`m_{vis}`$, which for the $`4b`$ final state is the invariant mass of the Higgs boson pair, corrected for energy loss of the $`b`$ jets. The $`m_{vis}`$ distributions of the signal for $`m_H=120`$ GeV and the QCD background at the LHC are shown in Fig. 1.

Even after all cuts, the QCD background is more than two orders of magnitude larger than the signal. For an integrated luminosity of 300 fb-1, about 55 signal events are expected in the SM. The absence of a Higgs boson self-coupling ($`\lambda _{HHH}=\lambda /\lambda _{SM}=0`$) results in a Higgs boson pair production cross section about a factor 1.6 larger than the SM result, whereas increasing $`\lambda `$ to twice the SM value decreases the rate by a factor 1.7. At VLHC energies, the cross sections of the signal and background are a factor 100 and 60 larger than those at the LHC, giving negligible improvement in the signal to background ratio, $`S/B`$.

The small signal cross section combined with the huge QCD $`4b`$ background make it essentially impossible to determine the Higgs boson self-coupling in $`pp4b`$. We quantify this statement by performing a $`\chi ^2`$ test on the $`m_{vis}`$ distribution, similar to that described in Ref. . Except for the Higgs boson self-coupling, we assume the SM to be valid. To approximately take into account the unknown NLO QCD corrections to $`pp4b`$, we multiply the QCD $`4b`$ differential cross section by a uniform $`K`$-factor of $`K=1.3`$ and allow for a normalization uncertainty of $`10\%`$ for the SM cross section. For $`m_H=120`$ GeV we then obtain $`1\sigma `$ bounds of

$`6.8<`$ $`\mathrm{\Delta }\lambda _{HHH}<10.1`$ $`(\mathrm{LHC}),`$ (8)
$`3.1<`$ $`\mathrm{\Delta }\lambda _{HHH}<6.0`$ $`(\mathrm{SLHC}),`$ (9)
$`1.3<`$ $`\mathrm{\Delta }\lambda _{HHH}<2.4`$ $`(\mathrm{VLHC}),`$ (10)
where

$$\mathrm{\Delta }\lambda _{HHH}=\frac{\lambda }{\lambda _{SM}}1.$$
(11)
For comparison, a 500 GeV linear collider with an integrated luminosity of 1 ab-1 could determine $`\lambda `$ with a precision of about $`20\%`$ for $`m_H=120`$ GeV . For $`m_H>120`$ GeV, the $`Hb\overline{b}`$ branching ratio drops quickly. Since the background cross section decreases only slightly, $`S/B`$, and thus the bounds on the $`\lambda `$, worsen with increasing values of $`m_H`$.

### B $`𝐩pb\overline{b}\tau ^+\tau ^{}`$

The insensitivity of $`4b`$ production to the Higgs boson self-coupling is largely due to the overwhelming QCD background. A more advantageous $`S/B`$ is conceivable if one of the Higgs bosons in $`ggHH`$ decays into a $`\tau `$ pair. In this case, the main contributions to the background arise from continuum $`b\overline{b}\tau ^+\tau ^{}`$ and $`t\overline{t}W^+W^{}b\overline{b}\tau ^+\nu _\tau \tau ^{}\overline{\nu }_\tau b\overline{b}`$ production. We calculate both processes using tree level matrix elements which include all decay correlations. Top quarks are generated on-shell. The calculation of the signal proceeds as for the $`4b`$ final state. We assume that both $`b`$-quarks are tagged. Because of its small mass, we simulate $`\tau `$ decays in the collinear approximation. All $`\tau `$ decays are calculated following the approach described in Ref. .

If both $`\tau `$ leptons decay leptonically, $`4b`$ production where two $`b`$-quarks decay leptonically represents an additional potentially large background, even if one requires that both leptons are isolated. In addition, the signal cross section is suppressed by the small branching ratio of about $`13\%`$ if both $`\tau `$ leptons decay leptonically. In the following we therefore only consider decays where at least one $`\tau `$ lepton decays hadronically.

To identify $`b\overline{b}\tau \tau `$ events in a hadron collider environment, one has to trigger on the $`\tau `$ pair. At a luminosity of $`=10^{34}\mathrm{cm}^2\mathrm{s}^1`$ this requires severe transverse momentum cuts on the $`\tau `$ decay jet . To ensure that an event is successfully recorded in which one $`\tau `$ lepton decays leptonically ($`\tau \mathrm{}\nu _{\mathrm{}}\nu _\tau `$, $`\mathrm{}=e,\mu `$) and the other hadronically ($`\tau h\nu _\tau `$), we impose the following transverse momentum and rapidity cuts on the $`\tau `$ decay products :

$`p_T(\mathrm{})>20\mathrm{GeV},|\eta (\mathrm{})|<2.5,`$ (12)
$`p_T(h)>50\mathrm{GeV},|\eta (h)|<2.5,`$ (13)
$`\mathrm{\Delta }R(\mathrm{},h)>0.4,\mathrm{\Delta }R(\mathrm{},b)>0.4,\mathrm{\Delta }R(h,b)>0.4.`$ (14)
We assume that hadronically decaying $`\tau `$-jets which satisfy Eq. (14) will be identified with an efficiency of $`ϵ_\tau =0.33`$ and discriminated from other jets with a rejection factor of 500 or more . The large rejection factor makes the $`b\overline{b}jj`$ background, where the two non-$`b`$ jets fake hadronically decaying $`\tau `$ leptons, negligible. If both $`\tau `$ leptons decay hadronically, even more severe $`p_T`$ cuts are required :

$`p_T(h_{1,2})>65\mathrm{GeV},|\eta (h_{1,2})|<2.5`$ (15)
$`\mathrm{\Delta }R(h_1,h_2)>0.6,\mathrm{\Delta }R(h_{1,2},b)>0.4,`$ (16)
where $`h_{1,2}`$ are the $`\tau `$ decay jets.

For the signal, the $`\tau `$-pair invariant mass can be reconstructed from the observable $`\tau `$ decay products and the missing transverse momentum vector of the event . To reduce the background, we therefore impose a cut on the reconstructed $`\tau `$-pair invariant mass,

$$m_H2\mathrm{\Delta }<m_{\tau \tau }^{\mathrm{rec}}<m_H+2\mathrm{\Delta },$$
(17)
where $`\mathrm{\Delta }`$ is the $`1\sigma `$ half-width for the $`H`$ peak. $`\mathrm{\Delta }`$ ranges from about 7.5 GeV for $`m_H=120`$ GeV to 15 GeV for $`m_H=140`$ GeV. Finally, we impose the following cuts on the $`b`$-jets:

$`p_T(b)>30\mathrm{GeV},|\eta (b)|<2.5,`$ (18)
$`\mathrm{\Delta }R(b,b)>0.4,m_H20\mathrm{GeV}<m_{b\overline{b}}<m_H+20\mathrm{GeV}.`$ (19)
Unlike for the $`4b`$ final state, we have chosen a symmetric window around $`m_H`$ for the $`b\overline{b}`$ invariant mass. We found that although the energy loss in the $`b`$-quarks creates a non-Gaussian tail for $`m_{b\overline{b}}<m_H`$, it makes little difference whether a symmetric or asymmetric $`m_{b\overline{b}}`$ cut is imposed for $`b\overline{b}\tau \tau `$ production. Since there are more $`b\overline{b}`$ combinations possible, the difference between a symmetric and asymmetric window for $`m_{b\overline{b}}`$ is more pronounced for $`4b`$ production. We also use the cuts of Eqs. (14) and (19) for the SLHC and VLHC. We note that since the $`p_T`$ distributions of the $`\tau `$ decay products fall steeply with increasing transverse momenta, the $`ggHHb\overline{b}\tau \tau `$ cross section depends sensitively on the cuts in Eqs. (14) and (16).

As before, the effects of next-to-leading order (NLO) QCD corrections are included in our calculation via multiplicative factors which are summarized in Table I. Note that this is one of the rare instances where the NLO corrections are known for the signal and all major backgrounds. The factorization and renormalization scale choices are taken to be $`m_H`$ for the signal and the $`b\overline{b}\tau \tau `$ background; for $`t\overline{t}`$ production we choose the top quark mass, $`m_t`$.

For the cuts of Eq. (14), the cross section for $`ppHHb\overline{b}\tau \tau `$ where both $`\tau `$ leptons decay hadronically is about a factor 7 smaller than that where one of them decays into leptons. In the following we therefore consider the latter only. The $`HH`$ signal in $`ppb\overline{b}\tau _{\mathrm{}}\tau _h`$ ($`\tau _{\mathrm{}}`$ and $`\tau _h`$ denote the leptonically and hadronically decaying $`\tau `$ leptons, respectively) and the continuum $`b\overline{b}\tau \tau `$ and $`t\overline{t}`$ backgrounds can again be discriminated using the visible invariant mass distribution, $`m_{vis}`$. For the $`b\overline{b}\tau _{\mathrm{}}\tau _h`$ final state, $`m_{vis}`$ is given by

$$m_{vis}^2=\left[E_b+E_{\overline{b}}+E_{\mathrm{}}+E_h\right]^2\left[𝐩_b+𝐩_{\overline{b}}+𝐩_{\mathrm{}}+𝐩_{had}\right]^2,$$
(20)
where $`E`$ and $`𝐩`$ denote the measured energy and momentum of a particle. Fig. 2 demonstrates that, for $`m_H=120`$ GeV, the signal peaks at significantly larger values of $`m_{vis}`$ than the background processes.

The QCD $`b\overline{b}\tau \tau `$ background peaks at smaller $`m_{vis}`$ values because the $`b\overline{b}`$ system does not form a heavy resonance. The $`t\overline{t}`$ background peaks below $`2m_t`$ due to the additional neutrinos produced in the $`W\tau \nu _\tau `$ decays. Although the shape of the visible invariant mass distribution provides a tool to discriminate signal and background in $`ppb\overline{b}\tau _{\mathrm{}}\tau _h`$, the combined QCD $`b\overline{b}\tau \tau `$ and $`t\overline{t}`$ background is much larger than the signal. In addition, the signal cross section is very small: at the SLHC (VLHC), one expects about 20 (140) signal events for $`m_H=120`$ GeV. The number of signal events decreases quickly with increasing Higgs boson mass, due to the rapidly falling $`Hb\overline{b}`$ and $`H\tau \tau `$ branching fractions. Since the background decreases much less, $`S/B`$, and thus the bounds on $`\lambda `$, worsen with increasing values of $`m_H`$.

To determine whether useful information on the Higgs boson self-coupling can be extracted from the $`b\overline{b}\tau _{\mathrm{}}\tau _h`$ final state, we again perform a $`\chi ^2`$ test on the $`m_{vis}`$ distribution. Since the signal cross section is too small to be observable at the LHC, we derive bounds only for the SLHC and a VLHC. Allowing for a normalization uncertainty of $`10\%`$ of the SM cross section, for $`m_H=120`$ GeV we find $`1\sigma `$ bounds of

$`1.6<`$ $`\mathrm{\Delta }\lambda _{HHH}<3.1`$ $`(\mathrm{SLHC}),`$ (21)
$`0.84<`$ $`\mathrm{\Delta }\lambda _{HHH}<0.96`$ $`(\mathrm{VLHC}).`$ (22)
While these bounds are a factor 1.5 – 2.5 more stringent than those which can be obtained from $`HH4b`$ (see Eq. (10)), they are a factor 5 – 10 less stringent than those one hopes to achieve with 1 ab-1 at a linear collider operating at 500 GeV . For $`m_H=140`$ GeV, we obtain limits which are more than a factor two weaker than those for $`m_H=120`$ GeV.

### C Supersymmetric Higgs Bosons

We close this section with a brief remark on Higgs boson pair production in supersymmetric models. In the MSSM, pair production of $`CP`$-odd Higgs bosons, $`ggAA`$, is enhanced by a factor $`\mathrm{tan}^4\beta `$ . $`AA`$ production may thus be observable in the $`b\overline{b}\tau _{\mathrm{}}\tau _h`$ final state at the LHC if $`\mathrm{tan}\beta `$ is sufficiently large and $`m_A`$ is small. In this region of supersymmetric parameter space the $`CP`$-odd Higgs boson, $`A`$, can of course be produced via bottom quark fusion, which is enhanced by a factor $`\mathrm{tan}^2\beta `$ compared to the usual gluon fusion process. However, it is difficult to observe the additional final state $`b`$-jets and to verify that the process indeed proceeds through an enhanced bottom Yukawa coupling. Observing the correspondingly huge increase of the pair production cross section would confirm the presence of a large $`\mathrm{tan}\beta `$ enhancement factor. For example, for $`m_A=120`$ GeV and $`\mathrm{tan}\beta =35`$, we find a cross section (including $`b`$-tagging and hadronic $`\tau `$ decay efficiencies, and using the same cuts as in the SM case) of about 0.06 fb at the LHC, yielding about 20 signal events for an integrated luminosity of 300 fb-1. The combined $`t\overline{t}`$ and QCD $`b\overline{b}\tau \tau `$ background is about 17 events. $`AA`$ production thus should be observable at the LHC with a significance of $`5\sigma `$ or more if $`\mathrm{tan}\beta >35`$. Unfortunately, since the Feynman diagrams involving the $`HAA`$ and $`hAA`$ couplings are only enhanced by a factor $`\mathrm{tan}^2\beta `$, the process $`ggAA`$ is very insensitive to these couplings.

It should be noted that, for the values of $`m_A`$ and $`\mathrm{tan}\beta `$ chosen here, $`A`$ and the heavy $`CP`$ even Higgs boson, $`H`$, are almost degenerate in mass ($`m_H125`$ GeV). The invariant mass resolution of the LHC detectors will make it impossible to separate the $`A`$ and $`H`$ bosons in this case, and one will in fact observe a combined $`AA+AH+HH`$ signal. For $`m_A=120`$ GeV and $`\mathrm{tan}\beta =35`$, the $`HH`$ cross section is approximately a factor 6 smaller than the $`AA`$ cross section, whereas the $`AH`$ cross section is negligible .

## IV A heavier Higgs boson at linear colliders

We now turn our attention to Higgs boson pair production in $`e^+e^{}`$ collisions. A detailed study of how well the Higgs boson self-coupling for $`m_H=120`$ GeV can be measured in $`e^+e^{}ZHH`$ at $`\sqrt{s}=500`$ GeV was presented in Ref. . Here we consider $`HH`$ production via both $`e^+e^{}ZHH`$ and $`e^+e^{}HH\nu \overline{\nu }`$ for a heavier Higgs boson, in particular for $`m_H150`$ GeV, where $`HWW`$ decays dominate; and for center of mass energies in the range $`0.51`$ TeV. For $`HH\nu \overline{\nu }`$ production, we take into account the $`WW`$ fusion diagrams considered in Ref. , as well as the diagrams contributing to $`e^+e^{}Z(\nu \overline{\nu })HH`$. The $`WW`$ fusion diagrams contribute only to the $`HH\nu _e\overline{\nu }_e`$ final state.

If $`m_H140`$ GeV, the dominant decay mode of the SM Higgs boson is $`Hb\overline{b}`$. In this mass range, Higgs bosons which are pair produced in $`e^+e^{}ZHH`$ can be identified with high efficiency via the $`b`$-quark content of the system recoiling against the $`Z`$ boson (which may either decay hadronically or leptonically). As demonstrated in Ref. , it is sufficient to require that only one $`b`$-quark is tagged. If one assumes a tagging efficiency for $`b`$-quarks of $`ϵ_b=0.8`$ , this can be done with an efficiency close to $`100\%`$. In $`HH\nu \overline{\nu }`$ production, on the other hand, the presence of neutrinos makes it necessary to fully reconstruct the event . In this case we require that both Higgs bosons decay into $`b`$-quark pairs, and that all $`b`$-quarks are identified.

For $`ZHH`$ production, where both Higgs bosons decay either into $`W`$ or $`Z`$ boson pairs, we consider $`HH8`$ jets, $`HH\mathrm{}\nu +6`$ jets ($`\mathrm{}=e,\mu `$) and $`HH\mathrm{}^+\mathrm{}^{}+6`$ jets. The first two final states have the largest individual branching ratios of all $`4V`$ ($`V=W,Z`$) channels. For $`e^+e^{}HH\nu \overline{\nu }`$, we restrict ourselves to the neutrino-less 8 jets and $`\mathrm{}^+\mathrm{}^{}+6`$ jets final states. The total $`e^+e^{}ZHH`$ and $`e^+e^{}HH\nu \overline{\nu }`$ cross sections times branching ratios for the final states discussed above are shown in Figs. 3 and 4 as a function of $`m_H`$.

Since the $`Hb\overline{b}`$ branching ratio decreases rapidly for increasing $`m_H`$, the $`e^+e^{}ZHH`$, $`HHb+`$jets cross section falls quickly. For $`m_H<140`$ GeV, the $`ZHH`$ cross section at a 1 TeV linear collider is smaller than that obtained at a machine operating at 500 GeV. For larger Higgs boson masses, phase space severely limits the cross section for $`\sqrt{s}=500`$ GeV. The combined $`ZHH`$, $`HH8`$ jets, $`\mathrm{}\nu +6`$ jets, and $`\mathrm{}^+\mathrm{}^{}+6`$ jets cross section peaks for $`m_H165`$ GeV, with only 18 (35) events/ab-1 produced at $`\sqrt{s}=500`$ GeV ($`\sqrt{s}=1`$ TeV) before any detection efficiencies are taken into account. The $`e^+e^{}HH\nu \overline{\nu }`$ cross sections are about a factor 4 to 10 smaller than those for $`ZHH`$ production for the center of mass energies considered here. In addition to $`ZHH`$ and $`HH\nu \overline{\nu }`$ production, there is the process $`e^+e^{}HHe^+e^{}`$. Its cross section is a factor $`35`$ smaller than that for $`e^+e^{}HH\nu \overline{\nu }`$ for values of collider energies and Higgs boson masses considered here. Thus, we ignore $`HHe^+e^{}`$ production.

The results shown in Figs. 3 and 4 are for unpolarized beams. Assuming a polarization of $`𝒫_{}=0.8`$ for the electron and $`𝒫_+=0.6`$ for the positron beam, the $`e^+e^{}ZHH`$ ($`e^+e^{}HH\nu _e\overline{\nu }_e`$) cross section is a factor 1.70 (2.88) larger than that obtained for unpolarized beams.

Since we are interested in determining the Higgs boson self coupling, we note that the sensitivity to $`\lambda `$ of both the $`ZHH`$ and $`HH\nu \overline{\nu }`$ cross sections decreases (increases) with increasing collider energy (Higgs boson mass). While the $`ZHH`$ cross section grows with rising $`\lambda `$ in the vicinity of the SM value, the $`WW`$ fusion cross section diminishes . These effects partially cancel in the $`e^+e^{}HH\nu \overline{\nu }`$ cross section and considerably reduce its sensitivity to the Higgs boson self-coupling.

### A $`𝐦_H140`$ GeV

Figures 3 and 4 demonstrate that $`ZHH`$ production followed by $`HHb+`$jets is the dominant source of $`HH`$ events in the SM if $`m_H140`$ GeV. The main backgrounds in this channel are top quark and $`W`$ pair production. These are efficiently suppressed by performing a neural net (NN) analysis. Such an analysis, including a detailed detector simulation, was presented in Ref.  for $`m_H=120`$ GeV and $`\sqrt{s}=500`$ GeV. It concluded that $`\lambda `$ can be determined with a precision of about $`23\%`$ if an integrated luminosity of 1 ab-1 can be achieved. As we have seen in Sec. III, the limits achievable at hadron colliders for $`m_H=120`$ GeV are significantly weaker. To see whether this statement also holds for other Higgs boson masses (with $`m_H140`$ GeV) and other collider energies, it is necessary to extend the result of Ref.  to larger Higgs boson masses and collider energies.

Since we do not have the tools available which enabled the authors of Ref.  to carry out their analysis, we use the following simple procedure to estimate bounds for $`\lambda _{HHH}`$. We calculate sensitivity limits from the number of $`ZHH`$, $`Z\mathrm{}\mathrm{},jj`$, $`HHb+`$jets ($`\mathrm{}=e,\mu `$) signal events and the number of background events for a NN output parameter of $`N_N>0.9`$. $`N_N`$ measures how “signal-like” events are with $`N_N=1`$ ($`N_N=0`$) corresponding to perfect signal-like (background-like) events. For $`m_H=120`$ GeV, $`\sqrt{s}=500`$ GeV and an integrated luminosity of 1 ab-1, these numbers are taken from Ref. . We calculate the number of signal events for larger Higgs boson masses and higher center of mass energies from the $`ZHH`$ cross section, assuming that the NN efficiency is independent of both in the ranges considered. We estimate the number of background events assuming that it scales with the top quark cross section as a function of the collider energy.

A slight complication arises from the functional form of the $`ZHH`$ cross section, which is a quadratic function of $`\lambda _{HHH}`$. It is possible that two separate ranges of $`\lambda _{HHH}`$ exist which are consistent with the measured cross section. In this case we select the range which includes the SM value, $`\lambda _{HHH}=1`$.

We show our results as a function of $`m_H`$ in Fig. 5.

For $`m_H=120`$ GeV and $`\sqrt{s}=500`$ GeV, we find bounds which are about a factor 1.2 weaker than those reported in Ref. . For $`\sqrt{s}=800`$ GeV and the same $`m_H`$ our estimated limits agree well with those found in Ref. . Since the $`ZHH`$ cross section and its sensitivity to $`\lambda `$ decrease with increasing collider energy, a linear collider operating at 500 GeV offers the best chance for a precise measurement of $`\lambda `$ for $`m_H140`$ GeV; the bounds we obtain for $`\sqrt{s}=500`$ GeV are up to a factor 1.4 (1.9) more stringent than those achievable for $`\sqrt{s}=800`$ GeV ($`\sqrt{s}=1`$ TeV). The advantage of operating at 500 GeV gradually disappears with increasing Higgs boson mass, due to the reduced phase space. For $`\sqrt{s}<500`$ GeV, the bounds on $`\lambda `$ degrade quickly, likewise due to the rapidly shrinking phase space. The sensitivity limits achievable weaken by a factor 1.8 (1.2) for $`\sqrt{s}=500`$ GeV ($`\sqrt{s}=1`$ TeV) if $`m_H`$ increases from 120 GeV to 140 GeV; for $`m_H=140`$ GeV one will not be able to probe $`\lambda `$ with a precision of better than $`50\%`$ for unpolarized beams. Since the bounds which could be obtained from $`ppb\overline{b}\tau ^+\tau ^{}`$ degrade by a similar amount in this range (see Sec. III), we conclude that a $`0.51`$ TeV linear collider offers a significantly better chance to probe $`\lambda `$ for the mass range from 120 GeV to 140 GeV. If both the electron and positron beams can be polarized, the bounds derived here improve by a factor 1.3, assuming $`80\%`$ polarization for the electron beam and $`60\%`$ for the positron beam, and the same integrated luminosity as for unpolarized beams.

### B $`𝐦_H>140`$ GeV

If $`m_H>140`$ GeV, the channels yielding the largest event rates are $`e^+e^{}ZHH`$ with $`Zjj`$ and $`HH8`$ jets or $`\mathrm{}\nu +6`$ jets. Channels where one of the Higgs bosons decays into $`b\overline{b}`$, as well as $`HH\nu \overline{\nu }`$ production, result in negligible cross sections. In this section, we therefore concentrate on the $`\mathrm{}\nu +8`$ jet and the 10 jet final states.

Final states of similar structure and complexity are encountered in $`t\overline{t}H`$ production. If $`HWW`$, one also expects $`\mathrm{}\nu +8`$ jet and 10 jet events . If the Higgs boson predominantly decays to bottom quarks, $`\mathrm{}\nu +6`$ jet and 8 jet events are produced. In contrast to $`ZHH`$ production, the $`\mathrm{}\nu +`$jets and all jets events originating from $`t\overline{t}H`$ production contain two or more $`b`$-quarks. The processes $`e^+e^{}t\overline{t}Hq\overline{q}b\mathrm{}\nu \overline{b}b\overline{b},q\overline{q}bq\overline{q}bb\overline{b}`$ were analyzed in detail in Ref. .

The main background processes contributing both to $`e^+e^{}ZHH`$ and $`e^+e^{}t\overline{t}H`$ are $`WW+`$ jets, $`t\overline{t}+`$jets and QCD multijet production. In the $`t\overline{t}H`$ case, the combined background cross section is several orders of magnitude larger than that of the signal. To reduce the background, one first imposes preselection cuts to remove as much background as possible. One optimizes $`S/B`$ via a NN analysis in a second step. We list the efficiencies and signal to background ratios found in Ref.  for $`t\overline{t}H`$ production for both steps in the analysis in Table II.

Before imposing any cuts, the $`e^+e^{}ZHH10`$ jets ($`\mathrm{}\nu +8`$ jets) cross section is about a factor 30 smaller than that for $`e^+e^{}t\overline{t}H8`$ jets ($`\mathrm{}\nu +6`$ jets). Due to the additional two jets in the final state, the background to $`ZHH`$ production is suppressed by a factor $`\alpha _s^2`$, resulting in a background cross section which is roughly one order of magnitude smaller than in the $`t\overline{t}H`$ case. The signal to background ratios before cuts for $`ZHH`$ and $`t\overline{t}H`$ production therefore are similar, and a NN analysis for $`ZHH`$ production will likely lead to reductions of the signal efficiencies and the background rates which are similar to those encountered in the $`t\overline{t}H`$ analysis of Ref. .

Exact sensitivity bounds for $`\lambda `$ in $`e^+e^{}ZHH\mathrm{}\nu +8`$ jets and $`e^+e^{}ZHH10`$ jets could be derived only after performing a detailed NN analysis, which is beyond the scope of this paper. Instead, we investigate how the sensitivity bounds for $`\lambda `$ depend on the signal efficiencies and the signal to background ratio. We then explore the prospects for determining the Higgs boson self-coupling in $`ZHH`$ production for $`m_H>140`$ GeV using the results of Table II, which we argued may be used as rough guidelines.

We perform our analysis assuming $`m_H=180`$ GeV, $`\sqrt{s}=1`$ TeV, and an integrated luminosity of 1 ab-1. Since the number of signal events is small, we combine the $`\mathrm{}\nu +8`$ jet and 10 jet final states and use the total cross section (including branching ratios and efficiencies) to derive sensitivity limits. We show the $`1\sigma `$ sensitivity limits for $`\mathrm{\Delta }\lambda _{HHH}`$ in Fig. 6 as a function of the efficiency of the semi-leptonic $`\mathrm{}\nu +8`$ jet final state, $`ϵ_{sl}`$, for several choices of $`S/B`$, assuming a fixed ratio of $`ϵ_{had}/ϵ_{sl}=1/3`$ for the efficiencies of the hadronic 10 jet and the semi-leptonic $`\mathrm{}\nu +8`$ jet channels.

This $`ϵ_{had}/ϵ_{sl}`$ is motivated by the NN results of Ref.  (see Table II). For larger (smaller) values of $`ϵ_{had}/ϵ_{sl}`$, somewhat more (less) stringent bounds are obtained.

Figure 6 demonstrates that the bounds achievable on $`\lambda `$ in $`e^+e^{}ZHH10`$ jets and $`e^+e^{}ZHH\mathrm{}\nu +8`$ jets depend strongly on $`ϵ_{sl}`$ and $`S/B`$. The latter dependence is more transparent in Fig. 7, where we show the $`1\sigma `$ sensitivity limits as a function of $`S/B`$ for the $`t\overline{t}H`$ preselection and NN efficiencies (see Table II).

Sensitivity bounds better than unity occur only for high efficiencies, similar to the $`t\overline{t}H`$ preselection efficiencies, and if $`S/B>0.5`$. For the more likely case that $`ϵ_{sl}`$, $`ϵ_{had}`$ and $`S/B`$ are similar to the values obtained in the $`t\overline{t}H`$ analysis, a first-generation LC could obtain only very loose bounds on $`\lambda _{HHH}`$. Using the values for the NN analysis listed in Table II for illustration purposes, one finds

$$4.1<\mathrm{\Delta }\lambda _{HHH}<1.0$$
(23)
at the $`1\sigma `$ level for $`m_H=180`$ GeV, $`\sqrt{s}=1`$ TeV and 1 ab-1. We find very similar constraints for both options of Table II, the cuts analysis and the NN analysis. To be sure, one should really perform a NN analysis for $`HH`$ that anti-tags $`b`$-jets, instead of tags them to confirm the presence of top quarks. However, as we argued previously, $`S/B`$ is already quite poor, and other backgrounds are of the same size as $`t\overline{t}`$, so the values in Table II can be taken to be fair approximations to what one might expect for the Higgs pair production signal. As noted earlier in this section, the limits achievable improve by about a factor 1.3 for electron and positron beam polarizations of $`80\%`$ and $`60\%`$, respectively, and if the same integrated luminosity as in the unpolarized case can be reached.

For comparison, the LHC (SLHC) $`\mathrm{\Delta }\lambda _{HHH}`$ can give $`1\sigma `$ constraints of $`0.3<\mathrm{\Delta }\lambda _{HHH}<1.6`$ ($`0.10<\mathrm{\Delta }\lambda _{HHH}<0.12`$ for $`m_H=180`$ GeV. The LHC with 300 fb-1 will thus be able to better constrain $`\mathrm{\Delta }\lambda _{HHH}`$ for negative values than a first generation linear collider with 1 ab-1 operating at 1 TeV. This is a fortuitous effect of the destructive interference of the two diagrams in the gluon fusion process. For positive values, the linear collider may enjoy a slight advantage over the LHC. We reach similar conclusions for $`m_H=160`$ GeV and $`\sqrt{s}=800`$ GeV. For $`m_H<160`$ GeV and $`m_H>180`$ GeV, fewer than 5 signal events would be seen if efficiencies are smaller than 0.5, disallowing bounds to be placed on $`\lambda _{HHH}`$.

## V Reconstructing the Higgs Potential at Lepton and Hadron Colliders

The results of the previous sections, together with those of Refs.  and , can be used to compare the capabilities of future lepton and hadron colliders to reconstruct the Higgs potential. In order to translate bounds on $`\mathrm{\Delta }\lambda _{HHH}=(\lambda \lambda _{SM})/\lambda _{SM}`$ into constraints on the Higgs potential which can be graphically displayed, it is convenient to consider the scaled Higgs potential

$$\frac{2}{v^2m_H^2}V(x)=x^2+\lambda _{HHH}x^3+\frac{1}{4}\stackrel{~}{\lambda }_{4H}x^4,$$
(24)
where

$$x=\frac{\eta _H}{v},$$
(25)
$`\stackrel{~}{\lambda }_{4H}=\stackrel{~}{\lambda }/\lambda _{SM}`$ is the four Higgs boson self-coupling normalized to the SM value ($`\lambda _{SM}`$ is given in Eq. (2)), $`\eta _H`$ is the physical Higgs field, and $`v=(\sqrt{2}G_F)^{1/2}`$ is the vacuum expectation value. In the following we assume $`\stackrel{~}{\lambda }_{4H}=1`$.

In Fig. 8 we show how well the scaled Higgs potential can be reconstructed for a light Higgs boson of mass $`m_H=120`$ GeV.

As demonstrated in Sec. III, hadron colliders have only very limited capabilities to probe $`\lambda `$ if $`m_H140`$ GeV. Higgs boson pair production with dual $`Hb\overline{b}`$ decays is swamped by the QCD $`4b`$ background. A slightly better chance is offered if one of the Higgs bosons decays into $`\tau `$ pairs, with one $`\tau `$ lepton decaying leptonically and the second decaying into hadrons. This channel will be invisible at the LHC, due to the small signal cross section. At the SLHC and VLHC a sufficient number of signal events is expected. However, the QCD $`b\overline{b}\tau \tau `$ and $`t\overline{t}`$ backgrounds limit the sensitivity to $`\mathrm{\Delta }\lambda _{HHH}`$ to $`𝒪(1)`$ and the Higgs potential can be reconstructed only poorly (dashed and dotted lines in Fig. 8). In contrast, at a 500 GeV linear collider with an integrated luminosity of 1 ab-1, $`\mathrm{\Delta }\lambda _{HHH}`$ can be measured with a precision of about $`20\%`$ , and the Higgs potential can be reconstructed fairly accurately.

We draw similar conclusions for other Higgs boson masses in the range $`120\mathrm{GeV}<m_H<140`$ GeV; the limits achievable for $`\mathrm{\Delta }\lambda _{HHH}`$ both at lepton and hadron colliders gradually weaken by about a factor 2 if $`m_H`$ is increased from 120 GeV to 140 GeV. While the constraints on the Higgs potential improve with increasing machine energy for hadron colliders, the opposite is true in $`e^+e^{}`$ collisions. Here, both the $`ZHH`$ cross section and its sensitivity to the Higgs boson self-coupling decrease with increasing values of the collider energy.

If the Higgs boson decays predominantly into a pair of $`W`$-bosons, i.e. if $`m_H150`$ GeV, a completely different picture emerges. Figure 9 displays how well the Higgs potential may be reconstructed at future colliders if $`m_H=180`$ GeV.

The dashed and dotted lines show the constraints on the Higgs potential which one may hope to achieve at the SLHC (LHC) in the $`(jj\mathrm{}^\pm \nu )(jj\mathrm{}_{}^{}{}_{}{}^{\pm }\nu )`$ channel. We derived these curves by converting the $`95\%`$ CL limits of Ref.  for $`\mathrm{\Delta }\lambda _{HHH}`$ into $`1\sigma `$ limits and using Eq. (24). While LHC experiments will only be able to put mild constraints on $`V(x)`$, a luminosity upgrade of the LHC will make it possible to reconstruct the Higgs potential quite precisely for this $`m_H`$ range.

At a linear collider with a center of mass energy in the $`0.81`$ TeV range and an integrated luminosity of 1 ab-1, the number of Higgs boson pair events is very limited. The dominant $`WW+`$ jets and $`t\overline{t}+`$ jets backgrounds are several orders of magnitude larger than the signal. As discussed in Sec. IV B, any analysis which attempts to improve $`S/B`$ to an acceptable level is likely to significantly reduce the signal efficiencies, and thus the sensitivity to $`\mathrm{\Delta }\lambda _{HHH}`$. As a result, it will be difficult to constrain the Higgs potential using linear collider data if $`m_H150`$ GeV. This point is illustrated by the dot-dashed lines in Fig. 9, which show how poorly $`V(x)`$ is constrained via $`e^+e^{}ZHH10`$ jets, $`\mathrm{}\nu +8`$ jets, at a 1 TeV $`e^+e^{}`$ collider with an integrated luminosity of 1 ab-1, if the efficiencies and the signal to background ratios would be equal to those obtained in Ref.  for $`e^+e^{}t\overline{t}H`$. Similar results are obtained for a wide range of efficiencies and $`S/B`$ values (see Fig. 7); the dot-dashed lines in Fig. 9 thus are representative. We obtain results similar to those shown in Fig. 9 for $`m_H=160`$ GeV. For Higgs boson masses between 150 GeV and 160 GeV, and for $`m_H>180`$ GeV, there are not enough signal events at an $`e^+e^{}`$ collider with center of mass energy in the $`0.81`$ TeV range to constrain the Higgs potential.

It should be noted that the prospects to determine the Higgs boson self-coupling and to reconstruct the Higgs potential at an $`e^+e^{}`$ collider for a Higgs boson with mass larger than 150 GeV improve dramatically at larger energies. The $`e^+e^{}HH\nu \overline{\nu }`$ cross section grows rapidly with energy , reaching about 0.5 fb for $`m_H=180`$ GeV and $`\sqrt{s}=3`$ TeV, the energy of the two beam linear collider CLIC proposed by CERN . At such a machine, with an integrated luminosity of 5 ab-1, it should be possible to determine $`\lambda _{HHH}`$ with a precision of about $`8\%`$ . The VLHC could achieve similar or better precision . As mentioned in Sec. II, one-loop electroweak radiative corrections change $`\lambda _{HHH}`$ by a similar amount . At CLIC or a VLHC it will thus be possible to probe the Higgs boson self-coupling at the quantum level. The constraints on the shape of the Higgs potential from $`HH\nu \overline{\nu }`$ production at CLIC and $`ggHH4W(jj\mathrm{}^\pm \nu )(jj\mathrm{}_{}^{}{}_{}{}^{\pm }\nu )`$ at a VLHC are shown in Fig. 10.

It will be possible to accurately reconstruct the Higgs potential at these machines.

## VI Discussion and Conclusions

A direct experimental investigation of the Higgs potential represents a conclusive test of the mechanism of electroweak symmetry breaking and mass generation. After the discovery of an elementary Higgs boson and the test of its couplings to fermions and gauge bosons, experimental evidence that the shape of the Higgs potential has the form required for breaking the electroweak symmetry will complete the proof that the masses of fermions and weak bosons are generated by spontaneous symmetry breaking. To probe the shape of the Higgs potential, one must determine the Higgs boson self-coupling.

Only Higgs boson pair production at lepton or hadron colliders can measure the Higgs boson self-coupling. Ref.  carried out a detailed study of how well this could be done for $`m_H=120`$ GeV in $`ZHH`$ production at a 500 GeV $`e^+e^{}`$ collider. Refs.  and  considered Higgs boson pair production at $`e^+e^{}`$ colliders operating in the $`25`$ TeV range. Refs.  determined the prospects at hadron colliders for $`150\mathrm{GeV}<m_H<200`$ GeV. In this paper, we tried to fill in gaps in the existing literature by considering Higgs boson pair production for a light Higgs boson with mass $`m_H140`$ GeV at hadron colliders, and for a Higgs boson of mass $`m_H>120`$ GeV at $`e^+e^{}`$ colliders, with particular emphasis on the range $`m_H150`$ GeV where decays into $`W`$ pairs dominate.

For pair production of a light Higgs boson at hadron colliders we considered the dominant $`4b`$ final state and the $`b\overline{b}\tau \tau `$ channel. The $`4b`$ final is swamped by the QCD background, which is more than two orders of magnitude larger than the signal. At the LHC, the number of $`b\overline{b}\tau \tau `$ signal events is too small to yield any useful information on the Higgs boson self-coupling. At the SLHC and VLHC, however, the improved signal to background ratio does yield somewhat better limits on $`\lambda `$ than the $`4b`$ final state, although the signal cross section in the $`b\overline{b}\tau \tau `$ channel is significantly smaller. Performing a $`\chi ^2`$ analysis for the visible invariant mass distribution, we found that it will be difficult to probe the Higgs boson self-coupling to better than about one, even at a 200 TeV VLHC.

If $`m_H140`$ GeV, the Higgs boson self-coupling can be determined with much greater precision in $`e^+e^{}ZHH`$, $`HHb+`$jets. We extrapolated the results of Ref.  to $`m_H>120`$ GeV and higher center of mass energies and found that, since both the $`ZHH`$ cross section and its sensitivity to $`\lambda `$ decrease with increasing center of mass energy, a 500 GeV $`e^+e^{}`$ collider operating is optimally suited to probe the Higgs boson self-coupling for $`120\mathrm{GeV}m_H140`$ GeV. The limits on the Higgs boson self-coupling for $`e^+e^{}`$ collisions at $`\sqrt{s}=500`$ GeV, assuming an integrated luminosity of 1 ab-1, are typically a factor 5 (10) more stringent than those that would come from a VLHC (SLHC) in this mass range. Data from a 500 GeV linear collider, however, will not be sufficiently sensitive to probe the electroweak one-loop corrections to $`\lambda `$. A multi-TeV $`e^+e^{}`$ collider will be the only machine capable of this if $`m_H140`$ GeV .

Due to phase space restrictions, a center of mass energy of at least 800 GeV would be needed to search for Higgs pair production in $`e^+e^{}`$ collisions if $`m_H150`$ GeV. For $`\sqrt{s}=0.81`$ TeV, $`e^+e^{}ZHH10\mathrm{jets},\mathrm{}\nu +8`$ jets via Higgs boson decays into weak boson pairs are the dominant Higgs pair production channels. The main contributions to the background originate from $`t\overline{t}+`$ jets and $`WW+`$ jets production, with cross sections several orders of magnitude larger than the signal. In such a situation, the only hope to improve the signal to background ratio, $`S/B`$, to an acceptable level is a NN analysis. We studied how the sensitivity bounds on $`\lambda `$ depend on the signal efficiencies and $`S/B`$. Using the results for $`e^+e^{}t\overline{t}H`$ production , where final states of similar complexity and $`S/B`$ are encountered, as guidelines, we conclude that it will be difficult to determine the Higgs boson self-coupling at a linear collider with $`\sqrt{s}=0.81`$ TeV with a precision equal to that which can be reached at the LHC with 300 fb-1. We reach this conclusion for a broad range of efficiencies and $`S/B`$ values; therefore, it does not depend on the specific values which were used. Experiments at both a multi-TeV $`e^+e^{}`$ collider (where $`HH\nu \overline{\nu }`$ production is the main source of Higgs pair events) and a VLHC will be able to probe the one-loop electroweak radiative corrections to $`\lambda `$ for $`m_H150`$ GeV.

Our results show that hadron colliders and $`e^+e^{}`$ linear colliders with $`\sqrt{s}1`$ TeV are complementary: for $`m_H140`$ GeV, linear colliders offer far better prospects in measuring the Higgs boson self-coupling, $`\lambda `$; for a Higgs boson in the range $`m_H150`$ GeV, the opposite is true. However, to actually perform a meaningful measurement at a hadron collider would demand precision Higgs boson properties input from a linear collider for the top quark Yukawa coupling, the $`HWW`$ coupling, and the total Higgs boson decay width.

Finally, we have explored how well various future colliders may constrain the shape of the Higgs potential, $`V(\eta _H)`$. To visualize how a non-standard Higgs self-coupling affects $`V(\eta _H)`$, we introduced a scaled version of the potential, expressed in terms of the dimensionless ratio $`x=\eta _H/v`$ (see Eq. (24)). Results for several machines and choices of $`m_H`$ are shown in Figs. 8 – 10.

###### Acknowledgements.

We would like to thank K. Desch, P. Gay, K. Jakobs, A. Juste, A. Nikitenko, T. Stelzer, A. Turcot, G. Wilson and P.M. Zerwas for useful discussions. One of us (U.B.) would like to thank the Fermilab Theory Group, where part of this work was carried out, for its generous hospitality. This research was supported in part by the National Science Foundation under grants No. PHY-9970703 and No. PHY-0139953.


format: text

text:
# Hadrons in Nuclei

## 1 Introduction

That hadrons can change their properties and couplings in the nuclear medium has been well known to nuclear physicists since the days of the Delta-hole model that dealt with the changes of the properties of the pion and Delta-resonance inside nuclei . Due to the predominant $`p`$-wave interaction of pions with nucleons one observes here a lowering of the pion branch with increasing pion-momentum and nucleon-density. A direct observation of this effect is difficult because of the strong final state interactions (in particular absorption) of the pions.

More recently, in-medium changes of vector mesons have found increased interest, mainly because these mesons couple strongly to the photon so that electromagnetic signals could yield information about properties of hadrons deeply embedded into nuclear matter. Indeed, the CERES experiment  has found a considerable excess of dileptons in an invariant mass range from $`300`$ MeV to $`700`$ MeV as compared to expectations based on the assumption of freely radiating mesons. This result has found an explanation in terms of a shift of the $`\rho `$ meson spectral function down to lower masses, as expected from theory (see, e.g.,  ).

However, the actual reason for the observed dilepton excess is far from clear. Both models that just shift the pole mass of the vector meson as well as those that also modify the spectral shape have successfully explained the data; in addition, even a calculation that just used the free radiation rates with their – often quite large – experimental uncertainties was compatible with the observations . There are also calculations that attribute the observed effect to radiation from a quark-gluon plasma .

I have therefore already some years ago proposed to look for the theoretically predicted changes of vector meson properties inside the nuclear medium in reaction on normal nuclei with more microscopic probes . Of course, the nuclear density felt by the vector mesons in such experiments lies much below the equilibrium density of nuclear matter, $`\rho _0`$, so that naively any density-dependent effects are expected to be smaller than in heavy-ion reactions.

On the other hand, there is advantage to these experiments: they proceed with the spectator matter being close to its equilibrium state. This is essential because all theoretical predictions of in-medium properties of hadrons are based on an equilibrium model in which the hadron (vector meson) under investigation is embedded in cold nuclear matter in equilibrium and with infinite extension. However, a relativistic heavy-ion reaction proceeds – at least initially – far from equilibrium. Even if equilibrium is reached this state changes by cooling through expansion and particle emission and any observed signal is built up by integrating over the emissions from all these different stages of the reaction.

In this paper I summarize results that we have obtained in studies of observable consequences of in-medium changes of hadronic spectral functions in reactions of elementary probes with nuclei. I demonstrate that the expected in-medium sensitivity in such reactions is as high as that in relativistic heavy-ion collisions and that in particular photonuclear reactions present an independent, cleaner testing ground for assumptions made in analyzing heavy-ion reactions.

## 2 Dilepton Production

The CERES experiment has received a lot of publicity for its observation of an excess of dileptons with invariant masses below those of the lightest vector mesons . Explanations of this excess have focussed on a change of in-medium properties of these vector mesons in dense nuclear matter. The radiating sources can be nicely seen in Fig. 1 that shows the latest dilepton spectrum obtained in a rather low-energy run at 40 AGeV.

The figure exhibits clearly the rather strong contributions of the vector mesons – both direct and through their Dalitz decay – at invariant masses above about 500 MeV. If this strength is shifted downward, caused by an in-medium change of the vector-meson spectral functions, then the observed excess can be explained as has been shown by various authors (see e.g. ).

As mentioned above such explanations always suffer from an inherent inconsistency: while the observed signal integrates over many different stages of the collision – nonequilibrium and equilibrium, the latter at various densities and temperatures – the theoretical input is always calculated under the assumption of a vector meson in nuclear matter in equilibrium. We have therefore looked for possible effects in reactions that proceed much closer to equilibrium and have thus studied the dilepton production in photon-induced reactions on the nucleus. It is not *a priori* hopeless to look for in-medium effects in ordinary nuclei: Even in relativistic heavy-ion reactions that reach baryonic densities of the order of 3 - 10 $`\rho _0`$ many observed dileptons actually stem from densities that are much lower than these high peak densities. Transport simulations have shown  that even at the CERES energies about 1/2 of all dileptons come from densities lower than $`2\rho _0`$. This is so because in such reactions the pion-density gets quite large in particular in the late stages of the collision, where the baryonic matter expands and its density becomes low again. Correpondingly many vector mesons are formed late in the collision (through $`\pi +\pi >\rho `$) and their decay to dileptons thus happens at low baryon densities.

The calculations are done in a combination of coherent initial state interactions that lead to shadowing at photon energies above about 1 GeV and incoherent final state interactions. The shadowed incoming photon produces, for example, a vector meson which then cascades through the nucleus. The latter process we describe by means of a coupled-channel transport theory. The details are discussed in ref. . A new feature of these calculations is that vector mesons with their correct spectral functions can actually be produced and transported consistently. This is quite an advantage over earlier treatments in which the mesons were always produced and transported with their pole mass and their spectral function was later on folded in only for their decay.

A typical result of such a calculation for the dilepton yield – after removing the Bethe-Heitler component – is given in Fig. 2.

Comparing this figure with Fig. 1 shows that exactly the same sources, and none less, contribute to the dilepton yield in a photon-induced reaction at 1 - 2 GeV photon energy as in relativistic heavy-ion collisions at 40 AGeV! The question now remains if we can expect any observable effect of possible in-medium changes of the vector meson spectral functions in medium in such an experiment on the nucleus where – due to surface effects – the average nucleon density is below $`\rho _0`$. This question is answered by the results of Fig. 3.

This figure shows the dilepton spectra to be expected if a suitable cut on the dilepton momenta is imposed. In the realistic case shown on the right, which contains both a collision broadening and a mass shift, it is obvious that a major signal is to be expected: in the heavy nucleus $`Pb`$ the $`\omega `$-peak has completely disappeared from the spectrum. The sensitivity of such reactions is thus as large as that observed in ultrarelativistic heavy-ion reactions.

An experimental verification of this prediction would be a major step forward in our understanding of in-medium changes111An experiment at JLAB is under way .. It would obviously present a purely hadronic base-line to all data on top of which all ’fancier’ explanations of the CERES effect in terms of radiation from a QGP and the such would have to live.

## 3 Hadron Formation

A major experimental effort at RHIC experiments has gone into the observation of jets in ultrarelativistic heavy-ion collisions and the determination of their interaction with the surrounding quark or hadronic matter . Such experiments are obviously very sensitive to hadron formation times. In addition they can yield information on interactions while the final hadron is still being formed.

A complementary process is given by the latest HERMES results at HERA for photon-induced hadron production at high energies . Here the photon-energies are of the order of 10 - 20 GeV, with rather moderate $`Q^22`$ GeV. Again, the advantage of such experiments is that the nuclear matter with which the interactions happen is at rest and in equilibrium.

In the high-energy regime the shadowing of the incoming photon, which is due to interference between interactions of the incoming bare photon and its hadronic components with the nucleons, becomes important. This coherence in the incoming state has to be combined with the incoherent treatment of the final state interactions in transport theory. For this purpose T. Falter has derived a novel expression for incoherent particle production on the nucleus  that allows for a clean-cut separation of the coherent initial state and the incoherent final state interactions which we again treat with our coupled-channel transport theory.

An example of the results obtained is given in Fig. 4. The figure clearly shows that the observed hadron multiplicities can be described only with formation times $`>0.3`$ fm. The curves obtained with larger formation times all lie very close together. This is a consequence of the finite size of the target nucleus: if the formation time is larger than the time needed for the preformed hadron to transverse the nucleus then the sensitivity to the formation time is lost. In  we have also shown that the $`z`$-dependence on the left side exhibits some sensitivity to the interactions of the leading hadrons during the formation; the curves show in Fig. 4 are obtained with a leading hadron cross section of $`0.25\sigma _h`$, where $`\sigma _h`$ is the ’normal’ hadronic interaction cross section.

## 4 Conclusions

In this talk I have illustrated with the help of two examples that photonuclear reactions can yield information that is important and relevant for an understanding of high density–high temperature phenomena in ultrarelativistic heavy-ion collisions. I have shown that the expected sensitivity of dilepton spectra in photonuclear reactions in the 1 - 2 GeV range is as large as that in ultrarelativistic heavy-ion collisions. I have also illustrated that the analysis of hadron production spectra in high-energy electroproduction experiments at HERMES gives information about the interaction of forming hadrons with the surrounding hadronic matter. This is important for any analysis that tries to obtain signals for a QGP by analysing high-energy jet formation in ultrarelativistic heavy-ion reactions.

## Acknowledgement

This talk is based on work done together with Martin Effenberger, Thomas Falter and Kai Gallmeister. The work on which it is based has been supported by the Deutsche Forschungsgemeinschaft, the BMBF and GSI Darmstadt.

* E-mail: mosel@physik.uni-giessen.de