text:
# 1 Introduction

## 1 Introduction

The arrival of top factories, LHC and TESLA, will bring a tremendous improvement in our knowledge of top quark properties . In particular, the large top samples produced will allow to perform precision studies of top rare decays. In this field, flavour-changing neutral (FCN) decays $`tcZ`$, $`tc\gamma `$, $`tcg`$, deserve special attention. Within the Standard Model (SM) they are mediated at lowest order in perturbation theory by penguin diagrams with quarks of charge $`Q=1/3`$ inside the loop. Due to the smallness of down-type quark masses compared to $`M_W`$, these decays are very suppressed by the GIM mechanism, in contrast with processes like $`bs\gamma `$, with diagrams with a top quark in the loop. This extra suppression results in decay rates $`O(10^{10})`$ or smaller . On the other hand, in several SM extensions the branching ratios for FCN top decays can be orders of magnitude larger. For instance, in two Higgs doublet models $`\mathrm{Br}(tcZ)10^6`$, $`\mathrm{Br}(tc\gamma )10^7`$, $`\mathrm{Br}(tcg)10^5`$ can be achieved . In supersymmetric models with $`R`$ parity conservation these branching ratios can reach $`\mathrm{Br}(tcZ)10^6`$, $`\mathrm{Br}(tc\gamma )10^6`$, $`\mathrm{Br}(tcg)10^5`$ .

Here we are interested in the possible enhancement of these rates in models with vector-like quark singlets. The addition of quark singlets to the SM particle content represents the simplest way to break the GIM mechanism consistently. In these models, the $`3\times 3`$ Cabibbo-Kobayashi-Maskawa (CKM) matrix is not unitary, and thus FCN couplings to the $`Z`$ boson appear at tree-level. FCN couplings between light quarks are experimentally constrained to be very small, but this is not the case for the top quark. Actually, top FCN vertices can mediate the decays $`tuZ`$ and $`tcZ`$, giving observable rates in models with up-type singlets . The largest branching ratios allowed by present experimental data are $`\mathrm{Br}(tuZ)=7.0\times 10^4`$, $`\mathrm{Br}(tcZ)=6.0\times 10^4`$ , much smaller than present direct limits $`\mathrm{Br}(tuZ),\mathrm{Br}(tcZ)0.08`$  but still observable at LHC  and TESLA . In this Letter we investigate the enhancement of the branching ratios for the two other FCN decays, $`tc\gamma `$ and $`tcg`$, in the presence of either up or down singlets. We find the rates of these processes allowed by present experimental constraints, and study how the GIM suppression takes place in these models. For completeness we also quote without discussion the results for $`tu\gamma `$ and $`tug`$, which in the SM are suppressed by the ratio $`|V_{ub}/V_{cb}|^2`$ with respect to the former, but in these SM extensions can have the same magnitude.

## 2 Overview of the Lagrangian

A full discussion of the Lagrangian in the weak eigenstate and mass eigenstate bases can be found for instance in Refs. . Here we only collect the terms of the Lagrangian in the mass eigenstate basis relevant for our study. We consider the SM extended with $`n_u`$ up singlets and $`n_d`$ down singlets, with $`n_u`$, $`n_d`$ arbitrary for the moment. The charged current Lagrangian is

$$_W=\frac{g}{\sqrt{2}}\overline{u}_L\gamma ^\mu Vd_LW_\mu ^++\mathrm{h}.\mathrm{c}.,$$
(1)
with $`V`$ the generalised CKM matrix, of dimension $`(3+n_u)\times (3+n_d)`$. The neutral-current Lagrangian describing the interactions with the $`Z`$ boson is

$$_Z=\frac{g}{2c_W}\left(\overline{u}_L\gamma ^\mu X^uu_L\overline{d}_L\gamma ^\mu X^dd_L2s_W^2J_{\mathrm{EM}}^\mu \right)Z_\mu ,$$
(2)
where $`X^u`$, $`X^d`$ are hermitian matrices of dimension $`(3+n_u)\times (3+n_u)`$ and $`(3+n_d)\times (3+n_d)`$, respectively. These matrices can be related to the CKM matrix by $`X^u=VV^{}`$, $`X^d=V^{}V`$. The interactions with the unphysical charged scalars $`\varphi ^\pm `$ are given by

$$_\varphi =\frac{g}{\sqrt{2}M_W}\overline{u}\left(^uVP_LV^dP_R\right)d\varphi ^++\mathrm{h}.\mathrm{c}.,$$
(3)
with $`^u`$ and $`^d`$ the diagonal mass matrices for the up and down quarks. The terms corresponding to the unphysical neutral scalar $`\chi `$ are

$`_\chi `$ $`=`$ $`{\displaystyle \frac{ig}{2M_W}}[\overline{u}(^uX^uP_LX^u^uP_R)u`$ (4)
$`\overline{d}(^dX^dP_LX^d^dP_R)d]\chi .`$
Finally, the terms describing the interactions with the Higgs boson are

$`_H`$ $`=`$ $`{\displaystyle \frac{g}{2M_W}}[\overline{u}(^uX^uP_L+X^u^uP_R)u`$ (5)
$`+\overline{d}(^dX^dP_L+X^d^dP_R)d]H.`$
In our analysis of $`tc\gamma `$ and $`tcg`$ we discuss the two simplest cases: $`n_u=1`$, $`n_d=0`$ (which will be called Model I) and $`n_u=0`$, $`n_d=1`$ (Model $`\mathrm{I}\mathrm{I}`$). These two cases correspond to CKM matrices of dimension $`4\times 3`$ and $`3\times 4`$, respectively, and in both models the CKM matrix is a submatrix of a unitary $`4\times 4`$ matrix.

## 3 Calculation of the decay rates

Using unbroken $`\mathrm{SU}(3)\times \mathrm{U}(1)`$ gauge invariance and the facts that final state particles are on-shell and the photon has transverse polarisation, the transition amplitude for $`tc\gamma `$ can be written with all generality as

$`(tc\gamma )`$ $`=`$ $`\overline{u}(p_c)\left[i\sigma ^{\mu \nu }q_\nu \left(A_\gamma +B_\gamma \gamma _5\right)\right]u(p_t)ϵ_\mu ^{}(q),`$ (6)
with $`p_t`$ and $`p_c`$ the momenta of the top and charm quarks, respectively, $`q=p_tp_c`$ the photon momentum and $`ϵ`$ its polarisation vector. This expression also assumes that the top quark is on-shell, which is an excellent approximation. A similar structure is valid for $`tcg`$, with form factors $`A_g`$ and $`B_g`$. In order to compute the amplitude the form factors are written in terms of Passarino-Veltman integrals  using FORM . The integrals are numerically evaluated using LoopTools . The Feynman diagrams relevant for $`tc\gamma `$ in the SM and Model $`\mathrm{I}\mathrm{I}`$ are depicted in Fig. 1. In the SM the down-type quarks $`d_i`$ in the loops are $`d_i=d,s,b`$, while in Model $`\mathrm{I}\mathrm{I}`$ there is an extra heavy quark $`B`$. The contributions of these diagrams to $`A_\gamma `$ and $`B_\gamma `$ in the ’t Hooft-Feynman gauge are collected in the Appendix. The diagrams relevant for $`A_g`$ and $`B_g`$ are the analogous to (1a) and (1b) in Fig. 1 but replacing the outgoing photon by a gluon.

In Model I there are extra diagrams with up-type quarks $`u_i=u,c,t,T`$ in the loops (see Fig. 2). These diagrams have one FCN vertex for $`u_i=c,t`$ and two for $`u_i=u,T`$, in which case they are very suppressed. The flavour-diagonal vertices are modified with respect to the SM value. For instance, the diagonal couplings of a quark $`q=u_i,d_i`$ to the $`Z`$ boson are

$`c_L^q`$ $`=`$ $`\pm X_{qq}2Q_qs_W^2,`$
$`c_R^q`$ $`=`$ $`2Q_qs_W^2,`$ (7)
as can be seen from Eq. (2), with the plus (minus) sign for up (down) quarks. The interactions with the unphysical scalar $`\chi `$ and the Higgs boson can be read from Eqs. (4,5). The contributions of these diagrams to $`A_\gamma `$ and $`B_\gamma `$ can be found in the Appendix.

We perform the computation keeping all quark masses. For external quarks we use the pole masses $`m_t=174.3`$ GeV, $`m_c=1.5`$ GeV. For internal quarks it is more adequate to use $`\overline{\mathrm{MS}}`$ masses at a scale $`O(m_t)`$, rather than pole masses. This is an important difference due to the strong dependence on the $`b`$ quark mass as a consequence of the GIM suppression. For a pole mass $`m_b=4.7\pm 0.3`$ GeV, $`\overline{m}_b(m_t)=2.74\pm 0.17`$ GeV .

In the limit $`m_c=0`$ the vector and axial form factors are equal: $`A_\gamma =B_\gamma `$, $`A_g=B_g`$. Since $`m_c`$ is small, $`A_\gamma B_\gamma `$, $`A_gB_g`$ and the effective couplings are predominantly right-handed. One important feature is that the form factors acquire imaginary parts from the contributions with $`d`$, $`s`$, $`b`$ quarks (and $`u`$, $`c`$ quarks in the extra diagrams present in Model I). These imaginary parts are one of the ingredients needed in order to have CP asymmetries $`\mathrm{\Gamma }(tc\gamma )\mathrm{\Gamma }(\overline{t}\overline{c}\gamma )`$, which will be analysed in detail later.

From Eq. (6), the partial widths of these processes are

$`\mathrm{\Gamma }(tc\gamma )`$ $`=`$ $`{\displaystyle \frac{1}{\pi }}\left[{\displaystyle \frac{m_t^2m_c^2}{2m_t}}\right]^3\left(|A_\gamma |^2+|B_\gamma |^2\right),`$
$`\mathrm{\Gamma }(tcg)`$ $`=`$ $`{\displaystyle \frac{C_F}{\pi }}\left[{\displaystyle \frac{m_t^2m_c^2}{2m_t}}\right]^3\left(|A_g|^2+|B_g|^2\right),`$ (8)
with $`C_F=4/3`$ a colour factor. In the SM, as well as in our models, the total width is dominated by the leading decay mode $`tbW^+`$, $`\mathrm{\Gamma }(tbW^+)=1.57|V_{tb}|^2`$ for $`m_t=174.3`$ GeV, $`M_W=80.39`$ GeV. The branching ratios are then

$$\mathrm{Br}(tc\gamma )=\frac{\mathrm{\Gamma }(tc\gamma )}{\mathrm{\Gamma }(tbW^+)},\mathrm{Br}(tcg)=\frac{\mathrm{\Gamma }(tcg)}{\mathrm{\Gamma }(tbW^+)}.$$
(9)
We do not use the next-to-leading order partial width $`\mathrm{\Gamma }(tbW^+)=1.42|V_{tb}|^2`$ for consistency, because our calculation for $`tc\gamma `$, $`tcg`$ is at leading order.

We have checked that using the set of input parameters of Ref.  our results agree with the results presented there. For the calculation of the branching ratios within the SM we take $`|V_{us}|=0.2224`$, $`|V_{ub}|=0.00362`$, $`|V_{cb}|=0.0402`$. These values are obtained performing a fit to the six measured CKM matrix elements, using $`3\times 3`$ unitarity. The phase $`\delta `$ in the standard parameterisation  is $`\delta =1.014`$, obtained with a fit to $`\epsilon `$, $`\epsilon ^{}/\epsilon `$, $`a_{\psi K_S}`$ and $`|\delta m_B|`$ (see Ref. ). The SM predictions are

$`\mathrm{Br}(tc\gamma )`$ $`=`$ $`(4.6{}_{1.0}{}^{+1.2}\pm 0.4{}_{0.5}{}^{+1.6})\times 10^{14},`$
$`\mathrm{Br}(tcg)`$ $`=`$ $`(4.6{}_{0.9}{}^{+1.1}\pm 0.4{}_{0.7}{}^{+2.1})\times 10^{12}.`$ (10)
The first uncertainty comes from the bottom mass, the second from CKM mixing angles and the third is estimated varying the renormalisation scale between $`M_Z`$ (plus sign) and $`1.5m_t`$ (minus sign). These figures are ten times smaller than the ones quoted in Ref. , where the pole mass is used for the internal $`b`$ quark ($`m_b=5`$ GeV is assumed). The uncertainty in the top mass does not affect these values, because the partial widths of $`tc\gamma `$, $`tcg`$ are proportional to $`m_t^3`$, and the partial width of $`tbW^+`$ is approximately given by

$`\mathrm{\Gamma }(tbW^+)`$ $`=`$ $`{\displaystyle \frac{g^2}{64\pi }}|V_{tb}|^2{\displaystyle \frac{m_t^3}{M_W^2}}\left[13{\displaystyle \frac{M_W^4}{m_t^4}}+2{\displaystyle \frac{M_W^6}{m_t^6}}\right].`$ (11)
Hence, the leading dependence on $`m_t`$ cancels in the ratios and the uncertainty in $`m_t`$ hardly affects the numbers quoted in Eqs. (10). The SM predictions for $`tu\gamma `$ and $`tug`$ are

$`\mathrm{Br}(tu\gamma )`$ $`=`$ $`(3.7{}_{0.8}{}^{+1.0}\pm 2.1{}_{0.4}{}^{+1.3})\times 10^{16},`$
$`\mathrm{Br}(tug)`$ $`=`$ $`(3.7{}_{0.8}{}^{+0.9}\pm 2.1{}_{0.5}{}^{+1.7})\times 10^{14},`$ (12)
suppressed by a factor $`|V_{ub}/V_{cb}|^28\times 10^3`$ with respect to top decays to a charm quark. The uncertainties have the same origin as in Eqs. (10).

## 4 CKM unitarity and GIM suppression

Let us discuss how the GIM mechanism suppresses these processes within the SM and how this suppression can be partially removed with the addition of vector-like singlets. We only study $`tc\gamma `$, the discussion of $`tcg`$ is formally identical. In the SM and Model $`\mathrm{I}\mathrm{I}`$ the form factors for the $`\gamma tc`$ vertex can be decomposed as

$`A_\gamma `$ $`=`$ $`{\displaystyle \underset{i}{}}f_{\gamma A}(m_i)\lambda _{ct}^i,`$
$`B_\gamma `$ $`=`$ $`{\displaystyle \underset{i}{}}f_{\gamma B}(m_i)\lambda _{ct}^i,`$ (13)
where $`i=1,2,3`$ in the SM and $`i=1\mathrm{}4`$ in Model $`\mathrm{I}\mathrm{I}`$, $`f_{\gamma A}(m_i)`$, $`f_{\gamma B}(m_i)`$ are functions of the internal quark mass and $`\lambda _{ct}^i=V_{ci}V_{ti}^{}`$ are CKM factors. We have dropped the bar over $`m_i`$, which are understood as $`\overline{\mathrm{MS}}`$ masses. Since $`f_{\gamma A}(m_i)f_{\gamma B}(m_i)`$, we only analyse $`A_\gamma `$. The mass dependence of the real and imaginary parts of $`f_{\gamma A}(m_i)`$ is shown in Fig. 3.

In order to estimate the branching ratio for $`tc\gamma `$ we make the approximation $`f_{\gamma A}(m_d)=f_{\gamma A}(m_s)=f_{\gamma A}(0)`$ for the moment. Then, using

$$\lambda _{ct}^d+\lambda _{ct}^s+\lambda _{ct}^b+\lambda _{ct}^B=0,$$
(14)
as implied by the unbroken row unitarity of the $`3\times 4`$ CKM matrix $`V`$, we have

$`A_\gamma `$ $`=`$ $`\left[f_{\gamma A}(m_b)f_{\gamma A}(0)\right]\lambda _{ct}^b+\left[f_{\gamma A}(m_B)f_{\gamma A}(0)\right]\lambda _{ct}^B`$ (15)
$``$ $`f_{\gamma A}^{}(m_b)\lambda _{ct}^b+f_{\gamma A}^{}(m_B)\lambda _{ct}^B.`$
Therefore, the decay amplitude is actually controlled by the shifted function $`f_{\gamma A}^{}(m_i)`$, plotted in Fig. 3 as well. The parameter $`\lambda _{ct}^B`$ measures the orthogonality of the $`c`$ and $`t`$ rows of the $`3\times 3`$ CKM submatrix $`V_{3\times 3}`$ (see Eq. (14)), i. e. the breaking of the GIM mechanism in this process. The SM limit is recovered setting, $`\lambda _{ct}^B=0`$, so the only contribution to the form factor is given by the small function $`f_{\gamma A}^{}(m_b)9.1\times 10^94.7\times 10^9i`$ multiplied by $`\lambda _{ct}^b0.04`$. With an extra down quark, there is a new term with a larger function $`f_{\gamma A}^{}(m_B)4.9\times 10^7+6.0\times 10^6i`$ (for $`m_B=200`$ GeV), which is however suppressed by $`\lambda _{ct}^B`$.

The parameter $`\lambda _{ct}^B`$ can be related to the breaking of the column unitarity of $`V_{3\times 3}`$. This is easily understood, because if the columns of this submatrix are orthogonal, so must be the rows. The explicit relation can be written using the extension of the Wolfenstein parameterisation  in Ref. . Assuming that $`X_{ds},X_{db},X_{sb}\lambda ^4`$, $`1X_{ss}\lambda ^3`$ and $`1X_{bb}\lambda ^3`$, we have

$$\lambda _{ct}^B=\underset{i=1}{\overset{3}{}}V_{ci}V_{ti}^{}=X_{sb}\lambda X_{db}+A\lambda ^2\left(X_{bb}X_{ss}\right)\frac{\lambda ^2}{2}X_{sb}+O(\lambda ^7).$$
(16)
This equation shows how the breaking of the orthogonality of the first three columns of $`V`$ “propagates” to the second and third rows. The effect of the new quark can be estimated with $`\lambda 0.22`$, $`A1`$ and the typical values $`X_{db}10^3`$, $`X_{sb}10^3`$, $`X_{bb}X_{ss}10^3`$ , obtaining $`\lambda _{ct}^BX_{sb}10^3`$. With this value the $`B`$ contribution is 20 times larger than the $`b`$ term, giving $`\mathrm{Br}(tc\gamma )10^{11}`$.

In Model I, neglecting for the moment diagrams with two FCN vertices, $`A_\gamma `$ can be decomposed as

$$A_\gamma =\underset{i=1}{\overset{3}{}}f_{\gamma A}(m_i)\lambda _{ct}^i+g_{\gamma A}X_{ct},$$
(17)
with $`g_{\gamma A}`$ the sum of the $`c`$ and $`t`$ diagram contributions, which is roughly of the same size as the $`f_{\gamma A}`$ functions. In this model we have the relation

$$\lambda _{ct}^d+\lambda _{ct}^s+\lambda _{ct}^b=X_{ct}$$
(18)
expressing the non-orthogonality of the second and third rows of the CKM matrix, of dimension $`4\times 3`$ in this case (compare with Eq. (14)). Hence, the form factor is written as

$`A_\gamma `$ $`=`$ $`\left[f_{\gamma A}(m_b)f_{\gamma A}(0)\right]\lambda _{ct}^b+\left[g_{\gamma A}+f_{\gamma A}(0)\right]X_{ct}`$ (19)
$``$ $`f_{\gamma A}^{}(m_b)\lambda _{ct}^b+g_{\gamma A}^{}X_{ct},`$
with $`g_{\gamma A}^{}=4.4\times 10^64.8\times 10^6i`$. In this model the FCN coupling $`X_{ct}`$ can be $`X_{ct}0.04`$ for $`V_{tb}0.6`$ , yielding a branching ratio $`\mathrm{Br}(tc\gamma )5\times 10^8`$.

We note that the larger branching ratio achieved in Model I is not a consequence of the presence of a tree-level coupling $`Ztc`$, which appears in the expressions of the form factors on the same footing as the parameter $`\lambda _{ct}^B`$. Moreover, the loop integrals of the new physics contributions of Model I, $`g_{\gamma A}^{}=4.4\times 10^64.8\times 10^6i`$ and of Model $`\mathrm{I}\mathrm{I}`$, $`f_{\gamma A}^{}(m_B)4.9\times 10^7+6.0\times 10^6i`$, are very similar. The only reason for the larger branching ratio in Model I is that $`X_{ct}\lambda _{ct}^B`$, that is, unitarity of $`V_{3\times 3}`$ can be broken to a lesser extent in Model $`\mathrm{I}\mathrm{I}`$ due to the strong requirements on FCN couplings between light quarks. Additionally, $`V_{tb}`$ can be smaller in Model I, and the total top width is reduced.

We are also interested in the CP asymmetry

$`a_\gamma `$ $`=`$ $`{\displaystyle \frac{\mathrm{\Gamma }(tc\gamma )\mathrm{\Gamma }(\overline{t}\overline{c}\gamma )}{\mathrm{\Gamma }(tc\gamma )+\mathrm{\Gamma }(\overline{t}\overline{c}\gamma )}}.`$ (20)
This interest is mainly academic, because if the branching ratios are unobservable, even less are the asymmetries. However, the latter show how large CP asymmetries at high energy are possible in these SM extensions. Here we analyse in detail $`a_\gamma `$ in Model $`\mathrm{I}\mathrm{I}`$, the results for Model I are similar but more involved. The form factors for $`\overline{t}\overline{c}\gamma `$ are

$`\overline{A}_\gamma `$ $`=`$ $`f_{\gamma A}^{}(m_b)\lambda _{ct}^b+f_{\gamma A}^{}(m_B)\lambda _{ct}^B,`$ (21)
and an analogous expression for $`\overline{B}_\gamma `$. The asymmetry can be written as $`a_\gamma =N_\gamma /D_\gamma `$, with

$`N_\gamma `$ $`=`$ $`2\mathrm{Im}\left[f_{\gamma A}^{}(m_b)f_{}^{}{}_{\gamma A}{}^{}(m_B)+f_{\gamma B}^{}(m_b)f_{}^{}{}_{\gamma B}{}^{}(m_B)\right]\mathrm{Im}\left[\lambda _{ct}^b\lambda _{ct}^B\right]`$
$`D_\gamma `$ $`=`$ $`\left[|f_{\gamma A}^{}(m_b)|^2+|f_{\gamma B}^{}(m_b)|^2\right]|\lambda _{ct}^b|^2+\left[|f_{\gamma A}^{}(m_B)|^2+|f_{\gamma B}^{}(m_B)|^2\right]|\lambda _{ct}^B|^2`$ (22)
$`+2\mathrm{Re}\left[f_{\gamma A}^{}(m_b)f_{}^{}{}_{\gamma A}{}^{}(m_B)+f_{\gamma B}^{}(m_b)f_{}^{}{}_{\gamma B}{}^{}(m_B)\right]\mathrm{Re}\left[\lambda _{ct}^b\lambda _{ct}^B\right].`$
A few comments are in order:

1. The CP asymmetry $`a_\gamma `$ is proportional to the imaginary part of the rephasing-invariant quartet $`\lambda _{ct}^b\lambda _{ct}^B=V_{cb}V_{cB}^{}V_{tb}^{}V_{tB}`$. This is expected from general grounds. In fact, it can be shown that in a model with an extra down singlet all CP violating observables at high energy (that is, when $`m_{u,d,s}0`$ compared to the scale of energy involved) must be proportional to $`\mathrm{Im}V_{cb}V_{cB}^{}V_{tb}^{}V_{tB}`$, $`\mathrm{Im}V_{cb}V_{cB}^{}X_{bB}`$, $`\mathrm{Im}V_{tb}V_{tB}^{}X_{bB}`$, or a combination of them .
2. The SM limit is recovered setting $`\lambda _{ct}^B=0`$, obtaining a vanishing CP asymmetry. It is well known that CP asymmetries at high energy are very small in the SM , due to: (i) the smallness of $`m_u`$, $`m_d`$ and $`m_s`$, what leads to a more efficient GIM cancellation; (ii) the small mixing between the top and the first two generations. When all the quark masses are kept in the computation a non-vanishing but a negligible asymmetry $`a_\gamma 6\times 10^6`$ is obtained. For the gluon case, the asymmetry $`a_g5\times 10^6`$ is also extremely small.
3. Since CPT invariance requires that the total width of the top and the antitop are equal, the different partial widths $`\mathrm{\Gamma }(tc\gamma )\mathrm{\Gamma }(\overline{t}\overline{c}\gamma )`$ must be compensated in other channel. In this case, the compensating decay channels are the SM leading modes $`tbW`$, $`tsW`$, $`tdW`$ and their conjugate processes .
4. The large phases in the functions $`f_{\gamma A}^{}`$, $`f_{\gamma B}^{}`$ allow to obtain relatively large CP asymmetries, provided $`\mathrm{Im}\lambda _{ct}^b\lambda _{ct}^B`$ is sizeable.

## 5 Results

We explore the parameter space of Models I and $`\mathrm{I}\mathrm{I}`$ to find the maximum values of $`\mathrm{Br}(tc\gamma )`$ and $`\mathrm{Br}(tcg)`$ allowed by present experimental measurements. The constraints on these models come from precision electroweak data, $`K`$ and $`B`$ physics and atomic parity violation (the details of the analysis can be found in Ref. ). We take all the quark masses into account, and require that the mass of the new quark is larger than 200 GeV to satisfy the limits from direct searches. In Model I, assuming that the new quark has a mass $`m_T=200`$ GeV, we find the maximum rates

$`\mathrm{Br}(tc\gamma )`$ $`=`$ $`4.5\times 10^8,`$
$`\mathrm{Br}(tcg)`$ $`=`$ $`8.9\times 10^7,`$ (23)
corresponding to $`|X_{ct}|=0.037`$, $`|V_{tb}|=0.58`$. (The branching ratios scale with $`|X_{ct}|^2`$ approximately.) For larger $`m_T`$, the allowed values of $`|X_{ct}|`$ are smaller , and these branching ratios decrease. The CP asymmetries corresponding to the figures in Eqs. (23) are negligible,

$`a_\gamma `$ $`=`$ $`0.0006,`$
$`a_g`$ $`=`$ $`0.002,`$ (24)
because the rates are dominated by the $`X_{ct}`$ term. The asymmetries can have values in the range $`0.5a_\gamma 0.4`$, $`0.9a_g0.6`$, but only reach the boundaries of these intervals for branching ratios much smaller than those in Eqs. (23). The results for decays to up quarks are a little larger,

$`\mathrm{Br}(tu\gamma )`$ $`=`$ $`4.6\times 10^8,`$
$`\mathrm{Br}(tug)`$ $`=`$ $`9.2\times 10^7.`$ (25)
In Model $`\mathrm{I}\mathrm{I}`$, assuming that the mass of the new quark is $`m_B=200`$ GeV, we have

$`\mathrm{Br}(tc\gamma )`$ $`=`$ $`4.5\times 10^{12},`$
$`\mathrm{Br}(tcg)`$ $`=`$ $`6.6\times 10^{11}.`$ (26)
These numbers are almost insensitive to the mass of the new $`Q=1/3`$ quark for $`m_B200`$ GeV, as can be seen from Fig. 1, and show a small increase with $`m_B`$. The corresponding asymmetries are

$`a_\gamma `$ $`=`$ $`0.05,`$
$`a_g`$ $`=`$ $`0.56.`$ (27)
In general, the CP asymmetries take values in the intervals $`1a_\gamma 1`$, $`1a_g1`$. The decay rates to up quarks are larger,

$`\mathrm{Br}(tu\gamma )`$ $`=`$ $`7.4\times 10^{12},`$
$`\mathrm{Br}(tug)`$ $`=`$ $`9.5\times 10^{11}.`$ (28)
The branching ratios in Eqs. (2328) are too small to be measurable in the near future. The estimated $`3\sigma `$ sensitivities of LHC to these decays are $`\mathrm{Br}(tc\gamma )=1.2\times 10^5`$ , $`\mathrm{Br}(tu\gamma )=3.0\times 10^6`$ , $`\mathrm{Br}(tcg)=2.7\times 10^5`$ and $`\mathrm{Br}(tug)=4.1\times 10^6`$ , with an integrated luminosity of 100 fb<sup>-1</sup>. The TESLA sensitivity to $`tc\gamma `$ is better but not enough, $`\mathrm{Br}(tc\gamma )=3.6\times 10^6`$  with a centre of mass energy of 800 GeV and a luminosity of 500 fb<sup>-1</sup>. Hence, we observe that in models with up-type singlets the rates for $`tqZ`$, $`q=u,c`$ can be observable but not the rates for $`tq\gamma `$ and $`tqg`$, which are four and three orders of magnitude smaller, respectively. This fact contrasts with the results for two Higgs doublet models or supersymmetric extensions of the SM, where the branching ratios for $`tcZ`$ and $`tc\gamma `$ are similar, and the branching ratio for $`tcg`$ is one order of magnitude larger. This difference would allow for a consistency check of the models, should a positive signal of top FCN decays be discovered.

Acknowledgements

J.A.A.S. thanks F. del Águila and M. Pérez-Victoria for comments and discussions. B.M.N. thanks D.F. Carvalho for useful discussions. We thank F. del Águila and G.C. Branco for reading of the manuscript. This work has been supported by the European Community’s Human Potential Programme under contract HTRN–CT–2000–00149 Physics at Colliders and by FCT through project CERN/FIS/43793/2001. The work of B.M.N. has been supported by FCT under the grant SFRH/BD/995/2000.

## Appendix A Form factors for $`tc\gamma `$ and $`tcg`$

The contributions to $`A_\gamma `$ and $`B_\gamma `$ of diagrams (1a) and (1b) with an internal quark $`d_i`$ are

$`A_{\gamma ,1\mathrm{a}}`$ $`=`$ $`{\displaystyle \frac{Q_ig^2e}{2}}{\displaystyle \frac{1}{16\pi ^2}}V_{2i}V_{3i}^{}\{(m_t+m_c)C_0+(2m_t+m_c)C_1+(m_t+2m_c)C_2`$ (29)
$`+m_tC_{11}+(m_t+m_c)C_{12}+m_cC_{22}\},`$
$`B_{\gamma ,1\mathrm{a}}`$ $`=`$ $`{\displaystyle \frac{Q_ig^2e}{2}}{\displaystyle \frac{1}{16\pi ^2}}V_{2i}V_{3i}^{}\{(m_tm_c)C_0+(2m_tm_c)C_1+(m_t2m_c)C_2`$ (30)
$`+m_tC_{11}+(m_tm_c)C_{12}m_cC_{22}\},`$
$`A_{\gamma ,1\mathrm{b}}`$ $`=`$ $`{\displaystyle \frac{Q_ig^2e}{4M_W^2}}{\displaystyle \frac{1}{16\pi ^2}}V_{2i}V_{3i}^{}\{m_c(m_t^2\overline{m}_{d_i}^2)C_1+m_t(m_c^2\overline{m}_{d_i}^2)C_2`$ (31)
$`+(m_t\overline{m}_{d_i}^2+m_cm_t^2)C_{11}+[m_t(m_c^2+\overline{m}_{d_i}^2)+m_c(m_t^2+\overline{m}_{d_i}^2)]C_{12}`$
$`+(m_tm_c^2+m_c\overline{m}_{d_i}^2)C_{22}\},`$
$`B_{\gamma ,1\mathrm{b}}`$ $`=`$ $`{\displaystyle \frac{Q_ig^2e}{4M_W^2}}{\displaystyle \frac{1}{16\pi ^2}}V_{2i}V_{3i}^{}\{m_c(m_t^2\overline{m}_{d_i}^2)C_1+m_t(m_c^2\overline{m}_{d_i}^2)C_2`$ (32)
$`+(m_t\overline{m}_{d_i}^2m_cm_t^2)C_{11}+[m_t(m_c^2+\overline{m}_{d_i}^2)m_c(m_t^2+\overline{m}_{d_i}^2)]C_{12}`$
$`+(m_tm_c^2m_c\overline{m}_{d_i}^2)C_{22}\}.`$
The $`C`$’s are functions of the external and internal masses, $`C(m_t^2,0,m_c^2,M_W^2,\overline{m}_{d_i}^2,\overline{m}_{d_i}^2)`$ in the notation of Ref. . For $`tcg`$ the contributions to the form factors $`A_g`$ and $`B_g`$ can be obtained replacing $`e`$ by $`g_s`$ and setting $`Q_i=1`$ in Eqs. (2932). The terms from diagrams (2a)–(2d) are

$`A_{\gamma ,2\mathrm{a}}`$ $`=`$ $`{\displaystyle \frac{g^2e}{4}}{\displaystyle \frac{1}{16\pi ^2}}V_{2i}V_{3i}^{}\{(2m_t+m_c)C_0+(4m_t+m_c)C_1+(m_tm_c)C_2`$ (33)
$`+2m_tC_{11}+2(m_tm_c)C_{12}\},`$
$`B_{\gamma ,2\mathrm{a}}`$ $`=`$ $`{\displaystyle \frac{g^2e}{4}}{\displaystyle \frac{1}{16\pi ^2}}V_{2i}V_{3i}^{}\{(2m_tm_c)C_0+(4m_tm_c)C_1+(m_t+m_c)C_2`$ (34)
$`+2m_tC_{11}+2(m_t+m_c)C_{12}\},`$
$`A_{\gamma ,2\mathrm{b}}`$ $`=`$ $`{\displaystyle \frac{g^2e}{4M_W^2}}{\displaystyle \frac{1}{16\pi ^2}}V_{2i}V_{3i}^{}\{m_c(m_t^2\overline{m}_{d_i}^2)C_1+(m_t\overline{m}_{d_i}^2+m_cm_t^2)C_{11}`$ (35)
$`+[m_t(\overline{m}_{d_i}^2m_c^2)+m_c(m_t^2\overline{m}_{d_i}^2)]C_{12}\},`$
$`B_{\gamma ,2\mathrm{b}}`$ $`=`$ $`{\displaystyle \frac{g^2e}{4M_W^2}}{\displaystyle \frac{1}{16\pi ^2}}V_{2i}V_{3i}^{}\{m_c(m_t^2\overline{m}_{d_i}^2)C_1+(m_t\overline{m}_{d_i}^2m_cm_t^2)C_{11}`$ (36)
$`+[m_t(\overline{m}_{d_i}^2m_c^2)m_c(m_t^2\overline{m}_{d_i}^2)]C_{12}\},`$
$`A_{\gamma ,2\mathrm{c}}`$ $`=`$ $`{\displaystyle \frac{g^2e}{4}}{\displaystyle \frac{1}{16\pi ^2}}V_{2i}V_{3i}^{}\left\{m_cC_0+m_cC_1+m_cC_2\right\},`$ (37)
$`B_{\gamma ,2\mathrm{c}}`$ $`=`$ $`{\displaystyle \frac{g^2e}{4}}{\displaystyle \frac{1}{16\pi ^2}}V_{2i}V_{3i}^{}\left\{m_cC_0m_cC_1m_cC_2\right\},`$ (38)
$`A_{\gamma ,2\mathrm{d}}`$ $`=`$ $`{\displaystyle \frac{g^2e}{4}}{\displaystyle \frac{1}{16\pi ^2}}V_{2i}V_{3i}^{}\left\{m_tC_2\right\},`$ (39)
$`B_{\gamma ,2\mathrm{d}}`$ $`=`$ $`{\displaystyle \frac{g^2e}{4}}{\displaystyle \frac{1}{16\pi ^2}}V_{2i}V_{3i}^{}\left\{m_tC_2\right\}.`$ (40)
Here the arguments of the functions are $`C(m_t^2,m_c^2,0,M_W^2,\overline{m}_{d_i}^2,M_W^2)`$. In Model I the contributions from diagrams (3a) and (3b) with internal quarks $`u_i`$ are

$`A_{\gamma ,3\mathrm{a}}`$ $`=`$ $`{\displaystyle \frac{Q_ig^2e}{4c_W^2}}{\displaystyle \frac{1}{16\pi ^2}}\{c_{LL}^i[(m_t+m_c)C_0(2m_t+m_c)C_1(m_t+2m_c)C_2`$
$`m_tC_{11}(m_t+m_c)C_{12}m_cC_{22}]+(c_{LR}^i+c_{RL}^i)\mathrm{\hspace{0.17em}2}\overline{m}_{u_i}[C_0+C_1+C_2]\},`$
$`B_{\gamma ,3\mathrm{a}}`$ $`=`$ $`{\displaystyle \frac{Q_ig^2e}{4c_W^2}}{\displaystyle \frac{1}{16\pi ^2}}\{c_{LL}^i[(m_tm_c)C_0(2m_tm_c)C_1(m_t2m_c)C_2`$
$`m_tC_{11}(m_tm_c)C_{12}+m_cC_{22}]+(c_{LR}^ic_{RL}^i)\mathrm{\hspace{0.17em}2}\overline{m}_{u_i}[C_0+C_1+C_2]\},`$
$`A_{\gamma ,3\mathrm{b}}`$ $`=`$ $`{\displaystyle \frac{Q_ig^2e}{8M_W^2}}{\displaystyle \frac{1}{16\pi ^2}}X_{2i}^uX_{i3}^u\{m_c(m_t^2\overline{m}_{u_i}^2)C_1+m_t(\overline{m}_{u_i}^2m_c^2)C_2`$ (41)
$`(m_t\overline{m}_{u_i}^2+m_cm_t^2)C_{11}\left[m_t(m_c^2+\overline{m}_{u_i}^2)+m_c(m_t^2+\overline{m}_{u_i}^2)\right]C_{12}`$
$`(m_tm_c^2+m_c\overline{m}_{u_i}^2)C_{22}\},`$
$`B_{\gamma ,3\mathrm{b}}`$ $`=`$ $`{\displaystyle \frac{Q_ig^2e}{8M_W^2}}{\displaystyle \frac{1}{16\pi ^2}}X_{2i}^uX_{i3}^u\{+m_c(m_t^2\overline{m}_{u_i}^2)C_1+m_t(\overline{m}_{u_i}^2m_c^2)C_2`$ (42)
$`(m_t\overline{m}_{u_i}^2m_cm_t^2)C_{11}\left[m_t(m_c^2+\overline{m}_{u_i}^2)m_c(m_t^2+\overline{m}_{u_i}^2)\right]C_{12}`$
$`(m_tm_c^2m_c\overline{m}_{u_i}^2)C_{22}\}.`$
The constants $`c_{LL}^i`$, $`c_{LR}^i`$ and $`c_{RL}^i`$ are products of left-handed and right-handed couplings between $`(c,i)`$ and $`(i,t)`$,

$`c_{LL}^i`$ $`=`$ $`\left(X_{ci}^u\delta _{ci}{\displaystyle \frac{4}{3}}s_W^2\right)\left(X_{it}^u\delta _{it}{\displaystyle \frac{4}{3}}s_W^2\right),`$
$`c_{LR}^i`$ $`=`$ $`\left(X_{ci}^u\delta _{ci}{\displaystyle \frac{4}{3}}s_W^2\right)\left(\delta _{it}{\displaystyle \frac{4}{3}}s_W^2\right),`$
$`c_{RL}^i`$ $`=`$ $`\left(\delta _{ci}{\displaystyle \frac{4}{3}}s_W^2\right)\left(X_{it}^u\delta _{it}{\displaystyle \frac{4}{3}}s_W^2\right),`$ (43)
with $`\delta _{ij}=1`$ if $`i=j`$ and zero otherwise. For these diagrams the arguments of the functions are $`C(m_t^2,0,m_c^2,M_Z^2,\overline{m}_{u_i}^2,\overline{m}_{u_i}^2)`$. Finally, the Higgs contribution in diagram (3c) reads

$`A_{\gamma ,3\mathrm{c}}`$ $`=`$ $`{\displaystyle \frac{Q_ig^2e}{8M_W^2}}{\displaystyle \frac{1}{16\pi ^2}}X_{2i}^uX_{i3}^u\{[2m_t\overline{m}_{u_i}^2+m_c(m_t^2+\overline{m}_{u_i}^2)]C_1`$ (44)
$`+\left[m_t(m_c^2+\overline{m}_{u_i}^2)+2m_c\overline{m}_{u_i}^2\right]C_2+(m_t\overline{m}_{u_i}^2+m_cm_t^2)C_{11}`$
$`+[m_t(m_c^2+\overline{m}_{u_i}^2)+m_c(m_t^2+\overline{m}_{u_i}^2)]C_{12}+(m_tm_c^2+m_c\overline{m}_{u_i}^2)C_{22}\},`$
$`B_{\gamma ,3\mathrm{c}}`$ $`=`$ $`{\displaystyle \frac{Q_ig^2e}{8M_W^2}}{\displaystyle \frac{1}{16\pi ^2}}X_{2i}^uX_{i3}^u\{[2m_t\overline{m}_{u_i}^2m_c(m_t^2+\overline{m}_{u_i}^2)]C_1`$ (45)
$`+\left[m_t(m_c^2+\overline{m}_{u_i}^2)2m_c\overline{m}_{u_i}^2\right]C_2+(m_t\overline{m}_{u_i}^2m_cm_t^2)C_{11}`$
$`+[m_t(m_c^2+\overline{m}_{u_i}^2)m_c(m_t^2+\overline{m}_{u_i}^2)]C_{12}+(m_tm_c^2m_c\overline{m}_{u_i}^2)C_{22}\},`$
where the functions are $`C(m_t^2,0,m_c^2,M_H^2,\overline{m}_{u_i}^2,\overline{m}_{u_i}^2)`$. We take $`M_H=115`$ GeV. For $`tcg`$ the extra contributions in Model I can be obtained replacing $`e`$ by $`g_s`$ and setting $`Q_i=1`$.


format: text

text:
# 1 Introduction

## 1 Introduction

The Standard Model (SM) in four dimensions has been well established as an effective theory below the weak scale. There have been many attempts to go beyond the SM. One attempt is the extension of the gauge group, while the other being the extension of the number of dimensions beyond the usual four space-time as motivated by the string theory.

In four dimensions, as an extension of the gauge group, models of $`SU(3)\times SU(3)_L\times U(1)_X`$ (3-3-1 model) were suggested . The models include extra gauge bosons, which are called bilepton gauge bosons. The gauge bosons couple to two leptons, and thus, they have two units of lepton number. The interesting feature of this model is that the anomaly is not canceled within each generation, and that the anomaly is canceled among the three generations. Thus, this model gives an explanation why the number of generation is 3. Unlike other extended models, the breaking scale of the $`SU(3)_L\times U(1)_X`$ have an upper bound about a few TeV due to the matching condition of the gauge coupling constants. Hence, it is very interesting to investigate the phenomenology of this model and to explore if the extra gauge bosons can be discovered (or ruled out) in the future high energy colliders such as LHC.

Recent realization that the string scale may be much smaller than the Planck scale, even as low as few tens of TeV, has inspired study of gauge symmetry in higher dimensions with the subsequent compactification to four dimension . Standard Model and its minimal supersymmetric extension has been formulated in five dimensions with compactification at an inverse TeV scale and their phenomenological consequences have been explored . In these studies, the usual Higgs mechanism has been used to break the gauge symmetry.

Compactification of this higher dimensional theory to four dimensions using a suitable manifold or orbifold, and choice of suitable boundary conditions, opens up a new mechanism for breaking gauge symmetry , an interesting alternative to the usual Higgs mechanism. A symmetry breaking occurs when the different components in a multiplet of the gauge group $`G`$ is assigned different quantum number for the discrete symmetry group of the compactifying orbifold . Such orbifold breaking of $`SU(5)`$ and $`SO(10)`$ Grand Unified Theories (GUT) and their implications as well as the symmetry breaking patterns of various group under orbifold compactifications have been studied . In these studies, orbifold compactifications have been used to break the GUT symmetry to the SM. Orbifold compactification has also been used to break left-right gauge symmetry group giving rise to interesting phenomenological implications .

In this work, we study the $`SU(3)_L\times U(1)_X`$ gauge theories in five dimension. The gauge symmetry is broken to $`SU(2)_L\times U(1)_X^{}\times U(1)_X`$ upon compactification on a $`S^1/Z_2\times Z_2^{}`$ orbifold. This is one of the minimal application of the orbifold breaking of a realistic gauge model. The next stage of symmetry breaking to the SM, $`SU(2)_L\times U(1)_Y`$ is achieved by using usual Higgs mechanism. We assume the gauge and the Higgs bosons propagate into the extra dimensions, while for the fermions we consider two cases: they can propagate into the bulk or they are localized at the 4 dimensional wall (3-brane). There are several interesting differences with the usual four dimensional 3-3-1 model. The most important qualitative differences are that the bilepton gauge bosons are Kaluza-Klein (KK) excited states, while the extra neutral gauge boson have KK zero mode, thus the mass of the neutral gauge boson is less than the mass of biletons, contrary to the 4 dimensional models. Thus the neutral gauge boson, $`Z^{}`$, can have larger contribution to the low energy neutral current. In the usual 4 dimensional 3-3-1 models, gauge coupling for $`U(1)_X`$ blows up around 3-4 TeV, thus we need some new physics around such scale. The scheme of the TeV scale extra dimension is well compatible to such situation, since the theory becomes five dimensional beyond this scale. In the same way to the usual 4 dimensional 3-3-1 models, we find that the compactification scale has an upper bound around 3-4 TeV. Thus, these new gauge bosons as well as their KK excitations can be explored at the upcoming LHC.

## 2 Formalism: Orbifold breaking of 3-3-1 gauge symmetry

We start with the 5-dimensional $`(x^\mu ,y)`$ gauge theory $`SU(3)_L\times U(1)_X`$ with coupling $`g`$, $`g_X`$. The fifth space-like dimensional coordinate $`y`$ is compactified in an orbifold $`S^1/Z_2\times Z_2^{}`$. The orbifold is constructed by identifying the coordinate with $`Z_2`$ transformation $`yy`$ and $`Z_2^{}`$ transformation $`y^{}y^{}`$, where $`y^{}=y+\pi R/2`$. Then the orbifold space is regarded as a interval $`[0,\pi R/2]`$ and 4 dimensional wall are placed at the folding point $`y=0`$ and $`y=\pi R/2`$.

Using the orbifold compactification, we break the gauge symmetry to $`SU(2)_L\times U(1)_X^{}\times U(1)_X`$. We impose the following transformation property for the five dimensional $`SU(3)_L`$ gauge fields under $`Z_2\times Z_2^{}`$.

$`W_\mu (x^\mu ,y)`$ $``$ $`W_\mu (x^\mu ,y)=PW_\mu (x^\mu ,y)P^1,`$ (1)
$`W_5(x^\mu ,y)`$ $``$ $`W_5(x^\mu ,y)=PW_5(x^\mu ,y)P^1,`$ (2)
$`W_\mu (x^\mu ,y^{})`$ $``$ $`W_\mu (x^\mu ,y^{})=P^{}W_\mu (x^\mu ,y^{})P^1,`$ (3)
$`W_5(x^\mu ,y^{})`$ $``$ $`W_5(x^\mu ,y^{})=P^{}W_5(x^\mu ,y^{})P^1.`$ (4)
Note that the five dimensional Lagrangian is invariant under the above transformations. We choose $`P=\mathrm{diag}(1,1,1)`$, $`P^{}=\mathrm{diag}(1,1,1)`$. For the $`U(1)_X`$ and $`SU(3)_c`$ gauge fields, we choose the $`P`$ and $`P^{}`$ to be identity matrix, so that $`U(1)_X`$ and $`SU(3)_c`$ gauge symmetries remain unbroken.

The five dimensional $`SU(3)_L`$ gauge filed $`W`$ can be written as

$$W=\lambda ^aW^a=\left(\begin{array}{ccc}W^3+\frac{1}{\sqrt{3}}W^8& \sqrt{2}W^+& \sqrt{2}Y^{++}\\ \sqrt{2}W^{}& W^3+\frac{1}{\sqrt{3}}W^8& \sqrt{2}Y^+\\ \sqrt{2}Y^{}& \sqrt{2}Y^{}& \frac{2}{\sqrt{3}}W^8\end{array}\right).$$
(5)
Then, we find that the gauge fields have following parities under $`Z_2\times Z_2^{}`$,

$`W_\mu ^3,W_\mu ^8,W_\mu ^\pm :(+,+)`$ , $`Y_\mu ^\pm ,Y_\mu ^{\pm \pm }:(+,),`$ (6)
$`W_5^3,W_5^8,W_5^\pm :(,)`$ , $`Y_5^\pm ,Y_5^{\pm \pm }:(,+).`$ (7)
The above five dimensional fields can be Fourier expanded as

$`\phi _{(+,+)}(x^\mu ,y)`$ $`=`$ $`\sqrt{{\displaystyle \frac{2}{\pi R}}}\left(\phi _{(+,+)}^{(0)}(x^\mu )+\sqrt{2}{\displaystyle \underset{n=1}{\overset{\mathrm{}}{}}}\phi _{(+,+)}^{(n)}(x^\mu )\mathrm{cos}{\displaystyle \frac{2ny}{R}}\right),`$ (8)
$`\phi _{(+,)}(x^\mu ,y)`$ $`=`$ $`{\displaystyle \frac{2}{\sqrt{\pi R}}}{\displaystyle \underset{n=1}{\overset{\mathrm{}}{}}}\phi _{(+,)}^{(n)}(x^\mu )\mathrm{cos}{\displaystyle \frac{(2n1)y}{R}},`$ (9)
$`\phi _{(,+)}(x^\mu ,y)`$ $`=`$ $`{\displaystyle \frac{2}{\sqrt{\pi R}}}{\displaystyle \underset{n=1}{\overset{\mathrm{}}{}}}\phi _{(,+)}^{(n)}(x^\mu )\mathrm{sin}{\displaystyle \frac{(2n1)y}{R}},`$ (10)
$`\phi _{(,)}(x^\mu ,y)`$ $`=`$ $`{\displaystyle \frac{2}{\sqrt{\pi R}}}{\displaystyle \underset{n=1}{\overset{\mathrm{}}{}}}\phi _{(,)}^{(n)}(x^\mu )\mathrm{sin}{\displaystyle \frac{2ny}{R}}.`$ (11)
We find the four dimensional fields $`\phi _{(+,+)}^{(n)}`$ and $`\phi _{(,)}^{(n)}`$ have masses $`2n/R`$, while $`\phi _{(+,)}^{(n)}`$ and $`\phi _{(,+)}^{(n)}`$ have masses $`(2n1)/R`$. Only $`\phi _{(+,+)}^{(0)}`$ is massless. Thus we find that the 4 dimensional bilepton gauge bosons $`Y_\mu ^\pm `$, $`Y_\mu ^{\pm \pm }`$ do not have massless modes, while $`W_\mu ^3`$, $`W_\mu ^8`$ and $`W_\mu ^\pm `$ have massless modes, hence, the $`SU(3)_L`$ symmetry is broken down to $`SU(2)_L\times U(1)_X^{}`$ symmetry in 4 dimension.

We break the remaining symmetry to the SM gauge symmetry, $`SU(2)_L\times U(1)_Y`$, by using the Higgs mechanism. The SM gauge symmetry is also broken down to $`U(1)_{EM}`$ by Higgs mechanism as usual. The appropriate Higgs multiplets for this scenario are

$$\mathrm{\Phi }:(1,3,1),\mathrm{\Delta }:(1,3,0),\mathrm{\Delta }^{}:(1,3,1),\eta :(1,6,0),$$
(12)
where the quantum numbers in parenthesis refer to $`SU(3)_c\times SU(3)_L\times U(1)_X`$ representations. We define their transformation property under $`Z_2\times Z_2^{}`$ as

$`\mathrm{\Phi }(x^\mu ,y)`$ $``$ $`\mathrm{\Phi }(x^\mu ,y)=P\mathrm{\Phi }(x^\mu ,y),`$ (13)
$`\mathrm{\Phi }(x^\mu ,y^{})`$ $``$ $`\mathrm{\Phi }(x^\mu ,y^{})=P^{}\mathrm{\Phi }(x^\mu ,y),`$ (14)
$`\mathrm{\Delta }(x^\mu ,y)`$ $``$ $`\mathrm{\Delta }(x^\mu ,y)=P\mathrm{\Delta }(x^\mu ,y),`$ (15)
$`\mathrm{\Delta }(x^\mu ,y^{})`$ $``$ $`\mathrm{\Delta }(x^\mu ,y^{})=P^{}\mathrm{\Delta }(x^\mu ,y^{}),`$ (16)
$`\mathrm{\Delta }^{}(x^\mu ,y)`$ $``$ $`\mathrm{\Delta }^{}(x^\mu ,y)=P\mathrm{\Delta }^{}(x^\mu ,y),`$ (17)
$`\mathrm{\Delta }^{}(x^\mu ,y^{})`$ $``$ $`\mathrm{\Delta }^{}(x^\mu ,y^{})=P^{}\mathrm{\Delta }^{}(x^\mu ,y^{}),`$ (18)
$`\eta (x^\mu ,y)`$ $``$ $`\eta (x^\mu ,y)=P\eta (x^\mu ,y)P^1,`$ (19)
$`\eta (x^\mu ,y^{})`$ $``$ $`\eta (x^\mu ,y^{})=P^{}\eta (x^\mu ,y^{})P^1.`$ (20)
The $`SU(3)_L`$ triplets and sextet are decomposed to $`SU(2)_L\times U(1)_X^{}`$ as $`3=(2,1)+(1,2)`$ and $`6=(3,2)+(2,1)+(1,4)`$. The decomposed fields have the following $`Z_2\times Z_2^{}`$ transformation property:

$`\mathrm{\Phi }_1,\mathrm{\Delta }_2,\mathrm{\Delta }_2^{},\eta _2`$ $`:`$ $`(+,+)`$ (21)
$`\mathrm{\Phi }_2,\mathrm{\Delta }_1,\mathrm{\Delta }_1^{},\eta _3,\eta _1`$ $`:`$ $`(+,)`$ (22)
where the suffix denotes $`SU(2)_L`$ representations. Thus, while $`\mathrm{\Phi }_2=(\varphi ^{++},\varphi ^+)`$ doublet acquires a KK mass of $`O(1/R)`$, $`\mathrm{\Phi }_1=\varphi ^0`$ (which electric charge is zero) have a massless mode, and thus can be assigned a vacuum expectation value (VEV) (we call its VEV $`u`$) to break the remaining gauge symmetry down to SM gauge symmetry. All the $`SU(2)_L`$ doublets is chosen to have VEVs in order to break SM gauge group and to give appropriate masses to fermions. If we denote the VEVs of neutral component of $`SU(2)_L`$ doublets, $`\mathrm{\Delta }_2`$, $`\mathrm{\Delta }_2^{}`$ and $`\eta _2`$ as $`v`$, $`v^{}`$ and $`w`$ respectively, the scale of the VEVs are assumed to be

$$uv,v^{},wO(100)\mathrm{GeV}.$$
(23)
The sextet $`\eta `$ is necessary in order to obtain acceptable masses for charged lepton in the case electron, positron and left-handed neutrino is in one $`SU(3)_L`$ triplet. The VEV in the neutral component of $`\eta _3`$ gives a large majorana mass to left-handed neutrino, so that the VEV should be zero or extremely small. In the four dimensional theory, a fine tuning in the Higgs potential is needed to achieve this. In our model, if we choose the doublet $`\eta _2`$ has a massless mode, $`\eta _3`$ automatically does not have a massless mode, and thus its neutral component will not have a VEV. This is a very nice feature of our orbifold breaking scenario.

## 3 Gauge Boson Masses

The Frampton-Pleitez type 3-3-1 models have an upper bound for extra gauge bosons due to the evolution of gauge couplings. To see that, we first study matching condition of gauge couplings.

The $`SU(3)_L`$ gauge symmetry is broken down to $`SU(2)_L\times U(1)_X^{}`$ by orbifold compactification. The VEV of Higgs $`\mathrm{\Phi }`$ breaks remaining $`U(1)_X^{}\times U(1)_X`$ symmetry down to $`U(1)_Y`$ symmetry. Then, the hypercharge is defined as

$$Y=2(\sqrt{3}T^8+XI),$$
(24)
and electric charge Q is defined as

$$Q=T^3+\frac{Y}{2}.$$
(25)
Here $`T^a`$ is the $`SU(3)_L`$ generator normalized as Tr $`(T^aT^b)=1/2`$ $`\delta ^{ab}`$, and $`I=`$ diag$`(1,1,1)`$. We define the covariant derivative for $`SU(3)_L`$ triplets as

$$D_\mu =_\mu igT^aW_\mu ^aig_XXIV_\mu .$$
(26)
The $`W_\mu ^a`$ and $`V_\mu `$ are the $`SU(3)_L\times U(1)_X`$ gauge fields. At this normalization, the coupling constant of $`U(1)_Y`$, $`g_Y`$, is given as

$$\frac{1}{g_Y^2}=\frac{1}{g_X^{}^2}+\frac{1}{g_X^2}.$$
(27)
where $`g_X^{}`$ is a coupling constant for $`U(1)_X^{}`$. At the unification scale for $`SU(3)_L`$, we have the unification condition

$$g_X^{}=\frac{1}{\sqrt{3}}g.$$
(28)
Since the $`SU(3)_L`$ gauge symmetry is broken on the brane at $`y=\pi R/2`$ in our model, we can add the brane gauge interaction which breaks the unification condition of gauge couplings. However, we can neglect the correction of brane interaction if the cutoff scale is much larger than compactification scale . From this matching condition, we obtain the relation at the compactification scale $`M_c`$,

$$\frac{g_X^2}{g^2}=\frac{\mathrm{sin}^2\theta _W(M_c)}{14\mathrm{sin}^2\theta _W(M_c)},$$
(29)
where the weak mixing angle is defined as $`\mathrm{tan}\theta _W=g_Y/g`$. Therefore, $`\mathrm{sin}^2\theta _W(M_c)`$ has to be less than 1/4. This constraint $`\mathrm{sin}^2\theta _W(M_c)<1/4`$ demands an upper bound of the compactification scale. If we demands that the gauge coupling $`g_X`$ at $`M_c`$ is less than $`N`$, we obtain the constraint as

$$\frac{14\mathrm{sin}^2\theta _W(M_c)}{\alpha _{EM}(M_c)}>\frac{1}{N}.$$
(30)
Some of the fermion such as new exotic quarks of this models decouple at the breaking scale of $`U(1)_X\times U(1)_X^{}`$. We call the scale $`M_D`$. Assuming that $`M_c>M_D`$, we obtain that the compactification scale is less than 4.4 TeV. We used the experimental data, $`\alpha _{EM}^1(M_Z)=127.9`$ and $`\mathrm{sin}^2\theta _W=0.2311`$ in the $`\overline{\mathrm{MS}}`$ scheme. We plot the upper bound of $`M_c`$ as a function of $`M_D`$ in the Fig.1.

Let us see the mass spectra of this model. We denote the VEVs of the neutral component of Higgs fields, $`\mathrm{\Phi }_1`$, $`\mathrm{\Delta }_2`$, $`\mathrm{\Delta }_2^{}`$, and $`\eta _2`$ as $`u/\sqrt{2}`$, $`v/\sqrt{2}`$, $`v^{}/\sqrt{2}`$, and $`w/\sqrt{2}`$, respectively. Then charged gauged bosons acquire masses

$`M_{W_{(n)}}^2`$ $`=`$ $`{\displaystyle \frac{1}{4}}g^2(v^2+v^2+w^2)+\left({\displaystyle \frac{2n}{R}}\right)^2(n=0,1,2,\mathrm{}),`$ (31)
$`M_{Y_{(n)}^+}^2`$ $`=`$ $`{\displaystyle \frac{1}{4}}g^2(u^2+v^2+w^2)+\left({\displaystyle \frac{2n1}{R}}\right)^2(n=1,2,3,\mathrm{}),`$ (32)
$`M_{Y_{(n)}^{++}}^2`$ $`=`$ $`{\displaystyle \frac{1}{4}}g^2(u^2+v^2+w^2)+\left({\displaystyle \frac{2n1}{R}}\right)^2(n=1,2,3,\mathrm{}).`$ (33)
For neutral gauge bosons, we can identify the photon field $`\gamma `$ and the massive bosons $`Z`$ and $`Z^{}`$ as follows.

$`\gamma `$ $`=`$ $`\mathrm{sin}\theta _WW^3+\mathrm{cos}\theta _W(\sqrt{3}\mathrm{tan}\theta _WW^8+\sqrt{13\mathrm{tan}^2\theta _W}V),`$ (34)
$`Z`$ $`=`$ $`\mathrm{cos}\theta _WW^3\mathrm{sin}\theta _W(\sqrt{3}\mathrm{tan}\theta _WW^8+\sqrt{13\mathrm{tan}^2\theta _W}V),`$ (35)
$`Z^{}`$ $`=`$ $`\sqrt{13\mathrm{tan}^2\theta _W}W^8+\sqrt{3}\mathrm{tan}\theta _WV.`$ (36)
In this basis, the mass squared matrix for $`Z_{(n)}`$ and $`Z_{(n)}^{}`$ $`(n=0,1,2,\mathrm{})`$ is given by

$$_{(n)}^2=\left(\begin{array}{cc}M_Z^2& M_{ZZ^{}}^2\\ M_{ZZ^{}}^2& M_Z^{}^2\end{array}\right),$$
(37)
with

$`M_Z^2`$ $`=`$ $`{\displaystyle \frac{1}{4}}{\displaystyle \frac{g^2}{\mathrm{cos}^2\theta _W}}(v^2+v^2+w^2)+\left({\displaystyle \frac{2n}{R}}\right)^2,`$ (38)
$`M_Z^{}^2`$ $`=`$ $`{\displaystyle \frac{1}{3}}g^2\left({\displaystyle \frac{\mathrm{cos}^2\theta _W}{14\mathrm{sin}^2\theta _W}}u^2+{\displaystyle \frac{14\mathrm{sin}^2\theta _W}{4\mathrm{cos}^2\theta _W}}(v^2+v^2+w^2)+{\displaystyle \frac{3\mathrm{sin}^2\theta _W}{14\mathrm{sin}^2\theta _W}}v^2\right)+\left({\displaystyle \frac{2n}{R}}\right)^2,`$
$`M_{ZZ^{}}^2`$ $`=`$ $`{\displaystyle \frac{1}{4\sqrt{3}}}g^2{\displaystyle \frac{\sqrt{14\mathrm{sin}^2\theta _W}}{\mathrm{cos}^2\theta _W}}\left(v^2+w^2{\displaystyle \frac{1+2\mathrm{sin}^2\theta _W}{14\mathrm{sin}^2\theta _W}}v^2\right).`$ (40)
The mass eigenstates $`Z_1`$ and $`Z_2`$ are

$`Z_1`$ $`=`$ $`\mathrm{cos}\varphi Z\mathrm{sin}\varphi Z^{},`$ (41)
$`Z_2`$ $`=`$ $`\mathrm{sin}\varphi Z+\mathrm{cos}\varphi Z^{},`$ (42)
where the mixing angle $`\varphi `$ is given by

$$\mathrm{tan}2\varphi =\frac{2M_{ZZ^{}}^2}{M_Z^{}^2M_Z^2}.$$
(43)
Assuming that $`u>v,v^{},w`$, we obtain

$$M_{Z_1}^2M_Z^2(1\frac{M_Z^{}^2}{M_Z^2}\varphi ^2),\varphi \frac{M_{ZZ}^2}{M_Z^2}.$$
(44)
The mixing angle have a range

$$\frac{1+2\mathrm{sin}^2\theta _W}{\sqrt{3}\sqrt{14\mathrm{sin}^2\theta _W}}\frac{M_{Z_1}^2}{M_{Z_2}^2}\varphi \frac{\sqrt{14\mathrm{sin}^2\theta _W}}{\sqrt{3}}\frac{M_{Z_1}^2}{M_{Z_2}^2}.$$
(45)
Note that while the bilepton gauge bosons, $`Y^{\pm \pm }`$ and $`Y^\pm `$, acquire masses at the compactification scale (see Eq. (32-33), the extra neutral gauge boson, $`Z_2`$ does not. Thus, $`M_{Z_2}M_Y`$. This is exactly opposite to the usual Higgs breaking of the four dimensional model in which $`M_{Z_2}M_Y`$. Thus orbifold breaking naturally predicts a low scale extra $`Z^{}`$ boson giving rise to interesting phenomenological implications as we will discuss later.

## 4 Fermions

In our model, the $`SU(3)_L`$ gauge symmetry in five dimension is broken by the boundary conditions. At the boundary, $`y=\pi R/2`$, the five dimensional gauge fields $`Y^\pm `$ and $`Y^{\pm \pm }`$ vanish. Thus, the orbifold fixed point $`y=\pi R/2`$ does not respect the $`SU(3)_L`$ symmetry. When we include the fermions in the theory, we have two choices: the fermions can propagate into the bulk or they are localized at the 4D wall orbifold fixed points.

We show two different anomaly-free fermion representations. The first anomaly free fermion contents are following .

$$\mathrm{}_a=(e_a,\nu _a,e_a^c)^T:(1,3^{},0),$$
(46)
$$q_i=(u_i,d_i,D_i)^T:(3,3,\frac{1}{3}),q_3=(d_3,u_3,T)^T:(3,3^{},\frac{2}{3}),$$
(47)
$$u_a^c:(3^{},1,\frac{2}{3}),d_a^c:(3^{},1,\frac{1}{3}),D_i^c:(3^{},1,\frac{4}{3}),T^c:(3^{},1,\frac{5}{3}),$$
(48)
where $`a=1,2,3`$ is a family index and $`i=1,2`$ is related to two of the three families.

The $`D_i`$ and $`T`$ are additional quark which have exotic electric charges, $`4/3`$ and $`5/3`$, respectively. In this model, the anomaly is not canceled within each generation, but canceled among three generations by choosing that two of the three generations are $`(3,3,1/3)`$ and the other one generation is $`(3,3^{},2/3)`$. There is an ambiguity which generation we assign the $`(3,3^{},2/3)`$ to.

Since both electron and positron are in the same $`SU(3)_L`$ triplet, we need Higgs sextet to give the physically acceptable masses to charged leptons: otherwise the mass matrix becomes anti-symmetric giving rise to $`m_e=0`$ and $`m_\mu =m_\tau `$. We can consider other assignment for positron, that is, the positron is additional $`SU(3)_L`$ singlet $`(1,1,1)`$ and $`e^c`$ component in the triplet $`\mathrm{}`$ get large by adding $`E:(1,1,1)`$. At this choice, we denote the multiplets as

$$\mathrm{}_a=(\nu _a,e_a,E_a^c)^T:(1,3^{},0),e_a^c:(1,1,1),E_a:(1,1,1).$$
(49)
We can have another type of anomaly-free fermion set.

$$\mathrm{}_a=(\nu _a,e_a,E_a^{})^T:(1,3,1),e_a^c:(1,1,1),E_a^{++}:(1,1,2),$$
(50)
$$q_i=(d_i,u_i,U_i)^T:(3,3^{},\frac{2}{3}),q_3=(u_3,d_3,B)^T:(3,3,\frac{1}{3}),$$
(51)
$$u_a^c:(3^{},1,\frac{2}{3}),d_a^c:(3^{},1,\frac{1}{3}),U_i^c:(3^{},1,\frac{4}{3}),B^c:(3^{},1,\frac{5}{3}).$$
(52)
Let us see the case that fermions are in the bulk. The bulk fermions have both chirality in the sense of four dimensional Weyl fermions. For example, for the lepton triplet, $`\mathrm{}=(e,\nu ,e^c)^T:(1,3^{},0)`$, the fermions have $`Z_2\times Z_2^{}`$ charge as

$$(e,\nu )_L:(++),e_L^c:(+),(e,\nu )_R:(),e_R^c:(+).$$
(53)
Only $`(e,\nu )_L`$ has zero mode. We have to have one more lepton triplet such as

$$(e^{},\nu ^{})_L:(+),e_L^c:(++),(e^{},\nu ^{})_R:(+),e_R^c:(),$$
(54)
and there is a positron in the triplet. Since electron and positron are in the different multiplet in this case, we don’t need Higgs sextet to construct a phenomenological viable charged lepton mass matrix.

For the quark fields, we also need to make them doubled in the same way in order to cancel the brane-localized chiral anomaly .

The most interesting feature of the bulk-fermion case, the lightest KK particles (LKP) become a candidate of dark matter, since the LKP become stable if all the particles are bulk fields.

## 5 Phenomenological Implications

In original 3-3-1 model, the VEV of Higgs field $`\mathrm{\Phi }`$ breaks $`SU(3)_L\times U(1)_X`$ down to SM gauge group directly. Thus, the mass of the neutral extra gauge boson, $`Z^{}`$, is related to the masses of new charged gauge bosons, $`Y^\pm `$ and $`Y^{\pm \pm }`$. The charged gauge bosons’ masses are constrained from collider experiments and muon decay. The constraint of the charged gauge boson pushes the lower bound of $`Z^{}`$ boson mass to be larger than $`O(1)`$ TeV. In the 5 dimensional 3-3-1 model, however, the $`SU(3)_L\times U(1)_X`$ symmetry cannot be broken down to SM directly. Therefore, there is no relation between the masses of new charged and neutral gauge bosons. As a result, the mass of $`Z^{}`$ can be lower than 1 TeV, and hence its contribution to the low energy experiments can be significant.

To see that, we first write out the interaction of $`Z^{}`$ boson and fermions. The gauge interaction is given as

$$(Z^{})=\frac{g}{\mathrm{cos}\theta _W}Z^\mu \overline{f}\gamma _\mu \left(g_L^{}(f)\frac{1\gamma _5}{2}+g_R^{}(f)\frac{1+\gamma _5}{2}\right)f.$$
(55)
The coupling coefficients are given as

$$g_{L,R}^{}(f)=\frac{\sqrt{14\mathrm{sin}^2\theta _W}}{2\sqrt{3}}Y(f_{L,R})+\frac{1\mathrm{sin}^2\theta _W}{\sqrt{3}\sqrt{14\mathrm{sin}^2\theta _W}}X(f_{L,R}).$$
(56)
The $`Z^{}`$ coupling to the fermion with non-zero $`X`$ charge is enhanced since the weak mixing angle is close to 1/4.

In fact, experimental data for forward-backward asymmetry $`A_{FB}^{(0,b)}`$ and atomic parity violation have more than 2$`\sigma `$ discrepancy compared with SM prediction. If we assign the $`X`$ charge of lepton multiplets to be zero, those predictions might be adjusted without changing the leptonic experimental data much.

We first concentrate our discussion to atomic parity violation. The value of the atomic parity violation for Cs is

$`Q_W(\text{SM prediction})`$ $`=`$ $`73.09\pm 0.03,`$ (57)
$`Q_W(\text{experiment})`$ $`=`$ $`72.06\pm 0.28\pm 0.34.`$ (58)
We find the contribution of this 3-3-1 model as

$`\delta Q_W`$ $``$ $`157{\displaystyle \frac{M_{Z_1}^2}{M_{Z_2}^2}}980\varphi 73\varphi ^2{\displaystyle \frac{M_{Z_2}^2}{M_{Z_1}^2}}(X_{Q_1}={\displaystyle \frac{2}{3}}),`$ (59)
$`\delta Q_W`$ $``$ $`45{\displaystyle \frac{M_{Z_1}^2}{M_{Z_2}^2}}300\varphi 73\varphi ^2{\displaystyle \frac{M_{Z_2}^2}{M_{Z_1}^2}}(X_{Q_1}={\displaystyle \frac{1}{3}}),`$ (60)
where $`X_{Q_1}`$ is the $`X`$ charge of the first generation of left-handed quarks. We can adjust the experimental data for the range $`M_{Z_2}`$ 1 TeV and $`|\varphi |\begin{array}{c}<\hfill \\ \hfill \end{array}10^3`$.

Next we consider the nucleon-neutrino scattering. The contribution of $`g_L^2`$ and $`g_R^2`$ are following. For the case that $`X`$ charge of lepton triplets, $`X_{\mathrm{}}`$, is 0,

$`\delta g_L^2`$ $``$ $`0.025{\displaystyle \frac{M_{Z_1}^2}{M_{Z_2}^2}}+0.0659\varphi +0.59\varphi ^2{\displaystyle \frac{M_{Z_2}^2}{M_{Z_1}^2}}(X_{Q_1}={\displaystyle \frac{2}{3}}),`$ (61)
$`\delta g_L^2`$ $``$ $`0.013{\displaystyle \frac{M_{Z_1}^2}{M_{Z_2}^2}}0.18\varphi +0.59\varphi ^2{\displaystyle \frac{M_{Z_2}^2}{M_{Z_1}^2}}(X_{Q_1}={\displaystyle \frac{1}{3}}),`$ (62)
$`\delta g_R^2`$ $``$ $`0.059{\displaystyle \frac{M_{Z_1}^2}{M_{Z_2}^2}}+0.36\varphi +0.058\varphi ^2{\displaystyle \frac{M_{Z_2}^2}{M_{Z_1}^2}}.`$ (63)
This correction is not enough to adjust NuTeV results  since we have a constraint of the $`Z`$-$`Z^{}`$ mixing angle $`|\varphi |\begin{array}{c}<\hfill \\ \hfill \end{array}10^3`$ from other experiments .

For the case $`X_{\mathrm{}}=1`$,

$`\delta g_L^2`$ $``$ $`0.49{\displaystyle \frac{M_{Z_1}^2}{M_{Z_2}^2}}+1.9\varphi +0.59\varphi ^2{\displaystyle \frac{M_{Z_2}^2}{M_{Z_1}^2}}(X_{Q_1}={\displaystyle \frac{2}{3}}),`$ (64)
$`\delta g_L^2`$ $``$ $`0.26{\displaystyle \frac{M_{Z_1}^2}{M_{Z_2}^2}}+1.7\varphi +0.59\varphi ^2{\displaystyle \frac{M_{Z_2}^2}{M_{Z_1}^2}}(X_{Q_1}={\displaystyle \frac{1}{3}}),`$ (65)
$`\delta g_R^2`$ $``$ $`1.14{\displaystyle \frac{M_{Z_1}^2}{M_{Z_2}^2}}+0.5\varphi +0.058\varphi ^2{\displaystyle \frac{M_{Z_2}^2}{M_{Z_1}^2}}.`$ (66)
In this case, we can adjust the NuTeV experiment for the range $`M_{Z_2}3`$ TeV and $`|\varphi |10^3`$ with $`\varphi <0`$.

We now briefly discuss the collider signals for this model. Since the $`Z^{}`$ remains massless after the orbifold breaking of the symmetry, this scenario predicts a low scale $`Z^{}`$. In the original four dimensional model, $`Z^{}`$ obtains mass at a scale higher than that of $`Y`$ and thus $`M_Z^{}M_Y`$. The exactly opposite is true in our orbifold breaking scenario, and unlike the usual four dimensional model, the mass of $`Z^{}`$ is not constrained from the low energy limit on the mass of $`Y`$. Such a low mass $`Z^{}`$, probably around a TeV or less, is likely to show up at the Tevatron Run2 and will be easily observed at the LHC. Another interesting signal is the production of $`Y^{++}`$ in association with exotic charged quark $`D`$ in the hadronic collider via the process $`u+gY^{++}+D`$ . $`Y^{++}`$ will decay to $`\mathrm{}^+\mathrm{}^+`$ $`(\mathrm{}=e,\mu ,\tau )`$, while $`D`$ will decay to $`u\mathrm{}^{}\mathrm{}^{}`$. This will lead to a spectacular signal of four charged leptons in the final state with a peak in $`\mathrm{}^+\mathrm{}^+`$ combination. Such a signal will be easily observed at the LHC for the mass range of $`Y`$ and $`D`$ predicted by our scenario.

## 6 Conclusions

We conclude summarizing our main results. We have considered the orbifold breaking of the 3-3-1 model in the five dimension. The first stage of symmetry breaking, $`i.e.`$ $`SU(3)_L`$ gauge symmetry to $`SU(2)_L\times U(1)_X^{}`$ is achieved via orbifold compactification to four dimension. The model has several distinguishing features from the usual four dimensional 3-3-1 model.

First, in the context of the four dimensional 3-3-1 model, we expect new physics since the gauge coupling of $`U(1)_X`$ blows up around 4 TeV. TeV scale compactification is a good candidate as the new physics. In other words, this model is very compatible with TeV scale unification since it predicts that $`\mathrm{sin}^2\theta _W`$ is less than 1/4 at unification scale.

Next, orbifold construction of 3-3-1 model has an advantage to forbid the VEV of $`SU(2)_L`$ triplet. In the minimal choice of the fermion representation, the lepton doublet and positron are included in one $`SU(3)_L`$ triplet $`\mathrm{}`$. In that case, we need the sextet Higgs, $`\eta `$, to give phenomenologically viable mass matrix of charged lepton. Then the Yukawa coupling $`\mathrm{}\mathrm{}\eta `$ contains the majorana mass of neutrino, if the $`SU(2)_L`$ triplet in $`\eta `$ acquire VEV. Thus, we have to choose the Higgs potential to prohibit the unwanted VEV. Orbifold compactification gives mass of $`1/R`$ to the $`SU(2)_L`$ triplet component naturally, if the sextet $`\eta `$ is a bulk Higgs.

Finally, phenomenological point of view, the mass of extra neutral gauge boson $`Z^{}`$ can be lower than 1 TeV in the orbifold breaking scenario. If the gauge symmetry $`SU(3)_L\times U(1)_X`$ is directly broken to SM gauge group, there is a rigid relation between the mass of bilepton gauge bosons and extra neutral gauge boson. We have a constraint for the mass of bileptons from muon decay or other experiments, which then lead to a constraint that the mass of $`Z^{}`$ should be larger than 1.3 TeV. However, in the orbifold breaking scenario, we have two independent brekaing scale, that is compactification scale and the VEV of triplet Higgs, and thus, there is no relation between mass of the bileptons and $`Z^{}`$. In that case, the mass of $`Z^{}`$ can be lower than 1 TeV, and is likely to be observed at the high energy collider such as Tevatron Run2 or LHC. Such a $`Z^{}`$ can also contribute the low energy neutral current experiments. In fact, this model gives interesting contribution to the atomic parity violation and nucleon-neutrino scattering.

## Acknowledgments

We thank K.S. Babu for useful discussions. I.G.and S.N. acknowledge the warm hospitality and support of the Fermilab Theory Group from their Summer Visitor Program during the initial stage of this work. S.N. also acknowledges the warm hospitality and support of the DESY and CERN Theory Groups during his summer visits there. This work was supported in part by US DOE Grants # DE-FG030-98ER-41076 and DE-FG-02-01ER-45684.