text:
# 𝑏-𝜏 unification and large atmospheric mixing: a case for non-canonical see-saw

\[

## Abstract

We study the second and third generation masses in the context of the minimal renormalizable SO(10) theory. We show that if the see-saw takes the non-canonical (type II) form, large atmospheric neutrino mixing angle requires $`b\tau `$ unification.

preprint: PACS: 98.80.Cq, IJS-TP-97/16,
hep-ph/yymmnn
\]

A. Introduction. A suspected quark-lepton symmetry is, as we know, badly broken by the difference in their mixing angles. Small $`V_{CKM}`$ mixing should be contrasted with the maximal mixing for atmospheric neutrinos and probably large mixing for solar neutrinos. Why is this so? This has become one of the major issues in the so-called fermion mass and mixing problem.

In this Letter we address this question in the minimal renormalizable SO(10) theory, without any additional symmetries or interactions. We focus only on the second and third generations for three reasons:

(i) in this case the neutrino mixing angle is maximal and experimentally established;

(ii) it is much likely that in the case of the first family we can not ignore higher dimensional operators;

(iii) in this simple $`2\times 2`$ case we can actually present analytic expressions.

Our main result is the following. We show that in the case of non-canonical see-saw, large neutrino mixing angle requires $`b\tau `$ unification. The rest of the paper is a proof of this statement and a discussion of its implications.

The choice of SO(10) theory is highly natural. It unifies a family of fermions; it unifies their interactions (except for gravity); it has a see-saw mechanism  of small neutrino mass naturally built in; it has charge conjugation as a gauge symmetry; and, in its supersymmetric version, leads naturally to a theory of R-parity .

The last result holds true in the renormalizable version of the theory with a $`126_H`$ dimensional Higgs supermultiplet used to give masses to the right-handed neutrinos.

B. Canonical (type I) versus non-canonical (type II) see-saw mechanism. The minimal Higgs that breaks SU(2)$`\times `$U(1) symmetry and gives mass to the fermions is under the Pati-Salam SU(2)$`{}_{L}{}^{}\times `$SU(2)$`{}_{R}{}^{}\times `$SU(4)C symmetry

$$10_H=(2,2,1)+(1,1,6),$$
(1)
and so $`10_H=(2,2,1)0`$ implies the well-known quark-lepton symmetric relation for fermion masses

$$m_D=m_E,$$
(2)
which works well for the $`3^{rd}`$ family, and fails badly for the first two. You can correct this by adding more Higgses, or appealing to higher dimensional operators (see for example ). However, a nice and important point was raised around twenty years ago . Ten years ago Babu and Mohapatra utilized it to study neutrino masses and mixings . With $`10_H`$ and $`126_H`$ the Yukawa sector of the Lagrangian is given by

$$_Y=10_H\psi Y_{10}\psi +126_H\psi Y_{126}\psi ,$$
(3)
where $`\psi `$ stands for the $`16`$ dimensional spinors which incorporate a family of fermions, and $`Y_{10}`$ and $`Y_{126}`$ are the Yukawa coupling matrices in generation space.

From

$$126_H=(3,1,10)+(1,3,\overline{10})+(2,2,15)+(1,1,6)$$
(4)
one has

$$M_{\nu _R}=Y_{126}(1,3,\overline{10})_{126},$$
(5)
where $`(1,3,\overline{10})_{126}=M_R`$, the scale of SU(2)R gauge symmetry breaking.

It can be shown that, after the SU(2)$`\times `$U(1) breaking through $`10_H=(2,2,1)M_W`$, the $`(3,1,10)`$ multiplet from $`126_H`$ gets a small vev

$$(3,1,10)_{126}\frac{M_W^2}{M_{parity}},$$
(6)
where $`M_{parity}`$ is the scale of the breakdown of parity. In general $`M_R`$ and $`M_{parity}`$ are not necessarily equal, but typically one breaks parity through the breaking of SU(2)R symmetry, in which case $`M_R=M_{parity}`$. This is what we take hereafter.

In turn, neutrinos pick up small masses

$$M_{\nu _L}=Y_{126}(3,1,10)_{126}+m_D^TM_{\nu _R}^1m_D,$$
(7)
where $`m_D`$ is the neutrino Dirac mass matrix. It is often assumed, for no reason whatsoever, that the second term dominates. This we call canonical (often called type I) see-saw. In what follows we explore the opposite case, which we call non-canonical (type II) see-saw. After all, it does not involve Dirac mass terms and so there is no reason a priori in this case to expect quark-lepton analogy of mixing angles. In this sense the non-canonical see-saw is physically more appealing. More than that, we will show that the large leptonic mixing fits perfectly with the small quark mixing, as long as $`m_b=m_\tau `$.

The crucial ingredient is the fact  that through a non-vanishing tadpole a $`(2,2,15)`$ field in $`126_H`$ also picks up a vev:

$$(2,2,15)_{126}\left(\frac{M_R}{M_{GUT}}\right)^2(2,2,1)_{10}.$$
(8)
In the supersymmetric version of the theory this requires a $`210`$ dimensional Higgs at the GUT scale.

C. Non-canonical see-saw: $`b\tau `$ unification and large atmospheric neutrino mixing. Most of the study throughout the years assumed the canonical see-saw, i.e. the second term dominates in (7). The original claim of  that the leptonic mixing matrix $`V_l`$ had a small $`23`$ element was questioned by using a non-minimal model  or the freedom to adjust the phases in the mixing matrices . Last year we studied  the opposite case, the non-canonical see-saw and noticed that it fitted nicely with a large $`23`$ mixing angle responsible for atmospheric neutrinos.

We give here a simple argument in favour of this. We show how maximal $`\mu \tau `$ mixing fits nicely with $`b\tau `$ unification.

To see this, notice that fermion masses take the following form

$`M_U`$ $`=`$ $`Y_{10}v_{10}^u+Y_{126}v_{126}^u,`$ (9)
$`M_D`$ $`=`$ $`Y_{10}v_{10}^d+Y_{126}v_{126}^d,`$ (10)
$`M_E`$ $`=`$ $`Y_{10}v_{10}^d3Y_{126}v_{126}^d,`$ (11)
$`M_N`$ $`=`$ $`Y_{126}(3,1,10)_{126},`$ (12)
where $`U`$, $`D`$, $`E`$, $`N`$ stand for up quark, down quark, charged lepton and neutrino, respectively, while $`v_{10}^{u,d}`$ and $`v_{126}^{u,d}`$ are the two vevs of $`(2,2,1)`$ in $`10_H`$ and $`(2,2,15)`$ in $`126_H`$, and the last formula is the assumption of the non-canonical see-saw. The result is surprisingly simple. Notice that

$$M_NY_{126}M_DM_E.$$
(13)
Now, let us study the $`2^{nd}`$ and $`3^{rd}`$ generations, and work in the basis of $`M_E`$ diagonal. The puzzle then is: why a small mixing in $`M_D`$ corresponds to a large mixing in $`M_N`$? For simplicity take the mixing in $`M_D`$ to vanish, $`\theta _D=0`$, and ignore the second generation masses, i.e. take $`m_s=m_\mu =0`$. Then

$`M_N\left(\begin{array}{cc}0& 0\\ 0& m_bm_\tau \end{array}\right).`$ (14)
Obviously, unless $`m_b=m_\tau `$, neutrino mixing vanishes. Thus, large mixing in $`M_N`$ (the physical leptonic mixing in the above basis) is deeply connected with the $`b\tau `$ unification. Notice that we have done no model building whatsoever; we only assumed a renormalizable SO(10) theory and the non-canonical see-saw.

Before we discuss (14) more carefully by switching on $`m_\mu `$, $`m_s`$ and the mixings, let us comment on the implication of our result. First, notice that it does not depend on the number of $`10_H`$’s. Notice also that it is not easily generalized to three generations, i.e. it is not easy to give the same reasoning why the solar neutrino mixing should be large (to be confirmed experimentally).

In short, our results should be taken as an argument in favour of the non-canonical see-saw: large atmospheric mixing angles and $`b\tau `$ unification seem to prefer clearly this form of the see-saw mechanism.

D. Quantitative analysis. Let us now be more quantitative and turn on $`m_s`$, $`m_\mu `$ and $`\theta _D`$. Notice that $`\theta _D`$ is not a $`23`$ $`V_{CKM}`$ mixing angle, but rather a difference between charged lepton and down quark mixing angles (recall that we choose $`M_E`$ diagonal).

An important comment. It is not that all the $`32`$ doublets in $`(2,2,1)`$ and $`(2,2,15)`$ remain light; with the minimal fine-tuning we end up with only two of them at $`M_Z`$. Let us denote their vevs by $`v_i`$ ($`i=u,d`$), where $`M_W=g\sqrt{v_u^2+v_d^2}/2`$ and we adopt as usual $`\mathrm{tan}\beta =v_u/v_d`$. Then we can write

$$v_{10}^i=v_i\mathrm{cos}\alpha _i,v_{126}^i=v_i\mathrm{sin}\alpha _i,(i=u,d),$$
(15)
where $`\alpha _i`$ are unknown angles. Defining

$$x=\frac{\mathrm{tan}\alpha _u}{\mathrm{tan}\alpha _d},y=\frac{\mathrm{cos}\alpha _d}{\mathrm{cos}\alpha _u}$$
(16)
(notice that either $`x^2y^21`$ or $`x^2y^21`$), it is a simple exercise to derive from (9)-(12)

$`Y_E`$ $`=`$ $`{\displaystyle \frac{1}{1x}}\left[4yY_U(3+x)Y_D\right],`$ (17)
$`Y_N`$ $`=`$ $`c\left(Y_EY_D\right),`$ (18)
where $`M_U=v_uY_U`$, $`M_D=v_dY_D`$, $`M_E=v_dY_E`$, $`M_NY_N`$, and $`c`$ is an unknown constant in this theory. Since $`Y`$’s are symmetric, we can write for species $`X`$

$$Y_X=XY_X^dX^T,$$
(19)
where $`Y_X^d`$ are diagonal Yukawa matrices and $`X`$ are in general unitary. In what follows we do not wish to play with the adjustment of phases and so take $`X`$ to be orthogonal matrices for simplicity and transparency.

Let $`\theta _l`$, $`\theta _D`$ and $`\theta _q`$ denote the rotation angles in $`E^TN`$, $`D^TE`$ and $`D^TU`$ respectively ($`\theta _l`$ and $`\theta _q`$ are the leptonic and quark weak mixing angles respectively). From (18) we get

$$\mathrm{tan}2\theta _l=\frac{\mathrm{sin}2\theta _D}{\frac{y_\tau y_\mu }{y_by_s}\mathrm{cos}2\theta _D}.$$
(20)
Next, we wish to connect $`\theta _D`$ with $`\theta _q`$ in order to have the dependence of $`\theta _l`$ with $`\theta _q`$. ¿From (17) one has ($`c_D=\mathrm{cos}\theta _D`$, $`c_q=\mathrm{cos}\theta _q`$, etc.)

$`\left(\begin{array}{cc}c_D^2y_\tau +s_D^2y_\mu y_b& c_q^2y_t+s_q^2y_c\\ s_D^2y_\tau +c_D^2y_\mu y_s& s_q^2y_t+c_q^2y_c\end{array}\right)\left(\begin{array}{c}x\\ 4y\end{array}\right)=`$ (21)
$`\left(\begin{array}{c}c_D^2y_\tau +s_D^2y_\mu +3y_b\\ s_D^2y_\tau +c_D^2y_\mu +3y_s\end{array}\right)`$ $`.`$ (22)
After introducing

$$ϵ_u=\frac{y_c}{y_t},ϵ_d=\frac{y_s}{y_b},ϵ_e=\frac{y_\mu }{y_\tau },ϵ=\frac{y_by_\tau }{y_b},$$
(23)
and after some computational tedium we get from (17) and (21)

$`(1ϵ_e)\mathrm{tan}\theta _D\left[(1ϵ_uϵ_d)\mathrm{tan}^2\theta _q+(ϵ_uϵ_d)\right]=`$ (24)
$`(1ϵ_u)\mathrm{tan}\theta _q\left[(1ϵ_eϵ_d)\mathrm{tan}^2\theta _D+(ϵ_eϵ_d)\right].`$ (25)
In the limit $`ϵ_i=0`$ ($`i=u,d,e`$) there are two solutions: $`\mathrm{tan}\theta _D=0`$ and $`\mathrm{tan}\theta _D=\mathrm{tan}\theta _q`$. The first solution can be shown to be unrealistic, whereas the second one gives the important relation between the physical mixing angles of quarks and leptons:

$$\mathrm{tan}2\theta _l=\frac{\mathrm{sin}2\theta _q}{2\mathrm{sin}^2\theta _qϵ}.$$
(26)
Since $`\theta _q=\theta _{bc}`$ of $`V_{CKM}`$, $`\theta _q10^2`$, (26) shows manifestly that $`\mathrm{tan}\theta _l1`$ requires $`ϵ0`$, i.e. $`y_by_\tau `$ as we argued repeatedly.

Let us now switch on the second generation masses, i.e. let us take $`ϵ_i0`$. From (24) one can see that the physically acceptable solution is

$$\mathrm{tan}\theta _D=𝒪(\delta ),\delta =ϵ_i,\mathrm{tan}\theta _q10^2.$$
(27)
From (20) is then obvious that $`b\tau `$ unification $`y_\tau =y_b+𝒪(\delta )`$ is sufficient to make the mixing angle large, i.e. $`\mathrm{tan}2\theta _l=𝒪(1)\delta `$. This is our main result, rather nontrivial in our opinion. A small quark mixing angle automatically leads to a large leptonic mixing in the $`23`$ case.

E. From high to low energy: running.

Our expressions are valid at the unification scale $`M_{GUT}`$. Thus we must run the physical parameters from $`M_{GUT}`$ to $`M_Z`$ in order to be precise. However, in this case the running is not so important as it may seem. Namely, in this letter we want to study the implications of the SO(10) symmetry (in its minimal renormalizable version) on fermion masses and mixings. What we said up to now is equally valid in ordinary and supersymmetric (with $`210_H`$ Higgs) SO(10) gauge theory. We wish to emphasize the generic feature of the model, that is the connection between the large $`\theta _{atm}`$ and $`b\tau `$ unification and do not worry so much about the precise numerical estimates. This requires specifying precisely the nature of the low energy effective theory. Still, it is instructive to see the impact of running. We thus discuss briefly the supersymmetric case and leave the complete discussion for a longer paper now in preparation.

The neutrino matrix elements $`M_{ij}`$ run at the 1-loop level and neglecting threshold effects according to

$$16\pi ^2\frac{d}{dt}M_{ij}=\left[y_\tau ^2(k_i+k_j)+6y_t^26g_2^2\frac{6}{5}g_1^2\right]M_{ij},$$
(28)
where $`t=\mathrm{ln}(Q/M_Z)`$, $`g_1`$ is normalized in the SU(5) fashion, $`i,j=2,3`$ stand for the second and third generations, and $`k_2=0`$, $`k_3=1`$. The neutrino mixing angle at the electroweak scale is

$`\mathrm{tan}2\theta _l|_{M_Z}`$ $`=`$ $`{\displaystyle \frac{2M_{23}(0)}{M_{22}(0)M_{33}(0)}}`$ (29)
$`=`$ $`{\displaystyle \frac{2M_{23}(t_{GUT})B_\tau }{M_{22}(t_{GUT})B_\tau ^2M_{33}(t_{GUT})}},`$ (30)
where $`t_{GUT}=\mathrm{ln}(M_{GUT}/M_Z)`$ and

$$B_\tau =\mathrm{exp}\left(\frac{1}{16\pi ^2}_0^{t_{GUT}}y_\tau ^2(t)𝑑t\right).$$
(31)
The elements $`M_{ij}(t_{GUT})`$ are exactly the ones discussed throughout the paper. We can thus recalculate (20) (valid at $`M_{GUT}`$) at $`M_Z`$:

$$\mathrm{tan}2\theta _l|_{M_Z}=\frac{B_\tau \mathrm{sin}2\theta _D}{\frac{y_\tau y_\mu }{y_by_s}\frac{1+B_\tau ^2}{2}\mathrm{cos}2\theta _D\frac{1B_\tau ^2}{2}\left(1+2\frac{y_\tau y_b}{y_by_s}\right)}.$$
(32)
All the parameters of the right-hand-side are to be evaluated at the GUT scale. For this reason the same equation (24) is again used to express $`\theta _D`$. Clearly, as before, large neutrino mixing angle comes out as soon as $`y_b`$ and $`y_\tau `$ unify at the GUT scale. Of course, the precise value of the neutrino mixing angle depends on this running, however the qualitative behaviour does not change.

A more detailed approach would require to use numerical techniques to account for (1) the running as function of $`\mathrm{tan}\beta `$; (2) the inclusion of threshold corrections ; and (3) first generation effects. However, threshold effects in SO(10) are bound to be important and high precision calculations may actually not be so useful, see for example .

What about the values of neutrino masses? We do not enter into this issue here since we have no new results beyond .

F. Summary and outlook.

The sharp contrast of quark and lepton mixings is often considered a deep puzzle. We argued here that it is actually quite natural in the minimal SO(10) renormalizable theory. All that is required is that the see-saw mechanism takes a non-canonical form free from Dirac masses. The approximate formula (26) expresses it clearly: a small quark mixing $`\theta _q=\theta _{bc}10^2`$ gives naturally a large $`\theta _l=\theta _{atm}`$ if $`ϵ0`$, i.e. $`y_by_\tau `$. Actually, the essence of our work lies in formulae (13)-(14). Formula (14), valid in the approximation of vanishing second generation masses and vanishing quark and lepton mixings speaks eloquently: unless $`m_b=m_\tau `$ at the large scale, we will have a vanishing atmospheric neutrino mixing. In short, the non-canonical see-saw marries nicely $`b\tau `$ unification with the maximal atmospheric neutrino mixing. This can be of great help in trying to pin-point the nature of the see-saw mechanism: our study points in favor of the non-canonical version.

Strictly speaking, a numerical study showed that in the $`3\times 3`$ case, by playing with CP phases, even the canonical see-saw can be made to work . However, in our case, the $`23`$ family study offers physical insight into the question, and after all the first family of fermions may suffer from the higher dimensional operators. The $`10_H`$ and $`126_H`$, the minimal Higgses needed to give masses to all fermions, work beautifully: $`10_H`$ offers $`m_b=m_\tau `$, and $`126_H`$ offers $`3m_s=m_\mu `$ at the GUT scale; and in this framework a small $`\theta _{cb}`$ ($`\theta _{ts}`$) and a large $`\theta _{atm}`$ become naturally connected. Thus, the observational evidence that quarks and leptons have sharply different mixing angles fits nicely with the belief that they are one and the same object at a fundamental level.

We are grateful to Rabi Mohapatra for his encouragement, and to Alejandra Melfo for useful comments and a careful reading of the manuscript. The work of B.B. is supported by the Ministry of Education, Science and Sport of the Republic of Slovenia. The work of G.S. is partially supported by EEC, under the TMR contracts ERBFMRX-CT960090 and HPRN-CT-2000-00152. We express our gratitude to INFN, which permitted the development of the present study by supporting an exchange program with the International Centre for Theoretical Physics.