Lepton Number Violation Higgs Decay at Muon Collider Fa-Xin Yang 1, Feng-Lan Shao 1, Zhi-Long Han 2, Fei Huang 2, Yi Jin 2,3, Honglei Li 2 Abstract In this paper, we consider the scalar singlet extension of type-I seesaw, where a scalar singlet S S and a heavy neutral lepton N N are further introduced. The Majorana mass term of heavy neutral lepton is generated through the Yukawa interaction with the scalar singlet, which then induces the lepton number violation decays of SM Higgs h h and heavy Higgs H H via mixing of scalars. As a pathway to probe the origin of heavy neutral lepton mass, we investigate the lepton number violation Higgs decay signature at the TeV-scale muon collider. The dominant production channel of Higgs bosons at the TeV-scale muon collider is via vector boson fusion. So we perform a detailed analysis of the signal process μ + ​ μ − → ν μ ​ ν ¯ μ ​ h / H → ν μ ​ ν ¯ μ ​ N ​ N \mu^{+}\mu^{-}\to\nu_{\mu}\bar{\nu}_{\mu}h/H\to\nu_{\mu}\bar{\nu}_{\mu}NN followed by N → μ ± ​ j ​ j N\to\mu^{\pm}jj , where the two jets from W W boson decay are treated as one fat-jet J J . With an integrated luminosity of 1 ​ ( 10 ) ​ ab − 1 1(10)~\text{ab}^{-1} , the 3 (10) TeV muon collider could discover the lepton number violation SM Higgs decay h → μ ± ​ μ ± ​ J ​ J h\to\mu^{\pm}\mu^{\pm}JJ signature for the Higgs mixing parameter sin ⁡ α > 0.05 ​ ( 0.009 ) \sin\alpha>0.05(0.009) . Meanwhile, a large parameter space can be detected by the lepton number violation heavy Higgs decay H → μ ± ​ μ ± ​ J ​ J H\to\mu^{\pm}\mu^{\pm}JJ signature for m H ≲ 1 ​ ( 3 ) m_{H}\lesssim 1(3) TeV and sin ⁡ α ≳ 0.03 ​ ( 0.005 ) \sin\alpha\gtrsim 0.03(0.005) at the 3 (10) TeV muon collider. Therefore, the lepton number violation SM and heavy Higgs decay signatures are both promising at the TeV scale muon collider. I Introduction The discovery of the 125 GeV Higgs boson at the large hadron collider (LHC) is a great achievement of the standard model (SM) [ 1 , 2 ] , which also provides a unique pathway to probe new physics beyond SM. Meanwhile, the neutrino oscillation experiments indicate tiny but non-zero neutrino masses [ 3 , 4 , 5 ] . If neutrinos are Dirac particles, their masses could originate from the Yukawa interaction with the SM Higgs doublet \Phi as y_{D}\bar{L}_{L}\tilde{\Phi}\nu_{R} . To explain the sub-eV neutrino masses, y_{D}\lesssim 10^{-12} is required, which is theoretically unnatural small compared to other SM Yukawa couplings. One natural pathway to explain the tiny neutrino mass is the seesaw mechanism [ 6 , 7 , 8 , 9 ] , which introduces heavy neutral lepton N . Due to the Majorana nature of N , one distinct signal is the lepton number violation signature at the collider [ 10 , 11 , 12 , 13 , 14 , 15 , 16 ] , such as pp\to W^{\pm(*)}\to\ell^{\pm}N\to\ell^{\pm}\ell^{\pm}jj [ 17 , 18 , 19 ] . In the well-studied phenomenological type-I seesaw model, the lepton number violation signature is determined by the mixing parameter V_{\ell N} between the light and heavy neutrinos [ 20 , 21 ] , which is mostly sensitive to the mass region with m_{N}\lesssim 100 GeV. Above the electroweak scale, V_{\ell N} is less constrained as |V_{\ell N}|^{2}\gtrsim 10^{-3} being disfavored via indirect constraints [ 22 ] . Currently, the LHC searches could exclude |V_{\ell N}|^{2}\gtrsim 2\times 10^{-6} at m_{N}=20 GeV for prompt [ 23 , 24 ] and |V_{\ell N}|^{2}\gtrsim 3\times 10^{-7} at m_{N}=12 GeV for displaced signature [ 25 , 26 , 27 ] . In the future, CEPC and FCC-ee could probe |V_{\ell N}|^{2}\gtrsim 10^{-10} with m_{N}\sim 50 GeV [ 21 ] , and the muon collider could test |V_{\ell N}|\gtrsim 10^{-6} for TeV scale m_{N} [ 28 , 29 , 30 ] . However, the natural theoretical predicted squared mixing parameter is |V_{\ell N}|^{2}\sim m_{\nu}/m_{N}<10^{-10} for m_{N}>1 GeV, which is even beyond the reach of future experiments [ 21 ] . The Majorana mass term m_{N}\bar{N}^{c}N is allowed by the SM gauge symmetry, so m_{N} is usually assumed to be large enough to suppress the neutrino mass in the canonical seesaw. If we assign the lepton number L=1 to N , the Majorana mass term clearly breaks the lepton number symmetry by two units. Since m_{N}\to 0 will restore the lepton number symmetry, the bare Majorana mass term should be theoretically preferred to be small [ 31 ] , just such as in the inverse seesaw mechanism [ 32 ] . On the other hand, the origin of the Majorana mass term is not clarified in the minimal seesaw either. This term can be realized by extending the SM gauge symmetry, such as the left-right symmetrical model [ 33 , 34 , 35 ] , the gauged U(1)_{B-L} model [ 36 , 37 , 38 , 39 ] , the gauged U(1)_{L_{\mu}-L_{\tau}} model [ 40 , 41 , 42 , 43 , 44 ] , and the general gauged U(1)_{X} model [ 45 , 46 , 47 , 48 , 49 ] . In such gauge symmetry extended models, heavy neutral lepton N has additional interactions with the new gauge boson and new scalars. Searches of heavy neutral lepton via decays of new gauge bosons have been performed by LHC [ 50 , 51 ] . For instance, the search for pair production of heavy neutral lepton via the process pp\to Z^{\prime}\to NN with N\to\ell^{\pm}jj by the CMS collaboration has excluded the parameter space with m_{Z^{\prime}}\lesssim 4.4 TeV and m_{N}\lesssim 1.3 TeV [ 51 ] . In the future, the HL-LHC could probe the parameter space with m_{Z^{\prime}}\lesssim 6 TeV and m_{N}\lesssim 3 TeV [ 49 ] . In this paper, we focus on the Higgs sector with an additional scalar singlet S [ 52 , 53 ] , which usually appears in the U(1) symmetric models. By assigning lepton number L=-2 to the new scalar S , the Majorana mass term then originates from the new Yukawa interaction S\bar{N}^{c}N , which is invariant under the U(1) symmetry. After the spontaneous symmetry breaking, both the scalar singlet and the neutral component of SM Higgs doublet \Phi obtain non-zero vacuum expectation values. The scalar interaction (\Phi^{\dagger}\Phi)(S^{\dagger}S) induces the mixing between scalars. In the mass eigenstate, we denote h to be the SM 125 GeV Higgs boson and H to be the additional heavy Higgs boson. Including the mixing between scalars, both the SM h and heavy Higgs boson H couple to the heavy neutral lepton N , which could induce the lepton number violation Higgs decay as h/H\to NN with N\to\ell^{\pm}jj at colliders when kinematically allowed, i.e., m_{N}0,~\lambda_{2}>0,~4\lambda_{1}\lambda_{2}-\lambda_{3}^{2}>0. (2) After the spontaneous symmetry breaking, the neutral component of scalars obtain vacuum expectation values as \langle\Phi\rangle=v_{0}/\sqrt{2} and \langle S\rangle=v_{s}/\sqrt{2} , where v_{0}=246 GeV. In the following studies, we fix v_{s}=1 TeV for illustration. The \lambda_{3}(\Phi^{\dagger}\Phi)(S^{\dagger}S) term then induces mixing between the scalars. The mass matrix for the two CP-even Higgs bosons \phi^{0} and s^{0} is given by \displaystyle M^{2}(\phi^{0},s^{0})=\begin{pmatrix}2\lambda_{1}v_{0}^{2}&\lambda_{3}v_{0}v_{s}\\ \lambda_{3}v_{0}v_{s}&2\lambda_{2}v_{s}^{2}\end{pmatrix}. (3) The mass eigenstate can be acquired by an orthogonal transformation \displaystyle\begin{pmatrix}h\\ H\end{pmatrix}=\begin{pmatrix}\cos\alpha&-\sin\alpha\\ \sin\alpha&\cos\alpha\end{pmatrix}\begin{pmatrix}\phi^{0}\\ s^{0}\end{pmatrix}, (4) where the mixing angle \alpha is derived as \displaystyle\tan(2\alpha)=\frac{\lambda_{3}v_{0}v_{s}}{\lambda_{2}v_{s}^{2}-\lambda_{1}v_{0}^{2}}. (5) Then, the masses for the two physical Higgs states h,H are \displaystyle m_{h}^{2}=\lambda_{1}v_{0}^{2}+\lambda_{2}v_{s}^{2}-\sqrt{(\lambda_{1}v_{0}^{2}-\lambda_{2}v_{s}^{2})^{2}+(\lambda_{3}v_{0}v_{s})^{2}}, (6) \displaystyle m_{H}^{2}=\lambda_{1}v_{0}^{2}+\lambda_{2}v_{s}^{2}+\sqrt{(\lambda_{1}v_{0}^{2}-\lambda_{2}v_{s}^{2})^{2}+(\lambda_{3}v_{0}v_{s})^{2}}, (7) where h is the 125 GeV SM Higgs, and H is the additional heavy Higgs. In this paper, we consider the scenario with m_{H}>m_{h} . The mixing between scalars modifies the coupling of SM Higgs by a factor of \cos\alpha , which then alters the Higgs signal strengths. In this paper, we consider the constraints from current LHC searches. Using the precision Higgs measurements data [ 75 , 76 ] , the global fit result of all channels requires |\sin\alpha|<0.29 [ 77 , 78 ] . More stringent constraints come from the direct searches for the heavy Higgs in channels of H\to ZZ [ 79 , 80 ] , H\to WW [ 81 , 82 ] , and H\to hh [ 83 , 84 , 85 , 86 ] . The combined analysis of all channels shows that |\sin\alpha|\lesssim 0.2 is allowed by direct searches [ 87 ] , where the explicit constraint on \sin\alpha depends on the mass of heavy Higgs. Meanwhile, the NLO calculations of W boson mass set the most strict constraints for m_{H} around the TeV scale, which could exclude |\sin\alpha|\gtrsim 0.16 [ 88 , 89 , 87 ] . The relevant Yukawa interactions in the scalar singlet extension of the type-I seesaw are -\mathcal{L}_{Y}=y_{D}\bar{L}_{L}\tilde{\Phi}N+\frac{y_{N}}{2}S\bar{N}^{c}N+\text{h.c.}, (8) where L_{L} denotes the left-handed lepton doublet. After the symmetry breaking, the Dirac and Majorana mass terms are generated as m_{D}=y_{D}v_{0}/\sqrt{2} and m_{N}=y_{N}v_{s}/\sqrt{2} . Through the seesaw mechanism, light neutrino mass is derived as m_{\nu}=m_{D}m_{N}^{-1}m_{D}^{T} . The mixing parameter between the light and heavy neutrino is V_{\ell N}=\frac{m_{D}}{m_{N}}\sim\sqrt{\frac{m_{\nu}}{m_{N}}}. (9) Without fine-tuning, the typical value of the mixing parameter is |V_{\ell N}|^{2}\sim m_{\nu}/m_{N}<10^{-12} for heavy neutral lepton above the electroweak scale. In this paper, we fix the mixing parameter as |V_{\ell N}|^{2}=m_{\nu}/m_{N}=10^{-12}\times(50~\text{GeV})/m_{N} with m_{\nu}=0.05 eV in the following studies. For instance, m_{N}=50(500) GeV corresponds to |V_{\ell N}|^{2}=10^{-12}(10^{-13}) . Such small mixing parameters are far beyond the reach of current experimental limits [ 23 , 24 ] . Including the mixing between light and heavy neutrinos, the interactions of heavy neutral leptons are \displaystyle\mathcal{L} \displaystyle\supset \displaystyle-\frac{g}{\sqrt{2}}W_{\mu}\bar{N}V_{\ell N}^{*}\gamma^{\mu}P_{L}\ell-\frac{g}{2\cos\theta_{W}}Z_{\mu}\bar{N}V_{\ell N}^{*}\gamma^{\mu}P_{L}\nu_{\ell}-\frac{gm_{N}}{2m_{W}}\cos\alpha~h\bar{N}V_{\ell N}^{*}P_{L}\nu_{\ell} (10) \displaystyle+\frac{m_{N}}{2v_{s}}\sin\alpha~h\bar{N}^{c}N-\frac{m_{N}}{2v_{s}}\cos\alpha~H\bar{N}^{c}N+\text{h.c.}, where \theta_{W} is the weak mixing angle. III Decay Properties In principle, the decay mode h\to N\nu_{\ell} is allowed if m_{N}m_{W} . Hence, the fully visible signature N\to\ell^{\pm}jj is always a dominant channel for m_{N}>5 GeV. Depending on the explicit decay mode of the heavy neutral lepton, there could be various signal channels from the cascade decays of the NN pair. Focusing on the final states with charged leptons, the explicit signatures could be \displaystyle NN \displaystyle\to \displaystyle\ell^{\pm}jj+\ell^{\pm}jj\to 2\ell^{\pm}+4j, (23) \displaystyle\to \displaystyle\ell^{\pm}jj+\ell^{\mp}jj\to\ell^{+}\ell^{-}+4j, (24) \displaystyle\to \displaystyle\ell^{\pm}jj+\nu_{\ell}\ell^{+}\ell^{-}\to 3\ell+2j+\cancel{E}_{T}, (25) \displaystyle\to \displaystyle\nu_{\ell}\ell^{+}\ell^{-}+\nu_{\ell}\ell^{+}\ell^{-}\to 4\ell+\cancel{E}_{T}. (26) A full study of these signatures is beyond the scope of this paper. As the smoking gun of Majorana neutrino, we investigate the lepton number violation signature 2\ell^{\pm}+4j in this work, which also has relatively small SM backgrounds compared to the lepton number conserving signature. IV Signature from SM Higgs We first consider the pair production of heavy neutral lepton N decaying from SM Higgs h . At the lepton colliders, there are two important production channels of SM Higgs, i.e., the Higgs-strahlung process \ell^{+}\ell^{-}\to Zh and the WW fusion process \ell^{+}\ell^{-}\to\nu_{\ell}\bar{\nu}_{\ell}h . It is well known that the cross section of the Higgs-strahlung process \ell^{+}\ell^{-}\to Zh decreases at high energies as \sim g^{4}/s , where \sqrt{s} is the center-of-mass energy. Meanwhile, the cross section of WW fusion process \ell^{+}\ell^{-}\to\nu_{\ell}\bar{\nu}_{\ell}h increases at high energies as \sim g^{4}\log(s/m_{h}^{2}) [ 94 ] . At the TeV scale muon collider, the WW fusion process is the dominant contribution [ 95 ] , whose cross section can be derived in the high energy limit as [ 94 ] \sigma(\mu^{+}\mu^{-}\to\nu_{\mu}\bar{\nu}_{\mu}h)\approx\frac{g^{4}}{256\pi^{3}}\frac{1}{v_{0}^{2}}\left(\log\frac{s}{m_{h}^{2}}-2\right). (27) In this paper, we use Madgraph5_aMC@NLO [ 93 ] to obtain more precise numerical results. Figure 4: Cross section of lepton number violation signature from SM Higgs decay as a function of heavy neutral lepton mass m_{N} in panel (a) and the center-of-mass energy \sqrt{s} in panel (b). The blue, green, and red lines are the results of the mixing \sin\alpha=0.2 , 0.15, and 0.1, respectively. In panel (a), the solid and dashed lines correspond to the results of 3 TeV and 10 TeV muon collider. In panel (b), we have fixed m_{N}=40 GeV. When the mass of heavy neutral lepton is small enough ( m_{N}50~{\rm{GeV}},|\eta(J)|<2.5, (33) which correspond to the default set of Delphes3 . In Figure 5 , we show the distributions of some variables for the signal and backgrounds at the 3 TeV muon collider after the pre-selection cuts, where m_{N}=25 GeV and 45GeV with \sin\alpha=0.2 are chosen as the benchmark points. Distributions of these variables at the 10 TeV muon collider are similar to the 3 TeV muon collider, hence they will not be shown in this paper. The distributions are normalized by using number of event in each bin divided by the total events. The normalized distributions are used to compare the difference between the signal and the backgrounds at the same level, based on which further selection cuts are applied. After the simulation, several cuts are further applied to suppress the SM backgrounds. To obtain the lepton number violation signature \mu^{\pm}\mu^{\pm}JJ , we first require exactly two same-sign muons in the final states. According to the distribution of P_{T}(\mu) in Figure 5 , the muons from heavy neutral lepton decays typically have a transverse momentum smaller than 100 GeV, while those from SM backgrounds could be much more energetic. So we select events with P_{T}(\mu) in the range of 20 to 100 GeV. The cuts on muons are summarized as \displaystyle N_{\mu}^{\pm}=2,20~{\rm{GeV}}0.1 . The precision results are shown in the upper-right panel of Figure 6 . Typically, the 3 TeV muon collider could have a precision of \mathcal{O}(10) % for \sin\alpha\gtrsim 0.1 . And the 10 TeV muon collider could reach the precision of \mathcal{O}(2) % for \sin\alpha\simeq 0.2 with 25 GeV \lesssim m_{N}\lesssim 55 GeV. In the lower panel of Figure 6 , we depict the 5\sigma discovery reach of the h\to\mu^{\pm}\mu^{\pm}JJ signature at the 3(10) TeV muon collider with an integrated luminosity of L=1(10)~\text{ab}^{-1} . When m_{N}\simeq 45~\rm{GeV} , the limit of the mixing parameter \sin\alpha can be as small as \sin\alpha=0.05(0.009) at the 3(10) TeV muon collider, which corresponds to the branching ratio of h\to NN as 4.1\times 10^{-3}(1.3\times 10^{-4}) . V Signature from Heavy Higgs Through mixing with SM Higgs h , the dominant production channel of heavy Higgs H at the TeV scale muon collider is also via vector boson fusion process \mu^{+}\mu^{-}\to\nu_{\mu}\bar{\nu}_{\mu}H . In the high energy limit, the corresponding cross section can be estimated as [ 104 ] \sigma(\mu^{+}\mu^{-}\to\nu_{\mu}\bar{\nu}_{\mu}H)\approx\frac{g^{4}\sin^{2}\alpha}{256\pi^{3}}\frac{1}{v_{0}^{2}}\left(\log\frac{s}{m_{H}^{2}}-2\right). (38) The numerical results are also obtained with Madgraph5_aMC@NLO [ 93 ] in the following discussions. When kinematically allowed m_{N}m_{W} . Hence, we expected the invariant mass of fat-jet satisfying m_{J}\sim m_{W} in this scenario, which can be used to distinguish the H\to\mu^{\pm}\mu^{\pm}JJ signature from the h\to\mu^{\pm}\mu^{\pm}JJ signal. Figure 7: Cross section of lepton number violation heavy Higgs decay. In panels (a), (b) and (c), we have fixed the mass relation as m_{N}=m_{H}/3 . In panel (b), we further set m_{H}=300 GeV. In panel (d), the Higgs mixing parameter is fixed as \sin\alpha=0.1 . In panels (a), (c), and (d), the solid lines are the results of the 3 TeV muon collider, while the dashed lines are the results of the 10 TeV muon collider. The theoretical cross section of the lepton number violation signature from heavy Higgs decay can be calculated as \sigma(\mu^{+}\mu^{-}\xrightarrow{H}\mu^{\pm}\mu^{\pm}JJ)\simeq\sigma(\mu^{+}\mu^{-}\to\nu_{\mu}\bar{\nu}_{\mu}H)\times\text{BR}(H\to NN)\times\text{BR}(N\to\mu^{\pm}jj)^{2}/2, (40) which depends on the heavy Higgs mass m_{H} , the collision energy \sqrt{s} , the Higgs mixing parameter \sin\alpha and the heavy neutral lepton mass m_{N} . In Figure 7 , we show the numerical results of \sigma(H\to\mu^{\pm}\mu^{\pm}JJ) at the muon collider. In panel (a), the cross section at 3 TeV and 10 TeV muon collider as a function of m_{H} are shown, where we set m_{N}=m_{H}/3 for illustration. We should mention that for \sin\alpha=0.15 , the current direct searches of heavy Higgs boson could exclude the mass region of 300 GeV \lesssim m_{H}\lesssim 500 GeV, meanwhile, all the mass range is allowed for \sin\alpha<0.1 [ 77 ] . The cross section decreases as the heavy Higgs mass increases. Typically, we have \sigma(H\to\mu^{\pm}\mu^{\pm}JJ)>0.01 fb when m_{H}\lesssim 1.5(5) TeV at the 3 (10) TeV muon collider. Hence, increasing the collision energy of the muon collider to 10 TeV is able to probe heavy Higgs above the TeV scale. In panel (b) of Figure 7 , we show the cross section of H\to\mu^{\pm}\mu^{\pm}JJ as a function of the center-of-mass energy \sqrt{s} , which clearly depicts the logarithmic dependence of collision energy as in Equation 38 . Another important parameter that alters the signal cross section is the Higgs mixing angle \alpha . In panel (c) of Figure 7 , we show the cross section of H\to\mu^{\pm}\mu^{\pm}JJ by varying \sin\alpha . Although the production cross section of heavy Higgs \sigma(\mu^{+}\mu^{-}\to\nu_{\mu}\bar{\nu}_{\mu}H) decreases as \sin\alpha becomes smaller according to Equation 38 , the branching ratio of H\to NN increases when \sin\alpha decreases as shown in Figure 2 . Therefore, the signal cross section \sigma(H\to\mu^{\pm}\mu^{\pm}JJ) is nearly a constant when \sin\alpha>0.1 . On the other hand, for \sin\alpha<0.05 , the branching ratio BR (H\to NN) is already close to one, so the signal cross section becomes proportional to \sin^{2}\alpha , and decreases quickly as \sin\alpha becomes smaller. In panel (d) of Figure 7 , the impacts of heavy neutral lepton mass m_{N} on the signal cross section are shown, where we have fixed m_{H}= 300 GeV, 600 GeV and 900 GeV for illustration. We find that when m_{N}\lesssim m_{H}/3 , the signal cross section \sigma(H\to\mu^{\pm}\mu^{\pm}JJ) is approximately a constant. This is because the branching ratio of H\to NN increases (due to larger Yukawa coupling y_{N}=\sqrt{2}m_{N}/v_{s} ) while the branching ratio of N\to\mu^{\pm}W^{\mp} decreases (as shown in Figure 3 ) when m_{N} becomes larger for certain m_{H} . Close to the threshold m_{N}\sim m_{H}/2 , the signal cross section will be suppressed by the phase space of H\to NN decay. After the simulation of signal and backgrounds events, the same pre-selection cuts as in Section IV are firstly applied \displaystyle|\eta(\mu)|<2.5,P_{T}(J)>50~{\rm{GeV}},|\eta(J)|<2.5. (41) Distributions of some variables for the signal and backgrounds at the 3 TeV muon collider are shown in Figure 8 . The benchmark points are m_{H}/3=m_{N}=200~\rm{GeV} and m_{H}/3=m_{N}=400~\rm{GeV} with \sin\alpha=0.1 . As shown in Figure 7 , the typical cross section of H\to\mu^{\pm}\mu^{\pm}JJ is less than 1 fb, which is smaller than the cross section of h\to\mu^{\pm}\mu^{\pm}JJ with the same value of \sin\alpha . In the upper panels of Figure 8 , we find that the transverse momentum of the muon P_{T}(\mu) tends to be larger when m_{N} increases. For the SM backgrounds, there are peaks around P_{T}(\mu)\sim 850 GeV. To keep more signal events, we have changed the selection cuts on muons as \displaystyle N_{\mu}^{\pm}=2,20~{\rm{GeV}}0.1 , although certain region with m_{H}\lesssim 600 GeV is already excluded by current direct searches of heavy Higgs, m_{H} around TeV scale is also promising with such large \sin\alpha . When \sin\alpha=0.05 , the promising region becomes 230~{\rm{GeV}}