Search for low-mass dilepton resonances in Higgs boson decays to four-lepton final states in proton–proton collisions at $\sqrt{s}=13\text{TeV}$

Abstract

A search for low-mass dilepton resonances in Higgs boson decays is conducted in the four-lepton final state. The decay is assumed to proceed via a pair of beyond the standard model particles, or one such particle and a $\mathrm{Z}$ boson. The search uses proton–proton collision data collected with the CMS detector at the CERN LHC, corresponding to an integrated luminosity of 137${\text{fb}}^{-1}$, at a center-of-mass energy $\sqrt{s}=13\text{TeV}$. No significant deviation from the standard model expectation is observed. Upper limits at 95% confidence level are set on model-independent Higgs boson decay branching fractions. Additionally, limits on dark photon and axion-like particle production, based on two specific models, are reported.

Introduction

Following the discovery of the Higgs boson ($\mathrm{H}$) by the ATLAS and CMS Collaborations [1–3] at the CERN LHC, a thorough program of precise measurements [4–6] has been carried out to uncover possible deviations from the standard model (SM) or to decipher the nature of the Higgs sector. In particular, various exotic decays of the Higgs boson have been considered, in which small deviations in the Higgs boson decay width or discovery of exotic decay modes could constitute evidence of beyond the SM (BSM) physics.

This paper describes a search for exotic decays of the Higgs boson $\mathrm{H}\to \mathrm{Z}\mathrm{X}$ or $\mathrm{H}\to \mathrm{X}\mathrm{X}$ in the four-lepton (electrons or muons) final state, using a sample of proton–proton collision data at a center-of-mass energy of 13$\text{TeV}$ recorded by the CMS experiment in 2016–2018. The analyzed data sample corresponds to an integrated luminosity of $137{\text{fb}}^{-1}$. Here $\mathrm{X}$ represents a possible BSM particle that could decay into a pair of opposite-sign, same-flavor (OSSF) leptons. In this paper, we consider two specific BSM models. In both models, leptonic decays of $\mathrm{X}$ and $\mathrm{Z}$ to either two muons or electrons give rise to the 4$\ell$ (where $4\ell$ may denote $4\mathrm{\mu }$, $2\mathrm{e}2\mathrm{\mu }$, or $4\mathrm{e}$) final states. Assuming narrow-width approximation decays of $\mathrm{X}$, only the mass range $m_{\mathrm{X}}<m_{\mathrm{H}}-m_{\mathrm{Z}}\approx 35\text{GeV}$ ($m_{\mathrm{X}}<m_{\mathrm{H}}/2\approx 62.5\text{GeV}$) is kinematically possible for $\mathrm{H}\to \mathrm{Z}\mathrm{X}$ ($\mathrm{H}\to \mathrm{X}\mathrm{X}$), where $m_{\mathrm{H}}$ and $m_{\mathrm{Z}}$ are the Higgs boson mass and Z boson mass, respectively. The decay channel $\mathrm{p}\mathrm{p}\to \mathrm{H}\to 4\ell$ has a large signal-to-background ratio. This channel allows a complete reconstruction of the kinematics of the Higgs boson based on final-state decay particles. In this analysis, a mass range of $4.0<m_{\mathrm{X}}<35.0\text{GeV}$ (62.5$\text{GeV}$) is considered.

The first model considered, hereby referred to as the “hidden Abelian Higgs model” (HAHM), concerns theories with a hidden “dark” sector [7–11], with the X particle identified as the dark photon (${\mathrm{Z}}_{\mathrm{D}}$), which mediates a dark $U{(1)}_D$ gauge symmetry, which is spontaneously broken by a dark Higgs mechanism. Interactions of the dark sector with SM particles can occur through a hypercharge portal via the kinetic-mixing parameter $\epsilon$, or through a Higgs portal via the Higgs-mixing parameter $\kappa$, as shown in Fig. 1. Details of this theory and subsequent phenomenological implications can be found in Ref. [7]. Several searches for ${\mathrm{Z}}_{\mathrm{D}}$ were previously performed by collider experiments, for example ATLAS [12, 13] and LHCb [14]. Other experiments, such as beam dump experiments, fixed target experiments, helioscopes, and cold dark matter direct detection experiments, provide complementary sensitivities to ${\mathrm{Z}}_{\mathrm{D}}$. A summary of the experimental coverage of the HAHM model can be found in Refs. [15, 16].

Fig. 1.

Feynman diagrams for Higgs boson decay via the kinetic-mixing (left) or Higgs-mixing mechanism (right) [7]. The symbol $\mathrm{h}$ represents the Higgs boson, and s represents the dark Higgs boson. The symbol $\epsilon$ represents the kinetic-mixing parameter while $\kappa$ represents the Higgs-mixing parameter

The second model involves axion-like particles (ALPs), with $\mathrm{X}$ being a pseudoscalar gauge singlet $\mathrm{a}$. Axions were originally proposed to address the strong CP problem [17]. Recently, ALPs were proposed to explain the observed anomaly in the magnetic moment of the muon [18]. Theoretical overviews of the ALP models can be found in Refs. [19, 20]. The models are formulated as an effective field theory of ALPs coupled to various SM particles. In particular, the theory allows the coupling between the Higgs boson, Z boson, and the ALP field, or the Higgs boson and the ALP field. These couplings are represented by the Wilson coefficients $C_{\mathrm{Z}\mathrm{H}}/\Lambda$ and $C_{\mathrm{a}\mathrm{H}}/{\Lambda }^2$, respectively, where $\Lambda$ is the decoupling energy scale in the effective field theory, or the mass scale of new physics. The former (latter) coefficient gives rise to the exotic decay of $\mathrm{H}\to \mathrm{Z}\mathrm{a}$ ($\mathrm{a}\mathrm{a}$). Various experimental searches for $\mathrm{H}\to \mathrm{a}\mathrm{a}$ have been performed [21–26]. Recently a direct search for $\mathrm{H}\to \mathrm{Z}\mathrm{a}$ has been performed targeting a signature with a light and hadronically decaying resonance $\mathrm{a}$ with $m_{\mathrm{a}}<4\text{GeV}$ [27]. The present search provides complementary coverage of the phase space of the ALP model with mass greater than 4$\text{GeV}$.

This paper is organized as follows. Section 2 describes the CMS detector and event reconstruction algorithms. Section 3 outlines the collision data used and various software packages used to generate the samples of simulated events. Section 4 summarizes the selection criteria and the categorization of signal events, and Sect. 5 describes the reducible background estimation method. Section 6 describes the various sources of systematic uncertainties in the search. Finally, results and interpretations are detailed in Sect. 7, and a summary is given in Sect. 8. Tabulated results are provided in HEPData [28].

The CMS detector and event reconstruction

The central feature of the CMS apparatus is a superconducting solenoid of 6$\text{m}$ internal diameter, providing a magnetic field of 3.8$\text{T}$. Within the solenoid volume are a silicon pixel and strip tracker, a lead tungstate crystal electromagnetic calorimeter (ECAL), and a brass and scintillator hadron calorimeter (HCAL), each composed of a barrel and two endcap sections. Forward calorimeters extend the pseudorapidity ($\eta$) coverage provided by the barrel and endcap detectors. Muons are detected in gas-ionization chambers embedded in the steel flux-return yoke outside the solenoid. A more detailed description of the CMS detector, together with a definition of the coordinate system used and the relevant kinematic variables, can be found in Ref. [29].

Events of interest are selected using a two-tiered trigger system [30]. The first level, composed of custom hardware processors, uses information from the calorimeters and muon detectors to select events at a rate of around 100$\text{kHz}$ within a fixed time interval of about 4$\mathrm{\mu }\text{s}$. The second level, known as the high-level trigger, consists of a farm of processors running a version of the full event reconstruction software optimized for fast processing, and reduces the event rate to around 1$\text{kHz}$ before data storage.

The candidate vertex with the largest value of summed physics-object $p_{\mathrm{T}}^2$ (where $p_{\mathrm{T}}$ is the transverse momentum) is taken to be the primary $\mathrm{p}\mathrm{p}$ interaction vertex. The physics objects are the jets, clustered using the jet finding algorithm [31, 32] with the tracks assigned to candidate vertices as inputs, and the associated missing transverse momentum, taken as the negative vector sum of the $p_{\mathrm{T}}$ of those jets.

The particle-flow (PF) algorithm [33] aims to reconstruct and identify each individual particle in an event (PF candidate), with an optimized combination of information from the various elements of the CMS detector. The energy of photons is obtained from the ECAL measurement. The energy of electrons is determined from a combination of the electron momentum at the primary interaction vertex as determined by the tracker, the energy of the corresponding ECAL cluster, and the energy sum of all bremsstrahlung photons spatially compatible with originating from the electron track. The energy of muons is obtained from the curvature of the corresponding track. The energy of charged hadrons is determined from a combination of their momentum measured in the tracker and the matching ECAL and HCAL energy deposits, corrected for the response function of the calorimeters to hadronic showers. Finally, the energy of neutral hadrons is obtained from the corresponding corrected ECAL and HCAL energies.

The missing transverse momentum vector ${\overrightarrow{p}}_{\mathrm{T}}^{\text{miss}}$ is computed as the negative vector sum of the transverse momenta of all the PF candidates in an event, and its magnitude is denoted as $p_{\mathrm{T}}^{\text{miss}}$  [34]. The ${\overrightarrow{p}}_{\mathrm{T}}^{\text{miss}}$ is modified to account for corrections to the energy scale of the reconstructed jets in the event.

Muons in the four lepton final state are measured in the range $|\eta |<2.4$, with detection planes made using three technologies: drift tubes, cathode strip chambers, and resistive plate chambers. The single-muon trigger efficiency exceeds 90% over the full $\eta$ range, and the efficiency to reconstruct and identify muons is greater than 96%. Matching muons to tracks measured in the silicon tracker results in a relative $p_{\mathrm{T}}$ resolution, for muons with $p_{\mathrm{T}}$ up to 100$\text{GeV}$, of 1% in the barrel and 3% in the endcaps [35].

Electrons in the four lepton final state with $p_{\mathrm{T}}>7\text{GeV}$ and $|\eta |<2.5$ are identified by a multivariate discriminant, which is constructed by observables related to the bremsstrahlung along the electron trajectory, ECAL energy measurements, electromagnetic showers, missing pixel detector hits, and the photon conversion vertex fit probability [36]. The electron momentum is estimated by combining the energy measurement in the ECAL with the momentum measurement in the tracker. The momentum resolution for electrons with $p_{\mathrm{T}}\approx 45\text{GeV}$ from $\mathrm{Z}\to \mathrm{e}\mathrm{e}$ decays ranges from 1.7 to 4.5%. It is generally better in the barrel region than in the endcaps, and also depends on the bremsstrahlung energy emitted by the electron as it traverses the material in front of the ECAL. The dielectron mass resolution for $\mathrm{Z}\to \mathrm{e}\mathrm{e}$ decays when both electrons are in the ECAL barrel (endcap) is 1.9% (2.9%).

This analysis focuses on promptly produced signal processes. To reduce the contributions from leptons arising from hadron decays within jets, a requirement is imposed on each lepton candidate using a variable defined as:

$$\begin{array}{c}\hfill I^{\ell }=\frac{\sum p_{\mathrm{T}}^{\text{charged}}+max[0,\sum p_{\mathrm{T}}^{\text{neutral}}+\sum p_{\mathrm{T}}^{\mathrm{\gamma }}-p_{\mathrm{T}}^{\text{PU}}]}{p_{\mathrm{T}}^{\ell }}\end{array}$$ 1

where the sums are over the PF candidates within a cone of radius $R=\sqrt{\Delta {\eta }^2+\Delta {\varphi }^2}<0.3$ (where $\varphi$ is the azimuthal angle in radians), $p_{\mathrm{T}}^i$ represents transverse momenta from each particle i, where i represents either charged hadrons, neutral hadrons, photons, or particles originating from overlapping proton–proton interactions (pileup) [37]. For muons, the isolation is required to be $I^{\mathrm{\mu }}<0.35$. For electrons, this variable is included in the multivariate discriminant for datasets in 2017 and 2018, while for the dataset in 2016, an isolation requirement $I^{\mathrm{e}}<0.35$ is imposed on each electron candidate. In addition, the three-dimensional impact parameter of electrons and muons is required to be consistent with the primary collision vertex. The requirement implies a negligible acceptance to signal models with long-lived $\mathrm{X}$.

An algorithm is utilized to correct for effects arising from final-state radiation (FSR) from leptons. PF-reconstructed photons are considered as FSR candidates if they satisfy the requirement $p_{\mathrm{T}}^{\mathrm{\gamma }}>2\text{GeV}$ and $I^{\mathrm{\gamma }}<1.8$, where $I^{\mathrm{\gamma }}$ is calculated similarly to the lepton isolation variable. Then each FSR candidate is assigned to the closest lepton in the event. The candidates are further required to have $\Delta R(\mathrm{\gamma },\ell )/{(p_{\mathrm{T}}^{\mathrm{\gamma }})}^2<0.012{\text{GeV}}^{-2}$ and $\Delta R(\mathrm{\gamma },\ell )<0.5$. These candidates are excluded from the calculation of the lepton isolation variables.

Lepton reconstruction and selection efficiencies are measured in data by a “tag-and-probe” technique with an inclusive sample of $\mathrm{Z}$ boson events [38]. The difference between the efficiencies in data and simulation are observed to be around 1–4%, depending on $p_{\mathrm{T}}$ and $\eta$ of the lepton considered. The differences are used to correct lepton efficiencies in simulation.

Data and simulated samples

Leading order (LO) signal samples for the physics processes $\mathrm{p}\mathrm{p}\to \mathrm{H}\to \mathrm{Z}{\mathrm{Z}}_{\mathrm{D}}({\mathrm{Z}}_{\mathrm{D}}{\mathrm{Z}}_{\mathrm{D}})\to 4\ell$, where $\ell =(\mathrm{e},\mathrm{\mu })$, are generated using the MadGraph 5_amc@nlo 2.2.2 (2.4.2) [39–41] generator for 2016 (2017 and 2018), with HAHM [7] at leading order. Cross sections for each ${\mathrm{Z}}_{\mathrm{D}}$ signal are calculated by multiplying the next-to-next-to-next-to-leading order (NNNLO) Higgs production cross section [42] by the branching fraction of $\mathrm{H}\to \mathrm{Z}{\mathrm{Z}}_{\mathrm{D}}$ and $\mathrm{H}\to {\mathrm{Z}}_{\mathrm{D}}{\mathrm{Z}}_{\mathrm{D}}$, respectively [7]. Final states with $\tau$ leptons are neglected as their contribution to the signal region yield is below $1\%$. Signal contributions from vector-boson fusion and associated production with a top quark pair or a vector boson are also omitted.

The SM Higgs boson simulation samples, which include gluon fusion, vector boson fusion, and associated production with a top quark pair or a vector boson, and the simulated $\mathrm{Z}\mathrm{Z}$ background from quark-antiquark annihilation are generated at next-to-leading order (NLO) in perturbative quantum chromodynamics with powheg  v2 [43–46]. The cross section for the dominant production mode, gluon fusion, is taken at NNNLO [42].

Decays of the Higgs boson to four leptons are simulated with JHUGen 7.0.2 [47, 48]. The non-resonant process of $\mathrm{g}\mathrm{g}\to \mathrm{Z}\mathrm{Z}$ process is simulated at LO with mcfm 7.0.1 [49]. NLO correction factors [50] are applied to the $\mathrm{g}\mathrm{g}\to \mathrm{Z}\mathrm{Z}$ process.

Minor backgrounds from $\mathrm{t}\overline{\mathrm{t}}\mathrm{Z}$ and triboson production processes are also simulated at LO and NLO, respectively, with the MadGraph 5_amc@nlo 2.2.2 (2.4.2) [39–41] generator for 2016 (2017 and 2018).

The set of parton distribution functions (PDFs) used was NNPDF3.0 [51] (NNPDF3.1 [52]) for the 2016 (2017 and 2018) simulation. Parton showering and hadronization are simulated using the pythia  8.230 generator [53] with the CUETP8M1 (CP5) underlying event tune for the 2016 (2017 and 2018) simulation [54, 55]. The response of the CMS detector is modeled using the Geant4 program [56, 57]. Simulated events are reweighted according to a specified instantaneous luminosity and an average number of pileup events.

Event selection

In the trigger system, events are required to have more than two leptons. The overall trigger efficiency is measured in data using a sample of $4\ell$ events from single-lepton triggers and agreements are observed with simulation within 5%, and is found to be larger than 99%.

A set of requirements is applied to maximize the sensitivity of the search for a potential signal in the $\mathrm{Z}\mathrm{X}$ and $\mathrm{X}\mathrm{X}$ event topologies. In both cases, at least four well-identified and isolated leptons from the primary vertex are required, possibly accompanied by an FSR photon. Each muon (electron) is required to have $p_{\mathrm{T}}>5\text{GeV}$ (7$\text{GeV}$). All four leptons must be separated from each other by $\Delta R({\ell }_i,{\ell }_j)>0.02$. The leading (subleading) lepton $p_{\mathrm{T}}$ is required to satisfy $p_{\mathrm{T}}>20\text{GeV}$ (10$\text{GeV}$). The four-lepton invariant mass $m_{4\ell }$ is required to be within $118<m_{4\ell }<130\text{GeV}$. To further suppress background contributions from hadron decays in jet fragmentation or from the decay of low-mass resonances, all opposite-charge leptons pairs, regardless of lepton flavor, are required to satisfy $m_{{\ell }^{+}{\ell }^{-}}>4\text{GeV}$.

For each event in the $\mathrm{Z}\mathrm{X}$ and $\mathrm{X}\mathrm{X}$ searches, dilepton pair candidates are formed by considering all OSSF leptons. The dilepton invariant mass $m_{{\ell }^{+}{\ell }^{-}}$ for each candidate is required to be within $4<m_{{\ell }^{+}{\ell }^{-}}<120\text{GeV}$, however the mass window around the ${\rm Y}$ $\mathrm{b}\overline{\mathrm{b}}$ bound states ($8.0<m_{{\rm Y}}<11.5\text{GeV}$) is also excluded.

Two dilepton candidates are then paired to form a $\mathrm{Z}\mathrm{X}$ or $\mathrm{X}\mathrm{X}$ event candidate. For the $\mathrm{Z}\mathrm{X}$ search, ${\mathrm{Z}}_1$ is the OSSF dilepton pair with an invariant mass closest to the $\mathrm{Z}$ boson mass [58] (representing $\mathrm{Z}$ in $\mathrm{Z}\mathrm{X}$), and ${\mathrm{Z}}_2$ is the other pair ($\mathrm{X}$). For the $\mathrm{X}\mathrm{X}$ search, ${\mathrm{Z}}_1$ is the OSSF dilepton pair with the larger invariant mass, and ${\mathrm{Z}}_2$ is the lower-mass pair. For the $\mathrm{Z}\mathrm{X}$ search, $m_{{\mathrm{Z}}_1}$ is required to be larger than 40$\text{GeV}$. For the $\mathrm{X}\mathrm{X}$ search, $m_{{\mathrm{Z}}_1}$ and $m_{{\mathrm{Z}}_2}$ must lie between 4 and 62.5$\text{GeV}$. For events with more than four selected leptons, the combination of four leptons with $m_{{\mathrm{Z}}_1}$ closest to the Z boson is used for the $\mathrm{Z}\mathrm{X}$ candidate, while the combination with the least value of $(m_{{\mathrm{Z}}_1}-m_{{\mathrm{Z}}_2})/(m_{{\mathrm{Z}}_1}+m_{{\mathrm{Z}}_2})$ is used to select $\mathrm{X}\mathrm{X}$ candidates with similar invariant masses.

Four final-state lepton categories can be defined as $4\mathrm{\mu }$, $2\mathrm{\mu }2\mathrm{e}$, $4\mathrm{e}$, $2\mathrm{e}2\mathrm{\mu }$, where the order of lepton flavors corresponds to Z1 and Z2 flavors. For the $4\mathrm{\mu }$ and $4\mathrm{e}$ final states, one alternative pairing of the four leptons is possible, labelled by ${\mathrm{Z}}_{\text{a}}$ and ${\mathrm{Z}}_{\text{b}}$. For the $\mathrm{Z}\mathrm{X}$ search, events with $m_{{\mathrm{Z}}_{\text{b}}}<12\text{GeV}$ and $m_{{\mathrm{Z}}_{\text{a}}}$ closer to the $\mathrm{Z}$ boson mass than ${\mathrm{Z}}_1$ are discarded to suppress background contributions from on-shell Z and low-mass dilepton resonances. For the $\mathrm{X}\mathrm{X}$ search, the $\mathrm{X}\mathrm{X}$ candidate with the smallest value of $(m_{{\mathrm{Z}}_1}-m_{{\mathrm{Z}}_2})/(m_{{\mathrm{Z}}_1}+m_{{\mathrm{Z}}_2})$ is chosen.

Background estimation

Irreducible background estimation

Irreducible backgrounds for this search come from processes including a SM Higgs boson, as well as nonresonant production of $\mathrm{Z}\mathrm{Z}$ via quark-antiquark annihilation or gluon fusion, and rare backgrounds such as $\mathrm{t}\overline{\mathrm{t}}+\mathrm{Z}$ and triboson production. These backgrounds are estimated from simulation. Details of the simulation used for each of the backgrounds are described in Sect. 3.

Reducible background estimation

The reducible backgrounds in the $4\ell$ final state can arise from the leptonic decays of heavy-flavor hadrons, in-flight decays of light mesons within jets, charged hadrons misidentified as electrons when in proximity of a ${\mathrm{\pi }}^0$, and photon conversions. These backgrounds primarily arise from the $\mathrm{Z}+\text{jets}$ process. Additional physics processes with kinematics similar to the signal include $\mathrm{t}\overline{\mathrm{t}}$, $\mathrm{Z}\mathrm{\gamma }$, and $\mathrm{W}\mathrm{Z}$.

Two dedicated control regions are used to estimate the contribution from these backgrounds. The first (second) control region consists of events with two (three) leptons passing the lepton identification and isolation requirements and two (one) leptons failing the requirements, and is denoted as the 2P2F (3P1F) region. Backgrounds with only two prompt leptons, such as $\mathrm{Z}+\text{jets}$ and $\mathrm{t}\overline{\mathrm{t}}$, are estimated by the 2P2F region, while backgrounds with three prompt leptons, such as $\mathrm{W}\mathrm{Z}$ and $\mathrm{Z}\mathrm{\gamma }$ with the photon converting to an electron pair, are estimated by the 3P1F region. Other than the lepton requirements, the 3P1F and 2P2F regions follow the same event selection and alternative pairing algorithms as in the signal region to closely mimic its kinematics.

The lepton misidentification rates $f_{\mathrm{\mu }}$ and $f_{\mathrm{e}}$ are measured as a function of lepton $p_{\mathrm{T}}$ and $\eta$ with a sample which includes a $\mathrm{Z}$ candidate, formed by a pair of leptons passing the selection requirement of the analysis, and an additional lepton passing a relaxed requirement. These rates are measured separately in the data samples from 2016, 2017, and 2018. In addition, the mass of the $\mathrm{Z}$ candidate is required to satisfy the condition $|m_{{\mathrm{Z}}_1}-m_{\mathrm{Z}}|<7\text{GeV}$ to reduce contributions from $\mathrm{W}\mathrm{Z}$ and $\mathrm{t}\overline{\mathrm{t}}$ processes, and $p_{\mathrm{T}}^{\text{miss}}$ is required to be less than 25$\text{GeV}$.

To estimate the background contribution in the signal region, events in the 3P1F and 2P2F control regions are reweighted by lepton misidentification probabilities. Each event i in the 3P1F region is weighted by a factor $f_4^i/(1-f_4^i)$, where $f_4^i$ corresponds to the lepton misidentification rate of the failed lepton in the event. Physics processes in the 2P2F control region can contribute to the 3P1F region and are estimated by reweighting 2P2F events with $f_3^i/(1-f_3^i)+f_4^i/(1-f_4^i)$, where $f_3^i$ and $f_4^i$ correspond to the lepton misidentification rates of the two failed leptons in the event. A minor contribution from $\mathrm{Z}\mathrm{Z}$ events to the 3P1F control region is estimated from simulation and subtracted. The expected yield for the signal region can then be estimated as:

$$\begin{array}{cc}\hfill N_{\mathrm{SR}}^{\text{reducible}}=& \left(1,-,\frac{N_{3P1F}^{\mathrm{Z}\mathrm{Z}}}{N_{3P1F}}\right)\hfill \\ \hfill & \times \sum _i^{N_{3P1F}}\frac{f_4^i}{1-f_4^i}-\sum _i^{N_{2P2F}}\frac{f_3^i}{1-f_3^i}\frac{f_4^i}{1-f_4^i}\hfill \end{array}$$ 2

where each sum is over all 3P1F and 2P2F events, respectively.

Furthermore, dedicated validation regions, which include adjacent $m_{4\ell }$ regions to the signal region ($70<m_{4\ell }<118\text{GeV}$, $130<m_{4\ell }<200\text{GeV}$), are defined to inspect the level of agreement between data and predictions.

Systematic uncertainties

Experimental sources of the systematic uncertainties applicable to all final states include the integrated luminosity uncertainty and the lepton identification and reconstruction efficiency uncertainty. The integrated luminosities of the 2016, 2017, and 2018 data-taking periods are individually known with uncertainties in the 1.2–2.5% range [59–61], while the total Run 2 (2016–2018) integrated luminosity has an uncertainty of 1.6% [62], the improvement in precision reflecting the (uncorrelated) time evolution of some systematic effects. Lepton efficiency uncertainties are estimated in bins of lepton $p_{\mathrm{T}}$ and $\eta$ using the tag-and-probe method, as described in Sect. 2. These uncertainties on each lepton candidate lead to variations from 2.5 to 16.1% on event yields, dependent on final-state lepton categories. In addition, the systematic uncertainties in the lepton energy scale are determined by fitting the $\mathrm{Z}\to \ell \ell$ mass distribution in bins of lepton $p_{\mathrm{T}}$ and $\eta$ with a Breit–Wigner parameterization convolved with a double-sided Crystal Ball function [63]. Systematic uncertainties in the estimation of the reducible background are derived from the level of agreement between data and predictions in the validation regions in each lepton category (23–48% depending on data taking period), arising from different background compositions between signal and control regions (30–38% depending on lepton category), and from misidentification rate uncertainties (35–100% depending on lepton category).

Theoretical uncertainties that affect both the signal and background estimation include uncertainties in the renormalization and factorization scales and the choice of the PDF set. The uncertainty from the renormalization and factorization scales is determined by varying these scales between 0.5 and 2 times their nominal value while keeping their ratio between 0.5 and 2. The uncertainty from the PDF set is determined by taking the root-mean-square of the variation when using different replicas of the default NNPDF set [64]. An additional uncertainty of 10% in the K factor used for the $\mathrm{g}\mathrm{g}\to 4\ell$ prediction is included [37]. To estimate the effect of the interference between the signal and background processes, three types of samples are generated using the MadGraph 5_amc@nlo 2.4.2 [39–41] generator: inclusive sample ($\mathrm{H}\to \mathrm{Z}{\mathrm{Z}}^{\ast }\to 4\ell$, $\mathrm{H}\to \mathrm{Z}\mathrm{X}/\mathrm{X}\mathrm{X}\to 4\ell$), signal-only sample $\mathrm{H}\to \mathrm{Z}\mathrm{X}/\mathrm{X}\mathrm{X}\to 4\ell$ and background-only sample $\mathrm{H}\to \mathrm{Z}{\mathrm{Z}}^{\ast }\to 4\ell$. The inclusive sample contains background, signal, and interference contributions. The effect of the interference on the normalization of the signal is estimated by taking the difference of the inclusive sample cross section and the sum of the cross sections of the signal and background samples. This difference is at 1–2% after the final event selection. Theoretical values of branching fractions $\mathcal{B}({\mathrm{Z}}_{\mathrm{D}}\to \mathrm{e}\mathrm{e}\text{or}\mathrm{\mu }\mathrm{\mu })$ are calculated in Ref. [7]. The calculations are based on experimental measurements of the ratio of the hadronic cross section to the muon cross section in electron-positron collisions $R_{\mathrm{\mu }\mathrm{\mu }}/R_{\text{had}}$ up to $m_{{\mathrm{Z}}_{\mathrm{D}}}=12\text{GeV}$ and a next-to-leading order theoretical calculation for $m_{{\mathrm{Z}}_{\mathrm{D}}}>12\text{GeV}$. To account for uncertainties in these theoretical estimates, a conservative 20% (10%) uncertainty is assigned to them for $m_{{\mathrm{Z}}_{\mathrm{D}}}<12\text{GeV}$ ($m_{{\mathrm{Z}}_{\mathrm{D}}}>12\text{GeV}$) [7]. Differences in the kinematic properties between the HAHM and ALP model have been inspected. For the determination of model-independent exclusion limits, differences in acceptances are included as systematic uncertainties, ranging from 10% ($m_{\mathrm{X}}\sim 4\text{GeV}$) to 30% ($m_{\mathrm{X}}\sim 35\text{GeV}$ for $\mathrm{Z}\mathrm{X}$, $m_{\mathrm{X}}\sim 60\text{GeV}$ for $\mathrm{X}\mathrm{X}$), while they are used to correct signal yields for the determination of ALP exclusion limits.

In the combination of the three data taking periods, the theoretical uncertainties and experimental ones related to leptons are correlated across all data taking periods, while all others from experimental sources are taken as uncorrelated. The sensitivity of this analysis is dominated by data statistical uncertainty rather than systematic uncertainties.

Results and interpretation

Dilepton mass distributions for the $\mathrm{Z}\mathrm{X}$ and $\mathrm{X}\mathrm{X}$ selections are shown in Figs. 2 and 3, respectively. The dilepton mass variable for the $\mathrm{X}\mathrm{X}$ selection shown in Fig. 3 is $m_{Z12}=(m_{{\mathrm{Z}}_1}+m_{{\mathrm{Z}}_2})/2$, which should peak at $m_{\mathrm{X}}$ in case of a signal $\mathrm{H}\to \mathrm{X}\mathrm{X}$. In all cases, the observed distributions agree well with standard model expectations within the assigned uncertainties.

Fig. 2.

Event yields against $m_{{\mathrm{Z}}_2}$ with the $\mathrm{Z}\mathrm{X}$ selection for the muon and electron channels. Numbers in the legend show the total event yields with the $\mathrm{Z}\mathrm{X}$ selection corresponding to data, and the expected yields for each background and signal processes, along with the corresponding statistical uncertainty coming from the amount of simulated data

Fig. 3.

Event yields against $m_{Z12}=(m_{{\mathrm{Z}}_1}+m_{{\mathrm{Z}}_2})/2$ with the $\mathrm{X}\mathrm{X}$ selection for the 4$\mu$, 2e2$\mu$, and 4e final states. Numbers in the legend show the total event yields with the $\mathrm{X}\mathrm{X}$ selection corresponding to data, and the expected yields for each background and signal processes, along with the corresponding statistical uncertainty coming from the amount of simulated data

These results are further interpreted as upper limits on model-independent branching fractions and model parameters for the dark photon and ALP models. For interpretations of the results of the $\mathrm{Z}\mathrm{X}$ selection, 351 mass hypotheses are considered. Each mass hypothesis $m_i$ is defined with an incremental step of 0.5%, as $m_i=4.20\times 1.{005}^i$, where $i=0,1,2,\dots ,424$, excluding the mass points around the ${\rm Y}$ $\mathrm{b}\overline{\mathrm{b}}$ bound states between $8.0<m_{{\rm Y}}<11.5\text{GeV}$ ($i=130,131,\dots ,201$). The incremental step is chosen so as not to miss any potential signal contribution due to detector resolution in $m_{{\mathrm{Z}}_2}$. For each mass hypothesis, the counting experiments are performed on the $m_{{\mathrm{Z}}_2}$ distribution, with the bin centered at each mass hypothesis. Because of the finite mass resolution of $m_{{\mathrm{Z}}_2}$, the choice of the bin width needs to be defined such that most of the signal contribution is included in the bin. The bin width is defined as $0.04(0.10)\times m_i$ for the $4\mathrm{\mu }$ and $2\mathrm{e}2\mathrm{\mu }$ ($4\mathrm{e}$ and $2\mathrm{\mu }2\mathrm{e}$) categories. This width is chosen as two times the $m_{{\mathrm{Z}}_2}$ resolution and includes $\approx$95% of signal events. The normalization of the Higgs background is allowed to float freely in the likelihood fit. For each mass hypothesis, events outside the mass window are included as a sideband to constrain the normalization parameter. No significant deviation with respect to the SM prediction is observed.

For interpretations of the results of the $\mathrm{X}\mathrm{X}$ selection, 462 mass hypotheses are considered instead. In contrast to the $\mathrm{Z}\mathrm{X}$ interpretations, the counting experiments are performed by constructing a rectangular region, centered at each mass hypothesis, in the ($m_{{\mathrm{Z}}_1}$,$m_{{\mathrm{Z}}_2}$) plane. The rectangular regions are effectively triangular as $m_{{\mathrm{Z}}_1}$ is defined as the larger invariant mass. The bin widths are defined in a similar manner as $0.04m_i$ ($0.10m_i$) for $m_{{\mathrm{Z}}_1}$ or $m_{{\mathrm{Z}}_2}$ formed by muon (electron) pairs.

The likelihood model for each mass hypothesis is formulated as

$$\begin{array}{cc}\hfill {\mathcal{L}}_m=& {\mathcal{L}}_{m,\mathrm{SR}}{\mathcal{L}}_{m,\text{SB}}\hfill \end{array}$$ 3

$$\begin{array}{cc}\hfill {\mathcal{L}}_{m,\mathrm{SR}}=& \prod _{\ell }\mathrm{Pois}(n_{m,\ell }|{\mu }_{\text{Higgs}}{n}_{\text{Higgs},m,\ell }\hfill \\ \hfill & +\sum _b{n}_{b,m,\ell }{\rho }_{b,m,\ell }+\mu n_{s,m,\ell }{\rho }_{s,m,\ell }),\hfill \end{array}$$ 4

$$\begin{array}{cc}\hfill {\mathcal{L}}_{m,\text{SB}}=& \prod _{\ell }\mathrm{Pois}(n_{\ell }|{\mu }_{\text{Higgs}}{n}_{\text{Higgs},\ell }+\sum _b{n}_{b,\ell }{\rho }_{b,\ell })\hfill \end{array}$$ 5

where the function $\mathrm{Pois}(n|x)$ is the Poisson probability to observe n events, when the expectation is x. The symbol m represents a particular mass hypothesis. The likelihood term ${\mathcal{L}}_{m,\mathrm{SR}}$ (${\mathcal{L}}_{m,\text{SB}}$) corresponds to the event yields within (outside) the mass window. The symbol $\mu$ is the signal strength parameter, ${\mu }_{\text{Higgs}}$ represents the free floating normalizing parameter on the SM Higgs boson process, $\ell$ represents each lepton category, b represents each background process, s represents a particular signal process and $n_{i,m,\ell }$ represents the yield in a mass window associated with the mass hypothesis m, from a source i and the lepton category $\ell$. In Eq. (5), the symbols $n_{\text{Higgs},\ell }$ and $n_{b,\ell }$ represent the yields of the SM Higgs boson and other backgrounds b outside the mass window for the lepton category $\ell$. Systematic uncertainties are included and profiled as nuisance parameters $\rho$ [65].

For each interpretation, $95\%$ exclusion limits are obtained with an asymptotic formulation of the modified frequentist ${\text{CL}}_{\text{s}}$ criterion as described in Refs. [65–68] with the ZX selection and full ${\text{CL}}_{\text{s}}$ approach for the XX selection.

Model-independent limits

Upper limits at 95% confidence level (CL) are derived on model-independent branching fractions with the $\mathrm{Z}\mathrm{X}$ and $\mathrm{X}\mathrm{X}$ selections assuming three decay channels: a flavor symmetric decay of $\mathrm{X}$ to a muon or an electron pair, exclusive $\mathrm{X}$ decays to a muon pair, and exclusive $\mathrm{X}$ decays to an electron pair. Acceptance effects arising from different signal models are included as systematic uncertainties in the signal yields after event selection. Little model dependence is expected as the event selection is defined without using angular information between the leptons. Figures 4 and 5 show the exclusion limits on the model-independent branching fractions with the $\mathrm{Z}\mathrm{X}$ and $\mathrm{X}\mathrm{X}$ selections, respectively. The weaker observed limit in the XX selection at $m_{\mathrm{X}}\approx 18\text{GeV}$ is due to one observed data event and does not represent a significant statistical deviation from the background hypothesis. Kinematic differences between the dark photon and ALP models are included as systematic uncertainties, as detailed in Sect. 6.

Fig. 4.

Expected and observed 95% CL limits on $\mathcal{B}(\mathrm{H}\to \mathrm{Z}\mathrm{X})\mathcal{B}(\mathrm{X}\to \mathrm{\mu }\mathrm{\mu })$ assuming $\mathrm{X}$ decays to dimuons only, $\mathcal{B}(\mathrm{H}\to \mathrm{Z}\mathrm{X})\mathcal{B}(\mathrm{X}\to \mathrm{e}\mathrm{e})$ assuming $\mathrm{X}$ decays to dielectrons only, and $\mathcal{B}(\mathrm{H}\to \mathrm{Z}\mathrm{X})\mathcal{B}(\mathrm{X}\to \mathrm{e}\mathrm{e}\text{or}\mathrm{\mu }\mathrm{\mu })$ assuming a flavor symmetric decay of $\mathrm{X}$ to dimuons and dielectrons. The dashed black curve is the expected upper limit, with one and two standard-deviation bands shown in green and yellow, respectively. The solid black curve is the observed upper limit. The red curve represents the theoretical cross section for the signal process $\mathrm{H}\to \mathrm{Z}\mathrm{X}\to 4\ell$. The discontinuity at 12$\text{GeV}$ in the uncertainty is due to the switch from experimental to theoretical uncertainty estimates of $\mathcal{B}({\mathrm{Z}}_{\mathrm{D}}\to \mathrm{e}\mathrm{e}\text{or}\mathrm{\mu }\mathrm{\mu })$, as described in Ref. [7]. The symbol $\epsilon$ is the kinetic-mixing parameter. The grey band corresponds to the excluded region around the $\mathrm{b}\overline{\mathrm{b}}$ bound states of ${\rm Y}$

Fig. 5.

Expected and observed 95% CL limits on $\mathcal{B}(\mathrm{H}\to \mathrm{X}\mathrm{X})\mathcal{B}{(\mathrm{X}\to \mathrm{\mu }\mathrm{\mu })}^2$ assuming $\mathrm{X}$ decays to dimuons only, $\mathcal{B}(\mathrm{H}\to \mathrm{X}\mathrm{X})\mathcal{B}{(\mathrm{X}\to \mathrm{e}\mathrm{e})}^2$ assuming $\mathrm{X}$ decays to dielectrons only, and $\mathcal{B}(\mathrm{H}\to \mathrm{X}\mathrm{X})\mathcal{B}{(\mathrm{X}\to \mathrm{e}\mathrm{e}\text{or}\mu \mu )}^2$ assuming a flavor symmetric decay of $\mathrm{X}$ to dimuons and dielectrons. The dashed black curve is the expected upper limit, with one and two standard-deviation bands shown in green and yellow, respectively. The solid black curve is the observed upper limit. The red curve represents the theoretical cross section for the signal process $\mathrm{H}\to \mathrm{X}\mathrm{X}\to 4\ell$. The discontinuity at 12$\text{GeV}$ in uncertainty is due to the switch from experimental to theoretical uncertainty estimates of $\mathcal{B}({\mathrm{Z}}_{\mathrm{D}}\to \mathrm{e}\mathrm{e}\text{or}\mathrm{\mu }\mathrm{\mu })$, as described in Ref. [7]. The symbol $\kappa$ is the Higgs-mixing parameter. The grey band corresponds to the excluded region around the $\mathrm{b}\overline{\mathrm{b}}$ bound states of ${\rm Y}$

Limits on dark photon model parameters

Upper limits at 95% CL are obtained on the Higgs-mixing parameter $\kappa$ and $\mathcal{B}(\mathrm{H}\to {\mathrm{Z}}_{\mathrm{D}}{\mathrm{Z}}_{\mathrm{D}})$ with the $\mathrm{X}\mathrm{X}$ selection, as shown in Fig. 6, assuming $\kappa \gg \epsilon$. The LHC provides unique sensitivity to the parameter $\kappa$ due to the presence of the Higgs boson. In addition, this analysis provides some sensitivity to $\epsilon$, but the upper limits are almost an order of magnitude weaker than those from the Drell–Yan search and from the LHCb Collaboration [14], and hence are not reported in this paper.

Fig. 6.

95% CL limits on the Higgs-mixing parameter $\kappa$, based on the $\mathrm{X}\mathrm{X}$ selection, as function of $m_{{\mathrm{Z}}_{\mathrm{D}}}$. The dashed black curve is the expected upper limit, with one and two standard-deviation bands shown in green and yellow, respectively. The solid black curve is the observed upper limit. The grey band corresponds to the excluded region around the $\mathrm{b}\overline{\mathrm{b}}$ bound states of ${\rm Y}$

Limits on the ALP model

Upper limits at 95% CL are calculated on the Wilson coefficients $C_{\mathrm{Z}\mathrm{H}}/\Lambda$ and $C_{\mathrm{a}\mathrm{H}}/{\Lambda }^2$, as shown in Fig. 7, where $C_{\mathrm{Z}\mathrm{H}}$ is the effective coupling parameter of the Higgs boson, $\mathrm{Z}$ boson, and the ALP, $C_{\mathrm{a}\mathrm{H}}$ is the effective coupling parameter of the Higgs boson and the ALP, and $\Lambda$ is the new physics scale. In both interpretations, the ALP is assumed to decay promptly with $\mathcal{B}(\mathrm{a}\to \mathrm{e}\mathrm{e}\text{or}\mathrm{\mu }\mathrm{\mu })=1$, with equal fractions to muons and electrons. The last six mass hypotheses are omitted in the calculation of upper limits on $C_{\mathrm{Z}\mathrm{H}}/\Lambda$ to match the $m_{\mathrm{a}}$ range adopted in Ref. [20]. Kinematic differences between the dark photon and ALP models are included as corrections on signal region yields, as detailed in Sect. 6.

Fig. 7.

95% CL limit on $C_{\mathrm{Z}\mathrm{H}}/\Lambda$ and $C_{\mathrm{a}\mathrm{H}}/{\Lambda }^2$ as function of $m_{\mathrm{a}}$. Black curves are the expected upper limits, with one and two standard-deviation bands shown in green and yellow, respectively. The solid black curves represent the observed upper limits. The grey band corresponds to the excluded region around the $\mathrm{b}\overline{\mathrm{b}}$ bound states of ${\rm Y}$

Summary

A search for dilepton resonances in Higgs boson decays to four-lepton final states has been presented. The search considers the two intermediate decay topologies $\mathrm{H}\to \mathrm{Z}\mathrm{X}$ and $\mathrm{H}\to \mathrm{X}\mathrm{X}$. No significant deviations from the standard model expectations are observed. The search imposes experimental constraints on products of model-independent branching fractions of $\mathcal{B}(\mathrm{H}\to \mathrm{Z}\mathrm{X})$, $\mathcal{B}(\mathrm{H}\to \mathrm{X}\mathrm{X})$ and $\mathcal{B}(\mathrm{X}\to \mathrm{e}\mathrm{e}\text{or}\mathrm{\mu }\mathrm{\mu })$, assuming flavor-symmetric decays of $\mathrm{X}$ to dimuons and dielectrons, exclusive decays of $\mathrm{X}$ to dimuons, and exclusive decays of $\mathrm{X}$ to dielectrons, for $m_{\mathrm{X}}>4\text{GeV}$. In addition, two well-motivated theoretical frameworks beyond the standard model are considered. Due to the presence of the Higgs boson production in LHC proton–proton collisions, the search provides unique constraints on the Higgs-mixing parameter $\kappa <4\times {10}^{-4}$ at 95% confidence level ($\text{CL}$) in a dark photon model with the $\mathrm{X}\mathrm{X}$ selection, in Higgs-mixing-dominated scenarios, while searches for ${\mathrm{Z}}_{\mathrm{D}}$ in Drell–Yan processes [14, 69] provide better exclusion limits on $\epsilon$ in kinetic-mixing-dominated scenarios. For the axion-like particle model, upper limits at 95% CL are placed on two relevant Wilson coefficients $C_{\mathrm{Z}\mathrm{H}}/\Lambda$ and $C_{\mathrm{a}\mathrm{H}}/{\Lambda }^2$. This is the first direct limit on decays of the observed Higgs boson to axion-like particles decaying to leptons.

Data Availability

This manuscript has no associated data or the data will not be deposited. [Authors’ comment: For CMS Release and preservation of data used by the CMS Collaboration as the basis for publications is guided by the CMS policy as stated in “CMS data preservation, re-use and open access policy” (https://cms-docdb.cern.ch/cgi-bin/PublicDocDB/RetrieveFile?docid=6032&filename=CMSDataPolicyV1.2.pdf&version=2).]

Declarations

Conflict of interest

The authors declare that they have no conflict of interest.