Observation of the ${\mathrm{\Lambda }}_{\text{b}}^0\to \text{J}/\mathrm{\psi }{\mathrm{\Xi }}^{-}{\text{K}}^{+}$ decay

Abstract

Using proton–proton collision data corresponding to an integrated luminosity of $140{\text{fb}}^{-1}$ collected by the CMS experiment at $\sqrt{s}=13\text{Te}\text{V}$, the ${\mathrm{\Lambda }}_{\text{b}}^0\to \text{J}/\mathrm{\psi }{\mathrm{\Xi }}^{-}{\text{K}}^{+}$ decay is observed for the first time, with a statistical significance exceeding 5 standard deviations. The relative branching fraction, with respect to the ${\mathrm{\Lambda }}_{\text{b}}^0\to \mathrm{\psi }(2\text{S})\mathrm{\Lambda }$ decay, is measured to be $\mathcal{B}({\mathrm{\Lambda }}_{\text{b}}^0\to \text{J}/\mathrm{\psi }{\mathrm{\Xi }}^{-}{\text{K}}^{+})/\mathcal{B}({\mathrm{\Lambda }}_{\text{b}}^0\to \mathrm{\psi }(2\text{S})\mathrm{\Lambda })=[3.38\pm 1.02\pm 0.61\pm 0.03]\%$, where the first uncertainty is statistical, the second is systematic, and the third is related to the uncertainties in $\mathcal{B}(\mathrm{\psi }(2\text{S})\to \text{J}/\mathrm{\psi }{\mathrm{\pi }}^{+}{\mathrm{\pi }}^{-})$ and $\mathcal{B}({\mathrm{\Xi }}^{-}\to \mathrm{\Lambda }{\mathrm{\pi }}^{-})$.

Introduction

Multibody decays of beauty hadrons present a rich laboratory to search for intermediate resonances in the decay products. When decay products contain a charmonium state, such intermediate resonances could decay into a charmonium meson and a hadron, which could be a manifestation of their exotic nature. An important turning point in exotic spectroscopy was achieved at the LHC, when the LHCb Collaboration reported the observation of statistically significant $\text{J}/\mathrm{\psi }\text{p}$ pentaquark-like structures in the decay of the lightest beauty baryon ${\mathrm{\Lambda }}_{\text{b}}^0\to \text{J}/\mathrm{\psi }\text{p}{\text{K}}^{-}$ [1]. Various interpretations of these structures have been proposed [2, 3], including tightly bound hidden-charm $[]$ pentaquark states [4, 5], loosely bound molecular baryon-meson states [6–8], or being due to a double triangle singularity [9]. More recently, additional exotic states have been reported by LHCb in the decays ${\mathrm{\Lambda }}_{\text{b}}^0\to \text{J}/\mathrm{\psi }\text{p}{\text{K}}^{-}$  [10], ${\mathrm{\Xi }}_{\text{b}}^{-}\to \text{J}/\mathrm{\psi }\mathrm{\Lambda }{\text{K}}^{-}$ [11],  [12], and ${\text{B}}^{-}\to \text{J}/\mathrm{\psi }\mathrm{\Lambda }\text{p}$ [13]. Up to now, the hidden-charm pentaquark candidates have been reported only in $\text{J}/\mathrm{\psi }\text{p}$ and $\text{J}/\mathrm{\psi }\mathrm{\Lambda }$ systems. Investigation of other channels with heavier baryons in the decay products, such as ${\mathrm{\Xi }}^{-}$ and ${\mathrm{\Omega }}^{-}$, could unveil the existence of doubly or triply strange pentaquarks [14, 15].

In this paper, we report on the search for the ${\mathrm{\Lambda }}_{\text{b}}^0\to \text{J}/\mathrm{\psi }{\mathrm{\Xi }}^{-}{\text{K}}^{+}$ decay, where the $\text{J}/\mathrm{\psi }\to {\mathrm{\mu }}^{+}{\mathrm{\mu }}^{-}$, ${\mathrm{\Xi }}^{-}\to \mathrm{\Lambda }{\mathrm{\pi }}^{-}$, and $\mathrm{\Lambda }\to \text{p}{\mathrm{\pi }}^{-}$ channels are used to reconstruct the intermediate decay products. Charge-conjugate states are implied throughout the text. The measurement of the ratio of branching fractions

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \mathcal{R}& \equiv \frac{\mathcal{B}({\mathrm{\Lambda }}_{\text{b}}^0\to \text{J}/\mathrm{\psi }{\mathrm{\Xi }}^{-}{\text{K}}^{+})}{\mathcal{B}({\mathrm{\Lambda }}_{\text{b}}^0\to \mathrm{\psi }(2\text{S})\mathrm{\Lambda })}=\frac{N({\mathrm{\Lambda }}_{\text{b}}^0\to \text{J}/\mathrm{\psi }{\mathrm{\Xi }}^{-}{\text{K}}^{+})}{N({\mathrm{\Lambda }}_{\text{b}}^0\to \mathrm{\psi }(2\text{S})\mathrm{\Lambda })}\hfill \\ \hfill & \times \frac{{ϵ}_{\mathrm{\psi }(2\text{S})\mathrm{\Lambda }}}{{ϵ}_{\text{J}/\mathrm{\psi }{\mathrm{\Xi }}^{-}{\text{K}}^{+}}}\frac{\mathcal{B}(\mathrm{\psi }(2\text{S})\to \text{J}/\mathrm{\psi }{\mathrm{\pi }}^{+}{\mathrm{\pi }}^{-})}{\mathcal{B}({\mathrm{\Xi }}^{-}\to \mathrm{\Lambda }{\mathrm{\pi }}^{-})}\hfill \end{array}\end{array}$$ 1

is also reported, where N is the measured ${\mathrm{\Lambda }}_{\text{b}}^0$ yield and $ϵ$ is the total efficiency. The normalization channel is chosen to be ${\mathrm{\Lambda }}_{\text{b}}^0\to \mathrm{\psi }(2\text{S})\mathrm{\Lambda }$, with the subsequent $\mathrm{\psi }(2\text{S})\to \text{J}/\mathrm{\psi }{\mathrm{\pi }}^{+}{\mathrm{\pi }}^{-}$ and $\text{J}/\mathrm{\psi }\to {\mathrm{\mu }}^{+}{\mathrm{\mu }}^{-}$ decays, because of its similar decay topology and kinematics to the signal decay, leading to the reduction of many systematic uncertainties. The branching fractions of the intermediate decays $\mathcal{B}(\text{J}/\mathrm{\psi }\to {\mathrm{\mu }}^{+}{\mathrm{\mu }}^{-})$ and $\mathcal{B}(\mathrm{\Lambda }\to \text{p}{\mathrm{\pi }}^{-})$ cancel in the ratio. Invariant mass distributions of the three two-body combinations for the signal channel are also presented in order to look for intermediate resonances.

The analysis uses proton–proton (pp) collision data recorded by the CMS experiment in 2016–2018, at $\sqrt{s}=13\text{Te}\text{V}$, corresponding to an integrated luminosity of $140{\text{fb}}^{-1}$ [16–18]. Tabulated results are provided in the HEPData record for this analysis [19].

The CMS detector and simulated event samples

The central feature of the CMS apparatus is a superconducting solenoid of 6 m internal diameter, providing a magnetic field of 3.8 T. Within the solenoid volume are a silicon pixel and strip tracker, a lead tungstate crystal electromagnetic calorimeter, and a brass and scintillator hadron calorimeter, each composed of a barrel and two endcap sections. Forward calorimeters extend the pseudorapidity coverage provided by the barrel and endcap detectors. Muons are measured in gas-ionization detectors embedded in the steel flux-return yoke outside the solenoid. A more detailed description of the CMS detector, together with a definition of the coordinate system used and the relevant kinematic variables, can be found in Ref. [20].

Muons are measured in the pseudorapidity range $|\eta |<2.4$, with detection planes made using three technologies: drift tubes, cathode strip chambers, and resistive-plate chambers. Matching muons to tracks measured in the silicon tracker results in a transverse momentum ($p_{\text{T}}$) resolution for muons with $p_{\text{T}}$ up to 100$\text{Ge}\text{V}$ of 1% in the barrel and 3% in the endcaps. The silicon tracker used in 2016 measured charged particles within the range $|\eta |<2.5$. For nonisolated particles of $1<p_{\text{T}}<10\text{Ge}\text{V}$ and $|\eta |<1.4$, the track resolutions were typically 1.5% in $p_{\text{T}}$ and 25–90 $\mu \text{m}$ in the transverse impact parameter [21]. At the start of 2017, a new pixel detector was installed [22]; the upgraded tracker measured nonisolated particles of $1<p_{\text{T}}<10\text{Ge}\text{V}$ up to $|\eta |<3$ with typical resolutions of 1.5% in $p_{\text{T}}$ and 20–75$\mu \text{m}$ in the transverse impact parameter [23].

Events of interest are selected using a two-tiered trigger system [24]. The first level, composed of custom hardware processors, uses information from the calorimeters and muon detectors to select events at a rate of around 100 kHz within a fixed latency of about 4$\mathrm{\mu }\text{s}$  [25]. The second level, known as the high-level trigger (HLT), consists of a farm of computing processors running a version of the full event reconstruction software optimized for fast processing, and reduces the event rate to around 1 kHz before data storage. All events used in this analysis are selected by a set of triggers requiring two identified muons of opposite charge plus an additional track to form a secondary vertex, displaced from the region of the pp interactions. The trigger demanded for each muon to have $p_{\text{T}}>4\text{Ge}\text{V}$ and to pass within 2 cm of the beam axis. The dimuon system was required to have $p_{\text{T}}>6.9\text{Ge}\text{V}$, invariant mass between 2.9 and 3.3$\text{Ge}\text{V}$, a vertex fit probability greater than 10%, a separation of the secondary vertex relative to the beam axis in the transverse plane larger than 3 standard deviations (s.d.), and a cosine of the angle in the transverse plane between the dimuon momentum vector and the vector joining the beam axis and the dimuon vertex greater than 0.9. The additional track was required to have $p_{\text{T}}>0.8(1.2)\text{Ge}\text{V}$ and an impact parameter with respect to the beam axis greater than 0 (2) s.d., for data collected in 2016 (2017–2018). Finally, the two muons and the additional track were required to originate from the same vertex with a ${\chi }^2$ per degree of freedom (dof) less than 10.

Monte Carlo (MC) simulated event samples are generated with Pythia v8.240 [26] using the CP5 underlying event tune [27]. The evtgen 1.6.0 [28] program models the beauty baryon decays ${\mathrm{\Lambda }}_{\text{b}}^0\to \text{J}/\mathrm{\psi }{\mathrm{\Xi }}^{-}{\text{K}}^{+}$ and ${\mathrm{\Lambda }}_{\text{b}}^0\to \mathrm{\psi }(2\text{S})\mathrm{\Lambda }$ with a phase space decay model, followed by the $\mathrm{\psi }(2\text{S})\to \text{J}/\mathrm{\psi }{\mathrm{\pi }}^{+}{\mathrm{\pi }}^{-}$ and $\text{J}/\mathrm{\psi }\to {\mathrm{\mu }}^{+}{\mathrm{\mu }}^{-}$ decays. Final-state radiation is included in evtgen using photos 3.61 [29]. The events are then passed through a detailed Geant4-based simulation [30] of the CMS detector, including also the decays of long-lived hyperons ${\mathrm{\Xi }}^{-}\to \mathrm{\Lambda }{\mathrm{\pi }}^{-}$ and $\mathrm{\Lambda }\to \text{p}{\mathrm{\pi }}^{-}$, followed by the trigger and reconstruction algorithms identical to those used for the collision data. The simulation includes additional interactions due to multiple pp collisions in each bunch crossing, with the same distribution as observed in the experiment.

Event reconstruction and selection

The reconstruction for all the decays considered in this analysis starts by finding two muons of opposite charge, which must match those that triggered the event readout and pass the soft-muon identification criteria [31]. The offline selection for both muons requires $p_{\text{T}}({\mathrm{\mu }}^{\pm })>3\text{Ge}\text{V}$, $|\eta ({\mathrm{\mu }}^{\pm })|<2.4$, ${\chi }^2$ fit probability to a common dimuon vertex $P_{\text{vtx}}({\mathrm{\mu }}^{+}{\mathrm{\mu }}^{-})>1\%$, and dimuon invariant mass $2.9<m({\mathrm{\mu }}^{+}{\mathrm{\mu }}^{-})<3.3\text{Ge}\text{V}$.

The $\mathrm{\Lambda }\to \text{p}{\mathrm{\pi }}^{-}$ candidates are selected from displaced two-prong vertices as described in Ref. [32]. The track with the higher momentum is assumed to be the proton one, and together with the pion track it is fit to a common vertex with their invariant mass constrained to the known $\mathrm{\Lambda }$ hyperon mass of $m_{\text{PDG}}(\mathrm{\Lambda })=1115.683\text{Me}\text{V}$ [33]. The ${\chi }^2$ fit probability for the $\mathrm{\Lambda }$ vertex is required to be $P_{\text{vtx}}(\text{p}{\mathrm{\pi }}^{-})>1\%$.

For the signal channel, to form the ${\mathrm{\Xi }}^{-}\to \mathrm{\Lambda }{\mathrm{\pi }}^{-}$ candidates, an additional high-purity [21] track assumed to be a pion is selected with $p_{\text{T}}>0.2\text{Ge}\text{V}$. This track and the selected $\mathrm{\Lambda }$ candidate are then fit to a common vertex with the $\mathrm{\Lambda }{\mathrm{\pi }}^{-}$ mass constrained to the known ${\mathrm{\Xi }}^{-}$ hyperon mass of $m_{\text{PDG}}({\mathrm{\Xi }}^{-})=1321.71$ $\text{Me}\text{V}$  [33]. To form the ${\mathrm{\Lambda }}_{\text{b}}^0\to \text{J}/\mathrm{\psi }{\mathrm{\Xi }}^{-}{\text{K}}^{+}$ candidate, a high-purity track is chosen with an assigned kaon mass and $p_{\text{T}}({\text{K}}^{+})>1.2\text{Ge}\text{V}$, which aligns with the HLT $p_{\text{T}}$ requirement. The final reconstruction step in the signal channel is the ${\mathrm{\mu }}^{+}{\mathrm{\mu }}^{-}{\mathrm{\Xi }}^{-}{\text{K}}^{+}$ vertex fit with a ${\chi }^2$ probability above 1%, where the dimuon mass is constrained to the world-average $\text{J}/\mathrm{\psi }$ meson mass of $3096.9\text{Me}\text{V}$ [33].

For the normalization channel, two high-purity tracks of opposite charges with $p_{\text{T}}>0.4\text{Ge}\text{V}$, assumed to be pions from the $\mathrm{\psi }(2\text{S})\to \text{J}/\mathrm{\psi }{\mathrm{\pi }}^{+}{\mathrm{\pi }}^{-}$ decay, are selected. One of them is required to have $p_{\text{T}}>1.2\text{Ge}\text{V}$ to match the HLT $p_{\text{T}}$ requirement. The ${\mathrm{\Lambda }}_{\text{b}}^0$ candidates are obtained by a vertex fit of the ${\mathrm{\mu }}^{+}{\mathrm{\mu }}^{-}{\mathrm{\pi }}^{+}{\mathrm{\pi }}^{-}\mathrm{\Lambda }$ system with a $\text{J}/\mathrm{\psi }\to {\mathrm{\mu }}^{+}{\mathrm{\mu }}^{-}$ mass constraint, as for the signal channel. The invariant mass of the $\text{J}/\mathrm{\psi }{\mathrm{\pi }}^{+}{\mathrm{\pi }}^{-}$ candidates is required to be in the range $3.60<m(\text{J}/\mathrm{\psi }{\mathrm{\pi }}^{+}{\mathrm{\pi }}^{-})<3.95\text{Ge}\text{V}$.

From all reconstructed pp collision points in each event, the primary vertex (PV) is chosen as the one with the smallest ${\mathrm{\Lambda }}_{\text{b}}^0$ pointing angle, which is the angle between the momentum of the ${\mathrm{\Lambda }}_{\text{b}}^0$ candidate and the vector from the PV to the reconstructed ${\mathrm{\Lambda }}_{\text{b}}^0$ candidate vertex. If any of the tracks used in the ${\mathrm{\Lambda }}_{\text{b}}^0$ candidate reconstruction were included in the fit of the chosen PV, they are removed, and the PV is refitted.

Selection criteria for the signal channel ${\mathrm{\Lambda }}_{\text{b}}^0\to \text{J}/\mathrm{\psi }{\mathrm{\Xi }}^{-}{\text{K}}^{+}$ are optimized using the Punzi figure of merit [34]. The signal efficiency is evaluated using simulated event samples. Estimation of the background yield involves combining the collision data from the ${\mathrm{\Lambda }}_{\text{b}}^0$ mass sideband, excluding the signal region which spans twice the mass resolution around the known ${\mathrm{\Lambda }}_{\text{b}}^0$ mass. Additionally, the wrong-sign candidates ($\text{J}/\mathrm{\psi }{\mathrm{\Xi }}^{-}{\text{K}}^{-}$ and $\text{J}/\mathrm{\psi }{\overline{\mathrm{\Xi }}}^{+}{\text{K}}^{+}$) from the full mass range are included, after ensuring that the mass distribution of the wrong-sign candidates matches that of the correct-sign ones. Combining these two background sources reduces the impact of the statistical uncertainty in the optimization procedure. The variables used in the optimization include the $p_{\text{T}}$ of all decay products; the flight length significance in the transverse plane of the ${\mathrm{\Lambda }}_{\text{b}}^0$, $\mathrm{\Lambda }$, and ${\mathrm{\Xi }}^{-}$ baryon candidates and the corresponding pointing angles; the impact parameter significance with respect to the PV in the transverse plane for the tracks; the vertex fit probabilities; and the mass windows for hyperon candidates. The order of the cuts is determined randomly, and in several rounds of optimization this order was different each time; all rounds have converged to the same final set of optimized cuts. The resulting criteria are summarized in Table 1. The background is reduced by a factor of 15 after the optimization, whereas the signal efficiency is 70% of the initial selection described above. The selection criteria in the normalization channel ${\mathrm{\Lambda }}_{\text{b}}^0\to \mathrm{\psi }(2\text{S})\mathrm{\Lambda }$ are chosen to be the same, wherever possible, as in the signal channel, to reduce the systematic uncertainties. The $\text{J}/\mathrm{\psi }{\mathrm{\pi }}^{+}{\mathrm{\pi }}^{-}$ mass is required to be within 11.1$\text{Me}\text{V}$ of the known $\mathrm{\psi }(2\text{S})$ meson mass of $3686.1\text{Me}\text{V}$ [33], which corresponds to approximately 2.5 times the mass resolution.

Table 1.

Optimized selection criteria for the signal decay mode ${\mathrm{\Lambda }}_{\text{b}}^0\to \text{J}/\mathrm{\psi }{\mathrm{\Xi }}^{-}{\text{K}}^{+}$. The first two requirements are applied using the momenta before the corresponding mass constraint

Variable	Selection
$|m(\text{p}{\mathrm{\pi }}^{-})-m_{\text{PDG}}(\mathrm{\Lambda })|$	<8 MeV
$|m(\mathrm{\Lambda }{\mathrm{\pi }}^{-})-m_{\text{PDG}}({\mathrm{\Xi }}^{-})|$	<6 MeV
$p_{\text{T}}({\mathrm{\Lambda }}_{\text{b}}^0)$	>11.5 GeV
$p_{\text{T}}(\text{J}/\mathrm{\psi })$	>6.5 GeV
$p_{\text{T}}({\mathrm{\Xi }}^{-})$	>2.6 GeV
$p_{\text{T}}(\mathrm{\Lambda })$	>2.2 GeV
$p_{\text{T}}({\text{K}}^{+})$	>1.2 GeV
${\mathrm{\mu }}^{+}{\mathrm{\mu }}^{-}{\mathrm{\Xi }}^{-}{\text{K}}^{+}$ vertex fit probability	>5%
$\mathrm{\Lambda }{\mathrm{\pi }}^{-}$ vertex fit probability	>5%
$\text{p}{\mathrm{\pi }}^{-}$ vertex fit probability	>1%
${\mathrm{\Xi }}^{-}$ vertex displacement from ${\mathrm{\Lambda }}_{\text{b}}^0$ vertex	>3 s.d.
$\mathrm{\Lambda }$ vertex displacement from ${\mathrm{\Xi }}^{-}$ vertex	>0 s.d.
${\mathrm{\Lambda }}_{\text{b}}^0$ vertex displacement from PV	>3 s.d.
Angle between ${\mathrm{\Xi }}^{-}$ momentum and displacement	<0.0447 rad
Angle between $\mathrm{\Lambda }$ momentum and displacement	<0.14 rad
Angle between ${\mathrm{\Lambda }}_{\text{b}}^0$ momentum and displacement	<0.0447 rad
PV impact parameter for pion from ${\mathrm{\Xi }}^{-}$ decay	>0.4 s.d.
PV impact parameter for kaon	>0.4 s.d.

For the measurement of $\mathcal{R}$ defined in Eq. (1), the pion from the ${\mathrm{\Xi }}^{-}$ decay is required to have $p_{\text{T}}>0.4\text{Ge}\text{V}$. Additionally, the HLT requirements are repeated offline by requiring $p_{\text{T}}(\mu )>4\text{Ge}\text{V}$, $p_{\text{T}}(\text{J}/\mathrm{\psi })>6.9\text{Ge}\text{V}$, $P_{\text{vtx}}({\mathrm{\mu }}^{+}{\mathrm{\mu }}^{-})>5\%$, and track (kaon for the signal channel, the harder of the two pions in the normalization channel) impact parameter above 2 s.d. with respect to the PV. These extra criteria ensure that events from potentially inadequately modeled phase space regions are avoided, as the reliability of the efficiency evaluation from simulated samples in those regions is questionable. Nevertheless, the reconstruction algorithm works reliably in those regions, and thus the corresponding events are used to study the mass distribution, as discussed in the following section.

In less than 5% of the events, multiple ${\mathrm{\Lambda }}_{\text{b}}^0$ candidates in the same channel are found. The rate is consistent in both channels and all candidates are used in the analysis. Selecting a single candidate has a negligible effect on the results.

Invariant mass distributions

The measured mass distribution of the $\mathrm{\psi }(2\text{S})\mathrm{\Lambda }$ candidates is shown in Fig. 1 (left) together with the results of an unbinned maximum likelihood fit. The signal is modeled with a Student’s t-distribution [35] with all parameters (mean, $\sigma$, n) free. The combinatorial background is described by an exponential function with a free slope parameter and normalization. The fitted mass of $5619.3\pm 0.3\text{Me}\text{V}$ is in agreement with the world-average ${\mathrm{\Lambda }}_{\text{b}}^0$ mass of $5619.60\pm 0.17\text{Me}\text{V}$ [33], and the mass resolution of $8.90\pm 0.40\text{Me}\text{V}$ is slightly larger than, yet in agreement with, its value of 8.52$\text{Me}\text{V}$ found in simulation. The measured yield is $N({\mathrm{\Lambda }}_{\text{b}}^0\to \mathrm{\psi }(2\text{S})\mathrm{\Lambda })=1744\pm 63$. The ${\chi }^2$ between the binned distribution and the fit function is 76.6 for 94 degrees of freedom, demonstrating the good quality of the fit.

Fig. 1.

Measured $\mathrm{\psi }(2\text{S})\mathrm{\Lambda }$ (upper) and $\text{J}/\mathrm{\psi }{\mathrm{\Xi }}^{-}{\text{K}}^{+}$ (lower) invariant mass distributions and overlaid fit results

The measured invariant mass distribution of the selected $\text{J}/\mathrm{\psi }{\mathrm{\Xi }}^{-}{\text{K}}^{+}$ candidates is shown in Fig. 1 (lower). A narrow peak at the ${\mathrm{\Lambda }}_{\text{b}}^0$ mass is seen on top of a smooth background. The ${\mathrm{\Lambda }}_{\text{b}}^0$ signal is modeled with a Student’s t-distribution with mean and $\sigma$ floating, but the n parameter fixed to the value found by fitting the simulated distribution, because of the limited signal yield of $N({\mathrm{\Lambda }}_{\text{b}}^0\to \text{J}/\mathrm{\psi }{\mathrm{\Xi }}^{-}{\text{K}}^{+})=46\pm 11$. The background is fitted with an exponential function. The ${\mathrm{\Lambda }}_{\text{b}}^0$ mass returned by the fit ($5625.9\pm 3.2\text{Me}\text{V}$) is within 2 s.d. of the world-average value [33]. The width of the signal peak ($\sigma$) is found to be $10.4\pm 3.3\text{Me}\text{V}$, consistent within 1.2 s.d. with the value found in simulation, $6.6\pm 0.2\text{Me}\text{V}$. The fit quality is good, as demonstrated by the ${\chi }^2/\text{dof}=30.1/45$ for the binned distribution.

The signal significance is evaluated using the likelihood ratio technique by applying the background-only and signal-plus-background hypotheses. In these two fits, a Gaussian constraint is applied on the background shape parameter to the one obtained from a fit to the wrong-sign data. Similarly, a Gaussian constraint is applied to the signal shape parameter n (from simulation) and the resolution $\sigma ={\sigma }_{\text{MC}}(8.90/8.52)$. The correction factor is extracted from the normalization channel and accounts for the difference in the widths of the peak between the measured and simulated event samples. The mean value of the peak is also Gaussian-constrained with a central value and uncertainty equal to the known ${\mathrm{\Lambda }}_{\text{b}}^0$ mass and its uncertainty [33], respectively. The fit with the signal-plus-background model with these constraints returns a signal yield of $36\pm 8$ and is presented in Appendix A. Since the conditions of Wilks’ theorem [36] are satisfied, the asymptotic formulae of Ref. [37] (Eqs. (12) and (52)) are used to determine the ${\mathrm{\Lambda }}_{\text{b}}^0\to \text{J}/\mathrm{\psi }{\mathrm{\Xi }}^{-}{\text{K}}^{+}$ signal significance, which is found to be 5.8 standard deviations. To evaluate the effect of the choice of the model for fitting the signal significance, several alternative models of signal and background were tested, including double-Gaussian or Johnson [38] functions for the signal and a second-degree polynomial or a modified threshold function for the background. An alternative without a constraint on the background shape was also tested. The significance obtained with the alternative models varies in the range from 5.3 to 5.9 standard deviations. This allows us to claim the first observation of the ${\mathrm{\Lambda }}_{\text{b}}^0\to \text{J}/\mathrm{\psi }{\mathrm{\Xi }}^{-}{\text{K}}^{+}$ decay.

The sensitivity of this analysis to potential pentaquark signals in the intermediate invariant mass distributions of the ${\mathrm{\Lambda }}_{\text{b}}^0\to \text{J}/\mathrm{\psi }{\mathrm{\Xi }}^{-}{\text{K}}^{+}$ decay is limited by the low signal yield. The background-subtracted two-body invariant mass distributions, obtained with the $_{\mathrm{s}}^{}\mathcal{P}\text{lot}$ technique [39], are shown in Fig. 2. The distributions do not show any clear peaks and agree, within uncertainties, with the predictions from the phase space simulation. The distributions are also consistent with the results of extracting the yields by fitting the ${\mathrm{\Lambda }}_{\text{b}}^0$ signal in each of the five intermediate invariant mass bins.

Fig. 2.

Intermediate invariant mass distributions of the ${\mathrm{\Lambda }}_{\text{b}}^0\to \text{J}/\mathrm{\psi }{\mathrm{\Xi }}^{-}{\text{K}}^{+}$ decay. The filled circles and empty squares show the measured background-subtracted distributions and the results from the simulation with a phase-space model, respectively

For the measurement of $\mathcal{R}$ (Eq. (1)), more stringent requirements are used, as explained at the end of Sect. 3, and the measured signal yields decrease to $1179\pm 47$ and $23\pm 7$ for the ${\mathrm{\Lambda }}_{\text{b}}^0\to \mathrm{\psi }(2\text{S})\mathrm{\Lambda }$ and ${\mathrm{\Lambda }}_{\text{b}}^0\to \text{J}/\mathrm{\psi }{\mathrm{\Xi }}^{-}{\text{K}}^{+}$ channels, respectively, using unconstrained fits as for Fig. 1. These are the baseline results referred to in Sect. 6. The corresponding mass distributions and fits are presented in Fig. 3.

Fig. 3.

Measured $\mathrm{\psi }(2\text{S})\mathrm{\Lambda }$ (upper) and $\text{J}/\mathrm{\psi }{\mathrm{\Xi }}^{-}{\text{K}}^{+}$ (lower) invariant mass distributions and corresponding fits used for the measurement of $\mathcal{R}$

Efficiencies

Efficiencies for the signal and normalization channels are calculated using simulated event samples. The total efficiency is calculated by factorizing into two components: detector acceptance and a combined trigger, reconstruction, and selection efficiency.

As only the ratio of the total efficiencies is needed to measure $\mathcal{R}$, the systematic uncertainties associated with the muon and track reconstruction are reduced. The obtained efficiency ratio is ${ϵ}_{\mathrm{\psi }(2\text{S})\mathrm{\Lambda }}/{ϵ}_{\text{J}/\mathrm{\psi }{\mathrm{\Xi }}^{-}{\text{K}}^{+}}=5.06\pm 0.29$, where the uncertainty reflects the limited size of the simulated samples. Efficiencies for different years of data-taking are combined with weights corresponding to the integrated luminosity collected in each year. The efficiency for the ${\mathrm{\Lambda }}_{\text{b}}^0\to \text{J}/\mathrm{\psi }{\mathrm{\Xi }}^{-}{\text{K}}^{+}$ channel is significantly lower than that for the ${\mathrm{\Lambda }}_{\text{b}}^0\to \mathrm{\psi }(2\text{S})\mathrm{\Lambda }$ channel for several reasons including the low energy release in the ${\mathrm{\Xi }}^{-}\to \mathrm{\Lambda }{\mathrm{\pi }}^{-}$ decay, resulting in a low-momentum pion track.

Systematic uncertainties

Many systematic uncertainties, related to the muon reconstruction and identification as well as to the trigger efficiency, partially cancel in the measured ratios. Since the signal and normalization channels have the same number of tracks in the final state, most uncertainties related to track reconstruction also cancel in the measured ratio $\mathcal{R}$. However, the $p_{\text{T}}$ spectrum of kaons from the ${\mathrm{\Lambda }}_{\text{b}}^0\to \text{J}/\mathrm{\psi }{\mathrm{\Xi }}^{-}{\text{K}}^{+}$ decay is observed to differ from that of the highest-$p_{\text{T}}$ pion in the ${\mathrm{\Lambda }}_{\text{b}}^0\to \mathrm{\psi }(2\text{S})\mathrm{\Lambda }$ channel used for normalization. Despite the signal and normalization channels having the same number of final-state tracks, an uncertainty of 2.3% [40] is included, which reflects the difference in tracking efficiency between the measured and simulated event samples. The MC event samples are validated using the normalization channel by comparing the measured distributions of variables used in the event selection, after background subtraction, to those found in simulation; no significant discrepancies are found in most of the distributions. A small discrepancy was observed in the $p_{\text{T}}({\mathrm{\Lambda }}_{\text{b}}^0)$ distribution, and the MC event samples for both channels were reweighted using $p_{\text{T}}({\mathrm{\Lambda }}_{\text{b}}^0)$-dependent weights so that the $p_{\text{T}}({\mathrm{\Lambda }}_{\text{b}}^0)$ distribution in the weighted simulation sample matches the background-subtracted distribution measured in the ${\mathrm{\Lambda }}_{\text{b}}^0\to \mathrm{\psi }(2\text{S})\mathrm{\Lambda }$ channel. The efficiency ratio evaluated using these weighted MC samples is found to be ${ϵ}_{\mathrm{\psi }(2\text{S})\mathrm{\Lambda }}/{ϵ}_{\text{J}/\mathrm{\psi }{\mathrm{\Xi }}^{-}{\text{K}}^{+}}=4.82\pm 0.39$, which is lower by 4.7%, yet still in agreement with the value reported in the previous section. An uncertainty of 4.7% is assigned to account for potential mismodeling of the $p_{\text{T}}({\mathrm{\Lambda }}_{\text{b}}^0)$ spectrum.

The systematic uncertainty related to the choice of the signal model is evaluated by testing three different models. For the normalization channel ${\mathrm{\Lambda }}_{\text{b}}^0\to \mathrm{\psi }(2\text{S})\mathrm{\Lambda }$ the signal shape parameters are floating, while for the signal channel ${\mathrm{\Lambda }}_{\text{b}}^0\to \text{J}/\mathrm{\psi }{\mathrm{\Xi }}^{-}{\text{K}}^{+}$ the mass resolution parameters are fixed to those found in simulation, after correcting the width of the peak for the ratio between the two resolutions in the measured and simulated event samples evaluated in the normalization channel. The tested models simultaneously vary the signal and normalization channels and use a Student’s t-distribution, a double-Gaussian, and a Johnson function [38] to model the signal. The largest deviation in the ratio of the ${\mathrm{\Lambda }}_{\text{b}}^0$ signal yields from the baseline value is taken as the systematic uncertainty.

The systematic uncertainty related to the choice of the background model is estimated in a similar way, with three alternative models: a second-degree polynomial, a threshold function [41, 42] multiplied by an exponential, and a threshold function multiplied by a first-degree polynomial.

By requiring the $\text{J}/\mathrm{\psi }{\mathrm{\pi }}^{+}{\mathrm{\pi }}^{-}$ invariant mass to be near the $\mathrm{\psi }(2\text{S})$ mass, we aim to select ${\mathrm{\Lambda }}_{\text{b}}^0\to \mathrm{\psi }(2\text{S})\mathrm{\Lambda }$ decays. Nevertheless, other ${\mathrm{\Lambda }}_{\text{b}}^0\to \text{J}/\mathrm{\psi }{\mathrm{\pi }}^{+}{\mathrm{\pi }}^{-}\mathrm{\Lambda }$ decays, either from different intermediate resonances or from four-body nonresonant decays may contribute. To estimate this contribution, we use the $_{\mathrm{s}}^{}\mathcal{P}\text{lot}$ technique to subtract the background under the ${\mathrm{\Lambda }}_{\text{b}}^0$ peak and plot the $\text{J}/\mathrm{\psi }{\mathrm{\pi }}^{+}{\mathrm{\pi }}^{-}$ mass in an expanded mass region. The $\text{J}/\mathrm{\psi }{\mathrm{\pi }}^{+}{\mathrm{\pi }}^{-}$ mass is fitted with a signal component for the $\mathrm{\psi }(2\text{S})$ and a background component for everything else. The integral of the background over the range used to select $\mathrm{\psi }(2\text{S})$ events yields 30 events, which is 2.5% of the total yield (1179 events). This value is used as the systematic uncertainty related to non-$\mathrm{\psi }(2\text{S})$ contributions in the normalization channel.

The uncertainty in the efficiency ratio due to the limited size of the simulated samples, calculated to be 5.6% in Sect. 5, is considered as a systematic uncertainty.

In order to assess the reliability of the efficiency evaluation from the simulated samples, the selection criteria on muon and $\text{J}/\mathrm{\psi }$ $p_{\text{T}}$, dimuon vertex probability, track impact parameter, and $p_{\text{T}}$ of the soft pion from ${\mathrm{\Xi }}^{-}$ decay are tightened, one at a time, until the signal efficiency decreases by 10 or 20% with respect to that obtained with the selection used for the $\mathcal{R}$ measurement. The analysis is repeated each time, and the value of $\mathcal{R}$ is re-calculated and compared to the baseline $\mathcal{R}$ value. The differences (d) between the two values and their uncertainties ($\delta d$), which also account for the correlation between the two values, are evaluated. The largest value of $\sqrt{d^2-{(\delta d)}^2}$ among the different variations of the selection criteria is found to be 14.3% and is used as the systematic uncertainty in the efficiency ratio.

Table 2 summarizes the previously discussed systematic uncertainties in the ratio $\mathcal{R}$. The total uncertainty is calculated as the sum in quadrature of the individual sources.

Table 2.

The relative systematic uncertainties in the measurement of $\mathcal{R}$

Source	Uncertainty (%)
Tracking efficiency	2.3
$p_{\text{T}}({\mathrm{\Lambda }}_{\text{b}}^0)$ spectrum	4.7
Signal model	3.9
Background model	6.7
Non-$\mathrm{\psi }(2\text{S})$ contribution	2.5
Limited size of MC samples	5.6
Selection efficiency	14.3
Total	18.2

Branching fraction ratio measurement

The branching fraction of the newly observed ${\mathrm{\Lambda }}_{\text{b}}^0\to \text{J}/\mathrm{\psi }{\mathrm{\Xi }}^{-}{\text{K}}^{+}$ decay, with respect to the ${\mathrm{\Lambda }}_{\text{b}}^0\to \mathrm{\psi }(2\text{S})\mathrm{\Lambda }$ one, is measured using Eq. (1) to be

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \mathcal{R}& \equiv \frac{\mathcal{B}({\mathrm{\Lambda }}_{\text{b}}^0\to \text{J}/\mathrm{\psi }{\mathrm{\Xi }}^{-}{\text{K}}^{+})}{\mathcal{B}({\mathrm{\Lambda }}_{\text{b}}^0\to \mathrm{\psi }(2\text{S})\mathrm{\Lambda })}\hfill \\ \hfill & =[3.38\pm 1.02\text{(stat)}\pm 0.61\text{(syst)}\pm 0.03(\mathcal{B})]\%,\hfill \end{array}\end{array}$$

where the last uncertainty is related to the uncertainties in the branching fractions $\mathcal{B}(\mathrm{\psi }(2\text{S})\to \text{J}/\mathrm{\psi }{\mathrm{\pi }}^{+}{\mathrm{\pi }}^{-})=34.68\pm 0.30\%$ and $\mathcal{B}({\mathrm{\Xi }}^{-}\to \mathrm{\Lambda }{\mathrm{\pi }}^{-})=99.887\pm 0.035\%$ [33].

Summary

The ${\mathrm{\Lambda }}_{\text{b}}^0\to \text{J}/\mathrm{\psi }{\mathrm{\Xi }}^{-}{\text{K}}^{+}$ decay is observed with a significance exceeding 5 standard deviations using $\sqrt{s}=13\text{Te}\text{V}$ proton–proton collision data corresponding to an integrated luminosity of $140{\text{fb}}^{-1}$ collected by the CMS experiment. The branching fraction is measured with respect to the ${\mathrm{\Lambda }}_{\text{b}}^0\to \mathrm{\psi }(2\text{S})\mathrm{\Lambda }$ decay to be $\mathcal{B}({\mathrm{\Lambda }}_{\text{b}}^0\to \text{J}/\mathrm{\psi }{\mathrm{\Xi }}^{-}{\text{K}}^{+})/\mathcal{B}({\mathrm{\Lambda }}_{\text{b}}^0\to \mathrm{\psi }(2\text{S})\mathrm{\Lambda })=[3.38\pm 1.02\text{(stat)}\pm 0.61\text{(syst)}\pm 0.03(\mathcal{B})]\%$. The distributions of intermediate invariant masses $m(\text{J}/\mathrm{\psi }{\mathrm{\Xi }}^{-})$, $m(\text{J}/\mathrm{\psi }{\text{K}}^{+})$, and $m({\mathrm{\Xi }}^{-}{\text{K}}^{+})$ from the ${\mathrm{\Lambda }}_{\text{b}}^0\to \text{J}/\mathrm{\psi }{\mathrm{\Xi }}^{-}{\text{K}}^{+}$ decay are also presented. This is the first discovered multibody decay containing the $\text{J}/\mathrm{\psi }{\mathrm{\Xi }}^{-}$ system, which opens the possibility to search for doubly-strange hidden-charm pentaquarks when more data are collected. The new results are important for understanding the strong interaction processes in hadronic decays of beauty baryons and the possible formation of exotic multiquark states.

Invariant mass distribution fit with constraints

The fit of the measured $\text{J}/\mathrm{\psi }{\mathrm{\Xi }}^{-}{\text{K}}^{+}$ invariant mass distribution with constraints on the background shape parameter, signal shape parameter, resolution, and the mean value of the peak is presented in Fig. 4. The fit quality is good, as demonstrated by ${\chi }^2/\text{dof}=35.6/44$ for the binned distribution.

Fig. 4.

Measured $\text{J}/\mathrm{\psi }{\mathrm{\Xi }}^{-}{\text{K}}^{+}$ invariant mass distribution and overlaid constrained fit result

Data Availability Statement

This manuscript has associated data in a data repository. [Author’s comment: Some data used for this analysis may be made available. Almost certainly not all data will be released near term.]

Code Availability Statement

Code/software cannot be made available for reasons disclosed in the code availability statement. [Author’s comment: Some software (the core software) is available. Not all code used is available.]

Declarations

Conflict of interest

The authors declare that they have no Conflict of interest.