Track fitting at the full LHC collision rate

Abstract

The LHCb experiment at the Large Hadron Collider underwent a major upgrade before the LHC Run 3 data-taking period, employing an all-software approach in its trigger system. Here we present a fast implementation of a Kalman filter, used in the first trigger stage since the 2025 data-taking period, allowing to determine parameter estimates of charged-particle trajectories at the full LHCb collision frequency of 30 MHz. This approach replaces computationally expensive magnetic field map lookups and numerical integration methods with fast analytical parameterisations while maintaining the mathematical framework of Kalman filtering. Implemented on approximately 500 GPUs within the first-level trigger, the algorithm has replaced the previous partial track fitting algorithm in the real-time trigger environment at the cost of a 2% increase in processing time. Compared to the previous fitter, this parameterised Kalman filter shows a significantly improved momentum resolution, resulting in a factor of two improvement in the invariant mass resolutions for reconstructed $D^0$ and $J/\psi$ hadrons. It additionally demonstrates greater robustness against detector misalignment effects and substantially sharpens the discrimination between genuine particle trajectories and accidental background, more than doubling the rejection of the latter at no cost to genuine-track efficiency, for a standard selection working point.

Introduction

The LHCb experiment [1–3] is one of the four big experiments at the Large Hadron Collider (LHC) at CERN. The original experiment and its upgrade are designed as general purpose detectors covering the forward direction. While LHCb’s primary focus is heavy-flavour physics, its physics program also includes measurements from electroweak to heavy-ion physics, as well as a fixed-target program [4].

The upgraded LHCb detector [2] enables data collection at an instantaneous luminosity of $2\times {10}^{33}{\text{,cm}}^{-2}{sec}^{-1}$, a fivefold increase compared to the instantaneous luminosity during the previous data-taking period from 2015 to 2018. For the upgrade, significant parts of the detector have been replaced or modified to be read out at the nominal LHC bunch-crossing frequency of 40 MHz: The tracking system has been fully replaced and consists of a silicon-pixel vertex detector (VELO) surrounding the pp interaction region [5], a large-area silicon-strip detector (UT) [6], located upstream of a dipole magnet with a bending power of about 4 Tm, and three stations of scintillating fibre detectors (SciFi) placed downstream of the magnet [6]. Charged hadrons are identified using information from two ring-imaging Cherenkov (RICH) detectors [7, 8]. In order to fully profit from the increased instantaneous luminosity, a novel all-software trigger system is implemented to process, in its first stage, the full detector readout at the LHCb bunch-crossing frequency of 30 MHz1 with a data rate of 4$\text{,TB/s}$. This real-time strategy significantly increases the yield of hadronic and electron final states beyond the luminosity increase by circumventing the limitation of the previous hardware-based trigger system, which exploited calorimeter deposits for hadrons and relied on high-$p_{\text{T}}$ signatures. The trigger consists of two stages: HLT1 is implemented on GPUs and performs a partial reconstruction with inclusive and exclusive kinematic and geometric selections based on approximately 50 different signatures, reducing the 30 MHz input rate to an output rate of about $1.2\text{MHz}$ in 2025 [2, 9, 10]. The events selected by HLT1 are stored on a large disk buffer, where they are used to perform real-time detector alignment and calibration, before being processed by the second trigger stage, HLT2. HLT2 is implemented on a farm of CPUs and processes the buffered output of HLT1, performing the full offline-quality reconstruction. HLT2 employs several thousand trigger selections to select a variety of physics processes, which are then either directly analysed, or passed on to a further offline selection step. This approach has enabled LHCb to record an integrated luminosity larger than 25${\text{,fb}}^{-1}$ at $\sqrt{s}=13.6\text{TeV}$ from 2024 to 2026, more than doubling the dataset collected during Run 1 and Run 2 of the LHC.

Central to the reconstruction in HLT1 is the reconstruction of charged particle trajectories (tracks) and displaced decay vertices, which are key signatures for selecting events containing heavy-flavour hadron decays. HLT1 is implemented using approximately 500 GPUs [11], within the Allen [12] software project, achieving the required computational throughput for real-time processing at 30 MHz. Accurate determination of track parameters using a track fit is important for the HLT1 selection performance. Up to the 2024 data-taking campaign, the HLT1 track fitting algorithm performed a simplified fitting procedure by applying a Kalman filter to only the VELO segment of each track. This choice was originally driven by the need to meet the tight throughput constraints. Starting with the 2025 data-taking campaign, a new track fitting algorithm has been added to the reconstruction sequence of charged particles, the Parameterised Kalman Filter, performing a fit using information from all involved tracking detectors.

This paper is organised as follows: After a brief introduction of the LHCb detector and its simulation, the principles of a Kalman Filter are reviewed in Sect. 3, with the specifics in LHCb discussed in Sect. 4. Section 5 describes the implementation on GPUs, and Sect. 6 analyses its performance in a simulated LHCb detector. A conclusion is presented in Sect. 7.

Detector and simulation

The LHCb detector [2] is a single-arm forward spectrometer covering the pseudorapidity range $2<\eta <5$, designed for the study of particles containing b or c quarks. A picture of its general layout can be seen in Fig. 1. Its tracking system consists of the VELO with 52 stations, each covering half the azimuthal acceptance (in the $x-y$ plane). The stations are arrayed such that 99% of all tracks from the region around the interaction point and within the nominal LHCb acceptance cross at least four VELO stations. The UT, a four layer silicon-strip detector, is situated downstream of the VELO in front of the large dipole magnet and is used to provide space point measurements before the volume with a large magnetic field. The SciFi is the main tracking detector downstream of the magnet. It consists of three stations with four layers each, two of them tilted at 5°  to provide spatial information of the y coordinate.2 The dipole magnet creates a magnetic field, dominantly in the y direction, with a peak field strength of about 1$\text{,T}$ and an integrated bending power of 4 Tm. In addition, LHCb also operates two Ring Imaging Cherenkov (RICH) detectors, RICH1 and RICH2. While RICH2 is located behind the SciFi tracker, RICH1 is located between VELO and UT and thus contributes significantly to the material budget to be considered in track fitting. The calorimetry systems and muon chambers, located behind RICH2, are used to identify electrons, muons, photons and hadrons.

Fig. 1.

Layout of the upgraded LHCb detector [2]

In the context of this work, simulation is used to determine the parameters of the Parameterised Kalman Filter and to evaluate its performance. In the simulation, pp collisions are generated using Pythia  [13, 14] with a specific LHCb configuration [15]. Decays of unstable particles are described by EvtGen  [16], in which final-state radiation is generated using Photos  [17]. The interaction of the generated particles with the detector, and its response, are implemented using the Geant4 toolkit [18, 19] as described in Ref. [20]. The studies presented here use simulated events of minimum bias pp-collisions, events containing a $D_{}^{\ast }{(2010)}^{+}\to (D^0\to K^{-}{\pi }^{+}){\pi }^{+}$ decay and events containing a $B_s^0\to (J/\psi \to {\mu }^{+}{\mu }^{-})(\varphi \to K^{+}{K}^{-})$ decay, together representing a large part of the LHCb physics programme in terms of kinematics and topology.

Track fitting with a Kalman filter

Efficiently selecting collisions containing b or c hadron decays requires precise estimates of track parameters, including track momentum and direction. Tracks are reconstructed in two steps: First the detector information is decoded and the resulting charged clusters in detectors with their spatial information, so-called hits, are combined to form particle trajectories using pattern recognition algorithms. Second, a track fit is performed to obtain the best estimate of the properties of the particle from the collection of associated hits. In LHCb this fit is performed using a Kalman filter.

First published in 1960 [21], Kalman filters have now become a commonly used method in high energy physics. The Kalman filter’s iterative nature has proven invaluable for reconstructing particle trajectories and reconstructing vertices from detector measurements, where the inherent measurement uncertainties and complex detector geometries present similar estimation challenges to those found in traditional non-particle physics applications [22, 23]. Kalman filters have also been used to fit the decay chains of heavy particles in the BaBar [24] and LHCb experiments, producing accurate uncertainties and correlations between quantities of interest.

Kalman filtering is an iterative process by which predictions from a physics model, here a charged particle moving through the detector and the magnetic field, are combined with measurement information from each detector layer to derive estimates of track parameters. The track information at a given position is called a state. LHCb is a forward detector, meaning all particles in the acceptance of the detector move in the positive z-direction. This simplifies the state description, as trajectories can be described as a function of the z-coordinate rather than the lifetime of the particle, and each detector element is roughly positioned at a fixed z-position. For LHCb a state is thus defined by the state vector ${\overrightarrow{x}}^{⊺}=(x,y,t_x,t_y,\frac{q}{p})$ and its covariance matrix P at given z positions, usually at the position of a detector element. The slopes $t_x=\frac{\mathit{dx}}{\mathit{dz}}$ and $t_y=\frac{\mathit{dy}}{\mathit{dz}}$ are given with respect to the z axis and $\frac{q}{p}$ is the particle charge divided by the magnitude of its momentum, which ensures a near-gaussian distribution for this quantity.3

A Kalman filter needs to be initialised with an estimate of the starting state and its covariance. Filtering is then a repetition of prediction and update steps. In a prediction step, the state ${\overrightarrow{x}}_{k-1|k-1}$ and covariance $P_{k-1|k-1}$ in the layer $k-1$ are extrapolated to the layer k using a function ${\overrightarrow{f}}_k$ and its Jacobian $F_k^{\mathit{ij}}=\frac{\partial f_k^i}{\partial x_k}$:

$$\begin{array}{cc}\hfill {\overrightarrow{x}}_{k|k-1}& ={\overrightarrow{f}}_k({\overrightarrow{x}}_{k-1|k-1}),\hfill \end{array}$$ 1

$$\begin{array}{cc}\hfill P_{k|k-1}& =F_k{P}_{k-1|k-1}{F}_k^{⊺}+Q_k,\hfill \end{array}$$ 2

where the noise matrix $Q_k$ models the expected uncertainty due to multiple scattering of the charged particles in the material of the detector. The predicted state ${\overrightarrow{x}}_{k|k-1}$ is then updated using the measurement ${\overrightarrow{m}}_k$ in the detector layer k to obtain the updated state ${\overrightarrow{x}}_{k|k}$ and covariance $P_{k|k}$. First, a residual ${\overrightarrow{r}}_k$ is obtained by projecting the predicted state onto the measurement space with the matrix $H_k$, encoding the relevant geometry:

$$\begin{array}{cc}\hfill {\overrightarrow{r}}_k& ={\overrightarrow{m}}_k-H_k{\overrightarrow{x}}_{k|k-1}.\hfill \end{array}$$ 3

Then a covariance $S_k$ of the residual ${\overrightarrow{r}}_k$ is calculated by projecting the state uncertainty $P_{k|k-1}$ onto the measurement space and adding the measurement covariance $R_k$:

$$\begin{array}{cc}\hfill S_k& =H_k{P}_{k|k-1}{H}_k^{⊺}+R_k.\hfill \end{array}$$ 4

The Kalman gain $K_k$ can be calculated from the state covariance $P_{k|k-1}$ and the covariance of the residual $S_k$ and describes how much the predicted state should be updated based on the residual:

$$\begin{array}{cc}\hfill K_k& =P_{k|k-1}{H}_k^{⊺}{S}_k^{-1}.\hfill \end{array}$$ 5

From the Kalman gain $K_k$ and the residual vector ${\overrightarrow{r}}_k$ the updated values ${\overrightarrow{x}}_{k|k}$ and $P_{k|k}$ can be calculated:

$$\begin{array}{cc}\hfill {\overrightarrow{x}}_k& ={\overrightarrow{x}}_{k|k-1}+K_k{\overrightarrow{r}}_k,\hfill \end{array}$$ 6

$$\begin{array}{cc}\hfill P_{k|k}& =(\mathbb{1}-K_k{H}_k)P_{k|k-1}.\hfill \end{array}$$ 7

The ${\chi }^2$ value of an update step can be calculated as:

$$\begin{array}{cc}\hfill {\chi }_k^2& ={\overrightarrow{r}}_k^{⊺}{S}_k^{-1}{\overrightarrow{r}}_k.\hfill \end{array}$$ 8

The reduced ${\chi }_{\text{red}}^2={\sum }_i^n{\chi }_i^2/\text{ndof}$, i.e. the sum of all ${\chi }^2$ values divided by the degrees of freedom (ndof) of the track fit, can be used as a measure of the goodness-of-fit.

Using this procedure only the last state has the best parameter estimates. Often, however, Kalman filters are used as smoothers, which enables the algorithm to provide state estimates also incorporating information downstream of the position of interest. If it is of interest to achieve a precise state measurement ${\overrightarrow{x}}_k$, where k is smaller than the total number of hits, n, an additional backwards pass can be performed. Starting at layer n going down to zero, the propagation and filtering process is repeated. If the state of interest and its covariance is saved both in forwards and backwards pass, they can then be combined to a smooth state ${\overrightarrow{x}}_{k|n}$, incorporating the information of all measurements. In real applications it can be necessary to repeat forward and backward passes for numerical stability.

The Parameterised Kalman Filter

Owing to the parallel nature of track fitting and the parallelisation capabilities of modern computing architectures, several studies have been performed with Kalman filter-based track fitting, see e.g. Refs. [25, 26]. The implementation discussed in the following however is the first full Kalman filter that was used in a production environment of an LHC experiment at the full collision rate.

The Parameterised Kalman Filter is a Kalman filter as defined in the previous section, but uses simplifications specific to typical implementations in a high-energy physics context which allows it to be executed with minimal time consumption. It was developed as a drop-in replacement for the previous track fitting algorithm, the VELO Only Kalman Filter, to provide better momentum (and therefore mass) resolution, more robustness against detector misalignments, and improved rejection against fake tracks (so-called ghosts) in the HLT1 stage of LHCb. The VELO Only Kalman Filter  performs a simplified Kalman filter track fit of only the track segment in the VELO detector under the assumption of a vanishing magnetic field, full decoupling of the x and y components and a parameterised scattering description. These simplifications were originally made to stay within the given throughput constraints. The Parameterised Kalman Filter extends the track fitting to all three tracking detectors: the VELO, UT and SciFi. First described as a fast alternative to a full Kalman filter [27], and prototyped on CPUs, this approach has proven to be well suited for the GPU-based HLT1 and provides improved state estimates while still fulfilling the real-time requirements.

As a Kalman filter performs extrapolations of particle trajectories between layers, the standard implementation of such an algorithm includes frequent look-ups of a magnetic field map and the detector geometry, as well as intensive calculations to predict the movement of a charged particle in the magnetic field by numerically solving the equations of motion. The cost of such an implementation is substantial, in the CPU-based HLT2 reconstruction, where a full Kalman filter is employed, the track fit accounts for approximately 14% of the total processing time. This approach is not suited for HLT1 as the field map and geometry exceed the GPU’s limited fast memory, while the position-dependent lookups and data-dependent integration steps produce irregular, non-coalesced memory accesses and control-flow divergence across threads. Almost all throughput improvements in the Parameterised Kalman Filter, compared to a standard implementation as described above, are achieved in changes to the state prediction step. Instead of extrapolating the state using numerical methods to solve the propagation of a charged particle in a magnetic field, a simple function is chosen to parameterise this extrapolation. The same is true for estimating multiple scattering contributions to the covariance matrix of the state, where simulation-derived parameterisations for each extrapolation steps are used. Different parameterisations are employed for each subdetector and for the steps between the subdetectors, with different sets of parameters for each layer, where the different propagation parameterisations are mainly driven by the strength of the magnetic field in the respective regions, see Fig. 2.

Fig. 2.

Measured magnetic field strength in the y direction inside the LHCb detector as a function of z[28]. Overlaid are the positions of the three tracking detectors

While the full parameterisation is explained in Ref. [27] and this implementation is largely using the same formulae, one instructive example is given here.

The propagation of state ${\overrightarrow{x}}_{k-1|k-1}$ in the layer $k-1$ to the layer k is given by the empirical vector-valued function ${\overrightarrow{f}}_{X\to Y}$, where X and Y denote the specific subdetectors of the involved layer, i.e. ${\overrightarrow{x}}_{k|k-1}={\overrightarrow{f}}_{X\to Y}({\overrightarrow{x}}_{k-1|k-1};p^{X\to Y,k-1})$.4 For example, the effect of the magnetic field is already sizeable between the UT layers, with a field strength in the y-component $B_y$ between 0.1$\text{,T}$ and 0.3$\text{,T}$. In this region the extrapolation function takes the form:

$$\begin{array}{c}\hfill {\overrightarrow{f}}_{UT\to UT}(\overrightarrow{x},\mathrm{\Delta }z)=\left(\begin{array}{c}x+\left[p_3,t_x,+,\left(1,-,p_3\right),f^{t_x},(\overrightarrow{x})\right]\mathrm{\Delta }z\\ y+\left[p_5,t_y,+,\left(1,-,p_5\right),f^{t_y},(\overrightarrow{x})\right]\mathrm{\Delta }z\\ t_x+\left[p_0,\frac{q}{p},+,p_1,{\left(\frac{q}{p}\right)}^3,+,p_2,y^2,\frac{q}{p}\right]\mathrm{\Delta }z\\ t_y+p_4\frac{q}{p}{t}_x\text{sign}(y)\\ \frac{q}{p}\end{array}\right)\end{array}$$ 9

Note that the x/y components rely on the predicted slopes, here written with the shorthand $f^{t_{x/y}}(\overrightarrow{x})$ and $\mathrm{\Delta }z$ is the extrapolation distance along the z axis, which is calculated based on the detector geometry.

The parameterisation of noise resulting from multiple scattering is the same for all extrapolation steps. The underlying assumption is that the scattering angle $\theta$ follows a central normal distribution $\mathcal{N}$ centred around 0 with a width inversely proportional to the momentum, i.e. $\theta \sim \mathcal{N}(0,\frac{\alpha }{p})$, where $\alpha$ is the parameter to be extracted from simulation. By extending this idea to two correlated dimensions and propagating the scattering angle to the state observable, the symmetric noise matrix Q can be derived:

$$\begin{array}{c}\hfill \begin{array}{cc}{\sigma }_x={\tilde{p}}_0\left|\mathrm{\Delta },z,\frac{q}{p}\right|,& {\sigma }_{\mathit{tx}}={\tilde{p}}_2\left|\frac{q}{p}\right|,\\ {\sigma }_y={\tilde{p}}_1\left|\mathrm{\Delta },z,\frac{q}{p}\right|,& {\sigma }_{\mathit{ty}}={\tilde{p}}_3\left|\frac{q}{p}\right|,\end{array}\end{array}$$ 10

$$\begin{array}{cc}\hfill \mathit{Q}& =\left(\begin{array}{ccccc}{\sigma }_x^2& 0& {\tilde{p}}_4{\sigma }_x{\sigma }_{t_x}& 0& 0\\ 0& {\sigma }_y^2& 0& {\tilde{p}}_5{\sigma }_y{\sigma }_{t_y}& 0\\ {\tilde{p}}_4{\sigma }_x{\sigma }_{t_x}& 0& {\sigma }_{t_x}^2& 0& 0\\ 0& {\tilde{p}}_5{\sigma }_y{\sigma }_{t_y}& 0& {\sigma }_{t_y}^2& 0\\ 0& 0& 0& 0& 0\end{array}\right).\hfill \end{array}$$ 11

This includes four parameters for the scaling of the scattering with momentum, ${\tilde{p}}_0,{\tilde{p}}_1,{\tilde{p}}_2,{\tilde{p}}_3$, and two parameters for the correlation between slope and position, ${\tilde{p}}_4,{\tilde{p}}_5$. For the scattering in the VELO, the parameters for x and y are assumed to be equal, halving the size of the parameter set. This is a reflection of the weak magnetic field in the extrapolation making a symmetric approximation possible. Outside the VELO this assumption no longer holds, as the magnetic field singles out the x-plane and the scattering is defined including the extrapolation. The fitted parameters for x and y differ by around 10%, showing that the difference cannot be neglected. While multiple scattering is taken into account as such, energy loss is not directly accounted for, i.e. $f_{X\to Y}^{\frac{q}{p}}(\frac{q}{p})=\frac{q}{p}$. The ratio $\frac{q}{p}$ is thus defined as the value at the production of the particle and the energy loss is absorbed into the parameterisation of the propagation of the remaining state components $(x,y,t_x,t_y)$ as well as the scattering matrices Q. The validity of this approach can be seen in the unbiased momentum estimates presented in Sect. 6. To propagate the covariance matrix P alongside the state from one detector position to another, the Jacobians of the extrapolation functions need to be calculated and are given by $F_{\mathit{ij}}=\frac{\partial f_i}{\partial x_j}$. These matrix elements can be calculated analytically as derivatives of the extrapolation functions.

For the Parameterised Kalman Filter to work correctly, the extrapolation functions need to be precise enough such that the parameterisation uncertainty ${\delta }_{\text{par}}=x_{\text{true extrap}}-x_{\text{param. extrap}}$ is smaller than the physical uncertainties of the state. There are two sources of state uncertainty included in the Parameterised Kalman Filter: the contribution from multiple scattering, Q, and the inherent resolution of the tracking detectors, e.g. from the non-zero pixel size. For all particles that did not experience any hard momentum changes, e.g. bremsstrahlung emission, multiple scattering (MS) is the dominant source of uncertainty on the state estimates, with limited detector resolution becoming significant for particles with momenta in excess of 100$\text{,Ge}\text{V!/}c$. The parameterisations therefore need to be accurate to a precision better than the MS contribution. As the asymptotic case for tracks of infinite momentum, i.e. $\frac{q}{p}\to 0$, is no bending in the magnetic field, the parametrisation functions need to recover a linear function in that limit to maintain an asymptotically correct ${\delta }_{\text{par}}$ distribution. Energy loss not described by MS, especially the emission of bremsstrahlung by electrons, is not accounted for in the fit model. The recovery of bremsstrahlungs emission is performed by a separate algorithm based on calorimeter information. The selection algorithms involving electrons are designed to take their energy-loss characteristics into account.

In the event processing of HLT1, only a single track state is used. This is the ‘closest-to-beamline’ state, i.e. the point of closest approach of the track extrapolation to the LHC beamline. This state is used in single-track trigger selections and to reconstruct secondary vertices. Saving only a single state enables further simplification of the usual Kalman filter process, which combines forward and backward passes and averages the intermediate states for the best possible estimates. The minimal configuration chosen is a full forward pass and a backward pass only over the VELO detector. For the partial backwards pass, the covariance matrix at the end of the forward pass is transported back to the end of the VELO using a cumulative transport matrix, then the backwards pass of the VELO is performed starting with the forwards pass state at that position with an updated momentum estimate. An accurate momentum estimate is achieved from the forward pass alone and the ‘closest-to-beamline’ state is almost completely determined by the VELO backward pass. Given this configuration, no outlier removal, i.e. the removal of hits above a threshold contribution to the track ${\chi }^2$, is performed. The output of the algorithm is reduced to the position (x, y, z), slopes ($t_x$, $t_y$), charge over momentum $\frac{q}{p}$, a simplified covariance ($P_{\mathit{xx}}$, $P_{x{t}_x}$, $P_{t_x{t}_x}$, $P_{\mathit{yy}}$, $P_{y{t}_y}$, $P_{t_y{t}_y}$) and track fit quality information (${\chi }^2$, ndof). This is all the information needed for an accurate determination of the four-momenta of the particles (assuming a mass) and the positions and uncertainties of the primary and secondary vertices, and their related quantities such as impact parameter significance.

Implementation

GPU architectures achieve high throughput with massive parallelism rather than low-latency operations. Computation is expressed in kernels, functions executed on the GPU by a large number of threads working in parallel. Threads are grouped into blocks of threads that can cooperate, and further divided into a warps, the basic unit of parallelisation, which perform operations in lockstep using 32 simple compute units. The quantity and quality of the basic compute units is dependent on the architecture: on the chipset used by HLT1 (NVIDIA RTX A5000) single precision floating point operations are preferred. When different threads within a warp need to perform different operations, the execution is serialised and the throughput drops. Thus, algorithms with uniform and data-independent execution paths are preferred. Memory access is an important bottleneck as there is a hierarchy of memory spaces with increasing latency. The registers file is the fastest available and directly connected to the compute units, the global off-chip memory can store large amounts of data, but accesses are associated with large latency. Additional on-chip caches with intermediate latencies exist. Latency is not only hidden by caches, but also by switching between warps to hide the latency of memory access.

The independence of each Kalman filter track fit from any other makes it easy to utilise the parallelism of the GPU architecture. At the same time, each individual track cannot easily be parallelised on a per-hit level since each step of the Kalman filter depends on the result of the previous step. Based on these two considerations the choice is made to have each individual thread perform one track fit at a time. This enables having tracks from different events processed in the same kernel block to fully utilise the available threads. As a direct consequence the runtime of the algorithm scales linearly with the number of tracks in an event. Further, it is necessary to consider the numerical precision limitations. The double floating point precision, commonly used in CPU applications, would be penalised with a performance factor of 1:64 on the specific chipset on which HLT1 is implemented in Run 3. The Parameterised Kalman Filter performs all calculation in single floating point precision by default in order to maximize throughput. This makes full use of the available processor cores optimised for single precision and reduces the memory resources allocated to a single track and thus increases opportunities for parallelisation. The reduction in numerical precision leads to small differences in the parameter estimates compared to the same algorithm running in double floating point precision, however the impact on the parameter estimates is acceptable for HLT1.

For the processing of each track a large amount of data is needed, specifically the number of floating point operations per byte of memory access is low, this means the kernel is limited by the available memory resources. Each thread needs to access the (unique) hit information and the common detector information. This adds up to several kilobytes per thread. The use of parameterisations, rather than numerically calculating the extrapolation, solves this issue by replacing unique access to a magnetic field map with a uniform access to parameters, e.g. the extrapolation of a track within the vertex detector is sufficiently described by six parameters regardless of the track position. These accesses can further be optimised by using the constant memory of the GPU. Constant memory is a 64 kB large read-only memory space with a broadcast ability that allows the distribution of data to all threads in a warp without additional latency. For the extrapolation between hits, each thread uses the same parameters in lockstep and the size of the parameterisation dataset is around 2 kB, so it can be optimised with the constant memory. The exception to this access pattern is the extrapolation through the main dipole magnet, where each thread needs to access a different subset of the much larger parameterisation dataset around 1 MB. This dataset is too large to be stored in the constant memory and the access pattern would not profit from the broadcasting capabilities. Instead the parameterisation dataset is stored in the global memory space, and in practice cached in on-chip memory. By designing the algorithm to interweave memory accesses with compute operations the latency of the memory can be partially hidden.

In general, the algorithm is designed to give as much opportunity as possible for the kernel to hide latency. Reducing the amount of data that is stored in the registers has led to major throughput improvements. When the algorithm needs more space than there is register space, register spilling occurs, this is the slow offloading of register-stored data into slower memory. The maximum register space used by a thread also determines the occupancy, i.e., how many warps can share the same compute units, and thus how effectively the GPU can hide latency. The goal is to keep track parameters, such as the state $\overrightarrow{x}$ and its covariance in the registers, while all other data structure are broken up into small objects only to be kept in fast memory for short periods. The data structures of external inputs, such as the track objects and the detector hits, are given by upstream algorithms and they are not necessarily optimal for the access pattern of the Kalman filter. For example, in the case of the SciFi and UT track segments, the order of hits, as stored in the track object, does not match the processing order in the Kalman filter. In these cases lightweight maps of hits to layers are created to avoid unnecessary reading processes. Since each track has the same structure, branching instructions can mostly be avoided and the loop structure over all hits in z order can be synchronised on a warp level.

The parameters used in the extrapolation are derived from simulated events. For each extrapolation step a dataset D is created, which contains the true state at the origin of the extrapolation step, ${\overrightarrow{x}}_{n,\text{origin}}$, the true state at the target of the extrapolation, ${\overrightarrow{x}}_{\text{target}}$, and a state extrapolated from the origin state to the target z-layer using a precise Runge–Kutta extrapolation, ${\overrightarrow{x}}_{n,\text{extrap}}$. The important difference between the latter two is that ${\overrightarrow{x}}_{n,\text{target}}$ contains a contribution of multiple scattering. The extrapolation parameters $p_i^{X\to Y}$ are obtained by minimising

$$\begin{array}{c}\hfill \prod _{n\in D}\mathcal{N}\left(f_{X\to Y}^a,({\overrightarrow{x}}_{n,\text{origin}};p),-,a_{n,\text{extrap}},,,\theta \right),\end{array}$$ 12

where $a\in \{x,y,t_x,t_y\}$ is the state component. This effectively ensures the parameterised extrapolation matches the previously calculated Runge–Kutta extrapolation. The width $\theta$ of the Gaussian used in the minimisation describes the parameterisation uncertainty. This minimisation is done for the individual dimension a of the state in the order that respects the dependencies in the extrapolation function, e.g. the predicted x value depends on the predicted $t_x$ value. The scattering parameters $\tilde{p}$ in x are extracted by fixing the extrapolation parameters p and minimising with only the scattering parameters $\tilde{p}$ left floating:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \prod _{n\in D}[{\mathcal{N}}_2(& f_{X\to Y}^a({\overrightarrow{x}}_{n,\text{origin}};\tilde{p})-a_{n,\text{target}},\hfill \\ \hfill & f_{X\to Y}^a({\overrightarrow{x}}_{n,\text{origin}};\tilde{p})-t_{a|n,\text{target}},\hfill \\ \hfill & Q_{[a{t}_a]})+c],\hfill \end{array}\end{array}$$ 13

where $a\in \{x,y\}$ denotes the spatial dimension, $Q_{[a{t}_a]}$ is the relevant $2\times 2$ sub matrix of Q (Eq. 11) and c is an empirical constant to help with convergence. Relatively small sample sizes of 10,000 simulated events lead to 100,000–1,000,000 data points in D, depending on the specific extrapolation function and are sufficient to determine all parameters. The required accuracy, better than the expected MS contribution, is achieved for all extrapolation functions, i.e. the mean squared deviation of the parameterised extrapolation is less than the expected uncertainty resulting from MS. Additionally, the parameterisation is extended to be valid for momenta as low as 1.5$\text{,Ge}\text{V!/}c$, rather than 3$\text{,Ge}\text{V!/}c$ as in the first report on this algorithm[27], matching the output of the HLT1 track reconstruction algorithms. As the Parameterised Kalman Filter uses the same set of parameters for both polarity settings of the main LHCb dipole magnet by applying a sign flip to extrapolation terms sensitive to the sign of $\frac{q}{p}$, the parameters fit also uses a dataset containing events of both magnet polarities. The parameter set comprises 240,466 floating-point values, of which 240,000 describe the extrapolation from the UT to the SciFi, given by a grid of size $50\times 60$, and each grid point associated with an 80-parameters polynomial. Each thread reads 946 of them: the 480 extrapolation parameters relevant to its track’s region via interpolation of 6 grid points in the UT to SciFi extrapolation[27], together with the remaining 466 values, which are common to all threads. These 466 shared values are placed in constant memory, where their broadcast access across threads is served efficiently.

The Parameterised Kalman Filter algorithm makes up roughly 3.5% of the total runtime of the 2025 HLT1 sequence, while the VELO Only Kalman Filter made up 1.4% of the runtime, in an otherwise equal sequence. This modest runtime increase from the Parameterised Kalman Filter does not impact the overall system throughput. While the evaluation was performed at the nominal pile-up of $\nu =7.6$, it is important to note that the throughput of the algorithm scales linearly with pile-up. LHCb expects an input rate to HLT1 of about 30 MHz, dictated by the collision rate delivered by the LHC to LHCb. With 500 processors working in parallel, the minimum viable throughput for a single GPU is about 60$\text{,kHz}$. However, some safety margin needs to be accounted for to deal with outages, so any viable configuration must achieve throughputs well over 60$\text{,kHz}$ per GPU.

The impact of the Parameterised Kalman Filter in terms of throughput is shown in Table 1. From all these numbers it is clear that the Parameterised Kalman Filter is well compatible with HLT1 ’s real-time requirements.

Table 1.

Comparison of throughput on a single production GPU (NVIDIA RTX A5000) running the default HLT1 sequence for 2024 with the two different track fits algorithms. The results and their uncertainty are extracted from 10 individual runs of the same setup, emulating the production environment and their variance

Sequence	Throughput (events/s/device)
VELO-only KF	$65940\pm 160$
Parameterised KF	$64310\pm 90$

Performance analysis

The primary benefit of fitting the full track comes from incorporating hits upstream and downstream of the LHCb magnet, which enables precise measurements of particle momenta. The resulting momentum resolution is much better than that achieved by the pattern recognition algorithms (using a parametric approach) and used by the VELO Only Kalman Filter. The improvement in momentum resolution is shown as a function of momentum in Fig. 3a, achieving a momentum resolution close to the one of the full reconstruction in HLT2 [2]. In addition the figure also displays the impact of detector misalignment on the momentum estimate. For this study the detector elements of the SciFi detector only have been shifted randomly in the simulation, with the size of the shifts being realistic for the 2024 data-taking period. One can see the large impact on the resolution for the VELO Only Kalman Filter, while the Parameterised Kalman Filter manages to fully recover the performance of a perfectly aligned detector. In Fig. 3b a similar effect can be seen for the mean bias of the momentum estimate. As both algorithm treat the fit of the VELO segment identically, there is no difference in sensitivity to the VELO alignment.

Fig. 3.

The momentum resolution is compared using the same simulated events of the Parameterised Kalman Filter and the algorithm it replaces the VELO Only Kalman Filter, both with and without misalignment injected into the simulated geometry of the sample. The momentum resolution of HLT2 is included as a comparison point. The momentum resolution is extracted from a Gaussian fit to $(p-p_{\text{true}})/p_{\text{true}}$ in the given bin of $p_{\text{true}}$, the ground truth value of the simulation

The ${\chi }^2$ value produced by the Kalman filter is also used to select tracks of high quality or reject tracks that are likely the combination of random hits, leading to ghost tracks. A neural-network based ghost rejection is already used as part of the track reconstruction, resulting in an average fraction of ghost tracks considered in the track fit of only around 3%. Further reduction of this fraction is performed with a cut on the quality-of-fit measure, ${\chi }_{\text{red}}^2$. The ${\chi }_{\text{red}}^2$ calculated by the VELO Only Kalman Filter only accounts for the VELO segments, compared to the Parameterised Kalman Filter which accounts for the full track. Many ghost tracks are combinations of real VELO segments with real SciFi segments from different particles. This type of ghost track cannot be identified by a fit of only the VELO segment, thus the Parameterised Kalman Filter creates a larger separation between the ${\chi }_{\text{red}}^2$ distributions of real tracks and ghosts compared to the VELO Only Kalman Filter, as seen in Fig. 4. Especially selections of low ${\chi }_{\text{red}}^2$ values result in a higher purity of the track sample. At the working points used by the high-rate single track selection, the VELO-only fit (${\chi }_{\text{red}}^2<2.5$) produces a ghost efficiency of 70% at a genuine track efficiency of 87%, while the Parameterised Kalman Filter (${\chi }_{\text{red}}^2<3.0$) improves to a ghost efficiency of 29% with the efficiency for genuine tracks simultaneously increasing to 91%.

Fig. 4.

Discrimination power between real and fake (ghost) tracks, compared using the same simulated events, with the Parameterised Kalman Filter and the algorithm it replaces, the VELO Only Kalman Filter. Shown are (top) the ${\chi }_{\text{red}}^2$ distributions and (bottom) the ghost-rejection power represented as an ROC curve. A ghost track is defined as a track for which less than 70% of its hits originate from a single simulated particle

The state estimates and their covariance are examined for two components in Fig. 5 by calculating the pulls. The pull distribution of the momentum estimate shows a strong improvement, though some non-gaussian behaviour is still apparent. However, covariance of the momentum is not used further in the HLT1 processing, and the bias and precision is better understood using Fig. 3. For the spatial component x, which is dominated by the VELO component of the fit, the improvements are smaller, as the VELO Only Kalman Filter has a similar approach for the VELO segment of the track fit. Improvements in momentum resolution and general state resolution are apparent in the improved resolution of mass peaks that are reconstructed in simulated signal samples. Figure 6 shows invariant mass spectra resulting from dihadron and dimuon combinations of tracks, which include $D^0$ and $J/\psi$ mass peaks. The widths of the mass peaks are fitted with double Gaussians. Significant improvements close to a factor 2 in the resolution can be seen compared to the VELO Only Kalman Filter. A similar comparison of the dimuon resolution using $J/\psi \to {\mu }^{+}{\mu }^{-}$ decays with data recorded in 2025 is available in Ref. [29].

Fig. 5.

The pulls of (top) the momentum component and (bottom) the x component of the estimated states. The histograms show the quantities using the same simulated event sample, once using VELO Only Kalman Filter and once using Parameterised Kalman Filter, but otherwise identical reconstruction sequences

Fig. 6.

Two-particle invariant mass histograms taken from running HLT1 lines over simulation of signal events. The histograms compare the reconstruction of the same simulated event sample, once using VELO Only Kalman Filter and once using Parameterised Kalman Filter, but otherwise identical reconstruction sequences. The data was selected using (top) oppositely charged dimuon mass pairs and (bottom) oppositely charged pairs of kaons and pions

Conclusion

This paper presents the implementation and deployment of the Parameterised Kalman Filter in the LHCb HLT1 trigger system. The algorithm has been operational since the beginning of the 2025 data-taking period, demonstrating the feasibility of performing full-track Kalman filtering at the 30 MHz LHC collision rate, using GPU computing. The core development lies in replacing computationally expensive magnetic field map lookups and Runge–Kutta methods with fast parameterisations, while maintaining the mathematical rigour of the Kalman filter framework. This approach reduces the algorithm to simple functions that are well-suited to GPU architecture limitations, particularly the constraints on fast memory access and the requirement for single-precision arithmetic.

The performance improvements are shown across multiple metrics. Momentum resolution shows strong improvement, with the Parameterised Kalman Filter almost matching the momentum resolution of the full reconstruction in HLT2. This is particularly evident in invariant mass distributions where $J/\psi$ and $D^0$ peak widths are almost halved. Crucially, the Parameterised Kalman Filter demonstrates robustness against detector misalignment effects that degrade the performance of momentum estimates based solely on pattern recognition, making it more reliable for stable physics performance throughout long data-taking periods. The algorithm also provides a better track-quality metric through improved ${\chi }^2$ distributions, enabling better separation between genuine particle tracks and combinatorial background. This enhanced track quality directly translates to improved trigger efficiency and a reduced number of background events, while from a computational perspective, the Parameterised Kalman Filter increases the HLT1 processing time only by about 2%.

The successful deployment of the Parameterised Kalman Filter in LHCb’s production trigger system demonstrates the potential for advanced reconstruction algorithms in real-time high-energy physics computing and is another step towards an offline-quality reconstruction on GPUs. The improved track parameter estimates enable more effective event selection in the trigger system, increasing the efficiency for recording events of physics interest and thereby enhancing the statistical precision of LHCb’s measurements in Run 3 and beyond.

Further developments include the implementation of outlier removal and the addition of state estimates for a future particle identification using reconstruction in the RICH detectors [30]. The Parameterised Kalman Filter and its future enhancements represent a step towards an offline-quality reconstruction at the full LHC collision frequency, which is envisaged for the next upgrade of the LHCb experiment [31].

Funding

M. De Cian acknowledges support from STFC, L. H. Uecker acknowledges support from BMFTR.

Data Availability Statement

This manuscript has associated data in a data repository. [Authors’ comment: Software/Code that is associated with this publication and that is publicly available is referenced within the publication content. Specific analysis software/code used to produce the results shown in the publication is preserved within the LHCb collaboration internally and can be provided on reasonable request, provided it doesn’t contain information that can be associated with unpublished results.]

Code Availability Statement

This manuscript has associated code/software in a data repository. [Authors’ comment: The software used during the study is available in the Allen repository, https://gitlab.cern.ch/lhcb/Allen/-/tree/v7r12/device/kalman/ParKalman.]