YRT-PET: An Open-Source GPU-accelerated Image Reconstruction Engine for Positron Emission Tomography

Abstract

Image reconstruction for positron emission tomography (PET) requires an accurate model of the PET scanner geometry and degrading factors to produce high-quality and clinically meaningful images. It is typically implemented by scanner manufacturers, with proprietary software designed specifically for each scanner. This limits the ability to perform direct comparisons between scanners or to develop advanced image reconstruction algorithms. Open-source image reconstruction software can offer an alternative to manufacturer implementations, allowing more control and portability. Several existing software packages offer a wide range of features and interfaces, but there is still a need for an engine that simultaneously offers reusable code, fast implementation and convenient interfaces for interoperability and extensibility.

In this work, we introduce YRT-PET (Yale Reconstruction Toolkit for Positron Emission Tomography), an open-source toolkit for PET image reconstruction that aims for flexibility, reproducibility, speed, and interoperability with existing research software. The toolkit is implemented in C++ with CUDA-enabled GPU acceleration, relies on a plugin system to facilitate the use with multiple scanners, and offers Python bindings to enable the development of advanced algorithms. It includes support for list-mode/histogram data formats, multiple PET projectors, incorporation of time-of-flight information, event-by-event rigid motion correction, point-spread function modeling. It can incorporate correction factors such as normalization, randoms and scatter, obtained from scanner-specific plugins or provided by the user. The toolkit also includes an experimental module for scatter estimation without time-of-flight.

To evaluate the capabilities of the software, two different scanners in four different contexts were tested: dynamic imaging, motion correction, deep image prior, and reconstruction for a limited-angle scanner geometry with time-of-flight. Comparisons with existing tools demonstrated good agreement in image quality and the effectiveness of the correction methods. The proposed software toolkit offers high versatility and potential for research, including the development of novel reconstruction algorithms and new PET scanner systems.

I. Introduction

POSITRON emission tomography (PET) has become an essential tool for the development and evaluation of imaging biomarkers in neuroscience, cardiology, oncology and others. Image reconstruction, which requires an accurate modeling of the system, is typically implemented and provided by the manufacturer, but the data format, the scanner geometry and the implementation details are usually proprietary. However, using custom image reconstruction engines can be advantageous when more control over the image reconstruction process is desired either to perform systematic comparisons between scanners, make use of state-of-the-art reconstruction algorithms, or when developing new scanners. Nevertheless, the lack of an established standard for raw PET data (until recent efforts [1]), differences in the availability and implementation of certain features, and the need for scanner-specific correction methods often hinder the dissemination of image reconstruction engines. More specifically, the handling of time-of-flight (TOF), depth-of-interaction (DOI), normalization and randoms estimations typically varies significantly from scanner to scanner. Additionally, motion correction is often not available in the manufacturer reconstruction engines, but may be a critical correction for quantitative PET imaging [2]. Motion information may be collected from a variety of sources (e.g., external motion tracking devices [3], [4], [5], simultaneous PET/MR [6], [7], [8], or from the data itself [9]) and may exploit different assumptions (e.g., rigid motion typically assumed in the brain, affine or deformable in other organs such as liver or heart) which affect the corresponding motion correction algorithms (e.g., event-by-event correction [3], [10] or image deformation in the forward model [11], [12]). Therefore, the implementation of a reconstruction algorithm oriented towards one specific PET scanner and imaging application might not be applicable in another context without major changes. This can be problematic, for instance, in early stages of scanner development when focus is placed on the hardware development and image reconstruction is needed to drive design decisions.

Several open-source packages for PET image reconstruction have been released to address this need, including STIR [13], CASToR [14], and PyTomography [15]. They aim to provide implementations of conventional and advanced reconstruction algorithms and simplify the data conversion from PET scanners or simulation software. STIR (Software for Tomographic Image Reconstruction) [13] is a comprehensive package written in C++ with MATLAB and Python bindings, implemented using the simplified wrapper and interface generator (SWIG) [16]. It offers a large collection of algorithms and operators and natively supports a wide range of scanners. This includes the several versions of the ordered-subsets expectation-maximization (OS-EM) [17] algorithm, corrections for attenuation, normalization and randoms, scatter estimation using the single scatter simulation (SSS) algorithm [18], kernel-based reconstruction [19], etc. It can leverage the PARALLELPROJ program [20], which implements a GPU version of the Joseph projector [21] for the geometry of the forward model. STIR mostly operates on sinograms, which enables efficient use of symmetries in the forward model but limits the implementation of event-by-event corrections such as motion correction [3]. CASToR (Customizable and Advanced Software for Tomographic Reconstruction) [14] is another widely used package for image reconstruction which offers alternative PET projectors such as the distance-driven model [22], [23] and conveniently interfaces with the GATE simulation software [24], enabling the conversion of GATE geometries and ROOT [25] files, which can greatly facilitate the design of novel scanners. While not part of CASToR, SSS is implemented in openSSS [26], which is compatible with CASToR. The main limitations of CASToR, as of version 3.2, include the lack of interface for the forward and back-projection (in the form of command-line executables or bindings to a high-level language) and the CPU-only implementation. Finally, PyTomography [15] is a Python-based reconstruction toolkit relying on PyTorch [27] and geared towards deep learning algorithms. It relies on PARALLELPROJ for its GPU-accelerated forward and back-projection and implements filtered back-projection and deep image prior [28], [29] algorithms. Its modular approach allows for easy integration with various deep learning algorithms. Note that other GPU projectors have been proposed [30], [31], but are not included in existing open source reconstruction engines.

In this work, we present the Yale Reconstruction Toolkit for Positron Emission Tomography (YRT-PET)¹, a free and open-source C++ reconstruction library for PET image reconstruction that focuses on flexibility, interoperability and speed. It includes support for both CPU and GPU parallelism using OpenMP [32] and Nvidia CUDA. Image reconstruction is performed using the OS-EM algorithm with Siddon [33], [34] or distance-driven [22], [23] projectors. YRT-PET supports event-by-event motion correction in list-mode [3], [4] and rigid image-domain transformation. Additionally, YRT-PET includes optional Python bindings, allowing access to internal objects and operators for use in algorithms implemented in Python. This allows users to seamlessly integrate YRT-PET’s projection operators, image transformations, or data manipulations in their customized algorithms with Python array manipulation libraries. Finally, the toolkit can potentially handle input data from any list-mode or histogram format as it is designed around a plugin system, which allows developers to extend the software to support data in arbitrary formats. The main features that differentiate YRT-PET from existing reconstructions engines are:

• A plugin system to easily add support for arbitrary scanners, including interfaces to read normalization factors and randoms estimate (from singles rate or delayed coincidences) from raw scanner files. This system enables sharing reconstruction code and features without divulging manufacturer intellectual property on scanner layout and data formats (keeping plugins closed-source when necessary).

• An end-to-end GPU implementation, where data remains on the GPU if memory permits or is split into batches, with efficient memory transfer, overlapped with execution of GPU kernels.

• Simultaneous support for event-by-event motion correction [3], time-of-flight, depth-of-interaction and point spread function modeling.

• An interface with Python for most objects and operators, enabling the incorporation of the projectors, operators (CPU and GPU) and solvers in advanced algorithms written in Python. This includes bindings for Pytorch to enable the development of deep learning-based algorithms (supporting full GPU processing or CPU-to-GPU transfers depending on memory requirements).

II. Methods

A. Formulation of imaging model

In this work, we formulate the image reconstruction problem as that of recovering the activity in an imaged object, represented as a vector $\mathbf{x}={[x_1,\dots ,x_J]}^{\mathsf{T}}$ where $J$ is the total number of image voxels, from measurements $\mathbf{m}={[m_1,\dots ,m_I]}^{\mathsf{T}}$ where $I$ is the number of measurements. In list-mode, $m_i=1,\forall i$ and $I$ is the number of collected coincidences, while in histogram mode, $I$ represents the total number of detector pairs of the system. In the rest of this paper, we denote by $\mathscr{H}$ the set of all the lines of response (LORs) of the system and by $ℳ$ the set of measurements, which is equal to $\mathscr{H}$ in histogram mode and is the list of collected LORs in list-mode. Given an image estimate $\mathbf{x}$, the measurement for a given line of response $i\in ℳ$ is modeled as:

$$m_i=\text{Poisson}\left(\sum _j{s}_i{a}_i{G}_{ij}{x}_j+r_i+{\sigma }_i\right),$$ (1)

where $s_i$ represents the sensitivity of the line of response $i$, $a_i$ represents the attenuation coefficient factor, and $r_i$ and ${\sigma }_i$ respectively represent the contribution from randoms and scatter associated with measurement $i$. The geometric system matrix element $G_{ij}$ represents the overlap between LOR $i$ and image voxel $j$. YRT-PET implements both the Siddon [33], [34] and the distance-driven [22], [23] projectors (both available on CPU and GPU). The Siddon projector supports a multi-ray option that randomly samples multiple rays on the plane orthogonal to the normal vector of the crystal surface, which allows to take into account the detector width and orientation. $G_{ij}$ is computed on-the-fly, and can optionally incorporate additional modeling, including the point spread function (PSF) of the system in both image and projection space [35], [36], time-of-flight information, and event-by-event rigid motion [37]. Image-space PSF is modeled as a spatially invariant convolution (separable along x, y and z) and projection-space PSF is modeled, in the distance-driven kernel, as a user-defined smoothing profile incorporated into the overlap function, which varies with the radial position of a LOR. Time-of-flight uses a Gaussian model with user- controlled width and truncation where each coincidence includes a time of arrival difference in ps. Randoms estimation is scanner-specific and can be performed using singles rates or delayed coincidences [38], depending on the plugin implementation. YRT-PET includes a limited scatter estimation routine implementing the SSS method [18].

B. Reconstruction algorithm

The reconstruction of the activity image $\mathbf{x}$ from measurements $\mathbf{m}$ is performed iteratively using the maximum-likelihood expectation-maximization algorithm [39] (ML-EM), which results in the following update of image $\mathbf{x}$ at pixel $j$ and iteration $n+1$:

$$x_j^{(n+1)}=\frac{x_j^{(n)}}{q_j}\sum _{i\in ℳ}{G}_{ij}\frac{m_i}{{\sum }_k{G}_{ik}{x}_k^{(n)}+\frac{r_i+{\sigma }_i}{s_i{a}_i}},$$ (2)

$$q_j=\sum _{i\in \mathscr{H}}{s}_i{a}_i{G}_{ij},$$ (3)

where $\mathbf{q}={[q_1,\dots ,q_J]}^{\mathsf{T}}$ is the sensitivity image, computed before the first iteration. The ML-EM algorithm can be accelerated using the ordered-subsets expectation-maximization algorithm [17] (OS-EM), resulting in the following update, where the iteration number $n$ and subset number $p$ have been combined into a single update counter denoted by $n^{\prime }$:

$$x_j^{(n^{\prime }+1)}=\frac{x_j^{(n^{\prime })}}{q_j^{(p)}}\sum _{i\in {ℳ}_p}{G}_{ij}\frac{m_i}{{\sum }_k{G}_{ik}{x}_k^{(n^{\prime })}+\frac{r_i+{\sigma }_i}{s_i{a}_i}},$$ (4)

where $q_j^{(p)}$ is the sensitivity image value at voxel $j$ for subset $p$, and ${ℳ}_p$ is the set of measurements associated with subset $p$. $q_j^{(p)}$ is therefore calculated differently for list-mode and histogram data:

$$\text{List-Mode}:q_j^{(p)}=\frac{1}{P}\sum _{i\in \mathscr{H}}{s}_i{a}_i{G}_{ij},$$ (5a)

$$\text{Histogram}:q_j^{(p)}=\sum _{i\in {\mathscr{H}}_p}{s}_i{a}_i{G}_{ij},$$ (5b)

where $P$ is the number of subsets and ${\mathscr{H}}_p$ is the set of LORs in the system for subset $p$. For histogram reconstructions, LORs are grouped geometrically, resulting in a different sensitivity image for each subset. For list-mode reconstructions, since the subsets are subdivided chronologically, which assumes that the LOR spatial distribution will be similar, the same sensitivity image is reused for all subsets.

C. Event-by-event motion correction

Event-by-event motion correction is implemented as described in [4]. This method assumes that rigid motion information is available for all frames, defined as sets of consecutive coincidences. This information is typically obtained from external tracking devices [3], [37], [40]) or from the data itself [41]. For each event, motion correction consists in displacing the endpoints of a line of response based on the corresponding transformation matrix. This method also involves segmenting the attenuation image into two components: the hardware attenuation, corresponding to objects that remain fixed with respect to the scanner (e.g., the bed), and the in vivo attenuation, representing the moving subject [4]. The resulting image reconstruction update, with event-by-event motion correction, can be expressed as:

$$x_j^{(n^{\prime }+1)}=\frac{x_j^{(n^{\prime })}}{{\check{q}}_j}\sum _{\begin{array}{c}\hfill i\in {ℳ}_p\hfill \\ \hfill i^{\prime }={ℒ}_f(i)\hfill \end{array}}{G}_{i^{\prime }j}\frac{\frac{1}{a_{i^{\prime }}^{(v)}}}{{\sum }_k{G}_{i^{\prime }k}{x}_k^{(n^{\prime })}+\frac{r_i+{\sigma }_i}{s_i{a}_i}},$$ (6)

where $a_i^{(v)}$ is the in vivo attenuation coefficient factor for the LOR associated with event $i$, $i^{\prime }={ℒ}_f(i)$ represents, with a shorthand notation, the displaced LOR for event $i$ based on its corresponding frame $f$. Additionally, the sensitivity image, denoted by $\check{\mathbf{q}}$ is generated by first calculating the sensitivity image in the reference frame and then averaging it over time, applying the motion transform of each frame, as described in [42]:

$${\check{q}}_j=\frac{1}{P}\sum _f\frac{{\tau }_f}{\tau }\sum _k{T}_{jk}^{(f)}\sum _{i\in \mathscr{H}}{G}_{ik}{a}_i^{(h)}{s}_i,$$ (7)

where $a_i^{(h)}$ is the hardware attenuation coefficient factors for $i$, $T_{jk}^{(f)}$ is the rigid motion transformation from voxel $k$ to $j$ in frame $f$, and $\tau$ and ${\tau }_f$ denote respectively the scan and frame duration. Note that $a_i=a_i^{(h)}{a}_i^{(v)}$.

D. Scatter correction

Finally, the vector $\sigma ={[{\sigma }_1,\dots ,{\sigma }_I]}^{\mathsf{T}}$ in (4) and (6) represents the scatter estimate for each LOR. YRT-PET includes an experimental, non-TOF and CPU-only implementation of the SSS algorithm [18]. It requires an initial activity image, reconstructed without scatter correction but with randoms correction, which is used, along with the attenuation map, to compute the volume integrals over the scattering object, yielding an estimate of the scatter distribution in a low- resolution histogram with radial, angular downsampling and considering only direct planes. A dense Histogram3D is then interpolated from the low resolution histogram. Tail-fitting is then performed to estimate a scatter scaling factor such that the upsampled scatter histogram matches the prompts (from which randoms have been subtracted) near the edges of the imaged object. The resulting histogram is then used in the reconstruction as in (4) and (6). The activity image and scatter estimates can be iteratively updated by alternating image reconstruction and SSS steps. The current implementation does not support sinogram compression, TOF or fine-tuning of SSS and tail fitting parameters. Future versions will include these features to offer better control for individual scanners.

III. Implementation

This section first describes the data structures used by YRT-PET, then the operators used for reconstruction, and finally, the design and implementation of the optimization algorithm.

A. Data structure

In order to allow the support of a wide range of PET scanners, geometries and data formats, YRT-PET relies on generic data structures to represent the geometry, projection data and images.

1). Scanner geometry:

The Scanner object is responsible for handling information related to the scanner geometry, including the layout of detectors, their dimensions and orientations. The scanner is defined by a text-based configuration file (using the JavaScript object notation (JSON) format), which contains basic information about the scanner geometry, and a lookup table (LUT) file with detector positions and orientations. If the scanner encodes DOI information, the LUT stores the different DOI layers as separate detectors, and the sensitivity image generation includes all the possible pairs of detectors across all DOI layers. By convention, the detectors are ordered anticlockwise, ring by ring, and layer by layer. The Scanner object then defines an interface to gather the spatial coordinates of a LOR based on any given pair of detector indices.

2). Projection data formats:

In YRT-PET, the term projection data describes a set of LORs associated to numerical values. Projection data can be either of list-mode or histogram type. The class diagram shown in Fig. 1 illustrates a hierarchy of the supported data types. The native projection data structures in YRT-PET respect the Python buffer protocol, enabling their usage within Python programs, via an interface implemented using pybind11². This design allows projection data objects to be bound to NumPy arrays [43], or PyTorch tensors [27] for seamless processing between Python and YRT-PET.

Fig. 1.

Software architecture diagram for the projection-data objects. The PluginListMode class is an example of a plugin reading data in a custom list-mode format.

List-mode objects store the LOR associated with each event chronologically, along with the event timestamp and additional information such as time-of-flight measurement (when available). YRT-PET’s native list-mode format (named ListModeLUT in Fig. 1) defines, for each event, the timestamp and the detector pair defining the LOR (i.e. the indices of the two detectors in the scanner’s LUT). Optionally, motion information can be stored with the list-mode data for event-by-event motion correction as described in Section II-C.

The base histogram objects differ from list-mode objects in their ability to store a value associated with each LOR and the fact that each LOR is stored only once. The actual handling of buffers is left for derived classes, but the assumption is that values for a given LOR can be retrieved with a $𝒪(1)$ complexity. YRT-PET also defines a 3D histogram format (named Histogram3D in Fig. 1) which encodes all the possible LORs in a given scanner into a 3D array with three logical coordinates: the in-plane angular and radial coordinates and the axial ring pair. Note that time-of-flight information is not encoded in this histogram format in the current implementation. This histogram format only stores the exact LORs allowed by the scanner without compression (mashing), as opposed to a sinogram, which describes LORs in a regularly-spaced grid. This histogram representation allows for sensitivity image computation as it defines $\mathscr{H}$. Another built-in histogram format, named SparseHistogram in Fig. 1, encodes a value (number of coincidences) associated with a detector pair, which can save memory when storing acquisitions of small objects with some PET systems.

3). Plugin system for custom projection data formats:

YRT-PET includes a compile-time plugin system, which allows the extension of the software to support arbitrary scanner data formats. A plugin consists of a C++ class, inheriting from either the list-mode or the histogram abstract class, and the implementation of the required interface methods. Plugins can also add desired format-specific command-line options to toolkit’s built-in command-line executables. The simplified diagram in Fig. 1 shows a hypothetical third-party plugin PluginListMode, which inherits from the ListMode class. The plugin needs to define a type identifier and a static factory function (namely create) added to a “Format Registry”. This registry consists of a hash table linking unique identifiers to corresponding factory functions, allowing for the transparent creation of a ProjectionData object. Each format gets added into the registry at compile-time. At runtime, the user provides, through the command-line parameters, a filename and a type identifier to create the proper object. Fig. 2 shows a diagram describing the inputs for an OS-EM list-mode reconstruction with motion correction in YRT-PET. Randoms estimates are extracted from the list-mode data (typically by plugins or passed as histogram format. Scatter must be provided in Histogram3D format. Attenuation correction is passed via hardware and in-vivo attenuation maps. Finally, normalization factors can be extracted by plugins or passed as Histogram3D objects. Note that the example in Fig. 1 can also be applied for a custom histogram format. Plugins can also add Python bindings and command-line executables, which will be compiled within YRT-PET. Note that a plugin adding support for the PET ETSI Raw Data (PETSIRD) list-mode format [1] is currently in development.

Fig. 2.

Data pipeline for a list-mode reconstruction with motion-correction. Note that the list-mode and histogram formats can be customized through additional plugins, as shown in figure 1. Reconstructions from histogram data follow a similar structure, but without the steps related to motion-correction.

This plugin system design has several advantages: 1) it enables the use of YRT-PET with scanners whose raw data format is proprietary, 2) it simplifies the addition of new formats, requiring minimal effort, as it is automatically performed by the CMake build system upon discovery, and 3) it isolates data loading code from the rest of the reconstruction code, enabling the use of new data formats with all the YRT-PET features, including CPU/GPU reconstruction, Python bindings and others.

4). Image format:

Images are read and saved in the NeuroImaging Informatics Technology Initiative (NIfTI) open file format (National Institutes of Health, MD, USA). Additionally, since YRT-PET’s image buffers also respects the Python buffer protocol, image objects can also be used in Python and bound to NumPy arrays [43], or PyTorch tensors [27]. Therefore, YRT-PET operators can be seamlessly applied to NumPy arrays or PyTorch tensors without data transfer, which greatly facilitates their integration in Python programs and scripts.

B. Operators

YRT-PET operator objects are responsible for performing data processing components of the imaging model (e.g., PET projector, PSF model, image-domain transformation, etc.). A YRT-PET operator object must implement a forward and adjoint routine to be used within an optimization algorithm. These operations may be implemented as matrix-vector products, following traditional notations, but are more frequently implemented as on-the-fly operations due to memory constraints. Operators act as the building blocks for the reconstruction algorithm. This design pattern increases readability and code portability and simplifies the development of advanced solvers (see for instance [44] where Python bindings were used to develop a complex optimization framework using YRT-PET projection operators). Available operators include the image-domain PSF operator, image-domain rigid-motion transformations, and PET projectors.

GPU-support for operators is implemented by subclassing the regular CPU operators and adding memory management infrastructure. This means that YRT-PET operators transparently implement both a CPU and GPU version (using CUDA). GPU operators can either operate directly on GPU buffers or perform data transfers between host and device, depending on the location of the input buffer. Figure 3 shows the software architecture of the projection operators in both CPU and GPU. For projection operators, LORs are gathered through the ProjectionData interface before processing. The Scanner properties and the motion information are used to prepare the processing of each LOR. In order to properly select the LORs in a subset and provide a transparent interface for list-mode and histogram operations, they are enumerated by a BinIterator object, which is created for each subset and is responsible for listing measurements in a given subset. For the GPU operators, the subsets are divided into batches and LORs are then transferred to device memory through the intermediary of ProjectionDataDevice. The batch loading process is run in parallel with the forward and backward projections. This means that while the CPU prepares the data and copies it into the GPU memory, the GPU can run the projection operators, avoiding bottlenecks. This optimization leads to significant speed-ups compared to conventional approaches, especially when dealing with large datasets. Image-domain operators assume that images fit in GPU memory.

Fig. 3.

Software architecture diagram for projection operators

Additionally, just as it is possible to bind a NumPy array to YRT-PET data structures and apply CPU operators from Python, GPU operators can be applied directly on PyTorch [27] GPU tensors. This memory aliasing enables the straightforward implementation of neural networks that incorporate PET projection operations, as studied in [45] or [46], while operating directly within device memory, greatly accelerating network training and inference.

C. Optimization algorithm

The OS-EM implementation follows (6) (which reduces to (4) when event-by-event motion correction is not used). The forward model applied to the current image estimate includes image-domain rigid motion and PSF modeling, followed by application of the PET projector, multiplicative corrections (for LOR sensitivity and attenuation correction), and additive corrections for randoms and scatter. The building blocks provided allow the development of more advanced algorithms in C++ or Python.

Note that YRT-PET’s GPU implementation of the OS-EM algorithm subdivides the subsets into multiple batches of LORs if the projection data does not fit on the GPU memory. More specifically, (4) and (6) are modified such that the sums over measurements change from ${\sum }_{i\in {ℳ}_p}$ to ${\sum }_{b=1}^B{\sum }_{i\in {ℳ}_{p,b}}$, where $b$ is the batch index, $B$ is the total number of batches, and ${ℳ}_{p,b}$ is the set of LORs in subset $p$ and batch $b$. The number of batches is determined by the amount of available GPU memory.

IV. Evaluation

This section illustrates the functionalities of YRT-PET in several applications using different input data formats and corrections. The goal is to demonstrate YRT-PET’s capabilities for dynamic reconstruction, motion correction, algorithm prototyping, and an arbitrary scanner geometry. Two acquisition systems were analyzed: the GE Discovery MI and GATE [24] simulations of a flat panel scanner under development [47], [48], [49]. Comparisons were performed against manufacturer software and existing reconstruction tools using the same reconstruction parameters.

A. Brain imaging using GE Discovery MI

Two evaluations were performed using the GE Discovery MI, the first one on a dynamic acquisition of a human subject and the second on a 3D Hoffman brain phantom [50] to demonstrate motion correction.

1). Dynamic acquisition:

This acquisition was performed on a cognitively normal human subject imaged with [¹⁸F]MK6240, a tracer developed to image tau aggregation. The imaging protocol is described in [51]. The human study was approved by the local institutional review board. All subjects provided informed consent.

Reconstruction was first performed using the manufacturer’s toolbox (Duetto v02.18, GE Healthcare) and compared against YRT-PET using the same image reconstruction parameters and projector (i.e., OS-EM with 17 subsets and three iterations, using the distance-driven projector without PSF modeling) on a 1.7 × 1.7 × 2.8 mm³ voxel grid. Correction for scatter and randoms was incorporated in the reconstruction, as described in (4). Since this dataset had negligible intra-frame motion, reconstructions were performed in histogram format frame-by-frame, using a YRT-PET in-house plugin to read the input data. The scatter estimate, randoms estimate, and the normalization factors were generated by the manufacturer’s toolbox. A total of 54 frames were reconstructed over a 2-hour scan time. YRT-PET reconstructions were rigidly registered to reconstructions from the GE toolbox averaging the first 10 minutes and using ANTs [52] in order to account for slight differences in geometry conventions. A comparison of reconstructed images between the GE reconstruction (top) and the YRT-PET reconstruction (bottom) for several frames is shown in Fig. 4. An isotropic 3D gaussian filter of 1.7 mm full width at half-maximum (FWHM) was applied in both images displayed. Images reconstructed using YRT-PET show good visual agreement with images reconstructed using the GE toolbox.

Fig. 4.

Reconstruction of a dynamic acquisition from the GE Discovery MI scanner using the manufacturer software and YRT-PET for frames (a) from 120 s to 135 s, (b) from 6 min to 7 min, (c) from 28 min to 30 min and (d) from 100 min to 105 min. Images and difference images are shown in units of kBq/mL.

To evaluate the quantitative accuracy of the reconstructed images, kinetic modeling was performed on both sets of images using the multilinear reference tissue model with fixed (population-based) $k_2^{\prime }$ value (MRTM2). The reference region was set to the cerebellum grey matter, $k_2^{\prime }$ was set to 0.0126 min⁻¹ and a $t^{\ast }$ value of 40 minutes was used. The distribution volume ratio (DVR) maps were computed in regions of interest (ROIs) segmented using the FreeSurfer software [53] and co-registered to the PET data using the FMRIB Software Library (FSL) [54]. The time activity curves (TAC) in figure 5 show little difference between the manufacturer and YRT-PET reconstructed images.

Fig. 5.

Time activity curves and their MRTM2 fitting for the manufacturer and the YRT-PET reconstructed images. The time $t^{\ast }$ at 40 minutes is illustrated by a vertical line.

The results demonstrate a strong agreement between DVR estimates obtained from both sets of reconstructed images. Figure 6 illustrates the correlation between the DVR values for each region when computed from YRT-PET’s images and the manufacturer’s images. Linear regression analysis of these parameters yielded a slope close to 1 (a = 0.97) and a high correlation coefficient ($R^2=0.99$), reinforcing that both methods produce highly consistent kinetic modeling results. Additionally, the Bland-Altman analysis [55] confirms the equivalence of the estimated kinetic parameters, indicating no significant bias between the two software tools.

Fig. 6.

Linear regression and Bland-Altman plot between the DVRs calculated using the images from the manufacturer and from YRT-PET

These results demonstrate that the proposed reconstruction software provides images that are quantitatively similar to the manufacturer’s images, validating its suitability for dynamic PET imaging and kinetic modeling.

2). Motion correction:

To evaluate event-by-event motion correction in YRT-PET, reconstructions were performed from two 15-minute acquisitions of a Hoffman [50] brain phantom as in [37]. The first acquisition was static and the second had manually induced motion ranging ±20° and ±40 mm along all 6 degrees of freedom. The motion information was captured using the Polaris Vega tracking camera (NDI, Ontario, Canada), which recorded transformation matrices at 60 Hz. Similar to the dynamic acquisition described earlier, the additive correction factors were generated by the manufacturer’s toolbox. Note that, although the additive corrections were stored in a 3D histogram, the measurement data was still in list-mode format. The reconstruction used a voxel size of 2 × 2 × 2.8 mm³, was performed with 34 subsets and two OS-EM iterations, following (6). Since the acquisition of the moving phantom was started shortly after the static acquisition, its reference frame was aligned with the static data, therefore no image registration was needed.

Images in Fig. 7 and the corresponding line plots show minimal differences between the static image and the motion-corrected image in arbitrary units (A.U.). To quantify the performance of motion correction, the structural similarity index measure (SSIM) [56] between the two images was measured. The SSIM is defined as:

$$\begin{gathered} \text{SSIM}({\mathbf{x}}_1,{\mathbf{x}}_2)= \\ \frac{(2\mu ({\mathbf{x}}_1)\mu ({\mathbf{x}}_2)+C_1)(2\mathrm{cov}({\mathbf{x}}_1,{\mathbf{x}}_2)+C_2)}{(\mu {({\mathbf{x}}_1)}^2+\mu {({\mathbf{x}}_2)}^2+C_1)(\sigma {({\mathbf{x}}_1)}^2+\sigma {({\mathbf{x}}_2)}^2+C_2)}, \end{gathered}$$ (8)

where ${\mathbf{x}}_1$ and ${\mathbf{x}}_2$ are the two images compared, the $\mu$, $\sigma$ and cov functions represent the mean, standard deviation and the covariance. $C_1$ and $C_2$ are constants included to avoid numerical instability and are set to a small fraction of the dynamic range of the images. The SSIM calculated between the motion-corrected reconstruction and the static reconstruction was 95.3%, while the SSIM between the static and uncorrected image was 48.6%. Additionally, the contrast-to-noise ratio (CNR) was measured to confirm the effect of motion correction. The CNR was calculated between a target region with high activity and a background region with low activity.

$$\text{CNR}(\mathbf{x})=\frac{\mu ({\mathbf{x}}_{\mathbf{tgt}})-\mu ({\mathbf{x}}_{\mathbf{bkg}})}{\sigma ({\mathbf{x}}_{\mathbf{bkg}})},$$ (9)

where ${\mathbf{x}}_{\mathbf{tgt}}$ and ${\mathbf{x}}_{\mathbf{bkg}}$ are the target and background regions, segmented from an aligned MRI image of the phantom, and correspond to the gray matter and white matter region voxels respectively [50]. Using the above equation, the CNR of the static image was 5.37, the motion-corrected image 5.35, and the non-motion-corrected image 1.04. Although the motion correction was able to recover a large portion of the contrast, the motion-corrected image showed more noise than the static image. This may be caused by the pre-correction of measurements $m_i$ from the in vivo attenuation coefficient factors $a_{i^{\prime }}^{(v)}$ from (6), which is an approximation [57] and invalidates the Poisson model assumption.

Fig. 7.

Hoffman phantom reconstructions acquired with the GE Discovery MI scanner: (a) No motion correction (No MC), (b) Static reference (Static), (c) YRT-PET motion correction (MC), (d) Horizontal line profiles through images.

B. Image reconstruction using Deep Image Prior

In order to demonstrate the usage of YRT-PET’s GPU operators within neural networks, the acquisition described in subsection IV-A2 was reconstructed using a Deep Image Prior (DIP) method [28] similar to that proposed by Hashimoto et al. [45]. This algorithm consists of calculating an image $\mathbf{x}$ using a convolutional neural network (CNN):

$$\mathbf{x}=f(\theta \mid \mathbf{z})$$ (10)

where $f$ represents the CNN, with weights represented by $\theta$, and with $\mathbf{z}$ representing the prior vector. We define the forward model $\overline{\mathbf{y}}$ as:

$$\overline{\mathbf{y}}=\mathbf{s}\odot \mathbf{a}\odot \mathbf{Gx}+\mathbf{r}+\sigma ,$$ (11)

with the $\odot$ operator representing element-wise multiplication. Supposing the measured acquisition $\mathbf{m}$ follows a distribution of $p(m_i\mid \mathbf{x})$ and the projection-space data is binned into histograms, the log likelihood for the measured data $\mathbf{m}$ is written as:

$$L(\mathbf{m}\mid \mathbf{x})=\sum _i\log p(m_i\mid \mathbf{x})=\sum _i{m}_i\log {\overline{y}}_i-{\overline{y}}_i-\log m_i!$$ (12)

This method estimates the image $\mathbf{x}$ by training the neural network, substituting the forward model into the network’s loss function. In this case, the negative log likelihood is used as loss function. The training is therefore performed as:

$$\widehat{\theta }=\underset{\theta }{\text{arg min}}\sum _i{\overline{y}}_i-m_i\log {\overline{y}}_i$$ (13a)

$$\text{s.t.}\overline{\mathbf{y}}=\mathbf{s}\odot \mathbf{a}\odot \mathbf{G}f(\theta \mid \mathbf{z})+\mathbf{r}+\sigma$$ (13b)

The final image is then computed from a final inference:

$$\widehat{\mathbf{x}}=f(\widehat{\theta }\mid \mathbf{z})$$ (14)

Since the loss function includes a forward projection $\mathbf{G}$ in (13b), the memory aliasing pattern allowed by the proposed software allows the usage of the Siddon or distance-driven projector within the neural network with minimal memory transfers between the CPU and GPU. Moreover, since both the forward and backward projectors are exposed through Python bindings, the PyTorch backpropagation routine can include YRT-PET’s back-projection operation transparently to the optimizer. The optimizer used is the limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) [58] with a learning rate of 0.1, and the CNN architecture is the same 3-D U-Net structure as in [45]. The implementation code for this demonstration is also available³, serving as an example of deep learning algorithm prototyping for PET reconstruction.

Figure 8 shows a visual comparison of the DIP method applied on the Hoffman brain phantom. The low count input was generated by randomly sampling 1/40th of the list-mode events and re-binning them into a 3D histogram. The projector used for $\mathbf{G}$ is Siddon for both the ML-EM and the DIP reconstructions. To quantitatively compare both reconstructions, the convergence curves in Figure 9 plot the contrast recovery coefficient (CRC) against the normalized standard deviation (NSTD), which are calculated as:

$$\text{CRC}(\mathbf{x},{\mathbf{x}}^{\mathbf{GT}})=\frac{\frac{\mu ({\mathbf{x}}_{\mathbf{tgt}})}{\mu ({\mathbf{x}}_{\mathbf{bkg}})}-1}{\frac{\mu ({\mathbf{x}}_{\mathbf{tgt}}^{\mathbf{GT}})}{\mu ({\mathbf{x}}_{\mathbf{bkg}}^{\mathbf{GT}})}-1},$$ (15a)

$$\text{NSTD}(\mathbf{x})=\frac{\sigma ({\mathbf{x}}_{\mathbf{bkg}})}{\mu ({\mathbf{x}}_{\mathbf{bkg}})},$$ (15b)

where ${\mathbf{x}}^{\mathbf{GT}}$ is the ground truth image defined here as the last iteration of the full-count ML-EM reconstructed image. ${\mathbf{x}}_{\mathbf{bkg}}$ and ${\mathbf{x}}_{\mathbf{tgt}}$ are defined the same way as previously, namely as the white matter and gray matter voxels respectively. Figure 9 shows that the DIP algorithm reaches a lower level of noise than the low-count ML-EM for the same contrast recovery level.

Fig. 8.

Hoffman phantom reconstructions of a static acquisition from the GE Discovery MI scanner using (a) ML-EM with full count, (b) ML-EM with 1/40th of the count, and (c) DIP with 1/40th of the count. The magnified region of the red dotted square are shown in the bottom row. The image intensity of the full-count reconstruction was rescaled to match the intensity of the low-count reconstructions.

Fig. 9.

GE Discovery MI Hoffman phantom reconstruction convergence curves from the static acquisition. The corresponding reconstructed images from Figure 8 are indicated by an “X”. The full count ML-EM curve shows every 4 images from iteration 4 to 500. The low count ML-EM curve shows every 4 images from iteration 4 to 300. The low count DIP curve shows every 30 images from epochs 20 to 3200.

C. Limited-angle reconstruction from GATE simulations

In order to validate YRT-PET for a non-cylindrical scanner geometry, GATE simulations [24] of the XCAT phantom [59] were performed for a long axial field-of-view flat panel scanner under development [47], [48], [60].

The simulated geometry used two flat panel detectors of dimension 120 × 60 cm, composed of 3 × 3 × 20 mm crystals with a TOF resolution of 200 ps. GATE simulation results were converted to binary list-mode format (excluding randoms and scattered coincidences for simplicity) and reconstructed with ML-EM using CASToR v3.2.1 [14], PyTomography v3.4.0 [15], and YRT-PET v1.3.1 using one subset and 50 iterations with a voxel size of 3 × 3 × 3 mm³. In order to model the axial sensitivity of the scanner in reconstruction, normalization factors were estimated using a Cauchy distribution model, fitted to the overlap between LORs and detectors with respect to the axial LOR angle [61]. The normalization correction factors were stored in the CASToR histogram format and populated into the list-mode file. In YRT-PET, the same normalization factors were accumulated into a 3D histogram and used for the reconstruction. All reconstructions were performed on an AMD EPYC 7763 64-Core processor using 32 threads and an NVIDIA A100-SXM4-80GB GPU.

Figure 10 shows a comparison of reconstructed images using CASToR, PyTomography and YRT-PET. The voxel values were scaled to match the simulated activity in the liver region, with the same scaling factor applied to all the reconstructed images. Images as well as the line plots suggest that YRT-PET, PyTomography, and CASToR produce comparable images and are in reasonable agreement with the ground truth activity map.

Fig. 10.

Comparison of reconstructed images from GATE simulation of XCAT phantom with flat-panel scanner: (a) ground truth (Ref.), (b) CASToR reconstruction, (c) PyTomography reconstruction, (d) YRT-PET reconstruction, and (e) horizontal line profile through images.

To further compare the two reconstruction engines, the normalized root mean squared error (NRMSE) was calculated for each reconstructed image:

$$\text{NRMSE}(\mathbf{x},{\mathbf{x}}^{\mathbf{GT}})=\frac{\sqrt{\frac{1}{J}{\sum }_j{(x_j-x_j^{GT})}^2}}{\mu ({\mathbf{x}}^{\mathbf{GT}})},$$ (16)

where $\mathbf{x}$ is the reconstructed image, ${\mathbf{x}}^{\mathbf{GT}}$ is the ground truth and $\mu$ is a function calculating the average intensity. To compute the CRC, (15a) is applied with the target region ${\mathbf{x}}_{\mathbf{tgt}}$ placed inside the liver and the background region ${\mathbf{x}}_{\mathbf{bkg}}$ in a uniform soft-tissue region. Quantitative metrics are summarized in Table I and confirm the qualitative analysis, showing comparable image quality between the three engines. Note that the SSIM metric in Table I compares each reconstruction against the ground truth.

TABLE I.

Metrics comparison of the XCAT reconstructions obtained from CASToR and YRT-PET (shown in Fig. 10).

Metric	CASToR	Pytomo.	YRT-PET
CRC	0.919	0.925	0.926
SSIM	88.1%	88.4%	88.3%
NRMSE	24.3%	23.9%	24.1%

Fig. 11 illustrates a breakdown of the time taken by each reconstruction engine. The timing comparison was divided in two parts, the sensitivity image generation and the ML-EM reconstruction. To avoid bias, the timings exclude the time spent reading the input files.

Fig. 11.

Bar plot comparing timings from the tested open-source reconstruction tools. For YRT-PET and CASToR, both the Siddon and the distance-driven projector were used.

YRT-PET on GPU outperforms CASToR in speed due to its usage of GPU acceleration. However, the CPU implementation of YRT-PET is slower than CASToR, due to its ability to re-use the elements of the geometric system matrix between the forward projection in the backward projection [14] rather than computing them on-the-fly twice.

When compared to PyTomography, which uses the GPU-based projector from the PARALLELPROJ program [20], YRT-PET still results in significant speedup (around 10x). This large difference is due to the size of the data, requiring splitting of the list-mode data in batches and because PyTomography’s [15] architecture is built on top of PyTorch, which handles memory management internally. The transfer between the host and the device is managed by PyTorch and blocks the execution, resulting in a severe bottleneck. YRT-PET parallelizes the data loading step with the projection operation, which avoids this bottleneck and leads to faster reconstruction. Note that the performance of PyTomography and YRT-PET is compared for a smaller image reconstruction problem in Supplementary Material S.2, showing a more modest, but still substantial, improvement around 2x for distance-driven and 5x for Siddon.

V. Discussions

The presented image reconstruction software offers a unique set of features, with GPU-acceleration, Python bindings, event-by-event motion correction, etc. and has been validated against existing, both proprietary and open-source, image reconstruction software.

A speed comparison was performed against open-source reconstruction software. Although the CPU implementation of YRT-PET still under-performs compared to a state-of-the-art reconstruction engine like CASToR, the GPU implementation outperforms both CPU-only and GPU-accelerated engines. The suboptimal performance of YRT-PET on CPU can be overcome in future works by integrating further optimizations such as the re-use of the geometric system matrix elements between forward and backward projection. The implementation difference that explains YRT-PET’s performance gain over PyTomography is, at its core, one of design choices. PyTomography has a more high-level design, since it is a Python library that interfaces with PyTorch for the vector operations and PARALLELPROJ for the Joseph projector CUDA kernel. On the other hand, YRT-PET is a C++ program that directly calls every CUDA kernel necessary for the reconstruction. This lower-level design, albeit more difficult to maintain, allows for far more control and optimization opportunities. The data flow in YRT-PET, which overlaps GPU kernels and memory transfers, results in a significant reduction of run time. Newer generations of high resolution and high sensitivity scanners are expected to produce a large amount of data per scan. This means that a more granular memory management becomes vital to achieve higher performance.

However, in order to achieve quantitative accuracy in image reconstruction, i.e. recover the activity level in each voxel, calibration factors must be determined for a given scanner and imaging protocol. Since scanner-dependent operations are handled by plugins, estimation of these calibration factors for quantitative accuracy must be performed by plugin developers and users. YRT-PET also allows for a global scaling parameter that can be used to scale images and recover accurate activity estimates. Similarly, the estimation of randoms is scanner-specific and is typically managed within plugins.

YRT-PET offers a provisional non-TOF implementation of the SSS algorithm for scatter estimation, evaluated for the GE DMI PET/CT scanner in Supplementary material S.1. The current implementation, however, has not been fully validated and might be unsuitable for some scanner architectures, such as flat panel or helmet-shaped scanners. Additional options will be added to allow more control over the scatter estimation process, allowing users to optimize it for different use cases.

Future work will address some of the current limitations of YRT-PET, including scatter estimation and improving the GPU implementation. More specifically, further optimizations to batch loading, memory management, and projector kernels are planned to boost speed and scalability. Additional developments will also include support for regularly-spaced sinograms (optionally with compression) and sinogram-based image reconstruction (using iterative reconstruction or filtered back-projection), handling of large datasets through disk swapping, and support for image-domain non-rigid deformations.

VI. Conclusion

This paper introduces the YRT-PET software, an open-source image reconstruction engine for PET which includes support for event-by-event motion correction, TOF, DOI, PSF modeling, GPU acceleration, and a provisional scatter estimation module. The plugin system aims to facilitate the extension of the software to support more PET scanners and data formats. Evaluations against existing tools showed good quantitative agreement for dynamic reconstructions, as evidenced by the high correlation between DVRs estimated from reconstructions using the manufacturer’s reconstruction engine and YRT-PET (slope: 0.97, $R^2=0.99$). Moreover, a substantial acceleration compared to state-of-the-art reconstruction engines was achieved thanks to its optimized GPU-accelerated implementation. The Python interface enables the use of the internal data structures and operators within complex algorithms including advanced reconstruction frameworks and integration with deep learning models.

Supplementary Material

Supplementary_S2

Fig. S.2. Histogram-space display of (a) prompts, (b) randoms estimates, (c) scatter estimate from the manufacturer toolbox and (d) scatter estimate from YRT-PET. A line plot is shown in the bottom graph. The manufacturer scatter estimate is slightly above the proposed software, which explains the slightly higher contrast observed in the reconstructed images.

Supplementary_S1

Fig. S.1. Reconstructed images from acquisition of Hoffman phantom on GE DMI PET/CT scanner: (a) without scatter correction (No SC), (b) with scatter correction using the GE toolbox estimation (SC (GE)) and (c) with scatter correction using YRT-PET’s estimation (SC (YRT-PET)). (d) shows the line plot through the three images.