Deep unrolled regularization for limited-angle dual-panel positron emission mammography

Abstract

A dual-panel Positron Emission Mammography (PEM) scanner is a breast-dedicated device with better spatial resolution and sensitivity than conventional Positron Emission Tomography (PET). However, the main limitation of dual-panel PEM is the limited-angle artifacts in the cross-plane slices of the reconstructed images, arising from the acquisition geometry. This work proposes incorporating deep unrolled regularization into the image reconstruction algorithm to mitigate these artifacts. A U-Net was trained to identify and correct the artifacts in cross-plane slices. Dual-panel PEM and ring-shaped dedicated breast PET Monte Carlo (MC)-generated images served as the input and ground truth, respectively; the latter was selected because it provides an artifact-free reference due to its full-angle geometry. Data augmentation was employed to expand the training dataset to 3840 image pairs, thereby improving the model’s generalization performance. The trained deep learning model was incorporated into the Forward-Backward Splitting Expectation-Maximization algorithm to regularize reconstruction and ensure agreement between the reconstructed images and the measured data. Images reconstructed with the proposed image reconstruction framework exhibited effective mitigation of limited-angle artifacts using MC-generated and experimentally measured data from a dual-panel PEM. Additionally, this framework outperformed OSEM and MAPEM as measured by metrics quantifying noise, contrast, peak-to-valley ratio, recovery coefficients, spillover ratios, and intensity profiles. To the best of our knowledge, this is the first study to address limited-angle artifacts in a dual-panel PEM incorporating deep learning-based regularization. At this stage, the present work constitutes a proof-of-concept; for clinical implementation, a model trained on anthropomorphic phantoms and further validation are required.

Introduction

Breast cancer is a malignant disease characterized by out-of-control growth of abnormal cells in breast tissue that leads to tumor formation [1]. If left untreated, these tumors can metastasize to other parts of the body and become life-threatening [2]. Globally, female breast cancer is the most commonly diagnosed cancer, with approximately 2.3 million new cases and 0.67 million deaths reported in 2022 [3]. Early detection remains a priority in global health care for reducing breast cancer mortality [4].

Nuclear imaging techniques play a significant role in cancer diagnosis, and several breast-specific imaging systems have been developed, including scanners with ring-shaped, C-shaped, and dual-panel detectors [5]. Positron Emission Mammography (PEM) is a highly valuable imaging modality for diagnosing breast cancer, monitoring patients in follow-up evaluations, assessing therapeutic responses, and detecting disease recurrence [6]. The dual-panel PEM prototype was first introduced in the 1990 s by Thompson et al. [7] and remains a focal point of active research and clinical application [4, 8, 9]. Dedicated breast imaging devices have been developed to address the limited sensitivity and spatial resolution of conventional Positron Emission Tomography (PET) in organ-specific studies, enabling the detection of smaller breast lesions [10, 11]. The spatial resolution reported for PEM commercial scanners like the PEM Flex Solo II in the in-plane images (slices parallel to the panels) is 2.4 mm at Full Width Half Maximum (FWHM) [12, 13]. Additionally, the dual-panel PEM is more sensitive than PET due to its closer proximity to the radiation source. Consequently, the dedicated-organ design enables shorter image acquisition times and/or lower patient radiation dose [11]. Furthermore, the device’s compact design reduces costs compared with a conventional PET system. However, due to the scanner geometry, data acquisition is limited to certain solid-angle ranges, a condition known as limited-angle tomography. Strong asymmetric and anisotropic spatially-variant deformations in the cross-plane images (slices perpendicular to the panels) have been reported due to the limited-angle coverage in the dual-panel PEM [9]. For that reason, only the in-plane slices are used for clinical evaluation [12, 13].

In traditional tomography, measurements are available across a full range of angles. However, recent organ-dedicated techniques require image reconstruction in limited-data acquisition scenarios [14, 15]. Image reconstruction in such cases is challenging because the available data is incomplete, which causes standard analytic reconstruction algorithms to produce less accurate images and introduces directional and streaking artifacts [14, 16]. To better constrain the solution, regularization techniques have been incorporated into image reconstruction to mitigate those artifacts [16, 17].

Additionally, in recent years, deep learning algorithms have been successfully applied in medical imaging [18–20]. Various strategies have been proposed for deep learning-based image reconstruction in emission tomography, including direct mapping from noisy sinograms to high-quality reconstructed images, denoising of measured data, post-reconstruction processing [21], and unfolding iterative image reconstruction algorithms, with deep learning models used as a form of regularization [22, 23]. In particular, Raj et al. [9] trained a 3D U-Net on synthetically generated data to mitigate limited-angle distortions in a dual-panel PEM in a post-reconstruction stage. Similarly, a recent work by Nezampour et al. [24] uses MC-generated data for a dual-panel PEM and a ring-shaped PEM to train a deep learning model to post-process reconstructed images. However, the dual-panel PEM used in that work rotates the detector by in a double acquisition to compensate for limited-angle distortions. In contrast, our research focuses on the use of deep unrolled regularization for a dual-panel PEM with a single acquisition, since rotation for the same image is not possible due to breast compression [12].

Deep learning regularization preserves our knowledge of physics and statistics and applies deep learning where we are less certain: the regularization component [22]. The main advantage of deep learning regularization, as opposed to post-reconstruction algorithms, is that it demands agreement with measured data. It can either use a pre-trained network or train the network during image reconstruction. Recently, several deep learning-based algorithms have been implemented for regularization in PET image reconstruction, including the Maximum A Posteriori Estimation Expectation Maximization Net (MAPEM-Net) [25], Block Coordinate Descent Net (BCD-Net) [26], and Forward-Backward Splitting Expectation Maximization (FBSEM) [27].

In particular, Mehranian and Reader [27] proposed the FBSEM optimization algorithm [28] accelerated with ordered subsets (OS) for Bayesian PET reconstruction, which generalizes De Pierro’s Maximum A Posteriori Expectation Maximization (MAPEM) algorithm [29] and unrolls the algorithm with a recurrent neural net (RNN). The work of Mehranian and Reader [27] aimed to reduce noise in PET images reconstructed from low-dose measurements. The algorithm assigns low probability to image solutions exhibiting large local intensity differences, assuming that these differences are due to noise [27].

In this work, we propose deep unrolled regularization to mitigate limited-angle artifacts in a dual-panel PEM prototype. To this end, we have incorporated the FBSEM algorithm into the image reconstruction framework for a dual-panel PEM, using a U-Net model that was previously trained to mitigate limited-angle distortions. The U-Net was trained with Monte Carlo (MC) simulated cross-plane images from a dual-panel PEM scanner and a ring-shaped dedicated-breast Positron Emission Tomography (dbPET) scanner as input and ground truth, respectively. This approach was selected because the limited-angle artifacts manifest as deformations in the cross-plane images, a problem not encountered in a ring-shaped dbPET because of its angle coverage. This image reconstruction framework was successfully tested with both MC-generated and experimental data of a dual-panel PEM prototype.

Methods

Dual-panel PEM prototype

Experimental data were acquired using the low-cost dual-panel PEM prototype currently under development in the Laboratorio de Imágenes Biomédicas (LIBi) of the Instituto de Física (IF) of the Universidad Nacional Autónoma de México, UNAM [4]. Each panel of the PEM scanner is a square measuring 176 mm on each side, composed of an array of 3× 3 modules. Each detector module is a monolithic scintillation crystal of mm, made of lutetium-yttrium oxyorthosilicate (LYSO). The detectors are coupled to an 8× 8 array of position-sensitive silicon photomultipliers (SiPMs) via a 6 mm light guide. The energy resolution (17.67%) and dead time (0.5 μs) were experimentally measured for our detectors. The energy window selected for data acquisition was [430, 750] keV, and the time window was 6 ns. The prototype does not measure the Depth of Interaction (DOI) due to current electronics limitations in our system. Compression paddles cover the panels, which are mounted on a gantry. The separation, height, and angle of the panels can be adjusted with a motorized mechanism. In this device, data acquisition is done in list mode.

Phantoms

To train the deep learning model and to test the proposed image reconstruction framework, we used uniformity (U), spillover ratio (SOR), spherical lesions (Lesions), recovery coefficients (RC), and Derenzo-like (MiniDerenzo) phantoms for data acquisition. This set of phantoms was previously proposed by Torres et al. [13] to evaluate the performance of a dual-panel PEM scanner, which was adapted from the widely accepted NEMA standards [30–32]. A diagram of the phantoms is shown in Fig. 1, where the numbers indicate the heights and diameters in mm.

Fig. 1.

Diagram illustrating the dimensions of the Uniformity, Spillover Ratios, Spherical Lesions, Recovery Coefficients, and MiniDerenzo phantoms. The values indicated in the figure correspond to heights and diameters in mm. The PMMA cylindrical container is shown in cyan, the activity distribution in yellow, and the lesions in red. The SOR phantom additionally includes chambers filled with water and air that contain no activity

Two additional phantoms -one containing cold lesions and another featuring low-uptake lesions- were simulated using dimensions identical to the Lesions phantom depicted in Fig 1. In the cold-lesions configuration, no activity was simulated in the spherical compartments. In the low-uptake lesions phantom, lesion-to-background concentration ratios of 3:1, 4:1, 5:1, and 6:1, were considered for the spherical lesions with diameters of 10, 8, 6, and 4 mm, respectively. These phantoms were included to compare the DeepPEM framework performance against a post-reconstruction correction approach when applied to out-of-distribution test phantoms.

Monte Carlo simulations

Detailed simulations of a dual-panel PEM scanner and a ring-shaped dedicated breast PET (dbPET) scanner have been developed in the open-source MC simulation toolkit Geant4 Application for Tomographic Emission (GATE) [33], developed within the OpenGATE Collaboration based on GEANT4 [34]. Monte Carlo simulations using all phantoms described in “Phantoms” Section were performed to generate synthetic data for training and testing the deep learning model and the image reconstruction framework.

An in-house developed MC simulation of the dual-panel PEM scanner, based on the characteristics of the experimental PEM prototype, was used to generate our dataset [4]. The mispositioning due to Compton scattering in the monolithic crystals was incorporated in our simulation [4], which generated image blurring in the MC-simulated data, consistent with that observed in the experimental images. The separation between panels was maintained at 6 cm, which is reported as a moderate level of breast compression in PEM studies [35]. The data was acquired in list mode. A diagram of the dual-panel PEM scanner is shown in Fig. 2a.

Fig. 2.

Diagram of the a dual-panel PEM and b ring-shaped dbPET scanners simulated in GATE

The ring-shaped dbPET scanner was simulated using the same detector modules as those employed in the dual-panel PEM system and the same energy window, dead time, and time window. We simulate a cylindrical scanner with dimensions based on a device previously described in the literature [36]. The dbPET scanner was simulated with an inner diameter of 184.62 mm and a length of 117 mm. To achieve that, the modules were arranged in a 2× 10 array. Figure 2b illustrates a diagram of the simulated dbPET scanner.

Positron sources of were simulated for 600 s for each phantom in each scanner. The acquired data were obtained with the phantoms positioned at the center of each scanner. The activity concentration (ρ ) for the RC and MiniDerenzo phantoms was 20 kBq/ml, while the activity concentration for the U, SOR, and the background of the three Lesions phantoms was 5 kBq/ml, since the background activity concentration reported in clinical studies is around 3 to 5 kBq/ml [12, 36]. The activity concentration ratio between the spherical lesions and the background was set to 8:1. The coincidence counts recorded for the U, SOR, Lesions, RC, and MiniDerenzo phantoms were 18.6, 17.6, 19.4, 3.24, and 5.97 million, respectively.

Acquisition of experimental data

Experimental measurements were obtained using the dual-panel PEM prototype of our laboratory to validate our MC simulation and the performance of the proposed image reconstruction framework. The MiniDerenzo phantom was filled with water containing , and an acquisition of 8 minutes was performed. A total of 9.7 million coincidences were recorded for the image reconstruction.

OSEM

Image reconstruction in emission tomography involves estimating the spatiotemporal distribution of the radiotracer’s activity concentration within the scanner’s field of view (FOV). The scanner data acquisition is modeled according to Eq. (1).

1

where is the mean number of photons detected in each pair of detectors, is the matrix of the system, is the mean number of disintegrations in each voxel of the FOV, M is the number of pairs of detectors for coincidence, and N is the number of voxels in the reconstructed image.

The maximization of the likelihood of measuring when the emission map is leads to the iterative scheme of Eq. (2), which characterizes the Maximum Likelihood Expectation Maximization (MLEM) algorithm [37].

2

where is the k estimation of the image, represents the backprojection operator, is a vector of ones for list-mode data and is the backprojection of a vector of ones, which corresponds to the sensitivity or normalization matrix.

The convergence of the MLEM algorithm can be accelerated by partitioning the measured data into several disjoint subsets, which produces the ordered subsets expectation maximization (OSEM) algorithm [38]. After cycling through all subsets, one complete OSEM iteration is completed.

MAPEM

The introduction of a ‘prior knowledge’ as a constraint that may favor convergence of the expectation maximization (EM) process is called regularization [39]. The prior is based on an assumption about the expected true image, typically chosen to penalize noisy images. A common Bayesian prior to enforce local smoothness is the Gibbs prior, given by , where U is an energy function and β is the weight of the prior that modulates the strength of regularization, and C is a normalizing constant. This prior modifies the likelihood function and its maximization leads to the iterative solution shown in Eq. (3) (see Bruyant et al [39] for further details on the derivation).

3

For our implementation of MAPEM, we use a quadratic prior that favors smooth solutions [39] and is defined in Eq. (4).

4

where is a set of voxels in the neighborhood of voxel j and is a weighting factor.

For , we considered the closest 26 neighbors of voxel j: 6 face-adjacent, 12 edge-adjacent, and 8 vertex-adjacent voxels. The distances between neighboring voxels were defined according to their connectivity: for face-adjacent voxels, for edge-adjacent voxels, and for vertex-adjacent voxels. Consequently, we define a weighting factor based on distance, as in Eq. (5).

5

where is either , or depending on the adjacent voxel.

DeepPEM

In this study, we propose integrating deep learning-based regularization into the LM-OSEM image reconstruction algorithm. The regularization objective is specifically designed to mitigate the limited-angle distortions of the dual-panel PEM scanner. Our implementation builds on the FBSEM algorithm [27] and requires a previously trained deep learning model. Hereafter, this integrated reconstruction framework is designated as DeepPEM.

The regularization approach based on the FBSEM algorithm requires unrolling the iterative image reconstruction algorithm. It involves three stages: regularization of the previous image estimate () that gives , EM update that gives , and the fusion of these two images according to (6) [27], which updates .

6

where γ is the regularization constant which controls the strength of regularization, which is determined empirically, with smaller values indicating an increased regularization [27].

The DeepPEM framework is summarized in Algorithm 1, where is the number of iterations, is the number of subsets, is the set of combinations of iteration, subiteration (i, j), in which regularization is applied and denotes the inference performed by the trained deep learning model on . The training procedure and inference methodology are described in detail in this subsection.

Algorithm 1.

DeepPEM algorithm

For the regularization stage, a U-Net [40] was implemented with six levels and three convolutions per block, followed by a ReLU activation function. The U-Net was implemented using the ADAM optimizer [41], a learning rate of , a batch size of 18, a Mean Absolute Error (MAE) loss function, and trained for 200 epochs. Figure 3 shows the architecture of the deep learning model.

Fig. 3.

Diagram of the U-Net trained to mitigate limited-angle artifacts in PEM [42]

To train the U-Net, a dataset of 15 pairs of 3D images was employed. The images were generated from reconstructions of MC-simulated data for both the dual-panel PEM and dbPET scanners. The dataset comprised 3 image pairs reconstructed with the LM-OSEM algorithm for each phantom of Fig. 1. The first pair consists of the image of the first iteration of the dual-panel PEM scanner and the image of the last iteration of the dbPET scanner; the second pair consists of the image of the second iteration of the dual-panel PEM scanner and the image of the last iteration of the dbPET scanner; and the third pair consists of the image of the last iteration of both scanners. This structure captured variations across iterations and noise levels, yielding improved results when applied to experimental data. The 3D images had voxels, with a voxel size of mm. For the training stage, the stacks were padded to voxels with linear ramp mode.

Each of the 15 pairs of 3D stacks yielded 128 pairs of 2D images of 64× 64 pixels. The images were extracted from the cross-planes; half of them were obtained from the coronal plane, while the other half were obtained from the sagittal plane. Consequently, the new dataset comprised 1920 image pairs.

To increase the diversity of the training data and to improve the generalization and performance of the deep learning model, data augmentation strategies were applied. Each image in the original dataset underwent random data augmentation operations. These operations included horizontal and vertical shifts of up to 20%, shear transformations of up to 20%, zooming in and out by up to 20%, and horizontal flipping. To address missing pixels resulting from these transformations, nearest-neighbor interpolation was used. Finally, the original dataset was concatenated with the augmented dataset, yielding a total of 3840 image pairs.

This dataset, composed of two-dimensional slices, was partitioned into training, validation, and testing subsets, with 70%, 15%, and 15% of the image pairs allocated to each set, respectively. The training was performed using cross-plane slices because limited-angle artifacts are more pronounced in these slices. The cross-plane images of the dual-panel PEM and dbPET were selected as input and ground truth, respectively. Images reconstructed with a ring-shaped dbPET scanner served as a reference free of limited-angle artifacts because this geometry enables full-angle acquisition.

When regularization was applied, the trained U-Net was applied to coronal and sagittal slices of the previous image estimation , and both outputs were averaged to avoid the formation of horizontal or vertical streak artifacts in the in-plane images. The U-Net output was combined with the current EM update , applying (6) [27]. The diagram of Fig. 4 illustrates the incorporation of regularization into a given subset and iteration of the OSEM algorithm.

Fig. 4.

Unfolding of the iterative image reconstruction algorithm with deep learning-based regularization

Image reconstruction parameters

Image reconstruction was performed using an in-house Python implementation of the image reconstruction algorithms. The projection and backprojection operators were implemented with the low-level operators of the Parallelproj library [43], which are based on the Joseph algorithm [44]. For normalization, an acquisition with a flood source located at the center of the scanner, aligned parallel to the detector panels, was performed for both the experimental and MC-simulated data. The normalization image was obtained by a single backprojection of the data onto the FOV [4, 45].

Since our prototype does not measure the depth of interaction (DOI), the z-positions of the photon detections were randomly sampled from a probability density function estimated from our MC simulations. The lack of DOI measurement in a dual-panel PEM has been reported to affect only the cross-plane spatial resolution [46], which is the primary focus of improvement in this work.

The Field of View (FOV) was restricted to mm, and the reconstructed images had a voxel size of mm. The image reconstruction algorithm implemented for this research does not include scatter, attenuation, or random corrections.

Evaluation of the reconstructed images

The images reconstructed using the proposed DeepPEM framework were compared with those reconstructed using MAPEM and OSEM. For each algorithm, performance across several hyperparameter settings was evaluated to select an optimal configuration that balanced multiple image quality metrics across all phantoms.

For the OSEM algorithm, the hyperparameters considered were the number of iterations and the number of subsets. For the MAPEM algorithm, we evaluated the reconstructed images as a function of the regularization strength β . For the DeepPEM image reconstruction framework, we evaluated the reconstructed images as a function of the regularization strength, γ , and of the combination of iterations/subgroups on which regularization is applied.

To select the optimal hyperparameters and compare the three algorithms using the top-performing settings, we used the image-quality metrics recommended for evaluating PEM scanner performance [13]. Additionally, in this work, we compare intensity profiles of cross-plane and in-plane slices to highlight the mitigation of limited-angle artifacts with the proposed DeepPEM image reconstruction framework.

Results

MC-generated data

For images reconstructed with the OSEM algorithm, optimal fidelity relative to the MC-source positions was achieved across all phantoms with iterations and subsets. Therefore, we kept the number of subsets and iterations the same for both MC-generated and experimental data across all algorithms. In the case of MAPEM, an optimal balance between noise suppression and spatial resolution preservation was achieved with β = 0.05 for MC-generated data and β = 0.20 for experimental data. For the DeepPEM framework, the optimal balance between enhancing the image quality of cross-plane slices while preserving that of in-plane slices was achieved by applying regularization to all subsets in the last iteration, with γ =2.0 for MC-generated data and γ =5.0 for experimental data.

Figure 5 compares in-plane and cross-plane images from MC-generated data of the phantoms of Fig. 1, reconstructed with OSEM, MAPEM, and DeepPEM. The first three columns illustrate the in-plane slices, whereas the subsequent three columns display the cross-plane slices. The dotted lines on the in-plane slices denote the position of the corresponding cross-plane slices, while the dotted lines on the cross-plane slices indicate the location of the corresponding in-plane slices. Based on the detection efficiency of our prototype, as determined by the MC simulation, the activity concentration was estimated in kBq/ml and displayed on the colorbar.

Fig. 5.

Comparison of MC-generated images for the phantoms of Fig. 1, reconstructed with OSEM, MAPEM, and DeepPEM. The first three columns illustrate the in-plane slices, whereas the subsequent three columns display the cross-plane slices. The dotted lines on the in-plane slices denote the position of the corresponding cross-plane slices, while the dotted lines on the cross-plane slices indicate the location of the corresponding in-plane slices. The colorbar indicates the estimate of activity concentration in kBq/ml

The uniformity phantom enables quantification of noise in the reconstructed images. Following the protocol proposed by Torres et al. [13], it can be calculated as the percentage standard deviation (%STD) measured in a cylindrical volume of interest (VOI) concentric to the source. For the images reconstructed with OSEM, we measure a . The use of MAPEM, an image reconstruction algorithm that incorporates regularization, effectively reduced the noise to . Reconstructions with the DeepPEM framework further reduced the noise to . For comparison, a value of was reported for images reconstructed using the commercial scanners PEM Flex Solo II with a higher activity concentration and larger voxel size [13]. Additionally, since the uniformity phantom is a cylindrical source, a rectangular shape is expected in a cross-plane slice. However, severe distortions are evident in images reconstructed with OSEM and even in those reconstructed with MAPEM. Incorporating deep learning-based regularization effectively mitigates these artifacts.

The reconstructed images of the SOR phantom show that limited-angle distortions are more pronounced near the scanner panels. In this phantom, the cylindrical air and water chambers without activity are fully contained inside the cylindrical source. Therefore, in a cross-plane slice, portions of the cylindrical source are expected to be identified above and below the hollow chambers. However, in the images reconstructed with the OSEM and MAPEM algorithms, this portion of the source is severely distorted. The inclusion of deep learning-based regularization yields images in which the borders of the imaged source are well-defined. This improvement can be quantified by measuring the SOR in each chamber following the protocol proposed by Torres et al. [13]. This metric measures the ratio of activity detected in the cold chambers to that in the uniform section of the phantom. These values are shown in the upper section of Table 1, where values closer to zero correspond to improved performance, as both chambers are expected to exhibit no activity. For the DeepPEM image reconstruction framework, a slightly better performance is identified.

Table 1.

Spillover ratio values for the water and air chamber of the SOR phantom and Recovery Coefficient values of each cylindrical source of the RC phantom for each image reconstruction algorithm

Algorithm	OSEM	MAPEM	DeepPEM
Chamber	SOR
Water	0.38 ± 0.07	0.38 ± 0.06	0.36 ± 0.03
Air	0.49 ± 0.12	0.50 ± 0.11	0.42 ± 0.03
ϕ (mm)	RC
1.5	0.13 ± 0.03	0.12 ± 0.01	0.14 ± 0.06
2.0	0.19 ± 0.03	0.18 ± 0.02	0.23 ± 0.09
3.0	0.34 ± 0.06	0.29 ± 0.04	0.71 ± 0.28
4.0	0.43 ± 0.06	0.38 ± 0.04	1.07 ± 0.31
5.0	0.50 ± 0.07	0.46 ± 0.06	0.94 ± 0.21
7.0	0.60 ± 0.08	0.54 ± 0.07	0.95 ± 0.14
9.0	0.64 ± 0.08	0.58 ± 0.07	0.96 ± 0.13

The spherical-lesion phantom simulates four spherical lesions on a warm background. In the images reconstructed with OSEM and MAPEM, the shape of the cylindrical warm background is affected in a manner similar to that of the uniformity phantom. This distortion is mitigated only by incorporating deep learning-based regularization into the image reconstruction algorithm. Additionally, the spherical sources exhibit a more well-defined shape when reconstructed with the DeepPEM image reconstruction framework. Figure 6 depicts a comparison of vertical intensity profiles of cross-plane images for the Lesions phantom, reconstructed with OSEM, MAPEM, and DeepPEM. Although the ideal activity concentration (green curve) is not fully recovered due to intrinsic limitations of the image reconstruction process, DeepPEM yields the closest estimate to it.

Fig. 6.

Comparison of vertical intensity profiles of cross-plane images for the Lesions phantom, reconstructed with OSEM, MAPEM, and DeepPEM. For reference, the ideal activity concentration is shown in green. White dotted lines on the images indicate the locations from which the intensity profiles were extracted

In the Lesions phantom, lesion contrast can be evaluated following the protocol proposed by Torres et al. [13]. In particular, the analysis in this work focuses on the contrast of the smallest lesion. For the image reconstructed using the OSEM algorithm, the contrast is . The use of MAPEM deteriorated the contrast, yielding a value of . In contrast, with DeepPEM, the contrast increased significantly, reaching . For comparison, in the commercial scanner PEM Flex Solo II, the contrast reported was [13]. Even though they use a higher activity concentration, their contrast is limited by the large slice thickness.

The cylindrical sources of the RC phantom exhibit distortions, which are evident in cross-plane slices of images reconstructed with the OSEM and MAPEM algorithms. Performing image reconstruction with DeepPEM yields images with sharper edges, reduced noise, and improved visibility of all resolved sources within the in-plane slice. Additionally, this improvement was quantified by measuring the RC according to the protocol proposed by Torres et al. [13]. This metric quantifies the ratio of the measured activity in each cylindrical source to the expected activity. The lower section of Table 1 reports the measured values for each cylindrical source with diameter ϕ , for images reconstructed with each algorithm. Ideally, these values should be 1; however, they deviate further from 1 for smaller sources due to the partial volume effect. We highlight an improvement in the RC for cylindrical sources reconstructed with DeepPEM, particularly those with diameters of 3 mm or greater.

The reconstructed images of the MiniDerenzo indicate that incorporating deep learning regularization improves spatial resolution. Spatial resolution can be quantified with the peak-to-valley ratio (PVR). Torres et al. [13] proposed to identify the rod sources with the smallest diameter that fulfill the criterion PVR≥ 1.5. The pie with cylindrical sources of diameter 2 mm is barely resolved in the OSEM-reconstructed images, with PVR=1.6± 0.1. Performing the image reconstruction with MAPEM reduces the PVR to 1.4± 0.1. In contrast, this pie is clearly resolved when deep learning-based regularization is incorporated into the image reconstruction algorithm (PVR=4.0± 0.5). For comparison, those sources are not resolved in the commercial scanner PEM Flex Solo II [13].

Experimental acquired data

Figure 7 compares the in-plane slices (top), cross-plane slices (middle), and vertical intensity profiles (bottom) for images of the MiniDerenzo phantom, reconstructed with OSEM, MAPEM, and DeepPEM. A notable improvement is observed in the quality of the cross-plane slices of the images reconstructed with DeepPEM. Furthermore, even in the in-plane images, an enhancement in spatial resolution is evident. The improvement in spatial resolution is quantified below using the PVR. The comparison of vertical intensity profiles from cross-plane images shows an important improvement in the edges of the cylindrical source in images reconstructed with DeepPEM, compared with those reconstructed with MAPEM and OSEM. Similar to Fig. 6, the ideal activity concentration (green curve) is not fully recovered; notably, DeepPEM again provides the closest estimate.

Fig. 7.

Comparison of in-plane images (top), cross-plane images (middle), and vertical intensity profiles (bottom) of the MiniDerenzo phantom (experimental data), reconstructed with OSEM, MAPEM, and DeepPEM. The dotted lines on the in-plane slices denote the position of the corresponding cross-plane slices, and the dotted lines on the cross-plane slices indicate the location of the corresponding vertical intensity profiles

Figure 8 depicts a comparison of intensity profiles of the in-plane images for the MiniDerenzo phantom, reconstructed with OSEM, MAPEM, and DeepPEM. This profile focuses on the pie of cylindrical sources with a diameter of 2 mm. The PVR for this pie, when reconstructed with the OSEM algorithm, is 2.0± 0.1. Performing image reconstruction with MAPEM affects spatial resolution, as it tends to favor smooth images, and the PVR of the analyzed pie was measured to be 1.3± 0.1. In contrast, incorporating deep learning-based regularization not only favors smooth images but also improves spatial resolution, as evidenced by a PVR of 4.2± 0.3 for this pie. The image shown in the inset of Fig. 8 is the average of the slices of the image reconstructed with DeepPEM.

Fig. 8.

Comparison of intensity profiles of in-plane images for the MiniDerenzo phantom, reconstructed with OSEM, MAPEM, and DeepPEM. The white line in the inset indicates the locations from which the intensity profiles were extracted

Discussion

In the MC-simulated data, the number of random events was analyzed. The highest count of random events was observed in the spherical lesions phantom, accounting for 0.24% of the coincidences. Furthermore, MC simulations were used to estimate the number of random coincidences expected in our experimental data acquisition, yielding a value of 0.18%. Therefore, random events at the reported activity concentrations are negligible.

The low-energy window (LEW) at 430 keV was selected to minimize scatter radiation while maintaining a reasonable percentage of true coincidences, as determined by our MC simulations. For the data of the MC-simulated phantoms, the scatter fraction (SF) with the selected LEW is 12.6%, while if a standard LEW of 350 keV had been used, we would have expected a SF=21.5%. However, our selected LEW rejected 5.5% of true coincidences that would have been accepted by a standard LEW. It is important to note that these values are primarily determined by the detectors’ energy resolution.

Attenuation correction is particularly challenging in dedicated devices that lack a transmission acquisition mechanism. Additionally, for list-mode data in limited-angle tomography, complete sinograms cannot be constructed. Indeed, the image reconstruction in commercial scanners such as the PEM Flex Solo II does not incorporate attenuation and scatter corrections [12, 13]; therefore, PEM is considered a semi-quantitative technique [12, 47]. According to our MC simulation, approximately 43.8% of true coincidences are lost due to attenuation within the phantoms. The relatively small diameter of the phantoms reduces the likelihood of cupping artifacts arising from the absence of attenuation correction, as observed in the uniformity phantom shown in Fig. 5. The use of relatively small phantoms mitigates attenuation and scatter, allowing the proposed DeepPEM image reconstruction framework to focus on limited-angle artifacts. Future work will address the implementation of attenuation and scatter correction methods for our dual-panel PEM prototype.

For the commercial scanner PEM Flex Solo II, the pixel size is 1.2 mm, with 12 slices (slice width of 5 mm for a separation of 60 mm between the panels) [12, 13]. A smaller voxel size is required for better spatial resolution, particularly in cross-plane slices, but it also increases image noise. Notably, the proposed DeepPEM image reconstruction framework yields less noisy images even with smaller voxel sizes and lower activity concentrations.

For OSEM and MAPEM, the processing time is approximately 0.2 seconds per million coincidences per iteration on a Workstation with 64 GB of RAM. Applying deep learning-based regularization in a Workstation with an NVIDIA RTX A5000 GPU with 24 GB of RAM takes approximately 2 s per each subset in which regularization is applied, in addition to the time for the EM iterations. Therefore, this framework can reconstruct complete studies in less than 30 s.

Reconstructing images using regularization according to the MAPEM algorithm reduces noise but affects spatial resolution and does not mitigate limited-angle artifacts. In contrast, the proposed DeepPEM framework effectively mitigated limited-angle artifacts, without introducing artifacts in the in-plane slices. Therefore, images reconstructed with DeepPEM can be used effectively as three-dimensional objects without requiring multiple acquisitions to evaluate different planes, while preserving the spatial resolution advantages of a breast-dedicated scanner. The improved image quality reported in this work has the potential to be translated into reduced scan times and/or decreased radiation exposure for patients. Furthermore, this approach provides an elegant framework for implementing a model-based image reconstruction framework with learned priors. In this manner, the progress and development of image reconstruction algorithms are maintained, while exploiting the data-analysis capabilities of deep learning, thereby providing a data-driven approach in the regularization stage [22]. The integration of deep learning as a regularization strategy, rather than as post-reconstruction processing, has the advantage of enforcing agreement with the measured data. Although hallucinations are a major concern when integrating AI into medical imaging, using a deep learning model alongside the image reconstruction algorithm reduces their likelihood.

Indeed, Fig. 9 compares in-plane and cross-plane slices from simulated data for two phantoms that differ from those used during training, considering images reconstructed with OSEM, corrected post-reconstruction (Post Rec.), and DeepPEM. In this comparison, the same deep learning model employed in DeepPEM is used to emphasize DeepPEM’s advantages over the post-reconstruction approach. For the data used during training (i.e., the phantoms shown in Fig. 1), post-reconstruction correction with the trained deep learning model achieved metrics comparable to those obtained with DeepPEM, consistent with previous reports from the literature [22]. However, when the model is tested on activity distributions that differ from those encountered during training and do not correspond to scenarios considered in the data augmentation process, hallucinations or artifacts may arise.

Fig. 9.

Comparison of in-plane and cross-plane slices from simulated data for two phantoms that differ from those used during training, considering images reconstructed with OSEM, corrected post-reconstruction (Post rec.), and DeepPEM. The white arrows highlight artifacts observed during post-reconstruction correction, which are mitigated by applying DeepPEM

A key advantage of DeepPEM is that the regularization strength can be controlled via the empirically determined parameter γ , thereby avoiding over-regularization. In particular, using a reduced regularization strength (γ =10) results in a considerable mitigation of limited-angle artifacts without inducing hallucinations, as shown in Fig 9. Particularly, for the cold-lesions phantom, the spherical chambers are distorted when post-reconstruction corrections are applied since only cylindrical chambers were encountered during training. For the low-uptake lesions phantom, artificial cold and hot horizontal and vertical line artifacts are generated in the background, as the model was trained with different lesion-to-background activity concentration ratios. For these lesions, the contrast was measured following the protocol proposed by Torres et al. [13]. For the smaller lesion, incorporating deep unrolled regularization increased the contrast from (23.31± 7.83) % to (45.77± 3.37) %. While this represents a significant enhancement, the magnitude of improvement was less pronounced than that observed for lesions with uptake levels consistent with the training dataset. Therefore, for scenarios not substantially different from those encountered during training, DeepPEM’s flexibility to control regularization strength yields robust mitigation of limited-angle artifacts without introducing hallucinations or other image artifacts and without requiring retraining of the deep learning model.

The improvement achieved by incorporating deep unrolled regularization for spatial resolution and cross-plane lesion detectability is presented in Appendix A. We found that lesions of 6 mm in diameter, with an 8 mm edge-to-edge separation, are barely resolved for images reconstructed with OSEM. In contrast, with DeepPEM, those lesions are better resolved (see Fig. 10). Additionally, due to limited-angle artifacts, a dual-panel PEM has reduced spatial resolution in cross-plane slices, as shown in Table 2. We note that these values are slightly better than those reported for the PEM Flex Solo II [13], and the in-plane spatial resolution is consistent with the MiniDerenzo reconstructions (see Figs. 5,  7, and  8). Regardless of the FOV location, DeepPEM improves spatial resolution in both cross-plane and in-plane slices. With a deep learning model trained on data with lesions of different sizes, shapes, intensities, and locations within the FOV, greater improvements in spatial resolution and lesion detectability are expected.

Fig. 10.

Comparison of cross-plane lesion detectability with OSEM and DeepPEM for two spherical sources of 6 mm in diameter, with an edge-to-edge separation of 8 mm

Table 2.

Cross-plane (c-p) and in-plane (i-p) spatial resolution as a function of FOV location, characterized by the FWHM of the PSF

FOV location	OSEM		DeepPEM
	FWHMc-p (mm)	FWHMi-p (mm)	FWHMc-p (mm)	FWHMi-p (mm)
Center	6.46 ± 0.03	2.02 ± 0.04	4.23 ± 0.19	1.56 ± 0.06
Side	6.46 ± 0.05	2.10 ± 0.03	6.22 ± 0.18	1.51 ± 0.07
Up	10.57 ± 0.83	3.55 ± 0.07	5.33 ± 0.13	1.22 ± 0.05
Corner	9.39 ± 0.63	2.45 ± 0.03	5.30 ± 0.18	1.25 ± 0.05

Limited-angle artifacts can be mitigated by incorporating time-of-flight (TOF) information into the image reconstruction algorithm [24, 48, 49]. However, that requires state-of-the-art TOF detectors due to the small FOV characteristic of dedicated breast PET systems. In contrast, the proposed approach does not rely on state-of-the-art TOF detectors for image reconstruction.

We recognize that the shapes of image-quality phantoms differ from the expected clinical situations. However, these were selected to test the proposed framework with phantoms and methods already used to evaluate image-quality metrics, as reported in the literature. Additionally, we propose ground-truth images from a ring-shaped breast scanner because, if access to both scanners of this type is available, the deep learning model could be trained on experimental data. Once the deep learning model is trained, it can be applied to reconstruct images from MC-generated or experimental data for phantoms with shapes similar to those used during training. Therefore, to apply this method to real patient data, the deep learning model must be trained and tested on a dataset comprising anthropomorphic breast phantoms that incorporate anatomical details, clinically derived uptake patterns, and lesions with varying shapes, intensities, volumes, and spatial locations within the FOV; this work is currently in progress. We also emphasize that these limitations are primarily associated with the trained deep learning model rather than with the image reconstruction framework itself, which constitutes the central focus of the present work.

Several studies [24, 50] have employed deep learning as a post-reconstruction strategy in dual-panel PEM devices. However, in such studies, acquisition comprises two discrete detector poses separated by , thereby increasing angular sampling. In contrast, the image reconstruction framework proposed in our work considers a dual-panel PEM operating with a single acquisition, corresponding to a severely limited-angle situation. Additionally, the work of Nezampour et al. [24] simulates TOF resolution, which mitigates further limited-angle artifacts.

The use of a conventional U-Net in this research was preferred over a 3D U-Net, such as the one reported in a related research [9], due to the limited amount of our training data. This limitation primarily arose from our training dataset being MC-generated, a computationally expensive process. Under these conditions, the training and validation of a conventional U-Net is faster and less computationally demanding than for a 3D U-Net.

The PEM prototype in our laboratory is currently under development; therefore, the quality of experimental data in future acquisitions is expected to improve. As this research progresses, both experimental and simulated images of anthropomorphic breast phantoms designed to capture expected breast anatomy will be acquired. This will enable the deep learning model to be trained on more realistic scenarios, thereby facilitating future clinical application of the proposed image reconstruction framework.

At this stage, the significance of this research lies in demonstrating a proof-of-concept for the improvements achieved by incorporating deep learning-based regularization into the image reconstruction algorithm of a dual-panel PEM system without the need for multiple acquisitions [24, 51] or detectors with sufficiently high performance to enable TOF measurements [24, 48, 49]. To the best of our knowledge, the integration of deep learning-based regularization into the image reconstruction process to mitigate limited-angle problems -ensuring consistency between the reconstructed image and the measured data- is the main novel aspect that has not been previously reported in the literature within the context of PEM imaging.

Finally, it is worth noting that the results obtained for the limited-angle tomography problem with deep learning-based regularization are not limited to the dual-panel PEM scanner. This approach may be extended to other dual-head devices, such as flat-panel total-body PET [52, 53] or PET monitoring devices in hadron therapy [54].

Conclusion

In summary, this work introduces a deep learning-based regularization approach integrated into the image reconstruction algorithm, serving as a proof-of-concept to mitigate limited-angle artifacts in a dual-panel PEM scanner. The developed framework, DeepPEM, was trained on MC-generated data from image-quality phantoms under simplified reconstruction conditions and subsequently tested on both MC-generated and experimentally acquired datasets. This methodology enabled the reconstruction of PEM images as authentic three-dimensional volumes. Consequently, DeepPEM offers the potential to enhance image quality, reduce acquisition times, and/or lower radiation doses for patients. To ensure suitability for clinical implementation, training the deep learning model on clinically derived anthropomorphic phantoms, incorporating attenuation and scatter correction, and further validation are required.

Apeendix A. Cross-plane lesion detectability and spatial resolution

The detectability of lesions in the cross-plane was investigated by simulating a cylindrical phantom with two 6 mm in diameter spherical lesions along the z-axis with an edge-to-edge separation of 8 mm, and a lesion-to-background ratio of 8:1 (see Fig. 10).

In-plane and cross-plane spatial resolution were characterized by the full width at half maximum (FWHM) of the point spread function (PSF). To this end, several point sources were simulated at multiple locations within the FOV. Subsequently, we compare the spatial resolution for images reconstructed with OSEM and DeepPEM, as shown in Table 2.

Author contributions

F. M. and A. M. contributed to the study conception and design. F. M. implemented the deep learning models. H. A. and M. R. lead the experimental data acquisition. The first draft of the manuscript was written by F. M., and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Funding

The authors declare that no funds, grants, or other support were received during the preparation of this manuscript.

Data availability

MC-simulated and experimentally acquired data sets from the current study are available from the corresponding author upon reasonable request.

Declarations

Conflict of interest

The authors declare no conflict of interest.

Ethical approval

This is a computer simulation study validated with laboratory measurements using medical imaging phantoms. The Research Ethics Committee of Instituto de Física, UNAM, has confirmed that no ethical approval is required.