Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2020 Oct 30.
Published in final edited form as: IEEE Trans Radiat Plasma Med Sci. 2018 Nov 30;3(4):400–409. doi: 10.1109/trpms.2018.2884176

Effect of PET-MR Inconsistency in the Kernel Image Reconstruction Method

Daniel Deidda 1, Nicolas Karakatsanis 2, Philip M Robson 3, Nikos Efthimiou 4, Zahi A Fayad 5, Robert G Aykroyd 6, Charalampos Tsoumpas 7
PMCID: PMC7596768  NIHMSID: NIHMS1533416  PMID: 33134651

Abstract

Anatomically-driven image reconstruction algorithms have become very popular in positron emission tomography (PET) where they have demonstrated improved image resolution and quantification. This work, consider the effect of spatial inconsistency between MR and PET images in hot and cold regions of the PET image. We investigate these effects on the kernel method from machine learning, in particular, the hybrid kernelized expectation maximization (HKEM). These were applied to Jaszczak phantom and patient data acquired with the Biograph Siemens mMR. The results show that even a small shift can cause a significant change in activity concentration. In general, the PET-MR inconsistencies can induce the partial volume effect, more specifically the ’spill-in’ of the affected cold regions and the ’spill-out’ from the hot regions. The maximum change was about 100% for the cold region and 10% for the hot lesion using KEM, against the 37% and 8% obtained with HKEM. The findings of this work suggest that including PET information in the kernel enhances the flexibility of the reconstruction in case of spatial inconsistency. Nevertheless, accurate registration and choice of the appropriate MR image for the creation of the kernel is essential to avoid artifacts, blurring, and bias.

Keywords: positron emission tomography (PET), iterative reconstruction, kernel method, image prior, expectation maximization (EM), anatomically-driven, hybrid kernel

I. Introduction

Machine learning techniques are being frequently exploited for positron emission tomography (PET) image reconstruction [1] Recently, techniques for PET image de-noising involving Neural network approaches have been proposed [2, 3], as well as deep learning [4] and support vector machine (SVM) [5] techniques. The latter with the kernel method has been frequently used to include anatomical information in the reconstruction [6]-[9].

Anatomically-driven algorithms have become popular in PET. The rationale supporting the use of these techniques is based on the assumption that the radiotracer uptake distribution follows specific anatomical patterns. As a consequence, the anatomical information from MR can be used as prior knowledge. Several studies have recently implemented and investigated different anatomically-driven techniques, showing that they can suppress noise while preserving resolution that is usually lost through the regularization process. There are two different ways of including anatomical information in the reconstruction algorithm: Bayesian techniques and the kernel method. Bayesian techniques are also subdivided into segmentation-based [10]-[12] and segmentation-free techniques [13]-[24], the latter avoids the potential error due to segmentation. The kernel method can be divided into hybrid [7, 8], where the kernel matrix is extracted from more than one source (for example, PET and MR) and non-hybrid [5, 6, 9, 25, 26] , where the kernel is estimated from one source.

In this work we focus on the kernel method, the aforementioned techniques are based on the kernel method, commonly used in machine learning and described in [27]. The method was first introduced in PET image reconstruction by Hutchcroft et al [6] and Wang and Qi [5] using one source of information for the kernel matrix, MR and PET respectively. Novosad and Reader [9] combined the method with temporal basis functions in order to perform full dynamic PET reconstruction. Ellis and Reader [26] proposed the use of kernelized expectation maximization (KEM) in the context of dual-dataset longitudinal PET studies, where a baseline scan reconstruction was used to define basis functions for a follow-up scan reconstruction. Gong et al [7] used a hybrid kernel method to perform direct Patlak reconstruction from dynamic PET using MR and PET information where the latter was obtained by combining different frames. Bland et al (2017) [28] studied the effect of KEM on simulated dose-reduced datasets, showing improved contrast to noise ratio, but at the cost of possible over-smoothing of features unique to the PET data. To overcome this issue Bland et al (2018) [29] proposed a method using a spatially constrained MR kernel in order to maintain the noise reduction properties of the conventional kernel method, whilst better retaining the features unique to the PET data. Spencer and Wang [30] proposed a dual-kernel approach for dynamic PET image reconstruction that combines the existing non-local kernel with a local convolution kernel and demonstrated, with simulated brain dynamic data, higher image quality than the standard single-kernel reconstruction approach.

Deidda et al [8] presented a list-mode hybrid kernelized OSEM (LM-HKEM) that does not require preliminary PET image reconstruction to create a PET kernel matrix by exploiting the iterative process to extract the PET information. In addition, both the proposed KEM and HKEM use a voxel-wise and spatially restricted kernel rather than a patch-wise one. The two techniques were evaluated for different count levels but with a focus on low count conditions corresponding to short acquisition times. The aim of the study was to improve quantification in terms of bias and contrast recovery coefficient (CRC) for the carotid arteries. The HKEM was also applied for the estimation of the image-derived input function from the aorta of rabbit models [31]

In this paper, we investigate the limitation of the anatomically-driven kernel methods in circumstances where there is an an evident spatial inconsistency between the MR and PET signal distribution even for perfectly co-registered PET and MR images. This is important because in some cases a PET lesion may be detected at the border between two different regions, as shown in [24], moreover, it has been shown that PET unique features can be severely over-smoothed [28] with the MR-guided kernel. Strul & Bendriem [33] investigated the limitations, due to MR segmentation and PET-MR registration, of different segmentation-based partial volume correction techniques. Although the segmentation does not represent a problem for the segmentation free techniques, registration can always represent a problem especially if the MR sequence is not acquired at the same time as PET. In this work, we show the effect on the PET image of PET-MR inconsistencies, particularly for cold and hot regions that are crossed by MR regions. HKEM and KEM from [8] were used to reconstruct images for four types of dataset: 5 s and 50 s acquisition with phantom data, and 30 s and 300s acquisition with patient neck data. All the data were acquired with the Biograph Siemens mMR. Both algorithm implementations use spatially restricted kernels and, thus, reduced smoothing of the PET unique features is expected compared to the standard patch-based kernel methods as shown by Bland et al (2018) [29]. The MR image was shifted by a different number of pixels along the x direction, and for each shift, the images were reconstructed for each augmented MR image.

II. Kernelized Image Reconstruction

The kernel approach can be applied to the LM-OSEM. The formulation in this paper follows the one in [8].

Figure 1 describes the process of the learning and reconstruction of the HKEM method, the KEM diagram can be easily obtained by removing the PET component in the creation of the kernel. In the traditional kernel method the learning part comes before the reconstruction, while the HKEM contains an iterative learning component which comes from the iterative reconstructed image. Using the kernel method the image λ can be written as follows:

λj=f=1Nαfkfj (1)

where kfj is the fjth element of the kernel matrix and α is the coefficient vector which is to be estimated during the expectation maximization process shown in the diagram. At the first sub-iteration our input images, m for the MR and α0 for the PET, are used to extract the relative feature vectors, υ and z0. For each voxel of the PET image the corresponding feature vectors, υj and zj(n), are extracted from the local neighborhood of the voxel from the MR image and the PET update image respectively. The number of elements in each feature vector was chosen as the total number of available voxels in the neighbourhood, which is 27. This was a result of a preliminary study which is reported in Supplementary Figure 2. The υj and zj(n) vectors represent the inputs for the method, and the kernel matrix is calculated using the following functions:

km(υf,υj)=exp(υfυj22σm2)exp(xfxj22σdm2) (2)

is the kernel coming from the MR image and

kp(zf(n),zj(n))=exp(zf(n)zj(n)22σp2)exp(xfxj22σdp2) (3)

is the part coming from the PET iterative update. The quantity xj is the coordinate of the jth voxel, υj and zj(n) are the feature vectors calculated respectively from the MR image and the nth PET update image, α, while σm, σp, σdm and σdp (the last two are in mm) are scaling parameters for the distances in (2) and (3). To make it simpler to choose the kernel parameters (such as σm and σp), the feature vector, υj, is normalized so that the equation (2) becomes

km(υf,υj)=exp(υfυj22(SDm)2σm2)exp(xfxj22σdm2) (4)

for km(υf, υj), where SDm is the standard deviation of the mean voxel value over the whole MR image. For the PET contribution, kp(zf(n),zj(n)), this normalization is slightly different. The differences in (3) are normalized with αj(n)

kp(zf(n),zj(n))=exp(zf(n)zj(n)22(αj(n))2σp2)exp(xfxj22σdp2). (5)

Fig. 1.

Fig. 1.

Learning and reconstruction framework for the hybrid kernel expectation maximization (HKEM) for a generic n+1 sub-iteration.

Once the kernel matrix is created, it is used in the first sub-iteration of HKEM to estimate α1, and as a consequence the PET image λ1. Consequently, the image α1 will be used to calculate the PET component of the kernel, while the MR component is stored in memory. The process will be repeated for every sub-iteration.

The iterative nature of the HKEM method allows the training sample to be updated with more accurate information. In addition, preliminary reconstructions, to obtain the PET input to the kernel estimation step, are not necessary.

III. Methods and Materials

A. Phantom Experiment

A phantom experiment was performed with a Jaszczak phantom for resolution studies and was acquired with the Siemens Biograph mMR scanner at Mount Sinai Hospital, NY, USA. The phantom consists of cold spheres with different diameters, 31.8 mm, 25.4 mm, 19.1 mm, 15.9 mm, 12.7 mm and 9.5 mm. The background represents the hot region, which was filled with 155 MBq of 18F-fludeoxyglucose (FDG), and data were acquired over 1 hour. The attenuation image was obtained from an MR volumetric interpolated examination (VIBE) acquisition, segmented into 2 tissue classes (air, water) [34]. The MR component of the kernel was obtained from a co-registered MR-VIBE sequence. The original voxel size was 0.35×0.35×1 mm3. The image is then aligned to the PET field of view (FOV) and re-sliced to match the PET native voxel size, 2.087×2.087×2.031 mm3, and FOV size, 344×344×127 voxels. The raw PET data were reconstructed in two different cases: 5 s and 50 s acquisition-time frames. These frames correspond to the beginning of the acquisition obtained from the full LM file.

B. Patient Experiment

The LM-HKEM method was also applied to dynamic data of the head and neck region of a patient. The acquisition was carried out using the Siemens Biograph mMR at Mount Sinai Hospital, NY, USA, and a consent form was signed by the patient. The patient was injected with [18F]FDG 184 MBq and scanned for 90 minutes. The attenuation images were obtained from the Dixon MR sequence using 4 tissue classes (fat, water, air, bone), and contains attenuation coefficients for bed and coils [34]. The LM data were divided into shorter frames, 30 s and 300 s. The scan started after 10 minutes from the injection of the tracer. Note that the two different durations are considered as two different datasets and the uptake in these datasets is not comparable due to different kinetic behaviour, especially because they are the first frames after injection. The MR part of the kernel matrix is obtained from a time-of-flight (TOF) MR angiography sequence (Figure 2), the original voxel size was 0.7×0.7×1 mm. It is then aligned to the PET field of view (FOV) and re-sliced to match the PET native voxel size, 2.087×2.087×2.031 mm3, and FOV size, 344×344×127 voxels. The MR TOF acquisition time is 540 s. A head and neck coil was employed: 3 slabs, each consisting of 60 slices of 1 mm thickness. This image sequence is particularly suitable to study the carotid because it provides high contrast between the carotid arteries and the surrounding tissues.

Fig. 2.

Fig. 2.

Slices of the MR images used to estimate the kernel matrix for (a) the phantom and (b) the patient studies.

C. Reconstruction Setup

All the datasets were reconstructed with 21 subsets and 3 iterations using HKEM and KEM. The subset division for the LM reconstruction is made by subdividing the events according to the view number. The values of the kernel parameters for this study are set to give the best trade-off between quantification accuracy and noise suppression while minimizing PET feature suppression: N = 3 × 3 × 3, σm=σp = 1, and σdm=σdp = 5. The best trade-off between quantification and noise suppression was estimated as the maximum CNR. For small value of σdm the HKEM will be always noisier than the KEM because the hybrid kernel contains also the noise from the PET image. When the σdm increases, the noise decreases as we are allowing similarities between further voxels in the neighbourhood. The number of neighbours we used was chosen after comparing the performance with 27, 125, and 343 neighbours and 27 allows faster image reconstruction without significant change in quantification. In fact, smooth images can be obtained without changing the neighbourhood. In addition, using a big neighbourhood has the effect of over smoothing small lesions This can be seen in the supplementary material in Supplementary Figure 1. A small neighbourhood leads to less noise suppression (bigger CoV) but at the same time a smaller bias. The calculated RMSE, which provides a trade-off between bias and noise, does not change significantly for regions where the MR does not contain meaningful information. On the other hand if the MR image provides boundaries then the RMSE improves with the size of the neighbourhood. The comparison is made at the 3rd iteration, as early stopping is routinely used in hospitals. The different frame durations are obtained by selecting only events which have occurred during the first 50 s and 5 s for the phantom, and 300 s and 30 s for the patient.

Scatter correction was performed with the method described in Tsoumpas et al [35] and Polycarpou et al [36]. Randoms were estimated from singles, which were calculated from delayed events [37]. The procedures for these evaluations including attenuation and normalisation corrections [38], make use of Software for Tomographic Image Reconstruction (STIR) library [39] version 3.0. All datasets were reconstructed using span 11. The MR image, for both experiments, was translated by 1, 2, 3, 5 and 10 voxels (as in Figure 4) along the x direction in order to study the effect of inaccurate registration between PET and MR images, and also to introduce differences so that different case scenarios can be explored. Note that the attenuation image was not translated so that we only study the effect of the anatomical image used for the kernel.

Fig. 4.

Fig. 4.

Representation of the introduced MR translations using the phantom: from left to right 1, 2, 3, 5, and 10 voxels translation. The blue sphere is the ROI used for the quantification.

D. Image Analysis

The comparison was carried out in terms of mean activity concentration. Region of interest (ROI) analysis was performed using: a sphere in a uniform cold region of the phantom represented by the blue circle in Figure 4, and extracted from the MR image; for the patient, the ROI was obtained using a few steps: firstly, the carotid was segmented using the MR image, and used as a mask on the HKEM image; a threshold was applied on the HKEM image only to segment the hot part of the carotid; finally it was cropped to match the shape of the lesion in both the HKEM and KEM image as the shape of the hot lesions are the same between HKEM and KEM (as it can be seen in Figure 5). In this study what is different between the HKEM and KEM lesion is the uptake. The ROI was not taken from the MR image because it provides the whole carotid, while the OSEM image is substantially affected by PVE. One ROI is used for the 300 s acquisition, and the other for the 30 s acquisition as the kinetics and possible motion make the 2 datasets very different (Figure 3).

Fig. 5.

Fig. 5.

Reconstructed images with OSEM, KEM using only MR and no shift, and HKEM with no shift, on all the datasets and count-levels. (a) Jaszczak phantom 50 s acqusition; (b) Jaszczak phantom 5 s; (c) patient 300 s acqusition; (d) patient 30 s acqusition acqusition.

Fig. 3.

Fig. 3.

Regions of interest (ROI) chosen for the patient study: (a) ROI shown on the MR image for the 300 s acquisition, (b) ROI shown on the PET image for the 300 s acquisition, (c) ROI shown on the MR image for the 30 s acquisition, (d) ROI shown on the PET image for the 30 s acquisition.

Quantitative comparison between algorithms was performed using the following figure of merit:

mean=iziV (6)

where zi is the value of voxel inside the ROI and V is the number of voxels included in the ROI. To estimate the bias images, showing the induced error for each MR translation, the reconstructed images with the correct MR is used as ground truth, also labelled as shift = 0. The difference between The CoV, which is defined as the percentage SD, was used in the measured ROI to asses the noise and the repeatability of the measure.

IV. Results

In this study, we show the effect of the anatomically-driven kernel method on the PET reconstructed images, with a focus on those occasions where MR and PET information do not match. To recreate these circumstances the MR image was translated along the x-axis creating some specific cases, such as a cold PET region partially crossed or surrounded by an MR well define region with the phantom data, and hot lesion partially crossed by MR regions with the patient data. All the shifts can be seen in Figure 4. The blue circles in the figure help to better understand and locate the area we are studying. This investigation was carried out using the hybrid and non-hybrid kernel method in [8] to explore the limitation of these methods as well as to study whether the hybrid method performs better when small differences between PET and MR images are introduced. The σm and σp from (2) and (3) were fixed at 1 to maximise CNR.

The values used for the two modulation parameters, σdm and σdp, were set to 5 which was the optimum value in terms of CNR. All the images shown correspond to the 3rd iteration. Figure 5 shows the images reconstructed with OSEM using 5 mm Gaussian post-filter, and KEM and HKEM with the correct MR image, for the Jaszczak phantom 50 s acquisition (a), Jaszczak phantom 5 s acquisition (b), patient data 300 s acquisition (c), patient data 30 s acquisition (d). Figure 6 shows the quantitative results for the 50 s acquisition phantom datasets. The x axis reports the shift in terms of the number of voxels, while the y axis is the mean value in the ROI, calculated with equation (6). The OSEM ROI mean value (with no post filtering) is also reported for reference. The colored bar reports the range of coefficient of variation (CoV) values in the plot. In Figure 7 an image-based comparison for the 50 s acquisition phantom datasets is shown, where it is possible to appreciate the effects of the different translations on HKEM and KEM (a); the bias in all the image voxels (b); and the zoomed cold spheres for the case where the MR image was translated by 10 voxels (c). The same experiment was repeated for the 5 s acquisition and the quantitative analysis is reported in Figures 8 and 9. Note that in Figure 9(c) The magnification for each sphere is different to allow a better visual comparison. This is why the size do not appear to be in a decreasing order.

Fig. 6.

Fig. 6.

Mean ROI activity concentration for each MR translation: comparison between reconstructed images with KEM using only MR, HKEM, and OSEM for the 50 s acquisition with the Jaszczak phantom. The x axis is the MR image shift in terms of number of voxels.

Fig. 7.

Fig. 7.

Reconstructed phantom images with KEM and HKEM for 50 s acquisition with the Jaszczak phantom. The different columns represent a different shift (the first one is without shift), in terms of number of voxels. (a) Comparison between reconstructed images; (b) Comparison between bias images. Reconstructed images with the correct MR are used as ground truth; (c) zoom of the cold spheres for the case with shift = 10 voxels.

Fig. 8.

Fig. 8.

Mean activity concentration comparison between reconstructed phantom image with KEM, HKEM, and OSEM for 5 s acquisition with the Jaszczak phantom. The x axis is the MR image shift in terms of number of voxels.

Fig. 9.

Fig. 9.

Reconstructed phantom images with KEM using only MR and HKEM on 5 s acquisition. The different columns represent a different shift (the first one is without shift), in terms of number of voxels. (a) Comparison between reconstructed images; (b) Comparison between bias images, reconstructed images with the correct MR are used as ground truth; (c) zoom of the cold spheres for the case with shift=10 voxels with HKEM and KEM.

Figure 10 reports the mean ROI value as a function of the shift for the patient data obtained from 300 s acquisition. The OSEM ROI mean value is also reported for reference. The colored bar reports the range of CoV values in the plot. In Figure 11 the image-based comparison between HKEM and KEM and OSEM is shown, in particular (a) shows the reconstructed images with each translation, while (b) reports the bias images estimated as the difference between the image reconstructed with the correct MR and the image obtained using each translated MR. The same is repeated for the 30 s acquisition, where image-based and quantitative results are shown in Figures 12 and 13 respectively.

Fig. 10.

Fig. 10.

Mean activity concentration comparison between reconstructed patient image with KEM using only MR and HKEM and OSEM on 300 s acquisition. The x axis is the MR image shift in terms of number of voxels.

Fig. 11.

Fig. 11.

Reconstructed patient images with KEM using only MR and HKEM on 300 s acquisition using different translations (a); difference images between the non-shift case and all the different translations for HKEM and KEM, reconstructed images with the correct MR are used as ground truth (b); The different columns represent a different shift (the first one is without shift).

Fig. 12.

Fig. 12.

Mean activity concentration comparison between reconstructed patient image with KEM using only MR and HKEM and OSEM on 30 s acquisition. The x axis is the MR image shift in terms of number of voxels.

Fig. 13.

Fig. 13.

Reconstructed patient images with KEM using only MR and HKEM on 30 s acquisition using different translations (a); difference images between the non-shift case and all the different translations for HKEM and KEM, reconstructed images with the correct MR are used as ground truth (b);The different columns represent a different shift (the first one is without shift), in terms of number of voxels.

V. Discussion

The images in Figure 5 show the improvements, in terms of contrast and resolution, provided by the two anatomically-driven kernelised method over the OSEM. With the Jaszczak phantom we investigated the effect of the PET-MR misalignment on cold spheres: in Figure 6 the ROI mean shows how the mean value, which is supposed to be zero, increases as the shift increases, with a significantly stronger effect for KEM. The maximum increase is 100% for KEM against the 37% of HKEM. This is due to the introduction of the PET information in the kernel, which makes HKEM more flexible allowing the PET borders to be restored. The increase in activity along with the shift seems to indicate that the activity was pushed from its original position to the area after the MR border. Figure 7 shows the same result visually, where the effect of the difference between PET and MR can already be seen for the 1 voxel shift for both HKEM and KEM. Bias appears close to the borders of the MR and PET images. In particular, we can see positive bias in the cold region and negative bias in the hot regions close to the borders. It is also possible to see that the smoothing effect is more significant for the smaller spheres.

For the 5 s acquisition, Figure 8 shows more moderate variation than the higher count case. In this case the maximum variation was 33% for KEM and 15% for HKEM with the 5 voxels shift. The fact that the short acquisition shows less quantitative change than the 50 s acquisition is probably due to the fact that the noise here has a more important impact and there is already an increased uptake even with the correct MR image. In fact, the OSEM value is close to the “wrong” HKEM value. Looking at the zoomed spheres in Figure 9 it is possible to see that the degradation is actually more visible for the low count.

With the patient data, we show the effect of the PET-MR inconsistency on hot lesions in the carotid artery. In contrast to the phantom, here it is more difficult to visually detect the artifacts as the human body is more complicated as there is a wider variety of visible tissue in the MR image. Figure 10 shows the translation consequences for the 300 s acquisition. We can see an instant drop of uptakes as soon as the shift takes place. The HKEM performs better for the small shifts, however, after the 2 voxels translation it behaves similarly to the KEM with a maximum variation of 7% for both. Although the ROI analysis for the hot regions show more moderate variation than the case of the cold spheres, in Figure 11 it is possible to see that the shape of the lesion changes with the shift and becomes gradually blurred. It is a consistent results over the different count-levels that when the inconsistency is too severe the HKEM behave similarly to KEM. In Supplementary Figure 3, a similar analysis is shown for a 5400 s acquisition dataset. Also, negative and positive bias can be seen all over the image, which can reach 50% in certain locations. The ROI quantification for the 30 s acquisition, as reported in Figure 12, shows slightly more significant variations with a maximum of 10% for KEM and 8% for HKEM. In this case, HKEM performs better in all translations except the 5 voxel translation. Similarly to the 300 s case, in Figure 13 it is possible to see the gradual blurring of the lesion and the variation voxel by voxel for all the different shifts. The dark blue and the dark red areas in Figure 11(b) and 13(b) represent a bias of up to 50% and which is more important for the low-count case. Note that, 2 cm of motion is probably too big to be missed, and this type of motion can be corrected by registration. However, in this case we studied the effect of the uniform MR area of the neck overlaying the carotid artery of the PET image. This represents, for example, cases where a high uptake region in PET does not show any signal in the MR.

In general, the PET-MR inconsistencies can induce partial volume effects, more specifically spill-in for the affected cold regions and spill-out for the hot regions. The more significant errors are introduced for the cold sphere than the hot lesion. In this case the HKEM appears more flexible thanks to the PET information included in the kernel. The average ROI error for the hot lesions was always smaller or equal to 10% and the HKEM outperformed KEM only in some cases while they behave similarly in other. Finally, although the PET-MR spatial inconsistencies introduce voxel variations and errors, it is a consistent result over our datasets that the application of a 5 mm Gaussian filter provide a significantly bigger PVE than the one created by any MR translation, while OSEM provide accurate but very noisy results. This can be seen from Figure 5, 6, 8, 10 and 12. The CoV estimated in this study showed up to 20% difference between HKEM and KEM when using the phantom. On the other hand, for the patient data the difference was always around or lower than 1% Note that if the PET and MR images have the same dimensions and voxel size, then the Euclidean distances in Equations (2) and (3) are the same. For the KEM there is no σdp but only σdm. The extra spatial Gaussian term and the Gaussian term with the PET information contribute to improve the PET unique feature preservation compared to KEM. Narrower Gaussian term for KEM can be obtained with smaller σdm, however this comes at the cost of CNR. In fact, the maximum CNR was obtained with σdm = 5. The findings highlight the improved flexibility of both HKEM and KEM compared to the gold standard, OSEM, and the importance of a well chosen MR sequence to use as anatomical information, as well as an accurate registration between PET and MR. In fact, to obtain the most accurate result, a study making use of anatomical information should be thoroughly planned according to the region of interest one wants to study to avoid unwanted MR regions crossing one or more PET regions.

VI. Conclusion

In this work, we showed the effect of introduced mismatches between PET and MR images, for cold and hot regions, when we use the kernel method with MR information for PET reconstructed images. We showed that even small changes in the MR can result in a change in quantification and blurring. Moreover, hybrid information can help to reduce this effects, especially in the case of cold regions. The ideal solution is to avoid MR sequences having tissue-borders crossing the PET ROI as well as a very accurate PET-MR registration. In addition, creating a protocol where the chosen MR sequence is acquired during the PET scan would minimize the PET-MR inconsistencies.

Supplementary Material

trpms-deidda-2884176-mm

Acknowledgment

The authors acknowledge the group in Mount Sinai Hospital for data support. We wish to thank Dr Kris Thielemans and the Collaborative Computational Project in PET-MR imaging (CCP-PET-MR), funded with EP/M022587/1 and EP/P022200/1 grant for enabling PET-MR reconstruction and for funding Daniel Deidda participation in an exchange program at University College of London (UCL). This work was undertaken on MARC1, part of the High-Performance Computing and Leeds Institute for Data Analytics (LIDA) facilities at the University of Leeds, UK. Finally, the authors would like to acknowledge the reviewers and the editor for dedicating their time to help significantly improve the manuscript.

This work is supported by the University Research Scholarship, University of Leeds and the research grant NIH/NHLBI R01HL071021. Part of this research is funded by the EPSRC Collaborative Computational Flagship Project (EP/P022200/1). Dr Tsoumpas is sponsored by a Royal Society Industry Fellowship (IF170011).

Contributor Information

Daniel Deidda, Biomedical Imaging Science Department, Leeds Institute of Cardiovascular and Metabolic Medicine (LICAMM), School of Medicine, and the Department of Statistics, School of Mathematics, University of Leeds, UK.

Nicolas Karakatsanis, Translational and Molecular Imaging Institute (TMII), Icahn School of Medicine at Mount Sinai, Department of Radiology, NY, USA; Division of Radio-pharmaceutical Sciences, Department of Radiology, Weill Cornell Medical College of Cornell University, NY, USA.

Philip M. Robson, Translational and Molecular Imaging Institute (TMII), Icahn School of Medicine at Mount Sinai, Department of Radiology, NY, USA

Nikos Efthimiou, School of Life Sciences, Faculty of Health Sciences, University of Hull, UK.

Zahi A. Fayad, Translational and Molecular Imaging Institute (TMII), Icahn School of Medicine at Mount Sinai, Department of Radiology, NY, USA

Robert G. Aykroyd, Department of Statistics, School of Mathematics, University of Leeds, UK

Charalampos Tsoumpas, Translational and Molecular Imaging Institute (TMII), Icahn School of Medicine at Mount Sinai, Department of Radiology, NY, USA; Biomedical Imaging Science Department, School of Medicine, University of Leeds, UK and with Invicro Ltd., UK.

References

  • [1].Wang G, Ye JC, Mueller K, and Fessler JA, “Image reconstruction is a new frontier of machine learning,” IEEE trans med imaging, vol. 37, no. 6, pp. 1289–1296, 2018. [DOI] [PubMed] [Google Scholar]
  • [2].Yang B, Ying L, and Tang J, “Artificial neural network enhanced bayesian pet image reconstruction,” IEEE trans med imaging, 2018. (in press). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [3].Gong K, Guan J, Kim K, Zhang X, Fakhri GE, Qi J, and Li Q, “Iterative pet image reconstruction using convolutional neural network representation,” arXiv preprint arXiv:1710.03344, 2017. [Google Scholar]
  • [4].Kim K, Wu D, Gong K, Dutta J, Kim JH, Son YD, Kim HK, El Fakhri G, and Li Q, “Penalized pet reconstruction using deep learning prior and local linear fitting,” IEEE trans med imaging, vol. 37, no. 6, pp. 1478–1487, 2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [5].Wang G and Qi J, “PET image reconstruction using kernel method,” IEEE Trans. Med. Imaging, vol. 34, pp. 61–71, 2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [6].Hutchcroft W, Wang G, and Qi J, “Anatomical-image aided PET reconstruction by the kernel method,” J. Nucl. Med, vol. 55, pp. suppl 1, abstract 44, 2014. [Google Scholar]
  • [7].Gong K, Cheng-Liao J, Wang G, Chen KT, Catana C, and Qi J, “Direct patlak reconstruction from dynamic PET data using the kernel method with MRI information based on structural similarity,” IEEE Trans Med Imaging, vol. 37, no. 4, pp. 955–965, 2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [8].Deidda D, Karakatsanis N, Robson PM, Efthimiou N, Fayad ZA, Aykroyd RG, and Tsoumpas C, “Hybrid PET-MR list-mode kernelized expectation maximization reconstruction for quantitative PET images of the carotid arteries,” NSS/MIC, Atlanta, Georgia, USA, 2017. [Google Scholar]
  • [9].Novosad P and Reader A, “MR-guided dynamic PET reconstruction with the kernel method and spectral temporal basis functions,” Phys. Med. Biol, vol. 61, pp. 4624–4645, 2016. [DOI] [PubMed] [Google Scholar]
  • [10].Baete K, Nuyts J, Van Laere K, Van Paesschen W, Ceyssens S, De Ceuninck L, Gheysens O, Kelles A, Van den Eynden J, Suetens P, and Dupont P, “Evaluation of anatomy based reconstruction for partial volume correction in brain FDG-PET,” Neuroimage, vol. 23, no. 1, pp. 305–17, 2004. [DOI] [PubMed] [Google Scholar]
  • [11].Goffin K, Van Paesschen W, Dupont P, Baete K, Palmini A, Nuyts J, and Van Laere K, “Anatomy-based reconstruction of FDG-PET images with implicit partial volume correction improves detection of hypometabolic regions in patients with epilepsy due to focal cortical dysplasia diagnosed on MRI,” Eur J Nucl Med Mol Imaging, vol. 37, no. 6, pp. 1148–55, 2010. [DOI] [PubMed] [Google Scholar]
  • [12].Nuyts J, Baete K, Bequ D, and Dupont P, “Comparison between MAP and postprocessed ML for image reconstruction in emission tomography when anatomical knowledge is available,” IEEE Trans Med Imaging, vol. 24, no. 5, pp. 667–75, 2005. [DOI] [PubMed] [Google Scholar]
  • [13].Leahy R and Yan X, “Incorporation of anatomical MR data for improved functional imaging with PET,” in Inf. Process. Med. Imaging, pp. 105–120, Springer, 1991. [Google Scholar]
  • [14].Lipinski B, Herzog H, Rota Kops E, Oberschelp W, and Mller-Grtner HW, “Expectation maximization reconstruction of positron emission tomography images using anatomical magnetic resonance information,” IEEE Trans Med Imaging, vol. 16, no. 2, pp. 129–36, 1997. [DOI] [PubMed] [Google Scholar]
  • [15].Bowsher JE, Johnson VE, Turkington TG, Jaszczak RJ, Floyd C, and Coleman RE, “Bayesian reconstruction and use of anatomical a priori information for emission tomography,” IEEE Trans. Med. Imaging, vol. 15, no. 5, pp. 673–686, 1996. [DOI] [PubMed] [Google Scholar]
  • [16].Tang J and Rahmim A, “Bayesian PET image reconstruction incorporating anato-functional joint entropy,” Phys. Med. Biol, vol. 54, no. 23, p. 7063, 2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [17].Somayajula S, Panagiotou C, Rangarajan A, Li Q, Arridge SR, and Leahy RM, “PET image reconstruction using information theoretic anatomical priors,” IEEE Trans Med. Imaging, vol. 30, no. 3, pp. 537–549, 2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [18].Vunckx K, Atre A, Baete K, Reilhac A, Deroose CM, Laere KV, and Nuyts J, “Evaluation of three MRI-based anatomical priors for quantitative PET brain imaging,” IEEE Trans. Med. Imaging, vol. 31, pp. 599–612, 2012. [DOI] [PubMed] [Google Scholar]
  • [19].Ehrhardt MJ, Thielemans K, Pizarro L, Atkinson D, Ourselin S, Hutton BF, and Arridge SR, “Joint reconstruction of PET-MRI by exploiting structural similarity,” Inv. Prob, vol. 31, no. 1, p. 015001, 2014. [Google Scholar]
  • [20].Mehranian A, Belzunce M, Prieto C, Hammers A, and Reader A, “Synergistic PET and SENSE MR image reconstruction using joint sparsity regularization,” IEEE Trans. Med. Imaging, vol. 37, no. 1, pp. 20–34, 2018. [DOI] [PubMed] [Google Scholar]
  • [21].Jiao J, Burgos N, Atkinson D, Hutton B, Arridge S, and Ourselin S, “Detail-preserving PET reconstruction with sparse image representation and anatomical priors.,” in Inf. Proces. Med. Imaging: Proc. Med. Image Comput. Comput. Assist. Interv, vol. 24, pp. 540–551, 2014. [DOI] [PubMed] [Google Scholar]
  • [22].Mehranian A, Belzunce M, Niccolini F, Politis M, Prieto C, Turkheimer F, Hammers A, and Reader A, “PET image reconstruction using multi-parametric anato-functional priors,” Phys. Med. Biol, vol. 62, no. 15, p. 5975, 2017. [DOI] [PubMed] [Google Scholar]
  • [23].Knoll F, Holler M, Koesters T, Otazo R, Bredies K, and Sodickson DK, “Joint MR-PET reconstruction using a multi-channel image regularizer,” IEEE Trans. Med. Imaging, vol. 36, no. 1, pp. 1–16, 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [24].Schramm G, Holler M, Rezaei A, Vunckx K, Knoll F, Bredies K, Boada FE, and Nuyts J, “Evaluation of parallel level sets and bowshers method as segmentation-free anatomical priors for time-of-flight pet reconstruction,” IEEE Trans Med Imaging, vol. 37, pp. 590–603, 2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [25].Hutchcroft W, Wang G, Chen KT, Catana C, and Qi J, “Anatomically-aided PET reconstruction using the kernel method,” Phys. Med. Biol, vol. 61, no. 18, p. 6668, 2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [26].Ellis S and Reader A, “Kernelised EM image reconstruction for dual-dataset PET studies,” NSS/MIC, Strasburg, France, pp. 1–3, 2016. [Google Scholar]
  • [27].Hoffman T, Scholkopf B, and Smola A, “Kernel methods in machine learning,” Ann. Stat, vol. 36, pp. 1171–1220, 2008. [Google Scholar]
  • [28].Bland J, Mehranian A, Belzunce MA, Ellis S, McGinnity CJ, Hammers A, and Reader AJ, “MR-guided kernel em reconstruction for reduced dose PET imaging,” IEEE Trans Rad Plas Med Sci, 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [29].Bland J, Belzunce M, Ellis S, McGinnity C, Hammers A, and Reader A, “Spatially-compact MR-Guided kernel EM for PET image reconstruction,” IEEE Trans Rad Plas Med Sci, 5 2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [30].Spencer B and Wang G, “Statistical image reconstruction for shortened dynamic PET using a dual kernel method,” NSS/MIC, Atlanta, Georgia, USA, 2017. (in press). [Google Scholar]
  • [31].Deidda D, Karakatsanis N, Calcagno C, Robson PM, Senders M, Mulder WJM, Fayad ZA, Aykroyd RG, and Tsoumpas C, “Hybrid PET-MR kernelised expectation maximisation reconstruction for improved image-derived estimation of the input function from the aorta of rabbits,” Contrast Media Mol Imaging, 2018. (in press). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [32].Ehrhardt MJ, Markiewicz P, Liljeroth M, Barnes A, Kolehmainen V, Duncan JS, Pizarro L, Atkinson D, Hutton BF, Ourselin S, et al. , “PET reconstruction with an anatomical MRI prior using parallel level sets,” IEEE Trans. Med. Imaging, vol. 35, no. 9, pp. 2189–2199, 2016. [DOI] [PubMed] [Google Scholar]
  • [33].Strul D and Bendriem B, “Robustness of anatomically guided pixel-by-pixel algorithms for partial volume effect correction in positron emission tomography,” J Cereb Blood Flow Metab, vol. 19, no. 5, pp. 547–559, 1999. [DOI] [PubMed] [Google Scholar]
  • [34].Karakatsanis N, Robson P, Dweck M, Abgral R, Trivieri M, Sanz J, Contreras J, Narula J, Padilla M, Gidwani U, et al. , “MR-based attenuation correction in cardiovascular PET/MR imaging: challenges and practical solutions for cardiorespiratory motion and tissue class segmentation,” J Nucl Med, vol. 57, no. supplement 2, pp. 452–452, 2016. [Google Scholar]
  • [35].Tsoumpas C, Aguiar P, Nikita K, Ros D, and Thielemans K, “Evaluation of the single scatter simulation algorithm implemented in the STIR library,” in NSS/MIC/RTSD, vol. 6, pp. 3361–3365, IEEE, 2004. [Google Scholar]
  • [36].Polycarpou I, Thielemans K, Manjeshwar R, Aguiar P, Marsden PK, and Tsoumpas C, “Comparative evaluation of scatter correction in 3D PET using different scatter-level approximations,” Ann. Nucl. Med, vol. 25, no. 9, pp. 643–649, 2011. [DOI] [PubMed] [Google Scholar]
  • [37].Jacobson MW and Thielemans K, “Optimizability of loglikelihoods for the estimation of detector efficiencies and singles rates in PET,” in NSS/MIC/RTSD, pp. 4580–4586, IEEE, 2008. [Google Scholar]
  • [38].Hogg D, Thielemans K, Spinks T, and Spyrou N, “Maximum-likelihood estimation of normalisation factors for PET,” in NSS/MIC, San Diego, California, vol. 4, pp. 2065–2069, IEEE, 2001. [Google Scholar]
  • [39].Thielemans K, Tsoumpas C, Mustafovic S, Beisel T, Aguiar P, Dikaios N, and Jacobson M, “STIR: software for tomographic image reconstruction release 2,” Phys. Med. Biol, vol. 57, pp. 867–883, 2012. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

trpms-deidda-2884176-mm

RESOURCES