PET Attenuation Correction using Synthetic CT from Ultrashort Echo-time MRI

Abstract

Rationale

Integrated PET (Positron Emission Tomography)/MR(magnetic resonance) systems are becoming increasingly popular in clinical and research applications. Quantitative PET reconstruction requires correction for γ photon attenuations using an attenuation coefficient map (μ-map) that is a measure of the electron density. One challenge of PET/MR, in contrast to PET/CT, lies in the accurate computation of μ-maps. Unlike CT, MRI measures physical properties not directly related to electron density. Previous approaches have computed the attenuation coefficients using a segmentation of MR images or using deformable registration of atlas CT images to the space of the subject MRI.

Method

In this work, we propose a patch-based method to generate whole head μ-maps from Ultra-short Echo Time (UTE) MR imaging sequences. UTE images are preferred to other MR sequences, because of the increased signal from bone). To generate a synthetic CT image, we use patches from a reference dataset, which consists of dual echo UTE images and a co-registered CT from the same subject. By matching patches between the reference and target images, corresponding patches from the reference CT are combined via a Bayesian framework. No registration or segmentation is required.

Results

For evaluation, UTE, CT and PET data, acquired from 5 patients under an IRB approved protocol, were used. Another patient (with UTE and CT only) was selected to be the reference to generate synthetic CT images for these five patients. PET reconstructions were attenuation corrected using (1) the original CT, (2) our synthetic CT, and Siemens (3) Dixon- and (4) UTE-based μ-maps, and (5) a deformable registration based CT. Our synthetic CT based PET reconstruction shows higher correlation (average ρ = 0.99, R² = 0.99) to the original CT based PET, as compared to the segmentation and registration based methods. Synthetic CT based reconstruction had minimal bias (regression slope 0.99) as compared to the segmentation based methods (regression slope 0.97). A peak signal-to-noise ratio of 36.0 dB in the reconstructed PET activity is observed, compared with 29.7, 29.3, 27.4 dB for Siemens Dixon, UTE, and registration based μ-maps.

Conclusion

A patch-matching approach to synthesize CT images from dual echo UTE images leads to significantly more accurate PET reconstruction as compared to actual CT scans. The PET reconstruction is improved over segmentation (Dixon and Siemens UTE) and registration based methods, even in subjects with pathology.

1. Introduction

Hybrid medical imaging systems, such as PET/CT and PET/MR, are routinely used as diagnostic tools for brain imaging in clinical and research environments. For PET/CT systems, X-ray attenuation coefficients from computed tomography (CT) images are used for attenuation correction of PET images. Recently, PET/MR systems have been introduced (1) in diagnostic applications. A significant challenge for PET/MR systems is that the intensities of a brain MR image is based on magnetic properties (e.g., proton density, the longitudinal and transverse relaxation times) that, unlike in CT, have no straightforward relation to electron density, which determines γ photon attenuation.

Different segmentation-based and atlas-based methods have been reported to synthesize attenuation coefficient maps (μ-maps) from MR images. Segmentation based methods rely on 3 or 4 class segmentations (2,3,4) of the MR image (e.g., soft tissue, fat, air and bone) either using a Dixon-based approach (5) or intensity-based segmentation of the MRI. Then typical CT intensities are assigned to the corresponding tissue labels to create a CT-like image. However, Dixon-based and intensity-based approaches often ignore bone or perform poorly with respect to reconstructing bone. This is because standard clinical MR scans do not show any signal for bone. Since bone has the highest attenuation of these tissue classes, it is an important structure to accurately represent for attenuation correction purposes (6). Ultra-short echo time (UTE) imaging (7) is a relatively new MRI technique that enables imaging of structures with short T2 relaxation times such as bone. Combined with an image with a longer echo time, where bone produces extremely low signal, a better bone segmentation can be obtained from dual echo UTE (8).

Most atlas based methods (9,10,11,12,13) rely on learning a regression from MR intensities to CT Hounsfield units (HU). Instead of just using voxel intensities, patches or sub-images are preferred to estimate CT numbers, because a patch encodes neighborhood information around a voxel (14). We define that one “atlas” consists of one MR image and its corresponding CT image. Multiple atlas MR images are first deformably registered to a subject's MR image. For a patch in the subject MR, multiple relevant patches from the atlas MR images are found. Then corresponding CT patches are combined to estimate the CT numbers for the subject patch. Atlas-based methods usually require that the atlas MR is well aligned to the subject. The deformable registration algorithms (15) that are used for this purpose can be computationally expensive and the final quality of the PET reconstruction invariably depends on the accuracy of the registration. This can be problematic when the geometry of the anatomy between the atlas and the subject substantially differ or the subject exhibits pathology.

In this paper, we propose an algorithm called GENErative Sub-Image Synthesis (GENESIS) to synthesize a CT image (or μ-map) from dual echo UTE images using reference or training data. Our “reference” data is distinguished from the “ atlases” in atlas-based methods in the manner that while an atlas is needed to be registered to the subject (12,13), we do not require any registration between a reference and the subject.

Our approach matches patches between the reference images and subject images based on a statistical model, similar to the idea of coherent point drift (16). The novelty of the method is three-fold. First, it does not require any segmentation of the MR images. Second, it does not require the reference images to be registered to the subject images. Third, unlike most algorithms that only utilize the MR images to determine the optimal matching patches in the reference data, our algorithm utilizes the reference CT image as well.

2. Method

Data Description

Data were acquired under an IRB approved protocol. The institutional review board (IRB or equivalent) approved this study and all subjects signed a written informed consent. Five patients were scheduled for FDG PET/CT and were recruited to have PET/MR immediately following the PET/CT. In addition to the MRI, for the purposes of this study, the PET data acquired on the PET/MRI and the CT data acquired on the PET/CT were used. Attenuation correction of the PET data was performed using four different methods of generating the attenuation map (μ-map) (see Section 2, Evaluation approach). Images from additional 30 subjects who had MRI with UTE sequences, 29 subjects with Dixon MRI, and 31 subjects with CTs, were also used for comparison later.

MR and PET images were acquired on a 3T Siemens Biograph mMR. The MR UTE images were of dimension 192 × 192 × 192 with 1.56 mm³ resolution (repetition time TR=11.94 s, echo time TE=70 μs/2.46 ms, flip angle = 10°). PET was acquired for five minutes approximately 90 minutes after injection of ~370 MBq 18F fluorodeoxyglucose. For comparison, MR Dixon images were acquired with dimension 192 × 126 × 128 and 2.08 × 2.08 × 2.34 mm³ resolution (TR=3.6 ms, TE=1.23/2.46 ms, flip angle 10°). Attenuation maps were inserted in the original MR-AC DICOM files and imported into a specialized computer workstation for PET retrospective image reconstruction. The image reconstruction was performed using a 3D ordered-subset expectation-maximization algorithm (17) with 3 iterations and 21 subsets on a 344 × 344 × 127 matrix with a 4-mm Gaussian filter.

CT images were acquired on a Siemens Biograph128 PET/CT scanner with a tube voltage of 120kVp, with dimension 512 × 512 × 149 and 0.58 × 0.58 × 1.5 mm³ resolution. CT images were rigidly registered to the corresponding MRI from the same subject. Real and synthetic CT images were transformed to μ-maps (unit cm⁻¹) using the following criteria (11),

$$\mu =\{\begin{array}{cc}9.6\times {10}^{-5}(\mathrm{h}+1000)\hfill & \mathrm{h}\le 47\mathrm{HU}\hfill \\ 5.1\times {10}^{-5}(\mathrm{h}+1000)+4.7\times {10}^{-2},\hfill & \mathrm{h}>47HU\hfill \end{array}\phantom{\}}$$ (1)

where h denotes CT intensities in HU.

Inputs for CT Synthesis

Our reference data is defined as a triplet of co-registered images {a₁, a₂, a₃} having the same resolution, where a₁ and a₂ denote dual echo UTE images, where typically the first echo shows signal in bone and the second echo does not. The variable a3 denotes the corresponding CT. See Fig. 1 top row for a set of reference images. The subject dual echo UTE images are denoted by b₁ and b₂. All the MR scans, a₁, a₂,b₁, b₂, are intensity normalized such that the mode of their white matter intensities are at unity. This intensity normalization step is performed automatically based on image histograms, and is required to allow the intensity scales to be of comparable magnitude. At each voxel of the reference images, 3D overlapping patches (of size p × q × r) are considered and stacked into 1D vectors of size d × 1, where d = pqr. The MR feature vector at the j^th voxel of the reference images is the concatenation of corresponding UTE patches, denoted by ${\mathbf{y}}_j\in {\mathbb{R}}^{2d}$. The CT feature vector is the CT patch, ${\mathbf{v}}_j\in {\mathbb{R}}^{2d}$, j = 1,...,M. Similarly, subject images b₁ and b₂ yield MR features denoted by x_i, i = 1,...,N, ${\mathbf{x}}_i\in {\mathbb{R}}^{2d}$, with N and M representing the number of voxels in the subject and the reference head, respectively. The unobserved CT subject patches are denoted by ${\mathbf{u}}_i\in {\mathbb{R}}^d$. We combine the patch triplets as 3d × 1 vectors ${\mathbf{p}}_i={\left[{\mathbf{x}}_i^{\mathrm{T}}{\mathbf{u}}_i^{\mathrm{T}}\right]}^{\mathrm{T}}$ and ${\mathbf{q}}_j={\left[{\mathbf{y}}_j^{\mathrm{T}}{\mathbf{v}}_j^{\mathrm{T}}\right]}^{\mathrm{T}}$. The subject and the reference patch clouds are defined as the collection of patch-triplets P = {p_i} and Q = {q_j}. For simplicity, we will use the word “patch” in place of “feature vectors” throughout the paper.

Figure 1.

Top two rows show dual echo UTE images (TE=70μs and 2.46ms) and corresponding original CT based μ-maps of a reference and a subject with a lesion. Bottom row shows Siemens Dixon, Siemens UTE based μ-map and our GENESIS result for the subject.

Synthesis Algorithm

The subject and reference patches represent a local pattern of intensities that have been scaled to a similar intensity range. Therefore, a reference patch y_j that has a pattern of intensities that is similar to a given subject patch x_i likely arises from the same distribution of tissues. In that case, the corresponding CT patch in the reference v_j can be expected to represent an approximate CT contrast of the subject patch. One could naively find a single patch-pair within the reference UTE images that is close (or closest) to the subject UTE patch-pair and then use the corresponding CT reference patch directly in synthesis. However, the nearest patch may not be a close representation because the patches are relatively sparse in their high-dimensional space (e.g., a 3 × 3 × 3 patch exists in a 27- dimensional space). We employ two techniques to address the sparsity. First, since the reference patches may not be plentiful enough to closely resemble all possible subject patches, we consider all convex combinations (or “linear interpolation”) of pairs of reference patches to obtain a better representation of a subject patch. Second, based on these interpolated reference patches, we construct a probability distribution with each interpolated pair acting as the mean of a component of a Gaussian mixture model.

In order to tie the MR and CT modalities together, we further assume that the subject's unknown CT patch is a random vector whose mean is also a convex combination of reference CT patches with the same weighting coefficients that generate the MR patches (18, 19). However, the CT patch has a covariance matrix that is unknown and different from the unknown MR patch covariance matrix.

To formalize these ideas mathematically, consider a subject patch p_i and two associated reference patches q_j and q_k. Then p_i is assumed to arise from a Gaussian distribution, given by

$${\mathbf{p}}_i\sim \mathrm{N}({\alpha }_{it}{\mathbf{q}}_j+(1-{\alpha }_{it}){\mathbf{q}}_k,{\Sigma }_t),t\equiv \{j,k\},{\alpha }_{it}\in (0,1)$$ (2)

where Σ_t is a covariance matrix associated with the j^th and k^th reference patches. The weighting coefficient α_it ∈ (0, 1) is larger when the i^th subject patch is more similar to q_j. Here we assumed that p_i is a Gaussian mixture of all possible pairs of reference patches (q_j and q_k). We define Ψ to be the set of all pairs of reference patch indices. From Eqn. 2, t is an element of Ψ, and $\mid \psi \mid =\left(\begin{array}{c}\hfill \mathrm{M}\hfill \\ \hfill 2\hfill \end{array}\right)$. Then each subject patch is assumed to follow an $\left(\begin{array}{c}\hfill \mathrm{M}\hfill \\ \hfill 2\hfill \end{array}\right)$- class Gaussian mixture model (GMM), where each of the mixtures contains two reference patches.

We maximize the probability of observing the subject patches p_i using expectationmaximization (EM) (20). The details of the estimation algorithm are provided in the supplementary material. Based on this model, synthetic CT patches u_j are estimated as,

$$E\left({\mathbf{u}}_i\right)={\Sigma }_{t\in \Psi }{w}_{it}\{{\alpha }_{it}{\mathbf{v}}_j+(1-{\alpha }_{it}){\mathbf{v}}_k\}$$ (3)

converge, the final values of w_it and α_it are used in Eqn. 3. Intuitively, it is observed that an estimated subject CT patch is a weighted average (weighted by w_it) of convex combinations (associated with α_it) of all reference CT patch-pairs (v_j and v_k , ∀ j, k). This is in accordance with our initial assumption that a subject UTE patch (x_i) is a Gaussian perturbation of convex combination of an reference UTE patch-pair (y_j and y_k). However, the weight depends on the similarity between the subject UTE patch (x_i) and relevant reference UTE patches (y_j, y_k), as well as the similarity between estimated subject CT patch (u_i) and reference CT patches (v_j, v_k).

Evaluation approach

The synthesis took ~ 1.5 hours on a 2.92 GHz Intel Xeon 12-core processor. To compare the synthetic CT (or μ-maps) images with the five real CT (or μ-maps), we use Pearson's linear correlation coefficient ρ and PSNR (peak signal to noise ratio) as error metrics. The correlation was calculated between two 1D vectors, each of them being a collection of non-zero voxels of the 3D image or μ-map volume. To compare reconstructed PET images, we used correlation, PSNR, coefficient of determination R² and least square linear regression slopes of volumes. Comparisons were made against three competing methods: Siemens Product Dixon, Siemens Work-In-Progress UTE, and an in-house implementation of an atlas registration based method (11). For the atlas registration method, each of the five patients with CT and dual echo UTE MR scans were used as atlases. Atlas UTE images were first deformably registered to the subject UTE (15). The transformations were then applied to the corresponding atlas CT images, and the transformed CT images were finally combined to generate a subject CT image.

In order to evaluate the performance of CT synthesis on a larger number of subjects, we examined the distribution of bone fraction and air fraction in 30 subjects that had dual echo UTE images but did not have a corresponding CT. In addition to the Siemens UTE μ-map, synthetic CTs were generated on these data using both GENESIS and the atlas-based registration approaches. Dixon based results from another set of 28 subjects and CT images from other 31 subjects were used for cross-sectional comparison. These subjects were acquired with the same scanners given earlier, but do not have UTE MR scans or Siemens UTE μ-maps. To remove differences in the field of view, which includes varying amounts of the neck, each μ-map or CT image (synthetic and real) was affine registered to a template CT image. The neck region was manually defined on the template by identifying an axial slice that corresponds to the neck and head boundary on the template. The neck regions were removed from each of the images using the corresponding axial slice. Bone and air volumes were computed using a threshold on the CT images (bone threshold 300 HU, air threshold -1000 HU), or directly from the μ maps for the Siemens generated results. The comparison on these cross-sectional data is described in the next section.

3. Results

Visual comparisons

An advantage of GENESIS is that the reference need not be registered to the subject. Since we match a subject patch to relevant patches in the reference irrespective of their spatial location, the synthetic CT quality does not suffer if the anatomy of the subject differs widely from that of the reference. An example is shown in Fig. 1, where synthetic μ-maps are generated for a patient with a large abnormality. The reference is chosen to be a patient with no similar lesion in the MR. In Fig. 1, the top two rows show dual echo UTE images and the corresponding μ-maps for the reference and the subject. Bottom row shows the results from the Siemens Dixon μ-map (5), the Siemens UTE μ-map (21), and our synthesis (GENESIS). The lesion is preserved in the GENESIS synthetic CT; furthermore, a more realistic recognition of bone is observed compared to the Dixon and UTE μ-maps. Comparison with the registration based method (11) are shown in Fig. 2. The white arrow in Fig. 2(B) shows the lesion synthesized by GENESIS, which cannot be seen in the registration based approach because none of the atlases used in registration have any lesions. Also, since deformable registration is never perfect, major misregistration error is observed on the registration result near cerebellum (in Fig. 2(C), orange arrow) and minor error is observed near the ventricles (in Fig. 2(C), green arrow).

Figure 2.

Corresponding axial sections of μ-maps of a subject from original CT (A), GENESIS (B), and deformable registration (C) demonstrate that, visually, the GENESIS μ-map is closer to the original CT based μ-map than is that obtained by deformable registration. Note the cystic lesion in the left frontal lobe (white arrow) is well represented by GENESIS but not by deformable registration. Similarly the dilation of the right lateral ventricle (green arrow) is not represented in the deformable registration. Finally, note that misregistration in the posterior fossa mislabels much of the cerebellum as bone (orange arrow).

Comparison of Original and Synthetic CT

A quantitative comparison of correlation and PSNR between the four MR based μ-maps and the CT μ-map is shown in Table 1 for the five patients with PET scans. All numbers for these and subsequent comparisons were computed on the whole head, ignoring background voxels. GENESIS consistently produces the highest correlation as well as the largest PSNR for all subjects, indicating that it is closest to the assumed ground truth. We note that since the subject with a lesion (Fig. 1) (subject #3 in Table 1) has a pathology that the reference does not contain, it has the lowest correlation and PSNR in GENESIS result.

Table 1.

CT based μ-maps are compared with GENESIS μ-map, Siemens Dixon, Siemens UTE, and registration based μ-map on 5 subjects (shown in each row). Correlation (ρ) and peak signal-to-noise-ratio (PSNR) in dB are chosen as error metric, assuming the CT μ-map as the truth. GENESIS produces higher correlation and PSNR than other three methods on all 5 subjects.

Subject ID
Metric		1	2	3	4	5	Mean±Std
Correlation	Dixon	0.89	0.44	0.54	0.74	0.36	0.61±0.21
	UTE	0.86	0.55	0.65	0.82	0.44	0.66±0.18
	Registration	0.95	0.63	0.67	0.85	0.66	0.75±0.14
	GENESIS	0.95	0.70	0.67	0.91	0.70	0.79±0.13
PSNR	Dixon	19.95	17.59	18.24	18.04	18.01	18.37±0.92
	UTE	16.62	16.43	17.50	16.73	16.32	17.12±0.96
	Registration	23.09	19.37	18.18	21.52	20.86	20.61±1.90
	GENESIS	23.36	21.17	20.62	22.38	22.04	21.92±1.07

Bold indicates largest correlation and PSNR

Fig. 3 shows percent air and bone fractions with respect to the relevant subject pool. Note that Siemens UTE μ-maps, GENESIS and registration has 30 subjects, Siemens Dixon contains 28 subjects, and true CT contains 31 subjects. Since Dixon images do not provide bone segmentation, they are only used for air fraction comparison. GENESIS provides similar (median 16%) percent of bone fractions to the original CT (median 17%). Siemens UTE μ-maps under-estimate (median 5%) and the registration method overestimates (median 23%) the bone in the head (p < 0.001 using Wilcoxon rank-sum test). During registration to atlases, a little misalignment between the MRI of the subject and atlas renders the bones in the corresponding registered CT images to become misaligned. Their combination produces blurred edges for the bones, as shown in Fig. 4(B) (white arrow). Thus a simple thresholding of CT images gives higher bone fraction. The bone edges are comparatively sharper in GENESIS. Similarly, Siemens UTE and Dixon μ-maps generally over-estimate the air fraction (p < 0.001), while the registration method under-estimates air fraction (p < 0.001). GENESIS is comparable to the original CT (p = 0.75 from Wilcoxon rank-sum test).

Figure 3.

Comparison of tissue classification results for (A) bone and (B) air across the different methods as compared to the gold standard original CT. GENESIS most closely corresponds to the gold standard. Note that the Siemens Dixon method does not allow for bone classification and hence is not represented in (A).

Figure 4.

Comparison of final attenuation correction process for a single subject using the different methods. Initial MRI UTE images (A) were converted into μ-maps (B), generated using the Siemens Dixon and UTE, deformable registration and GENESIS are compared to the gold standard CT. Note the blurring of bone introduced by the deformable registration method (white arrow in B). Although the attenuation corrected PET images (C) appear grossly similar, the images (D) representing the absolute difference between each of the 4 methodsand the original CT based attenuation corrected PET demonstrate marked differences. Note that the colorbar for the difference images represents a 10-fold increase in scale relative to that for the original images.

Comparison of PET Reconstruction

Fig. 4(B) shows the synthetic CT results on a subject comparing four methods. Since deformable registration is inaccurate, errors are seen near the eyes and nasal cavity (white arrow in Fig. 4(B)), while the GENESIS μ-map is visually closer to the truth. It also has better bone to soft tissue discrimination than the Siemens UTE based μ-maps. Reconstructed PET images from the five μ-maps are shown in Fig. 4(C). Assuming the true CT reconstructed PET as the ground truth, GENESIS provides the closest reconstructed images to the truth compared to the other three PET reconstructed images, also seen from the difference images in the bottom row Fig. 4(D). Visually, GENESIS produces a very similar reconstructed images inside the brain.

Scatter plots showing CT-based PET intensities vs. MR-based PET intensities at each voxel of the PET images (Fig. 5, subject ID #3) show that GENESIS is less biased. The solid magenta lines indicate unit slope and the dashed magenta lines indicate a robust linear fit of the points. Evidently, for UTE, Dixon, and registration, most of the points lie below the unit slope line, indicating that the PET intensities are clearly lower than the truth. This is also indicated by the slopes of the linear regression as 0.914, 0.912, 0.860, 1.006, for Siemens Dixon, UTE, registration, and GENESIS, respectively.

Figure 5.

Scatter plots of CT-based PET intensities vs. MR-based PET intensities (× 10⁴) at each voxel of the PET images are shown for Siemens Dixon, UTE, registration and GENESIS. Solid magenta lines indicate unit slope and dotted magenta lines are a robust linear fit of the data.

Table 2 shows comparison between four methods with CT reconstructed PET images, with respect to correlation, PSNR, regression slopes and R². Slopes of the linear regression (as in Fig. 5) should ideally be unity. For GENESIS, slopes across the four subjects are closer to unity than the other three methods. R² for all five subjects are closer to unity than the other three methods. We note that for the subject (#3) with lesion, the PSNR for GENESIS is smallest among the five subjects. This can be attributed to the fact that subtle lesions near the left ventricles were not synthesized in GENESIS, although the large lesion was synthesized well (white arrow in Fig. 2(B)).

Table 2.

Reconstructed PET images from CT based μ-maps are compared with PET images from MR based μ-maps on 5 subjects via correlation, PSNR (in dB), R² and linear regression slope from the scatter plots (such as Fig. 5), assuming CT based PET as the ground truth.

		Subject ID
Metric	Image Type	1	2	3	4	5	Mean±Std
Correlation	Dixon	0.994	0.993	0.992	0.990	0.994	0.993±0.001
	UTE	0.994	0.994	0.995	0.991	0.994	0.993±0.001
	Registration	0.954	0.982	0.964	0.997	0.875	0.954±0.048
	GENESIS	0.996	0.995	0.996	0.998	0.997	0.996±0.001
PSNR	Dixon	30.32	32.95	29.45	25.87	29.75	29.67±2.53
	UTE	29.63	33.03	30.28	25.15	28.62	29.34±2.86
	Registration	24.95	31.94	24.10	35.34	20.82	27.43±5.99
	GENESIS	35.34	37.78	34.57	36.59	35.61	35.98±1.24
Slope	Dixon	0.924	0.913	0.914	0.872	0.904	0.905±0.020
	UTE	0.912	0.899	0.913	0.870	0.888	0.896±0.018
	Registration	0.894	0.987	0.862	1.011	1.039	0.959±0.077
	GENESIS	0.983	0.992	1.014	0.986	0.971	0.990±0.016
R 2	Dixon	0.972	0.973	0.979	0.962	0.967	0.971±0.006
	UTE	0.974	0.967	0.982	0.945	0.971	0.968±0.014
	Registration	0.889	0.956	0.914	0.987	0.744	0.898±0.094
	GENESIS	0.992	0.989	0.994	0.993	0.985	0.991±0.004

Bold indicates largest correlation, PSNR, R² and slope

Does Choice of Reference Affect GENESIS Synthesis?

In this section, we investigate whether the choice of reference images influences the quality of our CT synthesis (or PET reconstruction). For one subject, we synthesize five synthetic CTs using five other subjects as references, as shown in Fig. 6. Visually, there is very little difference between them. The reconstructed PET images using the five different atlases were also compared with the one generated using the CT μ-map. The correlations of μ-maps from the five synthetic CT images were consistently around 0.9 as shown in Table 3. The correlations and PSNRs between each of the synthetic μ-maps and original CT μ-maps vary by only 0.49% and 1.24%, respectively. Similarly, the correlations and PSNRs of the PET images vary by only 0.01% and 0.64%. The percentage values are coefficient of variations, computed from the 5 numbers in each row of Table 3. Therefore the GENESIS results are insensitive to the choice of references.

Figure 6.

Comparison of GENESIS results using different reference data. Top row shows UTE and CT μ-map of a subject (ID #3). The similarity of all 5 images in the bottom row, each generated using a different atlas, indicates the robustness of the GENESIS method and its relative independence to choice of reference atlas.

Table 3.

For a subject, 5 synthetic μ-maps are generated using 5 different references, and are compared with CT based μ-map using correlation and PSNR. Corresponding PET reconstructions are also compared with the true CT based PET.

		Reference ID
Image type	Metric	1	2	3	4	5
μ-map	Correlation	0.9073	0.9017	0.9109	0.9017	0.9009
	PSNR	22.38	21.87	22.47	21.90	22.26
PET	Correlation	0.9979	0.9980	0.9979	0.9980	0.9982
	PSNR	36.59	36.98	36.41	36.90	36.86

4. Discussion

We have described a framework to synthesize CT images using dual echo UTE images from a reference. GENESIS does not employ deformable registration, which can sometimes suffer from poor performance when the atlas and subject images are geometrically dissimilar. Patch matching can also be susceptible to suboptimal performance if the reference data is not sufficiently rich. However, our approach compensates for this potential liability by not being limited to only patches within the reference data. GENESIS enriches the reference data by considering convex combinations of patches sampled from Gaussian mixture distributions.

The original registration based method (11) uses 27 atlases, while our implementation used only five. The accuracy of the synthetic CT increases with the number of atlases. The smaller number of atlases potentially decreases the accuracy of our implementation of the method. Nevertheless, GENESIS outperforms it even with a single reference image. Furthermore, deformable registrations usually take significant time as a preprocessing step (~1 hour with ANTS (15)). Nevertheless, a more detailed analysis comparing the performance of these two algorithms based on the number of atlases or reference images is warranted in future work.

Currently the algorithm is implemented as research software executed in post processing. Integration within a clinical workflow requires data to be pulled from a PACS or scanner, processed, and sent back to the PACS within the original study. A more seamless integration could be accomplished by optimizing the code for speed and implementing the algorithm within the Siemens Image Calculation Environment (ICE).

We have previously presented a CT synthesis method from a single T₁-w image for the sole purpose of image registration (18,19). However, standard T₁-w MR images do not have sufficient contrast to distinguish bone from air. Therefore the synthetic CT images were not as accurate the ones synthesized using dual-echo UTE. As the synthesis application was aimed to improve registration of MR and CT brain images, the imperfections in bone regions did not substantially impact the results. However, for brain PET attenuation correction, bone is critically important and this previous method would not be well suited. On the other hand, the current approach should perform quite well to improve MR-CT registration.

5. Conclusion

We have shown that by synthesizing a CT-like image from dual echo UTE images, better attenuation correction can be obtained for PET/MR systems. Our method produces synthetic μ-maps, which are closer to the CT μ-maps than both Siemens Dixon and UTE based μ-maps. We also compared with a recent registration based approach and demonstrated the limitations of registration based methods for MR to CT synthesis, particularly when the anatomy between atlas and subject significantly differ.

In this supplementary material, we derive the estimation of synthetic CT patches u_i. The Expectation-Maximization framework takes the perspective of an incomplete versus complete data problem to compute the best matching patch. We define z_it to be the indicator function that p_i comes from a GMM of the t = {j, k}^th patch pair, under the constraints that ${\sum }_{t\in \Psi }{z}_{it}=1\forall i$, z_it ∈ {j, k}^th. If we know the values of the indicator functions (the complete data), we would know what is the matching patch and we would also be able to estimate the unknown covariance matrices Σ_t and the unknown weighting coefficients α_it. The probability of observing p_i can be written as,

$$P({\mathbf{p}}_i\mid z_{it}=1;{\Sigma }_t,{\alpha }_{it})=\frac{1}{\sqrt{2\pi }\mid {\Sigma }_t\mid }\exp \left\{-\frac{1}{2}{\mathbf{h}}_{it}^T{\Sigma }_t^{-1}{\mathbf{h}}_{it}\right\},$$ (1)

where h_it = p_i− α_itq_j−(1− α_it)q_k, t ≡ {j, k}. Although we are not provided the indicator functions, the EM framework allows us to estimate the parameters by computing expectations of the indicator functions. We assume that the mixing coefficients are equal for each component of the mixture model.

We have experimentally found that any arbitrary positive definite Σ_t allows for too many degrees of freedom and is not robust to estimate. To simplify the problem, we assume it to be separable and diagonal and is given by

$${\Sigma }_t=\left[\begin{array}{cc}\hfill {\sigma }_{1t}^2\mathbf{I}\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill {\sigma }_{2t}^2\mathbf{I}\hfill \end{array}\right],$$

indicating that the variations of each voxel in a patch are the same around the means, although individual voxels can be of different tissues. Thus the joint probability becomes

$$\begin{array}{cc}\hfill & P(\mathbf{P},\mathbf{Z};ϴ)=D\prod _{t\in \Psi }\prod _{i=1}^N{\left[\frac{1}{{\sigma }_{1t}{\sigma }_{2t}}\exp \left\{-\frac{{\Vert {\mathbf{f}}_{it}\Vert }^2}{2{\sigma }_{1t}^2}\right\}\exp \left\{-\frac{{\Vert {\mathbf{g}}_{it}\Vert }^2}{2{\sigma }_{2t}^2}\right\}\right]}^{z_{it}},\hfill \\ \hfill & {\mathbf{f}}_{it}={\mathbf{x}}_i-{\alpha }_{it}{\mathbf{y}}_j-(1-{\alpha }_{it}){\mathbf{y}}_k,{\mathbf{g}}_{it}={\mathbf{u}}_i-{\alpha }_{it}{\mathbf{v}}_j-(1-{\alpha }_{it}){\mathbf{v}}_k.\hfill \end{array}$$ (2)

The set of unknown parameters are Θ = {σ_1t, σ_2t, _it}, and Z = {z_it , i = 1, . . . , N, t} ∈ ψ .

The maximum likelihood estimators of Θ are found by maximizing Eqn. 2 using EM. The E-step requires the computation of E(z_it|P, Θ^(m) = P (z_it|P, Θ^(m). Given that z_it is an indicator function, it can be shown that $E(z_{it}\mid \mathbf{P},{ϴ}^{\left(m\right)})=w_{it}^{\left(m\right)}$, where

$$w_{it}^{(m+1)}=\frac{\frac{1}{{\sigma }_{1t}^{\left(m\right)}{\sigma }_{2t}^{\left(m\right)}}\exp \left\{-\frac{{\Vert {\mathbf{f}}_{it}^{\left(m\right)}\Vert }^2}{2{\sigma }_{1t}^{{\left(m\right)}^2}}\right\}\exp \left\{-\frac{{\Vert {\mathbf{g}}_{it}^{\left(m\right)}\Vert }^2}{2{\sigma }_{2t}^{{\left(m\right)}^2}}\right\}}{{\sum }_{\ell \in \Psi }\frac{1}{{\sigma }_{1\ell }^{\left(m\right)}{\sigma }_{2\ell }^{\left(m\right)}}\exp \left\{-\frac{{\Vert {\mathbf{f}}_{i\ell }^{\left(m\right)}\Vert }^2}{2{\sigma }_{1\ell }^{{\left(m\right)}^2}}\right\}\exp \left\{-\frac{{\Vert {\mathbf{g}}_{i\ell }^{\left(m\right)}\Vert }^2}{2{\sigma }_{2\ell }^{{\left(m\right)}^2}}\right\}},$$ (3)

$w_{it}^{\left(m\right)}$ being the posterior probability of p_i originating from the Gaussian distribution of the t^th reference patches q_j and q_k. ${\mathbf{f}}_{it}^{\left(m\right)}$ and ${\mathbf{g}}_{it}^{\left(m\right)}$ are the expressions defined in Eqn. 2 but with ${\alpha }_{it}^{\left(m\right)}$ and ${\mathbf{f}}_{i\ell }^{\left(m\right)}$ denote the corresponding values with reference patches belonging to the ${\mathbf{g}}_{i\ell }^{\left(m\right)}$ pair, $\ell \in \Psi$, with ${\alpha }_{i\ell }^{\left(m\right)}$. The synthetic patches are obtained by the following expectation,

$$E({\mathbf{u}}_i\mid {ϴ}^{\left(m\right)})=\sum _{t\in \Psi }{w}_{it}^{\left(m\right)}\left({\alpha }_{it}^{\left(m\right)}{\mathbf{v}}_j+(1-{\alpha }_{it}^{\left(m\right)}){\mathbf{v}}_k\right).$$

At each iteration, we replace the value of u_i with its expectation. The M-step involves the maximization of the log of the expectation with respect to the parameters given the current w^(m)_it. The update equations are given by,

$${\sigma }_{1t}^{{(m+1)}^2}=\frac{{\sum }_{i=1}^N{w}_{it}^{\left(m\right)}\Vert {\mathbf{x}}_i-{\alpha }_{it}^{\left(m\right)}{\mathbf{y}}_j-{(1-{\alpha }_{it}^{\left(m\right)}{\mathbf{y}}_k\Vert }^2}{{\sum }_{i=1}^N{w}_{it}^{\left(m\right)}},$$ (4)

$${\sigma }_{2t}^{{(m+1)}^2}=\frac{{\sum }_{i=1}^N{w}_{it}^{\left(m\right)}\Vert {\mathbf{u}}_i-{\alpha }_{it}^{\left(m\right)}{\mathbf{v}}_j-{(1-{\alpha }_{it}^{\left(m\right)}{\mathbf{v}}_k\Vert }^2}{{\sum }_{i=1}^N{w}_{it}^{\left(m\right)}},$$ (5)

$$\begin{array}{cc}\hfill & {\alpha }_{it}^{\left(m\right)}:F\left({\alpha }_{it}^{\left(m\right)}\right)=0,\text{where}F\left(x\right)=A{x}^2(1-x)-Bx(1-x)+2x-1.\hfill \\ \hfill & A=\frac{{\Vert {\mathbf{y}}_k-{\mathbf{y}}_j\Vert }^2}{{\sigma }_{1t}^{{\left(m\right)}^2}}+\frac{{\Vert {\mathbf{v}}_k-{\mathbf{v}}_j\Vert }^2}{{\sigma }_{2t}^{{\left(m\right)}^2}},\hfill \\ \hfill & B=\frac{{({\mathbf{y}}_k-{\mathbf{x}}_i)}^T({\mathbf{y}}_k-{\mathbf{y}}_j)}{{\sigma }_{1t}^{{\left(m\right)}^2}}+\frac{{({\mathbf{v}}_k-{\mathbf{u}}_i)}^T({\mathbf{v}}_k-{\mathbf{v}}_j)}{{\sigma }_{2t}^{{\left(m\right)}^2}}.\hfill \end{array}$$ (6)

It should be noted that F (0) = −1, F (1) = 1, ∀ A, B, thus there is always a feasible ${\alpha }_{i\ell }^{\left(m\right)}\in (0,1)$. The EM algorithm is said to converge at iteration m, if $\Vert {\mathbf{u}}_i^{(m+1)}-{\mathbf{u}}_i^{\left(m\right)}\Vert <\delta$ for some small δ. Once the EM algorithm has converged, the expectation of the final u_i is considered the synthetic CT patch, and the center voxel of u_i is used as the CT replacement of the i^th voxel.

The model assumes all possible pairs $\left(\begin{array}{c}\hfill \mathrm{M}\hfill \\ \hfill 2\hfill \end{array}\right)$ of reference patches. Thus the complexity of the model is $\mathcal{O}\left(N{M}^2\right)$. For a typical 1mm³ UTE scan, M, N ~ 10⁷. Thus it is almost infeasible to solve with all possible reference patches. However, the Gaussian model is valid for those reference and subject patches that are close in intensity. Using a non-local type of criteria, for every subject patch x_i, we start with a feasible set of L reference patches such that they are the L nearest neighbors of x_i. Thus the i^th subject patch follows an $\left(\begin{array}{c}\hfill L\hfill \\ \hfill 2\hfill \end{array}\right)$-class GMM and the complexity becomes $\mathcal{O}\left(N{L}^2\right)$. In all our experiments, we choose 3 × 3 × 3 patches (d = 27), L = 20, and δ= 0.001 max(a₃).