Joint Estimation of Anatomy and Implants in X-ray CT using a Mixed Prior Model

Abstract

Medical implants are often made of dense materials and pose great challenges to accurate CT reconstruction and visualization, especially in regions close to or surrounding implants. Moreover, it is common that diagnostics involving implanted patients require distinct visualization strategies for implants and anatomy indvidually. In this work, we propose a novel approach for joint estimation of anatomy and implants as separate image volumes using a mixed prior model. This prior model leverages a learning-based diffusion prior for the anatomy image and a simple 0-norm sparsity prior for implants to decouple the two volumes. Additionally, a hybrid mono-polyenergetic forward model is employed to effectively accommodate the spectral effects of implants. The proposed reconstruction process alternates between two steps: Diffusion posterior sampling is used to update the anatomy image, and classic optimization updates to the implant image and associated spectral coefficients. Evaluation in spine imaging with metal pedicle screw implants demonstrates that the proposed algorithm can achieve accurate decompositions. Moreover, anatomy reconstruction between the two pedicle screws, an area where all competing algorithms typically fail, is successful in visualizing details. The proposed algorithm also effectively avoids streaking and beam hardening artifacts in soft tissue, achieving 15.25% higher PSNR and 24.29% higher SSIM compared to normalized metal artifacts reduction (NMAR). These results suggest that mixed prior models can help to separate spatially and spectrally distinct objects that differ from standard anatomical features in ordinary single-energy CT to not only improve image quality but to enhance visualization of the two distinct image volumes.

1. INTRODUCTION

Medical implants are essential devices used in various therapeutic and diagnostic applications to restore or enhance physiological function. Implants can range from orthopedic devices, such as joint replacements and spinal screws,^{1, 2} to cardiovascular devices, including stents and pacemakers.³ Dense alloys are often used to manufacture medical implants,⁴ which poses challenges to Computed Tomography (CT) imaging, because metal can severely attenuate the incident photons, resulting in significant beam-hardening⁵ and streaking artifacts which may extend to the whole CT image. Clinical diagnosis or surgery can be hampered by this kind of artifacts since the region-of-interest (ROI) is often close to metal.² Additionally, diagnostics associated with implants often involves independent visualization of the anatomy and the implant (in part due to the high dynamic range between metal and human tissue). Visualization can include overlay image of implant over anatomy and/or segmentation of implants to obtain boundary information which is overlaid, or independently analyzed.

Various methods have been investigated to improve CT reconstruction with implants.⁶ Many approaches treat this task as a missing data problem, trying to correct the metal-corrupted projections or reconstruct images from metal-free projections. While this strategy effectively removes most artifacts, it ignores information from the metal-corrupted projections, resulting in compromised performance around metal. Previous work called known-component reconstruction (KCR)^{7, 8} sought to decouple patient anatomy and implants using known models of the implants. This decoupling permits both independent modeling of the differing spectral characteristics of the implant versus anatomy - leading to improved image quality particularly in the vicinity of the implant, and also provides independent image volumes which facilitates various visualization strategies.

However, the precise prior knowledge KCR relies on, such as the exact shape and material composition of implants, is not always available in practice. Recent advances in score-based generative model⁹ provide an effective way to model an anatomical prior. In this work, we shift strong prior from implant to anatomy, and apply a mixed prior model for joint estimation of anatomy and implants. Specifically, we employ a sophisticated diffusion prior for the anatomical image, and a simple sparsity prior for the implant image. Additionally, a hybrid mono-polyenergetic forward model is employed to further accommodate the spectral effects of implants. The proposed algorithm is evaluated in spine imaging with pedicle screws. Simulation results demonstrate that the proposed algorithm can achieve accurate anatomy and implant reconstruction.

2. METHODS

2.1. Hybrid Mono-polyenergetic Forward Model

Consider a two-material measurement model for patient anatomy (e.g., water) with an implant (some other material):

$$y~𝒩\left(\bar{y},{\sigma }^2\right),\bar{y}=\int S\left(E\right)\text{exp}\left\{-\int x_a(E,\mathbf{r})\mathrm{d}\mathbf{r}-\int x_m(E,\mathbf{r})\mathrm{d}\mathbf{r}\right\}\text{d}E$$ (1)

where $S\left(E\right)$ is the spectral sensitivity of the system including x-ray spectrum, detector response, etc. as a function of energy, $E$. The terms $x_a$ and $x_m$ are the energy-dependent linear attenuation coefficients of the anatomy and implant. (We approximate the patient as being water only.) The polyenergetic spectrum and the energy-dependent material coefficients together result in beam-hardening⁵ and associated artifacts when reconstruction based on a monoenergetic model is applied. In practice, corrections^{10, 11} attempt to mitigate the beam hardening, resulting in an approximately monoenergetic forward model. However, the spectral effects of implants are much pronounced than normal tissue, and analogous processing for implants can be challenging due to unknown material composition and distribution. To address this issue, this work employs a model similar to that of previous work,⁸ which uses a monoenergetic model for the anatomical volume and a polyenergetic model for the implant volume:

$$\bar{y}\approx \underset{\mathit{\text{Mono}}\mathit{\text{model}}}{\underbrace{I_0\text{exp}\left\{-\int \overline{x_a}\left(\mathbf{r}\right)\mathrm{d}\mathbf{r}\right\}}}\underset{\mathit{\text{Poly}}\mathit{\text{model}}}{\underbrace{\text{exp}\left\{-\sum _{n=1}^N{c}_n{\left\{\int \overline{x_m}\left(\mathbf{r}\right)\mathrm{d}\mathbf{r}\right\}}^n\right\}}}$$ (2)

where $I_0=\int S\left(E\right)\text{d}E,\overline{x_a}$ and $\overline{x_m}$ are the effective (spectrum-independent) linear attenuation coefficients. Moreover, consider the case where the implant material is unknown, but the energy-dependent attenuation of the implant is modeled by a $N$th order polynominal function parameterized by “spectral” coefficients $\left\{c_n\mid n=\right.\mathrm{1,2}\dots N\}$. After discretization and log-normalization, the forward model can be rewritten as:

$$\mathbf{p}=\text{log}\left(I_0/\mathbf{y}\right)~𝒩(\overline{\mathbf{p}},\mathbf{\Sigma }),\overline{\mathbf{p}}=\text{log}\left(I_0/\mathbf{y}\right)=\mathbf{A}{\mathbf{x}}_a\odot \sum _{n=1}^N{c}_n{\left({\mathbf{A}\mathbf{x}}_m\right)}^n$$ (3)

where $\mathbf{A}$ is the system matrix. For independent measurements, covariance matrix $\mathbf{\Sigma }$ is approximated by diagonal matrix $\mathbf{D}\left\{\mathbf{y}\right\}$. We seek to estimate $x_a,x_m$, and $c_n$.

2.2. Mixed Prior Model for Anatomy and Implant

Joint estimation of anatomy ${\mathbf{x}}_a$ and implant ${\mathbf{x}}_m$ from measurements $\mathbf{p}$ is underdetermined. In this work, a mixed prior model, i.e., a deep learning prior for the anatomy and sparsity prior for the implant, is employed to facilitate this joint reconstruction. Specifically, the anatomy prior is captured by a diffusion model,¹² which samples from $p_{\theta }\left({\mathbf{x}}_a\right)\approx p\left({\mathbf{x}}_a\right)$ through a stochastic differential equation:⁹

$$\mathrm{d}\mathbf{x}=\left[-\frac{{\beta }_t}{2}\mathbf{x}-{\beta }_t{\mathbf{s}}_{\theta }(\mathbf{x},t)\right]\mathrm{d}t+\sqrt{{\beta }_t}\mathrm{d}\overline{\mathbf{w}},{\mathbf{s}}_{\theta }\left(\mathbf{x},t\right)\approx {\nabla }_{\mathbf{x}}\text{log}{p}_t\left(\mathbf{x}\right).$$ (4)

The gradient of log prior probability density ${\nabla }_{\mathbf{x}}\text{log}{p}_t\left(\mathbf{x}\right)$, referred to as the score function, is approximated by a neural network. We presume the metal implant to be relatively small and only occupy a limited number voxels in the volume, and thus, employ a sparsity prior for the implant volume:

$$-\text{log}p\left({\mathbf{x}}_m\right)\propto \lambda {‖{\mathbf{x}}_m‖}_0.$$ (5)

2.3. Joint Anatomy and Implant Reconstruction

Combining the hybrid mono-polyenergetic forward model and the mixed prior model, the objective function for joint anatomy and implant reconstruction is written as:

$$ℒ\left({\mathbf{x}}_a,{\mathbf{x}}_m,\mathbf{c};\mathbf{p}\right)=\underset{\mathit{\text{Likelihood}}}{\underbrace{{‖\mathbf{A}{\mathbf{x}}_a\odot \sum _{n=1}^N{c}_n{\left(\mathbf{A}{\mathbf{x}}_m\right)}^n-\mathbf{p}‖}_{\mathbf{y}}^2}}-\underset{\mathit{\text{Anatomy}}\mathit{\text{prior}}}{\underbrace{\text{log}{p}_{\theta }\left({\mathbf{x}}_a\right)}}-\underset{\mathit{\text{Implant}}\mathit{\text{prior}}}{\underbrace{\text{log}p\left({\mathbf{x}}_m\right)}}.$$ (6)

Note that the spectral coefficients $\mathbf{c}$ are jointly estimated with the anatomy and implant volumes. Eq.(6) is solved by alternating updates of ${\mathbf{x}}_a$ and $({\mathbf{x}}_m,\mathbf{c})$. Specifically, The implant volume and spectral coefficients are fixed in an anatomy volume update, and then vice versa.

The objective for the anatomy update can be written as:

$$ℒ\left({\mathbf{x}}_a;{\mathbf{x}}_m^{\left(k\right)},{\mathbf{c}}^{\left(k\right)},\mathbf{p}\right)={‖\mathbf{A}{\mathbf{x}}_a\odot \sum _{n=1}^N{c}_n{\left(\mathbf{A}{\mathbf{x}}_m^{\left(k\right)}\right)}^n-\mathbf{p}‖}_{\mathbf{y}}^2-\text{log}{p}_{\theta }\left({\mathbf{x}}_a\right)$$ (7)

where $k$ is the iteration index. Eq.(7) is solved via a Diffusion Posterior Sampling (DPS) CT reconstruction algorithm:¹³

$${\hat{\mathbf{x}}}_0=\frac{1}{{\sqrt{\bar{\alpha }}}_t}\left({\mathbf{x}}_t-\sqrt{1-{\bar{\alpha }}_t}{\mathit{ϵ}}_{\theta }\left({\mathbf{x}}_t,t\right)\right)$$ (8a)

$${\mathbf{x}}_a^{(k+1)}=\text{Adam}\left({‖\mathbf{A}{\mathbf{x}}_a\odot \sum _{n=1}^N{c}_n{\left(\mathbf{A}{\mathbf{x}}_m^{\left(k\right)}\right)}^n-\mathbf{p}‖}_{\mathbf{y}}^2,{\hat{\mathbf{x}}}_0,{\eta }_a,N_a\right)$$ (8b)

$${\mathbf{x}}_{t-1}=\frac{\sqrt{{\alpha }_t}\left(1-{\bar{\alpha }}_{t-1}\right)}{1-{\bar{\alpha }}_t}{\mathbf{x}}_t+\frac{\sqrt{{\bar{\alpha }}_{t-1}}{\beta }_t}{1-{\bar{\alpha }}_t}{\hat{\mathbf{x}}}_0+{\sigma }_t\mathbf{z}+{\mathbf{x}}_a^{(k+1)}-{\hat{\mathbf{x}}}_0$$ (8c)

Eq.(8a) is a one-step estimation of the anatomy using Tweedie’s formula.¹⁴ Eq.(8b) further modifies the anatomy volume to update the data consistency using the Adam optimizer.¹⁵ The parameters of Adam optimizer are (from right to left): the loss function, an initialization, step size, and number of iterations. Eq.(8c) computes the next diffusion state based on current state, estimated score, and anatomy update in Eq.(8b).

The objective for the implant and coefficients update is:

$$ℒ\left({\mathbf{x}}_m,\mathbf{c};{\mathbf{x}}_a^{(k+1)},\mathbf{p}\right)={‖\mathbf{A}{\mathbf{x}}_a^{(k+1)}\odot \sum _{n=1}^N{c}_n{\left(\mathbf{A}{\mathbf{x}}_m\right)}^n-\mathbf{p}‖}_{\mathbf{y}}^2+\lambda {‖{\mathbf{x}}_m‖}_0.$$ (9)

where $\lambda$ controls the strength of the sparsity prior. We introduce an auxiliary variable $\mathbf{u}$ for ${\mathbf{x}}_m$:

$$ℒ\left({\mathbf{x}}_m,\mathbf{c},\mathbf{u};{\mathbf{x}}_a^{(k+1)},\mathbf{p}\right)={‖\mathbf{A}{\mathbf{x}}_a^{(k+1)}\odot \sum _{n=1}^N{c}_n{\left(\mathbf{A}{\mathbf{x}}_m\right)}^n-\mathbf{p}‖}_{\mathbf{y}}^2+\lambda \Vert \mathbf{u}{\Vert }_0+\rho {‖{\mathbf{x}}_m-\mathbf{u}‖}_2^2$$ (10)

where $\rho$ controls the similarity between $\mathbf{u}$ and ${\mathbf{x}}_m$. Eq.(10) can be solved by alternating optimization: The implant volume and spectral coefficients are first updated as:

$$\left\{{\mathbf{x}}_m^{(k+1)},{\mathbf{c}}^{(k+1)}\right\}=\text{argmin}{‖\mathbf{A}{\mathbf{x}}_a^{(k+1)}\odot \sum _{n=1}^N{c}_n{\left(\mathbf{A}{\mathbf{x}}_m\right)}^n-\mathbf{p}‖}_{\mathbf{y}}^2+\rho {‖{\mathbf{x}}_m-{\mathbf{u}}^{\left(k\right)}‖}_2^2,$$ (11)

then the auxiliary variable $\mathbf{u}$ is updated by minimizing:

$${\mathbf{u}}^{(k+1)}=\text{argmin}\lambda \Vert \mathbf{u}{\Vert }_0+\rho {‖{\mathbf{x}}_m^{(k+1)}-\mathbf{u}‖}_2^2.$$ (12)

Eq.(11) enforces data consistency while maintaining similarity between ${\mathbf{x}}_m$ and $\mathbf{u}$. It is solved approximately using a fixed number of iterations for the likelihood minimization, and the similarity constraint can be implicitly added by starting the iteration from ${\mathbf{u}}^{\left(k\right)}$. Eq.(12) is considered as a 0-norm proximal operator on ${\mathbf{x}}_m^{(k+1)}$.¹⁶ Therefore, Eq.(10) is solved by:

$$\left\{{\mathbf{x}}_m^{(k+1)},{\mathbf{c}}^{(k+1)}\right\}=\text{Adam}\left({‖\mathbf{A}{\mathbf{x}}_a^{(k+1)}\odot \sum _{n=1}^N{c}_n{\left(\mathbf{A}{\mathbf{x}}_m\right)}^n-\mathbf{p}‖}_{\mathbf{y}}^2,\left\{{\mathbf{u}}^{\left(k\right)},{\mathbf{c}}^{\left(k\right)}\right\},\left\{{\eta }_m,{\eta }_c\right\},\left\{N_m,N_c\right\}\right)$$ (13a)

$${\mathbf{u}}^{(k+1)}=\left({\mathbf{x}}_m^{(k+1)}\odot {\mathbf{x}}_m^{(k+1)}\ge \lambda /\rho \right)\odot {\mathbf{x}}_m^{(k+1)}$$ (13b)

In this work, the reconstruction runs 150 iterations of Eq. 8 and Eqs.13, and hyperparameters are empirically chosen with: ${\eta }_a={\eta }_m={\eta }_c={10}^{-3},N_a=3,N_m=N_c=1,\lambda /\rho =2.5\times {10}^{-3}$.

2.4. Evaluation

Simulation Setup:

We simulated the abdomen phantom as shown in Fig. 1. The water and calcium basis are created by soft thresholding the single-energy CT as in Jiang et al.¹⁷ Two homogeneous pedicle screws (composition: 87% Ti, 6% Al, 7%Nb, density: 4.5 g/ml) are placed in the spine. The water/calcium density is set to zero where masked by the screws. Polyenergetic projections are simulated for this three-basis phantom, with the scanning technique set to 120 kVp, 0.4 mAs/view, 720 views.

Figure 1.

The abdomen phantom used in the simulation study. This phantom is composed of water and calcium bases for the anatomy, and a metal implant basis.

Comparison Algorithms:

The proposed algorithm was compared with filter backprojection (FBP), model-based reconstruction (MBIR), and normalized metal artifacts reduction (NMAR). (Monoenergetic) MBIR is formulated as:

$${\mathbf{x}}^{*}=\text{argmin}\Vert \mathbf{A}\mathbf{x}-p{\Vert }_{\mathbf{y}}+\lambda \mathbf{R}\left(\mathbf{x}\right)$$ (14)

$\mathbf{R}$ represents a simple quadratic regularization that penalizes the differences between first-order neighborhoods. Eq.(14) is solved using a separable paraboloidal surrogates (SPS) algorithm.¹⁸ For NMAR, the metal trace is generated by projecting the exact metal mask, and the corrected projection are reconstructed by FBP. All the FBP reconstructions are implemented with a Hamming window and a cutoff of 0.8×Nyquist frequency.

3. RESULTS

Fig. 2 displays reconstructions using different algorithms. The baseline image was generated by MBIR reconstruction on noiseless and implant-free projections. In the single screw case, due to photon starvation and beam hardening, FBP results exhibited shading and blooming artifacts around implant, as well as streaks throughout the image. While MBIR mitigated noise, it failed to correct the major streaking artifacts through the implant. NMAR replaced the noisy and hardened projection with interpolated value to largely eliminate streaking. However, the interpolation lost significant information, and led to a empty area (yellow arrow) around the implant. In contrast, the proposed algorithm most accurately depicted the tissue surrounding metal implant, including along the major axis of the screw where imaging was most challenging (blue arrow). Moreover, we note that the joint reconstruction provided an independent estimate of the implant volume without explicit segmentation. We used this information to overlay the pedicle screws in red.

Figure 2.

Reconstruction results using different algorithms. For the proposed method, the implant image is color-encoded and overlay with the anatomy image. The green box defines a zoomed spine ROI containing the implant. Brown and blue boxes indicate two soft tissue ROIs near the implant, Display window: [0.01,0.03]cm⁻¹.

Reconstruction of the case with two pedicle screws was more challenging, especially for the center of the spine, because almost all the projections pass through implants. With very limited metal-free projections, NMAR obscured almost all of the internal spinal structures. However, the proposed algorithm accurately depicted the spine with a relatively clear reconstruction of the spinal cord (purple arrow). In addition to the spine ROI containing implants, two ROIs around spine are displayed in Fig. 3 to investigate the soft tissue reconstruction.

Figure 3.

Soft tissue ROIs of the double screw case in Fig.2. PSNR and SSIM of each ROI are listed at top left corner. Display window: [0.015,0.025]cm⁻¹.

The proposed algorithm effectively removed artifacts and streaky noise without blurring the images, providing the best artery visualization and clearest muscle-fat boundary. Compared to NMAR, the proposed algorithm improved PSNR by 15.25% and 16.45% and improved SSIM by 24.29% and 10.47%, further demonstrating the superior accuracy of proposed algorithm. Fig. 4 compares results from a monoenergetic implant model, i.e., fix $\mathbf{c}$ as [1.0, 0.0, 0.0, 0.0], versus a polyenergetic implant model. The polyenergetic model exhibited less blooming and shading artifacts, particularly around the ends of screws as indicated by green arrows, which demonstrates that the polyenergetic metal model, though having additional unknown parameters to be estimated, was highly effective in improving the reconstruction.

Figure 4.

Comparison of proposed reconstructions with the monoenergetic metal model(left) and the polyenergetic metal model(right). Display window: [0.01,0.03]cm⁻¹.

4. CONCLUSION AND DISCUSSION

The joint reconstruction of anatomy and implant using single-energy projections is an underdetermined problem that requires prior information to solve. In this work, we proposed a novel estimation approach using a mixed prior model, which employs a learning-based diffusion prior for the anatomy and a simple 0-norm sparsity prior for implants. The reconstruction process alternates between two subproblems: one uses diffusion posterior sampling to update the anatomy volume, while classic optimization updates are applied to the implant volume and the spectral coefficients. The algorithm was evaluated on spine imaging with screw implants, demonstrating promising performance in accurate joint reconstruction, especially in regions surrounding the implants.

The combination of a spectrally parameterized forward model and a mixed prior model allows one to separate two distinct material types in an image volume. Parts of the image volume that are well-modeled with a monoenergetic form and that fit the learned prior distribution can be estimated as anatomy. In contrast, sparse elements of the volume that have similar transmission properties and that can only be well-modeled using a polyenergetic form are estimated in the implant volume. While this work demonstrates the success of the approach in reducing metal artifacts, there are likely other applications where spectrally distinct elements of the volume may be separated as part of the reconstruction process. Such applications would likely benefit from having sophisticated prior models for both materials - though the simple sparsity prior in the mixed model presented here was sufficient for this scenario.