Demons deformable registration of CT and cone-beam CT using an iterative intensity matching approach

Abstract

Purpose: A method of intensity-based deformable registration of CT and cone-beam CT (CBCT) images is described, in which intensity correction occurs simultaneously within the iterative registration process. The method preserves the speed and simplicity of the popular Demons algorithm while providing robustness and accuracy in the presence of large mismatch between CT and CBCT voxel values (“intensity”).

Methods: A variant of the Demons algorithm was developed in which an estimate of the relationship between CT and CBCT intensity values for specific materials in the image is computed at each iteration based on the set of currently overlapping voxels. This tissue-specific intensity correction is then used to estimate the registration output for that iteration and the process is repeated. The robustness of the method was tested in CBCT images of a cadaveric head exhibiting a broad range of simulated intensity variations associated with x-ray scatter, object truncation, and∕or errors in the reconstruction algorithm. The accuracy of CT-CBCT registration was also measured in six real cases, exhibiting deformations ranging from simple to complex during surgery or radiotherapy guided by a CBCT-capable C-arm or linear accelerator, respectively.

Results: The iterative intensity matching approach was robust against all levels of intensity variation examined, including spatially varying errors in voxel value of a factor of 2 or more, as can be encountered in cases of high x-ray scatter. Registration accuracy without intensity matching degraded severely with increasing magnitude of intensity error and introduced image distortion. A single histogram match performed prior to registration alleviated some of these effects but was also prone to image distortion and was quantifiably less robust and accurate than the iterative approach. Within the six case registration accuracy study, iterative intensity matching Demons reduced mean TRE to (2.5±2.8) mm compared to (3.5±3.0) mm with rigid registration.

Conclusions: A method was developed to iteratively correct CT-CBCT intensity disparity during Demons registration, enabling fast, intensity-based registration in CBCT-guided procedures such as surgery and radiotherapy, in which CBCT voxel values may be inaccurate. Accurate CT-CBCT registration in turn facilitates registration of multimodality preoperative image and planning data to intraoperative CBCT by way of the preoperative CT, thereby linking the intraoperative frame of reference to a wealth of preoperative information that could improve interventional guidance.

INTRODUCTION

The development of high-quality 3D intraoperative imaging is important to advancing surgical procedures presenting complex anatomical targets and infiltrative disease proximal to critical tissues, as in surgical management of head and neck disease.¹ Cone-beam CT (CBCT) implemented on a surgical C-arm shows particular promise as an imaging platform providing high-quality intraoperative 3D visualization of bony and soft-tissue anatomy, upon which advanced image-guided surgery (IGS) systems can be developed.^{2, 3, 4} C-arm CBCT has the advantage of an open geometry, dual-modality platform (fluoroscopy+CBCT) compatible with the surgical setup and, by providing up-to-date imaging that accounts for deformations and excisions imparted during the procedure, can enable increased surgical performance.^{5, 6, 7, 8}

The utility of intraoperative CBCT is improved by comparison with preoperative imaging and surgical planning data (e.g., definition of surgical trajectories, anatomical landmarks, critical structures, and targets). Conventional IGS systems which rely solely on preoperative imaging tacitly assume that patient anatomy does not change throughout the procedure and that a rigid transformation (translation and rotation) relates the anatomy in the preoperative images to that of the patient at the time of treatment. Complicated head and neck surgeries, including those using intraoperative 3D imaging for guidance relative to soft-tissue targets, challenge this assumption of static anatomy. Differences in patient anatomy between preoperative and intraoperative imaging arise due to changes in patient setup, deformation, introduction of foreign objects (e.g., intubation and instrumentation), and due to the changes imparted as a result of the surgical procedure itself. Therefore, to effectively interpret intraoperative imaging with respect to preoperative imaging and planning, the images must be brought into the same spatial and anatomical frame of reference in a manner that accounts for deformations and anatomical changes, i.e., a deformable registration algorithm that accurately accounts for anatomical changes while running within the limited time constraints of the operating theater. Deformable registration for image-guided procedures continues to be an active area of research and selection of an algorithm for a particular application often involves tradeoffs between speed, accuracy, robustness, matching ability, and smoothness of the registration transform.

The Demons algorithm⁹ is a popular technique for fast intensity-based registration and has been applied to a range of applications.^{10, 11, 12, 13, 14} In a previous work, we reported a variation of the algorithm that incorporated a “smart” convergence criterion for a multiresolution implementation, motivated by goals of fast, accurate registration.¹⁵ Results in cadaver and patient studies demonstrated CBCT-CBCT registration accuracy comparable to voxel size within ∼10–60 s (in a CPU-based implementation). The standard Demons algorithm can be considered an approximation of a second-order gradient descent on the sum of square of intensity differences metric¹⁶ and thus would not normally be considered a good choice for multimodality image registration where the assumption of constant image intensity (i.e., voxel value) for equivalent anatomical points between scans cannot be guaranteed. For surgical planning, the preoperative imaging modality of choice is often diagnostic CT and thus CT to CBCT registration is required for intraoperative guidance.

Although CBCT is based on the same physical principles as CT, the assumption of constant intensity values can fail due to physical factors and differences in the reconstruction algorithm. For example, scattered x-ray photons comprise a large portion of the detected signal in CBCT due to the large volumetric field of view (FOV) and variable use of an antiscatter grid. The high-scatter fluence results in tissue contrast reduction, shading artifacts, and, most importantly for this study, a reduction in CBCT voxel intensities. Such effects lead to intensity deviations in CBCT images typically greater than 30%, with intensity deviations larger than 100% possible under high-scatter conditions.¹⁷ A second physical factor typical in CBCT guidance is object truncation, in which the size of the patient (and operating table) exceeds the reconstruction FOV and causes a lack of normalization in voxel values, typically overestimated near the edge of the FOV and underestimated near the reconstruction center. Finally, CBCT intensity errors can arise from the reconstruction algorithm itself, e.g., through a lack of air calibration, a different calibration from that used in a particular diagnostic CT scanner, and∕or a lack of Hounsfield unit (HU) conversion altogether. While methods for improving HU accuracy in CBCT are under development, they are far from universally established and significant HU errors present a persistent challenge in state-of-the-art CBCT. Under such variable conditions, even within a given procedure, it becomes clear that CT-CBCT intensity differences pose a problem for intensity conservation assuming registration algorithms such as the Demons algorithm, with even relatively small HU uncertainties noted as a source of registration error.¹²

For image registration applications where intensity conservation assumptions cannot be made, previous works have shown the value of estimating an intensity correction and spatial transform simultaneously.^{18, 19, 20} Furthermore, to correct CBCT intensity inaccuracy, it has been shown as advantageous to segment the image into different “classes” representing different material types (e.g., air, soft tissue, and bone).²¹ In this work, we evaluate an implementation of the Demons algorithm that combines these ideas by iteratively estimating both a spatial transform and tissue-specific intensity correction while assuming a low-order model for the intensity relationship that exploits the inherent similarity (and common physical contrast mechanism) of CT and CBCT images. In this way, we hope to preserve the computational efficiency of the Demons approach while achieving accurate CT-CBCT registration for intraoperative use. We study the effect of CBCT intensity inaccuracy on registration performance in a series of simulations presenting various magnitudes of intensity inaccuracy and assess the performance of different correction methods, including (i) a naïve registration with no intensity match (NIM); (ii) a single histogram match (SHM) performed prior to registration; and (iii) the proposed iterative intensity match (IIM) approach. We then quantify the accuracy of the IIM method in cadaveric and patient data spanning a range of deformations from simple to complex and with real CBCT intensity variations.

METHODS

Registration method

Demons deformable registration

The Demons algorithm and its variants are a popular method of performing fast deformable 3D registration, particularly within a given modality where image intensity values (i.e., voxel values) are consistent. For deformable image registration in CBCT-guided head and neck procedures, we choose a variant of the Demons displacement force which uses gradient information from both the fixed and moving images. Previous work demonstrated that the use of the “symmetric” displacement equation performed better than the conventional (asymmetric gradient information) displacement equation in terms of overall registration quality and number of iterations required for convergence,²² which is consistent with other experimental and theoretical works.^{11, 13} After initialization by rigid registration of bony anatomy, the deformable registration algorithm consists of two steps repeated iteratively.²³ First, at every overlapping position x in the two images, a displacement vector d is calculated as

$$\stackrel{⃑}{d}\left(\mathit{x}\right)=\frac{2\left[I_0\left(x\right)-I_1\left(x\right)\right]\left[\stackrel{⃑}{\nabla }{I}_0\left(x\right)+\stackrel{⃑}{\nabla }{I}_1\left(x\right)\right]}{{\parallel \stackrel{⃑}{\nabla }{I}_0\left(x\right)+\stackrel{⃑}{\nabla }{I}_1\left(x\right)\parallel }^2+{\left[I_0\left(x\right)-I_1\left(x\right)\right]}^2∕K^2},$$ (1)

where I₀ is the moving image (the preoperative CT) and I₁ is the fixed image (the intraoperative CBCT). This choice of moving and fixed image is such that the preoperative image and planning data are brought into the context of the most up-to-date anatomy at the time of intervention. The respective gradient images are denoted $\stackrel{⃑}{\nabla }{I}_0$ and $\stackrel{⃑}{\nabla }{I}_1$, respectively, and the normalization factor K² is the mean-squared value of the image voxel size¹⁶

$$K^2=a_x^2+a_y^2+a_z^2,$$ (2)

where a is the voxel size in the x, y, and z directions of the fixed image. At iteration n, an update field U_n of displacement vectors arising from Eq. 1 at every voxel is added to the existing deformation field and, to ensure smoothness, the result is convolved with a 3D Gaussian kernel G_σ, with a kernel size of 1 mm chosen in this study. The resulting deformation field for the nth iteration is

$$D_n=G_{\sigma }\ast \left(D_{n-1}+U_n\right).$$ (3)

Equations 1, 3 are repeated until convergence. We make use of a previously reported convergence criterion based on the difference between successive deformation fields:¹⁵ After each iteration, the difference between D_n and D_n−1 is calculated by voxelwise vector subtraction and, when the magnitude of at least 99% of these difference vectors is smaller than one-tenth the voxel size, the registration is considered convergent. The multiresolution implementation used a morphological pyramid of input images downsampled at factors of 8, 4, 2, and 1, a well-known technique to improve the robustness of registration while decreasing overall computation time.²⁴ The registration process is illustrated in Fig. 1, with light gray portions describing the (previously reported) aspects mentioned above and the bold black portions relating to the IIM approach (described below).

Figure 1.

Flowchart illustrating the incorporation of iterative intensity matching of CT-CBCT voxel values within the Demons algorithm. The light gray parts of the flowchart illustrate nominal multiresolution registration with convergence criterion. The black parts of the flowchart illustrate the tissue-specific intensity match performed in each iteration.

Techniques for multimodality registration

Development of the Demons algorithm was based on heuristic arguments motivated by the desire for fast and efficient image registration with a force equation inspired by optical flow;^{9, 23} this gave rise to an algorithm that would later be understood as an approximation of an optimization process with mean-square difference as the image similarity metric.²⁵ A variety of image metrics exist with more relaxed requirements regarding the intensity relationship between equivalent anatomical points. These include, but are not limited to, cross-correlation (which assumes a linear intensity relationship),²⁶ rank correlation coefficient (which depends on a monotonic relationship),²⁷ correlation ratio which requires a functional relationship,²⁸ and many popular varieties of mutual information techniques (which assume only a statistical relationship between image intensities).²⁹ The image metric being optimized in the Demons algorithm is integrated within the force calculation and cannot simply be changed in a modular fashion as with certain other registration approaches. Recent works have begun to explore Demons variants incorporating such metrics for multimodality registration.^{30, 31} The ideal way to incorporate alternative metrics into Demons style algorithms is not yet established and remains an area of active research.

Alternatively, there exist free-form deformation algorithms featuring a more easily modified image similarity metric component and which generally perform explicit optimization of a parametrized transformation.³² Such techniques are popularly applied in applications such as CT-MR or MR-MR (differing pulse sequences) registration, where the contrast mechanisms are completely different. Unfortunately, such techniques sometimes involve increased computational complexity to a level no longer compatible with the timescales of intraoperative use.

While there may be a mismatch between CT and CBCT voxel values, the underlying contrast mechanism (attenuation coefficient) is the same, so we expect a monotonic relationship between the two (as opposed, for example, to registering T1 and T2 MR images). Therefore, in this study, instead of abandoning the mean-square based Demons algorithm, which has the advantages of computational efficiency and ease of implementation, we sought to develop and evaluate a version of the Demons algorithm which modifies the input images iteratively and simultaneously with the registration algorithm, such that strong assumptions of intensity similarity can still be exploited.

Iterative intensity matching for CT-CBCT registration

The approach investigated in this work involves an intensity correction within the iterative operation of the Demons algorithm as shown in Fig. 1. By performing an intensity match as part of the registration procedure, it is possible to exploit the spatial correspondence of overlapping voxels as estimated by the current deformation field, an estimate which was hypothesized to improve every iteration, allowing a subsequent improvement in the intensity correction step of the next iteration. In addition, an independent intensity correction is performed for different classes of materials (i.e., air, soft tissues, and bone). The relationship between CT intensity values (denoted I_CT) and CBCT intensity values (denoted I_CBCT) for each material class is modeled by a simple linear polynomial in light of a presumably monotonic relationship between CT and CBCT voxel values and the common underlying contrast mechanism. The resulting intensity correction function is therefore piecewise linear and possibly noncontinuous. Higher-order models could certainly be considered, but preliminary work showed a negligible difference over the linear assumption. The algorithm is therefore iterative in applying the CT-CBCT intensity match (denoted “IIM” for “iterative intensity match”) and tissue-specific in applying voxel-by-voxel intensity correction (i.e., bone voxels intensities are only matched to bone voxels and not air or soft tissue). Registration with IIM proceeds by the following steps, illustrated in the black portions of Fig. 1:

• (1)
  Prior to registration, CT and CBCT images are converted to the same physical units of image intensity, e.g., HU or attenuation coefficient (cm⁻¹). Below, we typically converted CT images to attenuation coefficient by the equation

  $$I_{\text{CT}}\left({\text{cm}}^{-1}\right)=I_{\text{CT}}{\left(\text{HU}\right)}^{\ast }\frac{{\mu }_w}{1000}+{\mu }_w,$$ (4)

  so that both CT and CBCT carried nominal units of attenuation coefficient (subject to the sources of error mentioned above).
• (2)Also prior to registration, distinct tissue classes were defined by intensity thresholding separately in CT and CBCT as detailed below. We used four tissue classes defined as air, soft tissue 1, soft tissue 2, and bone. Two soft-tissue classes were defined to better account for actual density differences (e.g., fat and muscle) and partial volume effects (which relegate tissue to the former class).
• (3)After each iteration of the Demons algorithm (Fig. 1), the input moving image is updated using the current deformation field.
• (4)
  Each pair of overlapping voxels in the two images are analyzed, and

  • (i)
    Voxel pairs consisting of different tissue types are ignored (e.g., air overlapping bone).
  • (ii)
    For voxel pairs belonging to the same tissue class, the CT-CBCT intensity relationship is estimated by a least-squares linear fit

    $$I_{\text{CT}}=m{I}_{\text{CBCT}}+b.$$ (5a)

    For four tissue classes, therefore, there are four fits performed at each iteration

    $$I_{\text{CT-air}}=m_{\text{air}}{I}_{\text{CBCT-air}}+b_{\text{air}},$$ (5b)

    $$I_{\text{CT-soft}1}=m_{\text{soft}1}{I}_{\text{CBCT-soft}1}+b_{\text{soft}1},$$ (5c)

    $$I_{\text{CT-soft}2}=m_{\text{soft}2}{I}_{\text{CBCT-soft}2}+b_{\text{soft}2},$$ (5d)

    $$I_{\text{CT-bone}}=m_{\text{bone}}{I}_{\text{CBCT-bone}}+b_{\text{bone}}.$$ (5e)
• (5)All voxels of the moving image (CT) are rescaled by the correction of Eqs. 5a, 5b, 5c, 5d, 5e.
• (6)A new deformation field is estimated using the intensity-scaled moving image and the process returns to step 3 for further iteration.

In step 4 above, four separate fits are performed. However, each pair of overlapping voxels only contributes to (at most) one fit as determined by their tissue class. As the least-squares solution to first-degree polynomial fits can be solved by a series of simple summations over the input data,³³ the computational cost added to each iteration by performing the intensity fits is linearly proportional to the size of the input images (equivalent to the computation complexity of the Demons force calculation step). In this study, thresholds were selected once manually for the IGS and RT datasets by visual inspection of the intensity histogram for a single case of each dataset and these same values were then used for other datasets in step 2 prior to registration, with nominal values for the C-arm CBCT cases of air: (0≤I_CT<0.04) and (−0.15≤I_CBCT<0.07); soft tissue 1: (0.04≤I_CT<0.16) and (0.07≤I_CBCT<0.14); soft tissue 2 (0.16≤I_CT<0.22) and (0.14≤I_CBCT<0.22); and bone (0.22≤I_CT<0.75) and (0.22≤I_CBCT<0.75) (all units cm⁻¹). In the datasets considered, some mismatch between the nominal threshold values and optimal values for each image were found to be tolerated well, likely due to the large number of voxel pairs in each class. Automatic selection of the threshold values could be implemented using well studied techniques³⁴ and may improve robustness further but was not investigated in this study.

Experimental methods

The effect of intensity mismatch on CT-CBCT registration accuracy and the ability of the proposed IIM approach to mitigate these effects were evaluated experimentally using a combination of simulations, cadaver studies, and real patient datasets drawn from CBCT-guided surgery and radiation therapy of the head and neck. An overview of the datasets is shown in Fig. 2 and Table 1. The simulations examined a broad range of intensity variations that could be exhibited under various conditions of CBCT imaging and the cadaver∕patient studies investigated the performance of the IIM approach in real data across a broad range of geometric variations ranging from fairly simple deformation to complex deformation and tissue change (e.g., weight loss or disease progression). Cases denoted “IGS” were imaged using the prototype C-arm, while those denoted “IGRT” were imaged using CBCT on a medical linear accelerator (Synergy XVI, Elekta Oncology, Stockholm, Sweden). Voxel size was (0.8×0.8×0.8) mm³ for C-arm CBCT, (1×1×2) mm³ for XVI CBCT, and varied widely depending on various clinical protocols for the preoperative CT. For simplicity, the preoperative CT was resampled to match the voxel size of the intraoperative CBCT before registration.

Figure 2.

Illustration of datasets. Each column represents a case ranging from minor anatomical deformation (case 1) to major deformation and tissue change (case 6). The first row illustrates the moving image (preoperative CT) and the second the fixed image (intraoperative CBCT). The third row illustrates the image overlap and field of view for each modality and the fourth row shows the difference image following rigid registration (overlaid on CT).

Table 1.

Summary of cases studied and registration performance for CT-CBCT registration using iterative intensity matching; “simple” cases at the top and more complex cases at the bottom.

Case	Subject	Description	NCC		Mean TRE (range) (mm)
			Rigid	IIM Demons	Rigid	IIM Demons
1	IGS cadaver	Endoscopic intervention displaced sinus anatomy	0.894	0.962	1.4±0.8 [0.7–2.5]	1.6±1.1 [0.4–3.4]
2	IGRT patient	Neck flexion	0.892	0.963	5.2±4.5 [1.2–15.6]	4.6±5.4 [0.7–18.1]
3	IGS patient	Inflammation of sinus structures	0.969	0.987	2.8±1.1 [1.3–4.2]	1.5±0.2 [1.2–1.8]
4	IGS cadaver	Nasal anatomy displaced, jaw flexion, large deformation throughout volume.	0.870	0.963	4.5±3.7 [1.0–10.7]	2.5±1.3 [1.1–4.8]
5	IGRT patient	Anatomical changes due to weight loss	0.889	0.958	3.0±0.8 [1.2–3.8]	1.9±1.0 [0.7–3.8]
6	IGS patient	Jaw flexion due to intubation. Sinuses filled with fluid.	0.885	0.953	3.0±1.4 [1.0–4.8]	1.7±1.1 [0.6–3.5]

Registration accuracy was evaluated in terms of normalized cross-correlation (NCC) and target registration error (TRE) as detailed below. The specific target points selected in each case are detailed in Sec. 2B3. Intraobserver variability in target selection was not quantified but was expected to be on the order of the image voxel size as measured in previous studies.¹⁵ NCC was chosen as a figure of merit for evaluating registration accuracy both for ease of interpretation (0 and 1 corresponding to completely uncorrelated images and perfectly registered images, respectively) and to keep the figure of merit distinct from the objective function that is optimized within the deformable registration algorithm. Patient setup for the intraoperative cases was not controlled, nor was the coordinate system of the prototype CBCT C-arm calibrated to match that of the preoperative scanner. For these reasons, rigid registration was an essential initialization step for the IGS cases to overcome large errors that would be implied without any registration (TRE >50 mm after basic alignment of coordinate systems). For consistency, all datasets were initialized by rigid registration. The accuracy of deformable registration was compared to that of rigid registration to characterize the amount of deformation present, as well as to evaluate the increase in accuracy obtained over image guidance systems utilizing preoperative imaging only, which implicitly assume a rigid transformation.

Simulation of image intensity errors

To evaluate the effect of different degrees of CBCT intensity variation on registration accuracy, a series of CBCT images was created with intensity variations of various magnitudes imparted. Simulations within a single image dataset isolated the effects arising from intensity variation and avoided possible confounding effects of differing anatomy and deformation. First, a diagnostic CT scan of a cadaveric head specimen was acquired (120 kVp; 225 mA s; 1.0 mm slice thickness; Toshiba Aquilion One, Toshiba Medical Systems, Tokyo, Japan). Subsequently, under endoscopic guidance, a surgeon induced a range of deformations in the sinuses typical of an endoscopic intervention, including displacement of the turbinates and displacement of the septum and orbital walls. The jaw was also exercised manually to induce a range of deformations throughout the volume. Finally, CBCT images of the deformed specimen were acquired using a prototype surgical C-arm previously reported.² The C-arm provides high-quality intraoperative CBCT and is the subject of ongoing evaluation in a number of surgical guidance applications.

Simulated CBCT images featuring intensity errors were created by imparting a constant scale factor to the image intensities across the entire 3D image. Simulation images were also generated using a spatially varying paraboloidal intensity scaling to approximate a “cupping” artifact, in which the edge of the image had no intensity deviation and the center of the image exhibited a multiplicative change in intensity given by the scale factor (varied from 0.5 to 1.5). This range is consistent with the magnitude of cupping artifact reported in the literature.¹⁷ As shown below, the results for the uniform and paraboloidal errors were largely indistinguishable and the former was taken as the nominal case throughout the results reported below except where otherwise noted. The simulated images were created by applying scaling factors ranging within ±50% of the originally reconstructed values. The range of intensity deviations was chosen to create a series of images representing various levels of intensity error (e.g., arising from various levels of x-ray scatter, truncation or, alternatively, an inaccurate reconstruction algorithm). No additional noise was added to the simulated images, but any underlying noise in the images was scaled with the image intensities.

The CT image was registered to each simulated CBCT, and accuracy was evaluated in terms of NCC, TRE, and the visual quality of the registered image. Between five and eight bone and soft-tissue anatomical points were selected for TRE analysis as detailed for each case in Sec. 2B3 below. For each degree of intensity mismatch imparted on the CBCT image, registration was performed using three deformable registration techniques: (i) A naive symmetric Demons implementation with no intensity match (denoted NIM); (ii) a simple histogram match followed by symmetric Demons registration (denoted SHM); and (iii) the symmetric Demons registration using the iterative intensity matching described in Sec. 2A3 (denoted IIM).

Evaluation with alternate image metrics and registration strategies

The behavior of the IIM Demons approach was further studied at each iteration and the overall output was evaluated in comparison to an alternate registration approach for the cadaver dataset described above, detailed as case 4. This case represents a well controlled (manual manipulation and intervention in a cadaver) but challenging case with deformations typical of an IGS procedure (sinus anatomy displacement and jaw and neck flexion). The linear intensity fits calculated for each tissue class were recorded. Also, the quality of the registration output at each iteration was evaluated in comparison to the intraoperative image in terms of rank correlation coefficient, a metric that has been used successfully for registration purposes in recent work.^{27, 35}

Comparison to a simple B-spline∕normalized mutual information registration

The choice of a Demons style algorithm for this particular application was evaluated in comparison to a registration method using a parametrized transform and an information theory based image similarity metric. As previous multi-institution studies have shown that the performance of different implementations of similar deformable registration methods vary depending on implementation choices and parameterization,^{36, 37} we utilized a readily available open-source software package with a wide range of registration choices: ELASTIX³⁸ (http://elastix.isi.uu.nl) was used for normalized mutual information based registration with a B-spline parameterized transformation and a four-level multiresolution registration scheme. Both the image resolution and grid spacing were downsampled by a factor of 2 at each multiresolution level, with a B-spline grid spacing of 10 mm at the final level. Optimization was by gradient descent with 100 iterations at each multiresolution level and 2000 samples taken from the images for calculation of normalized mutual information. These parameters were selected as reasonable choices following multiple registration trials and while the parameters gave good results, they should not be considered fully optimized. Case 4 was registered using both IIM Demons and the B-splines normalized mutual information approach. The resultant transformations were used to transform the preoperative CT image and registration quality was assessed by quantitative figures of merit (TRE) and qualitatively by difference images and visual inspection. With no defined gold standard for deformable registration for intraoperative use, this comparison served to evaluate the feasibility of a Demons style algorithm with iterative intensity matching for CT-CBCT registration as opposed to explicit optimization of a parametrized transform using a metric that makes less strict assumptions of image similarity.

Performance evaluation in real CT-CBCT data

A final study was performed to measure the accuracy of IIM registration across a broad variety of real CT-CBCT registration tasks ranging from simple deformations to complex tissue changes. Six image sets were chosen from cadaver studies, image-guided radiation therapy (IGRT) patients, and IGS patients that exhibited a range of deformations and intensity mismatches. IGRT and IGS patients were drawn retrospectively from imaging trials carried out under IRB-approved research protocols. For the IGRT patients, the most recently acquired (i.e., last treatment day) CBCT image was selected; conversely, for the IGS patients, the CBCT acquired at the beginning of surgery (viz., following intubation but prior to incision) was used for registration. Summarized as follows as well as in Fig. 2 and Table 1, cases 1–3 were considered modest deformations, while cases 4–6 exhibited complex tissue change.

• (i)Case 1. A cadaver imaged before and after endoscopic intervention carried out with the intention to displace internal sinus anatomy such as the turbinates and septum. Anatomical points selected for TRE analysis included superior aspect of the left and right coronoid process of the mandible; inferior aspect of the mastoid process; superior prominence of cervical vertebrae; and inferior prominence of the right middle turbinate.
• (ii)Case 2. An IGRT patient imaged using CBCT 81 days after the planning CT. Deformations were visible in the neck due to patient setup variations and neck flexion. A morphological change was evident in the soft palate and hypopharynx along with moderate weight loss, possibly in response to treatment. Anatomical points selected for TRE analysis included superior aspect of the left and right coronoid process of the mandible; inferior aspect of the mastoid process; and four conspicuous soft-tissue prominences in the epiglottis, pharynx, and left and right auditory canal.
• (iii)Case 3. An IGS patient imaged during resection of a sinonasal∕skull base tumor (esthesioneuroblastoma). The high-resolution preoperative CT covered a limited FOV and moderate deformations were visible throughout the volume in the intraoperative CBCT due to inflammation and preparation for surgical approach. Due to limited field of view, anatomical points picked for TRE measurements were limited to five anatomical points within and around the sinuses, including two aspects of the inferior turbinate, one aspect of the middle turbinate, the superior-posterior prominence of the hard palette, and the most superior point of the frontal sinus.
• (iv)Case 4. Cadaver specimen described in case 1 but with additional deformations imparted via manual flexion of the jaw (opening and closing of the mouth). A range of deformations was visible throughout the volume, including large deformations about the jaw. Anatomical points selected for TRE analysis included those identified for the specimen in case 1 and an additional two features in the soft tissue of the tongue base.
• (v)
  Case 5. An IGRT patient imaged with CBCT 60 days after the planning CT. The patient exhibited significant weight loss and related anatomical change due to treatment. Deformations owing to setup variations were also evident (head positioning and neck flexion). TRE points were identical to case 2.

  • (vi)
    Case 6. An IGS patient imaged in the course of maxillectomy. Significant tissue deformations were imparted by jaw flexion during intubation and large differences in neck flexion between preoperative CT and intraoperative CBCT. In addition, CBCT images showed nearly complete fluid filling of the nasal cavity and sinuses (sinusitis) compared to preoperative CT. Five points were selected for TRE measurement: Posterior-inferior aspect of the clivus, posterior and anterior aspects of the mandible, and two easily identifiable prominences within the sinuses.

RESULTS

Simulation of image intensity errors

Degradation in registration accuracy and distortion of anatomical features was seen due to the simulated intensity variation between the moving (CT) and fixed (CBCT) images described in Sec. 2B1. Figure 3a shows the central sagittal slice of an intraoperative CBCT (case 4) with a 20% error in voxel value and Figs. 3b, 3c, 3d show the preoperative CT deformably registered to match the CBCT. As seen in Fig. 3b, registration without intensity correction (NIM) produced significantly distorted anatomy, e.g., distorted soft tissues about the tongue, hypodense tissue boundaries, and uncontrolled distortion of rigid bony anatomy such as the cervical vertebrae (zoomed inset). A simple histogram match performed prior to registration (SHM) controlled some of these effects, but significant distortion of anatomy is still evident, e.g., bloating of the cervical vertebrae. The IIM, on the other hand, produced qualitatively undistorted images. These findings are quantified in Fig. 4a.

Figure 3.

CT-CBCT registration in the presence of intensity errors. (a) Intraoperative CBCT image showing the true (undistorted) anatomy of the fixed image. The inset shows a zoomed-in view of the cervical vertebrae. [(b)–(d)] Preoperative CT images following Demons deformable registration to (a) using (b) no intensity correction, (c) a single histogram match, and (d) an iterative intensity match. Note the tissue distortions in (b) and (c) in comparison to (a), whereas (d) appears robust against distortion. (e) NCC calculated within the inset region for the NIM, SHM, and IIM approaches.

Figure 4.

Effect of intensity variation on registration accuracy. (a) TRE and for the NIM, SHM, and IIM approaches with simulated uniform intensity errors. Both SHM and IIM provide robustness against intensity errors, but the latter better resolves distortions that can occur due to intensity mismatches. (b) TRE for the IIM approach with both uniform and spatially parabolic intensity errors showing similar results.

Without intensity correction (NIM), TRE degrades rapidly with errors in voxel intensity. Interestingly, the effect is more severe for erroneously low intensity values (which is usually the case for CBCT images dominated by x-ray scatter). A SHM corrects this to a large extent but exhibits higher TRE than the IIM, as expected from the anatomical distortions evident in Fig. 3. Although the difference in TRE between SHM and IIM cases is small (<1 mm), it is indicative of gross anatomical distortion throughout the volume in a manner not necessarily evident in a small sample of target registration points. The effect is evident in the zoomed-in region of Fig. 3a about the cervical vertebrae. The NCC analyzed from that region of interest is plotted in Fig. 3e for the NIM, SHM, and IIM approaches. The NIM case degrades quickly with errors in voxel intensity, more rapidly for erroneously low voxel values. The SHM approach gives NCC ∼0.991 and the IIM approach gives NCC ∼0.996 and is robust against voxel intensity error. Simulation study results were similar with the application of a uniform intensity error [Fig. 4a] and a parabolic intensity error approximating a cupping artifact [Fig. 4b].

Iterative intensity match

Evolution of intensity matching fits

Figure 5 shows an example of the fit parameter evolution during IIM for each tissue class (showing only the rescale parameter m for purposes of brevity). We note that in our implementation, the intensity correction fit is calculated on the original moving image each iteration (after updating it with the current deformation field); therefore, it is not expected that the rescale slope would converge to a value of 1.0, but instead that it would converge to a value proportional to the intensity differences in the CT-CBCT images for that class. The different magnitudes of the rescale parameter at convergence (m∼0.4–0.8) show the advantage of correcting intensities separately for each tissue class. The advantage gained by calculating the intensity correction iteratively depends on the tissue type, as the estimate for some materials is more stable with each iteration (e.g., air) than others (e.g., bone). It can be observed that changes in pyramid levels are associated with marked changes in the rescale slope fit. This can be caused both by different magnitudes of voxel intensity averaging associated with the different levels of downsampling and may also be indicative of instability of the registration transform and∕or the intensity correction parameters immediately following changes in the image pyramid, an effect that is rectified within a few iterations. We note also that rescale slope for the air during the first pyramid level can be driven by voxels at the edge of the CBCT image reconstructed with negative attenuation coefficient values. This effect is mitigated in later iterations when the deformation field for these voxels tends to map outside the FOV of the image. Another observation is the relatively large difference in the intensity rescale parameters for the two classes of soft tissue. One “soft-tissue” class was expected to include voxels featuring partial volume effect and the magnitude of partial volume averaging on voxel intensities would be different for CT and CBCT images due to the different underlying voxels sizes of the images. The rescale fit parameter for the tissue class featuring partial volume would thus reflect this additional numerical factor.

Figure 5.

Evolution of intensity rescale parameters computed at each iteration of the IIM approach. Four tissue classes are labeled. Levels 1–4 refer to the morphological pyramid of Fig. 1, with downsampling factors of 8, 4, 2, and 1.

Effect on computation time

IIM increased the required time per registration iteration (due to computing the fit parameters and rescaling the moving image); the total registration time was not necessarily increased for every case because the convergence behavior is distinct and potentially improved for IIM. For example, in case 3, NIM converged in 80 iterations, whereas IIM converged in 63 iterations, each requiring ∼100 s total. In the worst case, IIM increased total registration time by ∼30% [viz., case 6: 62 iterations (∼100 s) for NIM compared to 74 iterations (∼130 s) for IIM)]; however, NIM yielded gross image distortion in this case, whereas IIM performed well. The average time per iteration averaged over 100 iterations for registration of two (256×256×256) voxel images at floating point precision at level (4, 3, 2, 1) of the morphological pyramid was (0.02, 0.07, 0.48, 3.56) s for NIM Demons compared to (0.02, 0.11, 0.75, 5.46) s for IIM Demons. None of the algorithms were optimized for computational speed in the current work, and superior implementations can be envisioned.

Evaluation with alternate image metrics and registration strategies

Figure 6a shows IIM registration converging to a rank correlation coefficient value of ∼0.895. Consistent with a previous work,¹⁵ the quality of registration as measured by the image metric appears to have reached a plateau at convergence and further iterations would increase registration time without measurably increasing registration performance. While the results shown in Fig. 6a do not necessarily guarantee that IIM registration converged to a global minimum, they indicate that the addition of the iterative intensity matching step does not negatively affect the convergence behavior of the algorithm. The question of the overall quality of the registration is given by the more in-depth patient studies presented in Sec. 3C.

Figure 6.

Evaluation of rank correlation coefficient per iteration of IIM Demons registration and comparison to B-spline parametrized normalized mutual information based registration. The results show (a) registration convergence behavior starting from (b) rigid registration and (d) final product in comparison to (c) a simple B-spline parametrized normalized mutual information registration. All images show a central sagittal slice of intraoperative CBCT in grayscale with yellow-green differences from the registered preoperative CT overlaid.

Comparison to B-spline∕normalized mutual information registration

As seen in Fig. 6, Demons registration with iterative intensity matching produced results comparable to that of the B-spline∕normalized mutual information registration for the dataset studied with deformations typical of a difficult case (sinus anatomy displacement and jaw and neck flexion). The difference images (case 4 from Table 1, central sagittal slice) show that both methods accounted for displacement of the cervical vertebrae and surrounding anatomy arising from neck flexion, but the normalized mutual information registration did not account quite as well for jaw flexion nor turbinate displacement. Conversely, visual inspection confirmed that the B-spline∕normalized mutual information transform resulted in a smoother output deformation field and suffered less “deformation artifact” at the edges of the CBCT FOV, a known issue with Demons and similar algorithms.³⁹ Quantitatively, IIM Demons presented lower mean TRE for case 4 (2.5 mm compared to 3.3 mm for B-Spline∕normalized mutual information). Computation time for this particular B-spline∕normalized mutual information registration was in the hours as, for this evaluation, we did not use a more optimized, stochastic implementation. While the performance of the normalized mutual information based approach will vary depending on exact parameter choices and a more complete optimization is certainly possible, the qualitative and quantitative results confirm that the relatively unconstrained output of a Demons style algorithm is a reasonable choice for the deformations encountered in this application and that a simple extended version of a mean-square difference metric can perform comparably to more sophisticated image metrics while maintaining computational simplicity.

Performance evaluation in real CT-CBCT data

The mean TRE averaged across all target points and all six cases in Fig. 2 and Table 1 for rigid registration, NIM, SHM, and IIM Demons were (3.5±3.0), (5.5±6.1), (2.8±2.5), and (2.5±2.8) mm, respectively. The median TREs were 2.9, 2.9, 2.2, and 1.7 mm, respectively, for rigid registration, NIM, SHM, and IIM Demons. For NIM Demons registration (no intensity matching), it is clear from Fig. 7 that the large mean TRE is driven by certain high error points, specifically two cases that suffered from poor HU calibration. Mean TRE results were averaged across points selected individually for each case, with care taken to choose equivalent anatomical points whenever possible. Detailed description of the selected targets points for each case is given in Sec. 2B3. As evident in Fig. 7, the improvement in TRE with IIM vs rigid registration is statistically significant and the IIM approach reduced the number of high TRE outliers.

Figure 7.

Registration accuracy measured across all anatomical target points and datasets. Results are grouped according to simple deformation (cases 1–3) compared to “complex” deformation and tissue change (cases 4–6) and the final group pools all datasets. Box-and-whisker plots represent the median TRE (horizontal centerline), the 25th and 75th percentiles (box edges), and total range (whiskers) excluding outliers (+ symbols).

Overall, registration with IIM was qualitatively acceptable: TRE was comparable to the voxel size; results were robust against CBCT voxel value errors over a broad range of realistic conditions; and the registered images were qualitatively free of distortion errors that were clearly evident in insufficiently matched (NIM and SHM) images. Points of success and failure evident in the six cases of Fig. 2 and Table 1 are summarized as follows.

Simple deformations (cases 1–3)

Case 1. Registration of preoperative CT to intraoperative C-arm CBCT images acquired in the course of endoscopic intervention suggested excellent resolution of deformations imparted in tissues of the sinonasal space with no obvious points of failure. IIM Demons mean TRE was (1.6±1.1) mm.

Case 2. Registration of planning CT to online IGRT CBCT suggested resolution of the majority of deformations arising from neck flexion and bulk anatomical changes around the neck due to weight loss. The epiglottis showed a large change in position between scans which was not fully corrected by either rigid (as expected) or IIM deformable registration. We observed in this case that the epiglottis displacement was almost entirely in the superior-inferior direction while the immediately surrounding anatomy was mainly fixed or displaced transaxially. Further experimentation indicated that relaxing the smoothing parameter appeared to reduce epiglottis misregistration but at the expense of less realistic deformations in other areas of the image. For this study, we only report quantitative results with the same fixed smoothing parameters for all cases. IIM Demons mean TRE was (4.6±5.4) mm and largely influenced by misregistration of the epiglottis.

Case 3. Registration of diagnostic CT to intraoperative C-arm CBCT in a surgical patient demonstrated resolution of deformation due to jaw position and neck flexion. However, registration performance in areas about inflamed tissues in the sinuses was qualitatively worse, specifically in areas showing tissue “growth” or inflammation instead of deformation. IIM Demons mean TRE was (1.5±0.2) mm.

Complex deformations and disease (cases 4–6)

Case 4. The deformable registration process succeeded in capturing the deformations induced by the closing of the jaw and the associated neck flexion. Moderate deformations induced in the sinuses were also handled well. Less accurate registration was observed in proximity to image artifacts arising from dental fillings, which in this case presented differently on CT and CBCT. IIM Demons mean TRE was (2.5±1.3) mm.

Case 5. The majority of deformations arising from significant weight loss, setup changes, and varying presentation of pharynx anatomy were handled well by deformable registration. Following IIM registration, a modest degree of misalignment (a few mm) was visible in the soft palate where certain anatomical points were more prominent in CBCT than in CT. IIM Demons mean TRE was (1.9±1.0) mm.

Case 6. Excellent resolution of deformations arising from significant jaw flexion and variations in patient setup of the head and neck were observed. The most visible points of lower registration accuracy included anatomy within sinuses which were almost totally surrounded by sinus filling∕inflammation in one scan but visible in the other. Other challenges were noted in proximity to the intubation tube and streak artifacts. IIM Demons mean TRE was (1.7±1.1) mm.

DISCUSSION AND CONCLUSIONS

Over a range of large magnitudes of voxel intensity inaccuracy, it was observed that applying Demons registration with insufficient intensity correction can lead to severe image distortion and a loss of registration accuracy. Although the ability to consistently reconstruct CBCT images with accurate voxel values (despite high x-ray scatter and object truncation) is the subject of the ongoing work, it remains a challenging area; furthermore, in some applications (e.g., dental and image-guided surgery), HU accuracy may be of less concern compared to diagnostic radiology and radiotherapy planning applications. CBCT-guided surgery presents a collection of challenges in this regard. First, a need for accurate registration of preoperative imaging (and planning data) to intraoperative CBCT; second, a need for fast registration consistent with operating workflow; and third, a dynamic environment in which object truncation and x-ray scatter (and therefore HU inaccuracy) are the norm. The use of fast, intensity-based registration, such as the Demons algorithm, is enticing, but inaccuracy in CBCT voxel intensity confounds the assumption of intensity equivalence between anatomy presented in preoperative CT and intraoperative CBCT. In this work, we presented a modification to the well-known Demons algorithm, making it robust to intensity mismatch in CT-CBCT registration. By iteratively estimating both a spatial transform and tissue-specific intensity correction, the IIM Demons approach performed robustly over the range of intensity inaccuracies studied.

The flexibility of the relatively unconstrained transformations generated by the IIM Demons approach proved advantageous in capturing the large range of deformations expected in head and neck surgical applications. Conversely, explicit optimization of a parametrized transform tended to yield smoother transformations at the expense of capturing fewer deformations. Registration using the latter modular approach has the advantage of more easily changing or tuning the image metric for a particular application. Conversely, with no explicit image similarity metric component in the traditional Demons registration approach, we have chosen to incorporate an intensity matching step within the iterative progress of the algorithm.

The computational cost of iterative intensity matching concurrent with registration did not necessarily increase the amount of time required for registration. Although it did increase the time per iteration, the total time did not increase proportionally, since the IIM approach in some cases required fewer iterations to converge. This behavior was case-specific, dependent on the underlying anatomy of the two images, and showed that unmatched (or SHM) approaches often spent additional iterations due to intensity mismatches (not actual deformation). The increased cost per iteration for IIM was not negligible, however, and deserves further computational optimization∕acceleration.

Similar considerations relate to tradeoffs between registration accuracy and the computational complexity in estimating the CT-CBCT intensity relationship. In this study, the CT-CBCT intensity relationship was modeled by a simple four-class piecewise linear function; while this model does not necessarily represent the true underlying relationship between intensities in the two images, it appears sufficiently accurate at a given iteration step within the algorithm to enable more accurate registration in the next. A more sophisticated approach could use a higher-order model of the intensity relationship and∕or employ a spatially varying intensity fit model, consistent with the typical cupping nature of CBCT artifacts, in conjunction with a more sophisticated method of parameter estimation than least-squares fitting. Initial investigation of these approaches did not show a significant increase in accuracy for the data in this study and they were not studied further due to the increased computational expense.

Other areas of ongoing and future work related to registration in the presence of image dissimilarity other than deformation include the presence of image artifacts (e.g., dental streak artifacts aligned differently in preoperative and intraoperative images according to head angulation); the introduction of interventional devices (e.g., intubation or other hardware) in the intraoperative image; and, perhaps most importantly, the ability to account for tissue growth∕inflammation and shrinkage∕excision during the course of a surgical procedure. In the datasets studied, qualitative loss of registration accuracy was observed near sites of inflammation and filling of the sinuses and it remains to be determined whether the loss of registration performance is due to inability to accommodate tissue growth, a loss of contrast in one image due to tissue filling, or a combination of both factors. Similarly, it is expected that registration performance will suffer in the presence of significant tissue excision. In this work, IGS registration studies were limited to preoperative CT scans registered to CBCT images acquired immediately after patient setup but before incision. Such initial scans are envisioned as the first step in a series of low-dose scans acquired intraoperatively at specific stages of the surgical procedure.³ Registration of the preoperative CT and associated surgical planning data to the initial intraoperative scan can allow the surgical approach and the first stages of the procedure to be carried out in the context of up-to-date anatomical information. Future work will concentrate on the difficulties associated with registration to intraoperative CBCT scans featuring large amounts of excised tissue, such that similar guidance relative to preoperative scans and planning can be provided during later stages of the procedure.

The role of 3D intraoperative imaging in guiding surgical procedures continues to be defined in overcoming conventional barriers of precision, convenience, and workflow. While high-quality intraoperative CBCT can provide an up-to-date view of the patient anatomy on demand during the procedure, it is imperative to register preoperative imaging and planning data into this most up-to-date context. The ability to do so quickly and accurately in a manner that is robust against variations in CT-CBCT image intensity is an important step, resolved in this work through the incorporation of an iterative intensity match within the well-known Demons algorithm. The approach facilitates not only registration of the preoperative CT image with the intraoperative CBCT, but also any planning data that were defined in the preoperative CT (e.g., surgical trajectories and segmentations of target and normal tissues) as well as any other preoperative imaging modality (e.g., PET or MR) that is registered to preoperative CT (e.g., as part of the surgical planning process). In this way, CT-CBCT registration provides a “bridge” between intraoperative CBCT and any preoperative image or planning data. One may even envision scenarios where the intraoperative CBCT provides only an unseen, underlying framework for registration and only the (deformably registered) preoperative CT, MR, and∕or PET image is visualized in the course of guidance. In this work, we presented a scheme for fast deformable registration of preoperative CT to intraoperative CBCT that addresses intensity mismatches simultaneous to the process of spatial alignment, a technique that could facilitate richer integration of multimodality preoperative information with intraoperative CBCT to enhance surgical decision making, increase surgical precision, and improve patient safety.