Automated bone segmentation from dental CBCT images using patch-based sparse representation and convex optimization

Abstract

Purpose: Cone-beam computed tomography (CBCT) is an increasingly utilized imaging modality for the diagnosis and treatment planning of the patients with craniomaxillofacial (CMF) deformities. Accurate segmentation of CBCT image is an essential step to generate three-dimensional (3D) models for the diagnosis and treatment planning of the patients with CMF deformities. However, due to the poor image quality, including very low signal-to-noise ratio and the widespread image artifacts such as noise, beam hardening, and inhomogeneity, it is challenging to segment the CBCT images. In this paper, the authors present a new automatic segmentation method to address these problems.

Methods: To segment CBCT images, the authors propose a new method for fully automated CBCT segmentation by using patch-based sparse representation to (1) segment bony structures from the soft tissues and (2) further separate the mandible from the maxilla. Specifically, a region-specific registration strategy is first proposed to warp all the atlases to the current testing subject and then a sparse-based label propagation strategy is employed to estimate a patient-specific atlas from all aligned atlases. Finally, the patient-specific atlas is integrated into a maximum a posteriori probability-based convex segmentation framework for accurate segmentation.

Results: The proposed method has been evaluated on a dataset with 15 CBCT images. The effectiveness of the proposed region-specific registration strategy and patient-specific atlas has been validated by comparing with the traditional registration strategy and population-based atlas. The experimental results show that the proposed method achieves the best segmentation accuracy by comparison with other state-of-the-art segmentation methods.

Conclusions: The authors have proposed a new CBCT segmentation method by using patch-based sparse representation and convex optimization, which can achieve considerably accurate segmentation results in CBCT segmentation based on 15 patients.

INTRODUCTION

Segmentation of the cone-beam computed topographic (CBCT) image is an essential step for generating three-dimensional (3D) models in the diagnosis and treatment planning of the patients with craniomaxillofacial (CMF) deformities. It requires segmentation of bony structures from soft tissues, as well as separation of mandible from maxilla. CBCT scanners are becoming popularly used in the clinic, even in the private practice settings, due to its lower cost and lower dose compared to the conventional spiral multislice CT (MSCT) scanners.¹ However, the quality of CBCT images is significantly lower than that of spiral MSCT [Fig. 1a]. Also, there are many imaging artifacts [Fig. 1b], including noise, beam hardening, inhomogeneity, and truncation, that are inherent in CBCT units due to the nature of imaging acquisition and reconstruction process.² For example, CBCT machines for the purpose of dose reduction are often operated at milliamperes that are approximately one order of magnitude below those of medical CT machines. Thus, the signal-to-noise ratio of CBCT is much lower than that of CT. These artifacts affect image quality, and eventually the accuracy of subsequent segmentation.^{3, 4} Furthermore, based on the clinical requirement for correctly quantifying the deformity, CBCT scans are often acquired when the maxillary (upper) and mandibular (lower) teeth are in maximal intercuspation, i.e., the upper and lower teeth closely bite together as shown in Fig. 2. This leads to the display of both upper and lower teeth on the same cross-sectional slice as in Fig. 2c, thus making it extremely difficult to separate them in the same slice.⁵ To date, in order to use CBCT clinically for the diagnosis and treatment planning, the segmentation has to be done completely manually by the experienced operators.

Figure 1.

Comparison between (a) spiral MSCT and (b) CBCT images. Comparing to the MSCT, CBCT scans have severe image artifacts, including noise, beam hardening, inhomogeneity, and truncation.

Figure 2.

(a) Maximal intercuspation. (b) Zoomed view of the teeth. (c) Both upper and lower teeth are displayed on the same cross-sectional slice.

However, manual segmentation is tedious, time-consuming, and error-prone.^{6, 7} Although automated segmentation methods have been previously developed for CBCT segmentation, they were mainly based on simple thresholding and morphological operations,⁸ thus sensitive to the presence of the artifacts. Recently, statistical shape model (SSM) has been utilized for robust segmentation of the mandible.^{9, 10} However, these approaches are only applicable to the objects with relatively regular and simple shapes (e.g., mandible). They are not applicable to the objects with complex shapes (e.g., maxilla). On the other hand, interactive segmentation approaches^{5, 11} were also proposed to integrate automatic segmentation with manual guidance. For example, Le et al. proposed an interactive geometric segmentation technique to separate upper and lower teeth in CT images.⁵ However, these interactive methods^{5, 11} also have difficulty in producing plausible segmentations due to the presence of artifacts such as strong metal artifacts of dental implant.⁵ In Ref. 12, Duy et al. proposed a fully automatic method for tooth detection and classification in CT image data. However, they only focus on the upper teeth extraction in the CT image and their separation algorithm does not work as well on CBCT as it does on CT due to the low contrast in CBCT.¹³ To the best of our knowledge, our work is the first study aiming to fully automatically segment (and separate) the mandible from the maxilla on CBCT images.

Recently, there is a rapidly growing interest in using patch-based sparse representation.^{14, 15, 16} This approach makes an assumption that image patches can be represented by sparse linear combination of image patches in an overcomplete dictionary.^{17, 18, 19, 20} This strategy has been applied to a good deal of image processing problems, such as image denoising,^{17, 18} image in-painting,²¹ image recognition,^{19, 22} image super-resolution,²⁰ and deformable segmentations,²³ achieving promising results.

In this paper, we propose a new method for fully automated CBCT segmentation by using patch-based sparse representation to (1) segment bony structures from the soft tissues and (2) further separate the mandible from the maxilla. Specifically, we first propose a region-specific registration strategy to warp all the atlases to the current testing subject and then employ a sparse-based label propagation strategy to estimate a patient-specific atlas from all aligned atlases. Then, the estimated patient-specific atlas is integrated into a convex segmentation framework based on maximum a posteriori probability (MAP) for accurate segmentation. In this paper, the atlases consist of both spiral MSCT and CBCT images. The reasons for also using spiral MSCT images as a part of the atlases include (1) although image formation process is different between spiral CT and CBCT, they share the same patterns of anatomical structures (e.g., the bones generally have high intensities while the soft tissues have low intensities), which are captured in a patch fashion in our method to estimate the probability; (2) the spiral MSCT images have better contrast and less noise than the CBCTs, thus needing less processing time to construct atlases by manual segmentations (i.e., taking around 30 min for each MSCT image), compared to the CBCT images (i.e., requiring around 12 h for each CBCT image even by an experienced operator).

In summary, the novelty of our work includes the following:

• To clinically use CBCT for the dental diagnosis and treatment planning, we propose a fully automated segmentation method for the dental CBCT images, which currently has to be done completely manually by the experienced operators (taking around 12 h as mentioned above). This automated method shows its potential in solving the important clinical problem based on our diverse data from 15 patients with different ages and conditions.

• We also propose using the MSCT samples to estimate a patient-specific atlas for each CBCT image by employing a recently proposed patch-based sparse representation technique, since the basic patterns in MSCT and CBCT were found to be very similar.

• We further integrate our estimated patient-specific atlas into a convex segmentation framework, based on maximum a posteriori probability, for more accurate CBCT segmentation. This type of atlas-guided level sets segmentation method has not been previously developed for the challenging CBCT image segmentation.

• Most importantly, the performance of our proposed method is much better than any of other state-of-the-art methods, including auto segmentation, and multi-atlas based labeling with majority voting (MV) or conventional patch-based method (CPM).

Note that partial results in this paper were reported in our recent conference paper.²⁴ The remainder of this paper is organized as follows. The proposed method is introduced in Sec. 2. The experimental results are then presented in Sec. 3, followed by the discussion and conclusion in Sec. 4.

METHOD

In this study, we aim to segment a CBCT scan into three structures/regions: mandible, maxilla (the skull without the mandible), and background. Let Ω be the image domain and C be a closed subset in Ω, which divides the image domain into three disjoint partitions ${\Omega }_{i=1}^3$, such that $\Omega ={\bigcup }_{i=1}^3{\Omega }_i$. For every voxel χ ∈ Ω, we first estimate its patient-specific probability belonging to each class $p_i\left(\chi \right)\genfrac{}{}{0pt}{}{\Delta }{=}p\left(\chi \in {\Omega }_i\right)$, and then use this patient-specific probability map (also called as atlas) as a prior to segment the CBCT image based on maximum a posteriori probability. Finally, a convex segmentation framework is proposed for precise segmentation. The flowchart of the proposed method is shown in Fig. 3.

Figure 3.

The flowchart of the proposed method.

Subjects

The CBCT scans of 15 patients (6 males/9 females) with nonsyndromic dentofacial deformity and treated with a double-jaw orthognathic surgery were included in this study. Their average age at the time of surgery was 26 ± 10 years (range: 10–49 years). These CBCT scans were acquired with a matrix of 400×400, a resolution of 0.4 mm isotropic voxel, and the time of exposure of 40 s. The details of imaging protocol are provided in Table 1. On the other hand, additional 30 subjects with normal facial appearance scanned at maximal intercuspation were randomly selected from our HIPAA deidentified MSCT database. Their ages were 22 ± 2.6 years (range: 18–27 years). The MSCT images were acquired with a matrix of 512×512, a resolution of 0.488×0.488×1.25 mm,³ and the time of exposure of less than 5 s. All the MSCT and CBCT images were HIPAA deidentified prior to the study. The study was approved by The Methodist Hospital Institutional Review Board. These 30 CTs and 15 CBCTs were manually segmented to serve as the ground-truth segmentations by the two experienced CMF surgeons using Mimics 10.01 software (Materialise NV, Leuven, Belgium).

Table 1.

Summary of dental imaging protocols for MSCT and CBCT used in this study.

			FOV, mm		Voxels, mm		Dimensions
Image	kV	mA (s)	X, Y	Z	X, Y	Z	X, Y	Z
MSCT: GE
LightSpeed RT	120	120	250	300	0.488	1.25	512	240
CBCT: i-CAT	120	0.25	160	160	0.4	0.4	400	400

Region-specific registration with guidance of landmarks

Traditionally one can align the template image with the to-be-segmented image based on image intensity. However, it imposes two limitations in the application of MSCT/CBCT registration: (1) Head MSCT is usually acquired with different field of view (FOV) from CBCT [cf. Figs. 4a, 4b]. If we directly register them using image intensity, the registration error could be very large. (2) Precise separation of the closely bitted mandibular and maxilla teeth requires accurate registration of two images on the teeth region. Since teeth region only takes a small part of the entire image, by using global image registration, the registration accuracy of teeth region is usually limited. To overcome these limitations, we propose a region-specific, landmark-guided registration, with the landmark guidance coming from the two levels as described next. In the whole head level, we automatically detect 15 anatomical landmarks [Fig. 4a] using a robust landmark detection algorithm²⁵ to affine align MSCT atlases [e.g., Fig. 4b] with the CBCT testing image [Fig. 4c]. In this way, the affine alignment is not subject to different FOVs, since it is purely estimated from the detected landmarks. After landmark-guided affine alignment, MSCT/CBCT would have the same FOV and thus we can use deformable registration algorithm to further improve the registration accuracy. In fact, there are many registration methods^{26, 27, 28, 29, 30} that we can employ. In this paper, B-spline registration algorithm based on the mutual information³¹ is adopted for deformable registration. This algorithm was implemented in the free-access Elastix toolbox.³² Figures 4d, 4e show the warped results without and with head-level landmarks, respectively. It can be seen that the result without guidance of head-level landmarks miss matching with the testing CBCT. In the teeth level, we also rely on the automatically detected anatomical landmarks to improve the registration accuracy. Specifically, to align teeth regions of MSCT/CBCT, we use only eight teeth landmarks [Fig. 4a] for estimating the affine registration. Ideally, after head-level landmark-guided affine registration, we could adopt deformable registration to further improve the registration accuracy. However, as shown in Fig. 2, due to the beam hardening and complex bone structure, it is difficult for the current state-of-the-art deformable registration algorithms to work well. Figure 4e shows a typical result after deformable registration on the teeth region, which is worse than the registration result using only teeth-landmark-guided registration as shown in Fig. 4f. In Sec. 2C, we will use two separate registrations to estimate the patient-specific atlas at two different regions, i.e., teeth region and other remaining regions. Specifically, to estimate the patient-specific atlas at teeth region, we will use only those eight teeth landmarks to derive the registration. To estimate the patient-specific atlas for other regions, we will use all 15 landmarks to derive an initial affine registration and then adopt Elastix³² for further refinement.

Figure 4.

(a) Fifteen anatomical landmarks: #1–#15 are used to estimate the affine matrix to warp the atlases onto the testing subject; #8–#15 are used to estimate the affine matrix to warp the teeth joint regions between the template images and the testing image. (c) is the same as (a). (d) and (e) show the registration results without and with head-level landmarks, respectively. (f) shows the registration result with both head-level and teeth level landmarks.

Estimating a patient-specific atlas

Atlas-based segmentation has demonstrated its robustness and effectiveness in many medical image segmentation problems.³³ Specifically, a population-based atlas, in the context of this work, is defined as the pairing of a template image and its corresponding tissue probability maps.³⁴ The template image often refers to an intensity image, equally or weighted averaged from a number of aligned intensity images of the training subjects. Given a population-based atlas, image registration could be used to warp this atlas to the query image, to yield a transformation that allows the atlas's tissue probability maps to be transformed and treated as tissue probabilities for the query subject. However, a population-based atlas often fails to provide useful guidance, especially in the regions with high intersubject anatomical variability, thus leading to unsatisfactory segmentation results [Fig. 5b]. One way to overcome this problem is to integrate the patient-specific information^{35, 36, 37} in the atlas construction. To this end, we propose to construct a patient-specific atlas by combining both population and patient information as detailed below.

Figure 5.

(a) Original image. Comparison between population-based atlas (b) and patient-specific atlas (c). The proposed patient-specific atlas produces a more accurate estimation than the population-based atlas, especially in the regions indicated by the arrows.

Specifically, we propose to estimate the patient-specific atlas by using a patch-based representation technique.^{38, 39} The rationale is that an image patch generally provides richer information, e.g., anatomical pattern, than a single voxel. First, N intensity images of atlases I_j(j = 1, …, N) and their corresponding segmentations S_j are aligned onto the space of the testing image I according to the registration method described in Sec. 2.B. Then, for each voxel χ in the testing image I, its corresponding intensity patch with size of w × w × w can be represented as a column vector Q(χ). Similarly, at the same location, we can obtain patches Q_j(χ) from the jth aligned atlas. An initial codebook B can be constructed using all these atlas patches, i.e., B(χ) = [Q₁(χ), Q₂(χ), …, Q_N(χ)]. To alleviate the effect of possible registration errors, the initial codebook B is extended to include more patches from the neighboring search window (i.e., a w′ × w′ × w′ cubic) of all N aligned atlases, thus generating an overcomplete codebook with superior representation capacity.⁴⁰ In the codebook B(χ), each patch is represented by a column vector and normalized with unit ℓ2 norm.^{40, 41} To represent the patch Q(χ) by the codebook B(χ), its coding vector c could be estimated by many coding schemes, such as vector quantization, locality-constrained linear coding,⁴² and sparse coding.^{43, 44} In this paper, we utilize the sparse coding scheme^{43, 44} to estimate the coding vector c by minimizing a non-negative Elastic-Net problem,⁴⁵

$$\underset{\mathit{c}\ge 0}{\min }\frac{1}{2}{\Vert \mathit{Q}\left(\chi \right)-\mathit{B}\left(\chi \right)\mathit{c}\Vert }_2^2+{\lambda }_1{\Vert \mathit{c}\Vert }_1+\frac{{\lambda }_2}{2}{\Vert \mathit{c}\Vert }_2^2,$$ (1)

where the first term is the least square fitting term, the second term is the ℓ1 regularization term used to enforce the sparsity constraint on the reconstruction vector c, and the last term is the ℓ2 smoothness term used to enforce the similarity of coding coefficients for similar patches. Equation 1 is a convex combination of ℓ1 lasso⁴⁶ and ℓ2 ridge penalty, which encourages a grouping effect while keeping a similar sparsity of representation.⁴⁵ Each element of the coding vector c, i.e., c_j(y), reflects the similarity between the target patch Q(χ) and the patch Q_j(y) in the codebook. Based on the assumption that the similar patches should share similar labels, we use the sparse coding c to estimate the prior probability of the voxel χ belonging to the ith structure/region (e.g., mandible, maxilla, soft-tissue, or background), i.e.,

$$p_i\left(\chi \right)=\frac{1}{Z}\sum _j\sum _{y\in W\left(\chi \right)}{c}_j\left(y\right){\delta }_i\left(S_j\left(y\right)\right),$$ (2)

where Z is a normalization constant to ensure ${\sum }_i{p}_i\left(\chi \right)=1$, and δ_i(S_j(y)) = 1 if the label S_j(y) = i; otherwise, δ_i(S_j(y)) = 0. By visiting each voxel in the testing image I, we can finally build a patient-specific atlas p_i. The constructed patient-specific atlas for the image shown in Fig. 5a is provided in Fig. 5c, which captures more accurate patient-specific anatomical structures than the traditional population-based atlas [Fig. 5b], which was constructed using Joshi et al.'s groupwise registration method.⁴⁷ Finally, the patient-specific atlas can be used as a prior to accurately guide the subsequent level-sets based segmentation.

Convex segmentation based on MAP

The patient-specific atlas could provide a rough segmentation if simply thresholding the priori probability for each voxel. However, it may result in spatially inconsistent segmentation since the patient-specific atlas is estimated in a voxelwise manner. Moreover, in the construction of the patient-specific atlas, it is difficult to enforce the geometrical constraints, e.g., no overlapping between mandible and maxilla. To address these issues, we propose the following strategy to integrate the patient-specific atlas into a level-set framework based on the maximum a posteriori probability rule.

Specifically, to accurately label each voxel χ in the image domain Ω of the testing image, we jointly consider its neighboring voxels $y\in \mathcal{O}\left(\chi \right)$, where $\mathcal{O}\left(\chi \right)$ is the neighborhood of voxel χ. In fact, we can use a Gaussian kernel K_ρ with scale ρ to control the size of the neighborhood $\mathcal{O}\left(\chi \right)$.⁴⁸ The regions {Ω_i} produce a partition of the neighborhood $\mathcal{O}\left(\chi \right)$, i.e., ${\left\{{\Omega }_i\cap \mathcal{O}\left(\chi \right)\right\}}_{i=1}^3$. We first consider the segmentation of $\mathcal{O}\left(\chi \right)$ based on maximum a posteriori probability. According to the Bayes rule:

$$\begin{array}{ccc}\hfill & & p\left(y\in {\Omega }_i\cap \mathcal{O}\left(\chi \right)|I\left(y\right)\right)\hfill \\ \hfill & & =\frac{p\left(I\left(y\right)|y\in {\Omega }_i\cap \mathcal{O}\left(\chi \right)\right)p\left(y\in {\Omega }_i\right)p\left(y\in \mathcal{O}\left(\chi \right)\right)}{p\left(I\left(y\right)\right)},\hfill \end{array}$$ (3)

where $p\left(I\left(y\right)|y\in {\Omega }_i\cap \mathcal{O}\left(\chi \right)\right)$, denoted by p_i,χ(I(y)), is the structure probability density in region ${\Omega }_i\cap \mathcal{O}\left(\chi \right)$. Note that p(y ∈ Ω_i), i.e., p_i(y), is the a priori probability of y belonging to the region Ω_i, which has been estimated in Sec. 2B. Note that the prior probability was not utilized in the previous work⁴⁹ (all partitions were assumed to be equally possible) and only population-based probability was utilized.⁵⁰$p\left(y\in \mathcal{O}\left(\chi \right)\right)={\mathbf{1}}_{\mathcal{O}\left(\chi \right)}\left(y\right)$ is the indicator function, and p(I(y)) is independent of the choice of the region and can therefore be neglected. Accordingly, Eq. 3 can be simplified as

$$p\left(y\in {\Omega }_i\cap \mathcal{O}\left(\chi \right)|I\left(y\right)\right)\propto p_{i,\chi }\left(I\left(y\right)\right)p_i\left(y\right).$$ (4)

Assuming that the voxels within each region are independent, the MAP will be achieved only if the product of p_{i, χ}(I(y))p_i(y) across the regions $\mathcal{O}\left(\chi \right)$ is maximized: ${\prod }_{i=1}^3{\prod }_{{\Omega }_i\cap \mathcal{O}\left(\chi \right)}{p}_{i,\chi }\left(I\left(y\right)\right)p_i\left(y\right)$. Taking a logarithm transformation and then integrating all the voxels χ, the maximization can be converted to the minimization of the following energy:

$$\begin{array}{c}\hfill {\mathcal{E}}^{\mathcal{D}}\left(C\right)=-\sum _{i=1}^3\underset{\Omega }{\int }\underset{{\Omega }_i}{\int }{K}_{\rho }\left(x-y\right)\log p_{i,\chi }\left(I\left(y\right)\right)p_i\left(y\right)dydx.\end{array}$$ (5)

In this way, multiple level set functions could be used to represent the regions {Ω_i} as in the work of Li et al.⁴⁸ However, the minimization problem with respect to the level set function is usually a nonconvex problem and there is a risk of being trapped in the local minima. Alternatively, based on the work of Goldstein and Bresson,⁵¹ multiple variables could be used by taking values between 0 and 1 to derive a convex formulation. Since, in our project, there are only three different regions of interest: mandible, maxilla, and background, we need just two segmentation variables u₁ ∈ [0 1] and u₂ ∈ [0 1] to represent the partitions {Ω_i}:M₁ = u₁,M₂ = u₂,M₃ = (1 − u₁)(1 − u₂). Therefore, we convert Eq. 5 as follows:

$$\begin{array}{ccc}\hfill {\mathcal{E}}^{\mathcal{D}}\left(u_1,u_2\right)& =& -\sum _{i=1}^3\underset{\Omega }{\int }\underset{\Omega }{\int }{K}_{\rho }\left(x-y\right)\hfill \\ & & \times \log p_{i,x}\left(I\left(y\right)\right)p_i\left(y\right)M_i\left(y\right)dydx.\hfill \end{array}$$ (6)

There are many options to estimate p_{i, x}(I(y)). In this paper, we utilize a Gaussian distribution model with the local mean μ_i(x) and the variance ${\sigma }_i^2\left(x\right)$ (Ref. ⁵²) to estimate it:

$$\begin{array}{ccc}\hfill & & p_{i,x}\left(I\left(y\right)\right)\hfill \\ \hfill & & =\exp \left(-{\left({\mu }_i\left(x\right)-I\left(y\right)\right)}^2/2{\sigma }_i^2\left(x\right)\right)/\left(\sqrt{2\pi }{\sigma }_i\left(x\right)\right).\hfill \end{array}$$ (7)

Based on the assumption that there should be no overlap between mandible and maxilla, we propose the following penalty constraint term:

$${\mathcal{E}}^{\mathcal{P}}\left(u_1,u_2\right)=\int u_1\left(x\right)u_2\left(x\right)dx.$$ (8)

In addition, the length regularization term⁵³ is defined as the weighted total variation of functions u₁ and u₂,

$${\mathcal{E}}^{\mathcal{R}}\left(u_1,u_2\right)=\int g\left(I\left(x\right)\right)\left(|\nabla u_1\left(x\right)|+|\nabla u_2\left(x\right)|\right)dx,$$ (9)

where g is a nonedge indicator function that vanishes at object boundaries.⁵³

Finally, we define the entire energy functional below, which consists of the data fitting term ${\mathcal{E}}^{\mathcal{D}}$, the overlap penalty term ${\mathcal{E}}^{\mathcal{P}}$, and the length regularization term ${\mathcal{E}}^{\mathcal{R}}$:

$$\underset{u_1,u_2\in \left[01\right]}{\min }\left\{\mathcal{E}\left(u_1,u_2\right)={\mathcal{E}}^{\mathcal{D}}+\alpha {\mathcal{E}}^{\mathcal{P}}+\beta {\mathcal{E}}^{\mathcal{R}}\right\},$$ (10)

where α and β are the positive coefficients. Based on Ref. 51, the energy functional in Eq. 10 can be easily minimized in a fast way with respect to u₁ and u₂.

EXPERIMENTAL RESULTS

Evaluation metrics

In the following, we mainly employ Dice ratio to evaluate the segmentation accuracy, which is defined as

$$DR=2\left|A\cap B\right|/\left(\left|A\right|+\left|B\right|\right),$$

where A and B are two segmentation results of the same image. We also evaluate the accuracy by measuring the average surface distance error, which is defined as

$$\begin{array}{ccc}\hfill D\left(A,B\right)& =& \frac{1}{2}(\frac{1}{n_A}\sum _{a\in surf\left(A\right)}dist\left(a,B\right)\hfill \\ & & +\frac{1}{n_B}\sum _{b\in surf\left(B\right)}dist\left(b,A\right)),\hfill \end{array}$$

where surf(A) is the surface of segmentation A, n_A is the total number of surface points in surf(A), and dist(a, B) is the nearest Euclidean distance from a surface point a to the surface B.

Parameter optimization

The parameters in this paper were determined based on 30 MSCT subjects via cross-validation. For example, we tested the weight for ℓ1-term λ₁ = {0.01, 0.1, 0.2, 0.5}, the weight for ℓ2-term λ₂ = {0, 0.01, 0.05, 0.1}, the patch size w = {5, 7, 9, 11, 13}, and the size of neighborhood w′ = {3, 5, 7, 9}. We then compare the sum Dice ratios of mandible and maxialla with respect to the different combinations of these parameters, and found that the best accuracy is achieved with the parameter λ₁ = 0.1, λ₂ = 0.05, w = 9, w′ = 5. Cross-evaluation was also performed to determine other parameters, such as ρ = 3 for the Gaussian kernel K_ρ, and the weights α = 10 for the overlap penalty term ${\mathcal{E}}^{\mathcal{P}}$ and β = 10 for the length regularization term ${\mathcal{E}}^{\mathcal{R}}$. Empirically, we found the performance of our algorithm is insensitive to the small perturbation of these parameters. However, the above range for each parameter was empirically chosen in our experiments, which could lead to local minimum results as we will discuss later.

Atlas selection

In this section, we evaluate the performance of segmentation accuracy using different sets of atlases, i.e., using (1) 15 CBCT subjects (excluding the testing one), (2) 30 MSCT subjects, and (3) 15 CBCT + 30 MSCT subjects (excluding the testing one) as atlases. The Dice ratios on mandible and maxilla, and the average surface distance errors on mandible, with respect to different sets of atlases, are shown in Figs. 67, respectively. It can be seen that the use of CBCT atlases alone result in slightly lower Dice ratios and higher surface distance errors than those obtained using spiral MSCT atlases. One reason may be due to the limited number of CBCT atlases. It is not surprising that the combination of MSCT and CBCT achieves the highest Dice ratios and also the smallest surface distance errors, due to inclusion of more atlases. Therefore, in this paper, we finally choose CBCT+ MSCT as the atlases.

Figure 6.

Evaluation of Dice ratios on mandible and maxilla with respect to different sets of atlases: 15 CBCT subjects (excluding the testing one), 30 MSCT subjects, and 15 CBCT + 30 MSCT subjects (excluding the testing one).

Figure 7.

Evaluation of average surface distance errors on mandible with respect to different sets of atlases: 15 CBCT subjects, 30 MSCT subjects, and 15 CBCT + 30 MSCT subjects.

Number of atlases vs accuracy

In the following, we explore the relationship between the number of atlases and the segmentation accuracy. Figure 8 shows the Dice ratios as a function of the number of atlases. As we can see, increasing the number of atlases generally improves the segmentation accuracy, as the average Dice ratio increased from 0.803 (N = 5) to 0.894 (N = 44) for mandible, and 0.732 (N = 5) to 0.855 (N = 44) for maxilla. However, the inclusion of more atlases would also bring in larger computational cost. Meanwhile, we found that, in our tests, when the number of atlases reaches 40, the performance of CBCT segmentation converges.

Figure 8.

Changes of Dice ratio of segmentation with respect to the number of atlases used. Experiment is performed by leave-one-out strategy on a set of 45 atlases.

Performance for the case of maximal intercuspation

As shown in Fig. 2b, due to the closed-bite position, the maxillary and mandibular teeth are touched to each other, which makes the automatic segmentation very difficult to achieve, as mentioned in the Introduction. To demonstrate the importance of employing the landmark-guided region-specific registration for separation of the maxillary (upper) and mandibular (lower) teeth, we first show the comparison results of the proposed method (Step 1: estimation of the patient-specific atlas) without and with landmark guidance for the teeth region in Fig. 9. The surface distance errors between the manual segmentations and automatic segmentations are calculated for better comparison. We also rotate the upper and lower teeth in different directions for a better visualization of the opened mouth. It can be clearly seen that the proposed method with the landmark-guided registration produces much more accurate results than that without landmark guidance. To show the advantage of the convex formulation in segmentation, we further show the result of the proposed method with both Step 1 (estimation of the patient-specific atlas) and Step 2 (convex segmentation) in the third column of Fig. 9. The average surface distance errors for the mandibular teeth and the maxillary teeth on 15 subjects are plotted in Fig. 10, which clearly demonstrates the advantage of the landmark-guided registration in estimation of the patient-specific atlas and the subsequent convex segmentation.

Figure 9.

Comparison of results by the proposed method without or with landmark guidance.

Figure 10.

Average surface distances between the surfaces obtained by three different settings of our method and the manual segmented surfaces, on 15 subjects in the teeth joint regions.

Comparisons with the state-of-the-art methods

We first demonstrate the performance of different methods for the cases with maximal intercuspation [shown in Figs. 2b, 2c] in Fig. 11. We compare with other multi-atlases-based automatic segmentation methods,^{38, 39, 54, 55} using majority voting scheme, and conventional patch-based method.³⁹ Note that, for a fair comparison, we perform a similar cross-validation as in Sec. 3.B to derive the optimal parameters for the CPM, thus obtaining finally the patch size of 9 × 9 × 9 and the neighborhood size of 5 × 5 × 5. The segmentation results on a selected slice and two surfaces by different methods are shown in the first and second rows, respectively. It can be seen that the proposed method successfully delineates the maxillary and mandibular teeth. We further calculate the surface distance errors between the manual segmentations and automatic segmentations in the fourth and fifth rows, which demonstrates that the proposed method achieves the best accuracy. Note that, for fair comparison, the results from the proposed method were directly derived from the patient-specific atlas (Step 1) by simple thresholding, similar to the CPM.

Figure 11.

Segmentation results of three different methods for the case with maximal intercuspation. The first and second rows show the comparison on slice and surfaces, respectively. In the third row, we rotate the upper teeth and lower teeth to virtually make the mouth open for better visualization. The last two rows show the surface distance errors by three different methods.

We then demonstrate the performance of different methods on a typical subject as shown in Fig. 12. In the first row, from left to right show the volume rendering of the original intensity image, and the surfaces obtained by MV, CPM,³⁹ the proposed method by directly using the maximum class probability from Step 1 (prior estimation) as segmentation result, the proposed method with both Step 1 (prior estimation) and Step 2 (convex segmentation), and the manual segmentation. For better visualization, the corresponding results on slices and zoomed views are also provided in the bottom rows. Due to the errors in image registration process, the surface by MV is far from accurate, due to incorrect labeling of some upper teeth as lower teeth. Due to closed-bite position and the large intensity variations, the patch-based fusion method³⁹ cannot accurately separate the mandible from maxilla, thus mislabeling the upper teeth and lower teeth. Overall, the proposed method produces much more reasonable results.

Figure 12.

Comparison of segmentation results by four different methods on a typical CBCT image. Note that the fourth row shows the segmentation of four different methods on the image shown in Fig. 1b.

We finally use Dice ratio to quantitatively evaluate the overlap ratio between manual segmentations and automatic segmentation on 15 subjects, as shown in Fig. 13 and Table 1. As can be observed, by integrating both patient-specific atlas and convex segmentation, the proposed method achieves the highest Dice ratios. The upper row of Fig. 14 shows the surface distances from the surfaces obtained by these methods to the manual segmented surfaces, and the lower row shows the surface distances from the manual segmented surfaces to the automatically segmented surfaces. As can be seen from Fig. 14, the proposed method achieves superior results over all other methods. The average surface distance errors on 15 subjects are plotted in Fig. 15. Additionally, the Hausdorff distance

$$D_H\left(A,B\right)=max\left\{dist\left(A,B\right),dist\left(B,A\right)\right\}$$

was also used to measure the maximal surface-distance errors of each of 15 subjects. The average Hausdorff distance on all 15 subjects are shown in Table 2, which again demonstrates the advantage of our proposed method.

Figure 13.

Dice ratios of segmented mandible and maxilla by four different methods on 15 CBCT subjects.

Figure 14.

The upper row shows the surface distances in mm from the surfaces obtained by four different methods to the manual segmented surfaces. The lower row shows the surface distances from the manual segmented surfaces to the surfaces segmented by four different methods.

Figure 15.

Average surface distances from the surfaces obtained by four different methods to the manual segmented surfaces on 15 subjects.

Table 2.

Average Dice ratios and surface distance errors (in mm) on 15 subjects (The best performance is indicated in boldface).

		MV	CPM (Ref. 39)	Proposed (Step 1)	Proposed (Step 1+2)
Dice ratio	Mandible	0.82 ± 0.04	0.88 ± 0.02	0.90 ± 0.03	0.92 ± 0.02
	Maxilla	0.74 ± 0.05	0.82 ± 0.03	0.86 ± 0.02	0.87 ± 0.02
Average distance error	Mandible	1.30 ± 0.33	0.87 ± 0.25	0.72 ± 0.20	0.65 ± 0.19
Hausdorff distance error	Mandible	3.71 ± 1.68	2.45 ± 1.43	1.25 ± 0.62	0.96 ± 0.53

DISCUSSIONS

Normal subjects vs patients

The success of applying spiral MSCT atlases of the normal subjects to the CBCT images of the patients with CMF deformity can be mainly attributed to the following two factors: (1) The deformation between the subject with CMF deformity and the normal subject is first alleviated by the image registration; (2) After registration, the probabilities for the CBCT subject with CMF deformity are then robustly estimated by the proposed patch-based sparse technique in Sec. 2.C.

Image quality between MSCT and CBCT

Although the quality is different between MSCT and CBCT, the influence of the image quality is minimized due to the following reasons. First, our method works on image patches, where similar local patterns can be captured although the whole images may have large contrast differences. Second, in the following sparse representation, all the patches are normalized to have the unit ℓ2-norm to alleviate the intensity scale problem.^{40, 41} Third, the testing patch from CBCT can be well represented by the overcomplete patch dictionary constructed from the MSCT, as they share the same patterns of anatomical structures (e.g., the bones generally have high intensities while the soft tissues have low intensities).

Patch-based sparse representation vs conventional patch matching method

The conventional patch-based matching methods^{38, 39, 56, 57, 58} are less dependent on the accuracy of registration and this technique has been successfully validated on brain labeling³⁸ and hippocampus segmentation³⁹ with promising results. However, these methods^{38, 39, 56, 57, 58} mainly use a simple intensity difference based similarity measure (Sum of the Squared Difference, SSD). Consequently, they are sensitive to the variance of tissue contrast and luminance, which is often viewed in CBCTs. By contrast, in the proposed method, we assume that image patches can be represented by sparse linear combination of image patches in an overcomplete dictionary. It is a representation problem instead of a directly patch-matching problem as in Refs. ³⁸ and ³⁹. In the proposed method, all the patches are normalized to have the unit ℓ2-norm to alleviate the variance of tissue contrast and luminance.^{40, 41} The testing patch is well represented by the overcomplete patch dictionary with the sparse constraint. The derived sparse coefficients are utilized to (1) measure the patch similarity, instead of directly using the intensity difference similarity as used SSD in Refs. ³⁸ and ³⁹ and (2) estimate a patient-specific atlas, instead of a population-based atlas.

Computational cost

In our implementation, we use the LARS algorithm,⁵⁹ which was available in the SPAMS toolbox (http://spams-devel.gforge.inria.fr), to solve the Elastic-Net problem. The average computational time is around 5 h for segmentation of a 400×400×400 image with a spatial resolution of 0.4 mm isotropic voxel on a linux server with 8 CPUs and 16G memory. In our future work, we will further optimize the code to reduce the computational time.

Summary

We proposed a new method for automatic segmentation of CBCT images. We first estimated a priori probability from spiral MSCT atlases by using a sparse label fusion technique. Then, the priori probability was integrated into a convex segmentation framework based on MAP. Comparing to the state-of-the-art label fusion methods, our method achieved more accurate segmentation results. In our future work, we will validate the proposed method on more subjects and will also improve its robustness by increasing the variability of the atlases, such as including more training subjects with different CMF deformities.