Deformable multisurface segmentation of the spine for orthopedic surgery planning and simulation

Abstract.

Purpose: We describe a shape-aware multisurface simplex deformable model for the segmentation of healthy as well as pathological lumbar spine in medical image data.

Approach: This model provides an accurate and robust segmentation scheme for the identification of intervertebral disc pathologies to enable the minimally supervised planning and patient-specific simulation of spine surgery, in a manner that combines multisurface and shape statistics-based variants of the deformable simplex model. Statistical shape variation within the dataset has been captured by application of principal component analysis and incorporated during the segmentation process to refine results. In the case where shape statistics hinder detection of the pathological region, user assistance is allowed to disable the prior shape influence during deformation.

Results: Results demonstrate validation against user-assisted expert segmentation, showing excellent boundary agreement and prevention of spatial overlap between neighboring surfaces. This section also plots the characteristics of the statistical shape model, such as compactness, generalizability and specificity, as a function of the number of modes used to represent the family of shapes. Final results demonstrate a proof-of-concept deformation application based on the open-source surgery simulation Simulation Open Framework Architecture toolkit.

Conclusions: To summarize, we present a deformable multisurface model that embeds a shape statistics force, with applications to surgery planning and simulation.

1. Introduction

1.1. Motivation

Lower back pain is the second most common neurological ailment in the United States after a headache. According to the American Association of Orthopedic Surgeons, $\sim 80\%$ of Americans will suffer from back pain at least once in their lifetime. High incidence cases associated with back pain include intervertebral disc degeneration (IID) or disc herniation, in the spinal lumbar region, as well as sciatica, pain in the legs due to IID.^1,2 Imaging studies indicate that 40% of patients suffering from chronic back pain showed symptoms of IID.³ Primary treatment planning for lower back pain consists of nonsurgical treatment. If nonsurgical treatments are ineffective, a surgical procedure may be required to treat IID, known as spinal discectomy. Approximately 300,000 discectomy procedures, which accounts for over 90% of all spinal surgical procedures,⁴ are performed each year, totaling up to $11.25 billion in cost per year.⁵ Other spinal surgeries include treatment for metastatic spinal tumors and spinal cord injury.

A patient-specific, high-fidelity spine anatomical model that faithfully represents any existing spine pathologies can be utilized as input to finite element model (FEM) for biomechanical load and displacement modeling of a healthy and degenerated spine, in surgery planning and navigation, for use by expert surgeons, as an anatomical model for surgery simulation for training surgical residents, and finally to facilitate the fusion of several spine medical images into a probabilistic intensity atlas of the spine, which could provide intensity priors corresponding to various spine anatomical structures and thus support the identification of pathology in a minimally supervised manner. This paper describes a shape-aware multisurface simplex deformable model for the segmentation of healthy as well as pathological lumbar spine in medical image data. It provides an accurate and robust segmentation scheme for the identification of intervertebral disc pathologies to enable the minimally supervised planning and patient-specific simulation of spine surgery, in a manner that combines multisurface and shape statistics-based variants of the deformable simplex model.

The initial step toward determining the cause of lower back pain is acquiring and analyzing medical image scans of the patient. An x-ray-based computed tomography (CT) scan of the lumbar spine assists the physician in determining any degeneration or fractures of the spine’s bony structures, namely vertebrae. A magnetic resonance imaging (MRI) scan is normally acquired to analyze the soft tissue structures and detect disc herniation in the spine.

The standard procedure for detecting abnormalities in the spinal structures is through visual inspection of the medical images, which is subject to the expertise of the radiologist in charge of the patient. Spine treatment planning requires a patient-specific 3-D anatomical model of the spine capable of correctly representing the salient anatomical features, such as vertebrae, the intervertebral discs, the spinal cord, and surrounding nerves. This anatomical modeling requires the identification of nonoverlapping, homogeneous anatomical structures in medical images, a process referred to as image segmentation. The complexity of the anatomy of the spine poses a segmentation problem due to several connected structures. Low image resolution, low contrast between soft tissues, and image noise hinder the detection of these complex structural boundaries, affecting the accuracy of the constructed model. Moreover, if the elaboration of a surgery planning model is also left to a clinician, the traditional approach to anatomical modeling requires exhaustive slice-by-slice segmentation of the structures of interest, which requires several hours of work.

1.2. Review of the Spine Segmentation Literature

In this section, current research in spine segmentation is represented. Most methods have adopted a hierarchical segmentation approach that exploits prior knowledge by utilizing either shape and appearance-based statistical models or probabilistic models to identify the region of interest. Recent work has emphasized machine learning based on convolutional neural networks, much of it in 2D.

Aslan et al.^6,7 proposed a hierarchical 3-D vertebra segmentation method incorporating graph cuts with statistical shape and appearance features to aid in analysis of vertebra osteoporosis and fracture analysis. A 3-D shape model is constructed by extracting training shapes through manual segmentation of 3000 clinical CT images performed by an expert. The training images are aligned by rigid registration and binarized to form a shape volume composed of the vertebra object, the background, and the allowed variability. Their method calculates variability within the dataset by calculating the marginal density of the object and background, denoted the distance probability model. Aslan et al. mitigated user intervention by identifying the region of interest using a matched filter, with an average identification success rate of 85%.

Gosh et al.⁸ presented an automatic, multistep lumbar vertebra segmentation method in CT images to detect lumbar wedge compression fractures. They utilized a two-level probabilistic model to label the intervertebral discs in the sagittal plane of the CT based on image intensity as well as the disc location features. They further detected the vertebral skeleton in the sagittal CT images by performing thresholding and morphological operations using the prior knowledge that bone is typically 400 houndsfield units or higher. Individual vertebrae are segmented by applying the generalized Hough transform on the gradient of the vertebral skeleton image and performing morphological and hole-filling operations. This method can identify wedge compression fractures with a sensitivity of 91.7% but is limited to a representation in the shape of a rectangle and does not segment the sharp vertebra edges, such as the spinous process.

Khallaghi et al.⁹ proposed a volumetric registration framework of CT lumbar vertebral images to ultrasound images to assist with spinal needle injections. They constructed a statistical shape model (SSM) of 35 CT scans through semiautomatic segmentation and resampling to achieve shapes in correspondence. They apply a two-step registration approach for shape model construction as well as multimodal registration by applying rigid registration using six random parameters and then multiresolution B-spline deformable model using 12 random parameters. Results show about 80% success rate with $<3.5\mathrm{mm}$ acceptable registration accuracy, with least registration accuracy for the sagittal vertebral images, featuring sharp distinct edges.

Rasoulian et al.¹⁰ improved SSM construction and vertebral registration by proposing a probabilistic surface-based group-wise registration method. They apply point-to-point correspondence for the training dataset using the expectation–maximization (EM) method to solve a probability density estimation problem. This EM method assumes that voxel intensities on a shape are independent samples from a probability distribution class, which can be classified to a particular region by solving the likelihood function of the posterior probability of that voxel belonging to a region. Given a set of allowed transformations within the mean shape, they determine the likelihood of a point located on the mean shape being mapped onto a point in each of the training shapes. The constructed SSM is rigidly registered to the extracted bone surface density in the ultrasound volumetric images.

Klinder et al.¹¹ proposed an automatic 3-D segmentation of the spinal column using SSMs coupled with an initialization of the region of interest based on the generalized Hough transform. They propose a two-step segmentation method that initially detects the global location and orientation of the spinal column shape by representing it as a collection of objects encapsulated by rigid transformations against a defined vertebra coordinate system (VCS). In the second step, nonrigid shape-constrained deformation of the individual triangulated mesh corresponding to each vertebra object is performed. Although the method promises 1 mm accurate results, the results may be biased as the reference model is constructed using the same method with user guidance. Moreover, the VCS construction requires placement of landmarks by a domain expert that are necessary for mean model generation and achieving correspondence.

Ma et al.¹² presented a hierarchical deformable surface-based thoracic vertebra segmentation method in CT images. 20 training samples comprising 12 thoracic vertebrae were manually segmented and their discrete triangulated mesh models based on the marching cubes algorithm. A probabilistic edge detection method was used to sample 5 points along the normal direction for each mesh face, where the intensity and gradient projection feature vectors corresponding to each sample point were computed. These feature vectors were used to train the probabilistic model for detecting a vertebra edge. A coarse-to-fine template deformation mesh approach was adopted, where a vertebra was divided into 12 salient subregions whose edges were detected through deformation of a template. These subregions are deformed to accurately capture the object boundary through piecewise nonrigid registration. This method achieved grouped vertebra segmentation of over 90% success rate, however, finding the optimal transformations for deformation is computationally expensive and dependent on the coarsest mesh resolution.

Howe et al.¹³ proposed a hierarchical segmentation scheme for 273 cervical and 262 lumbar x-ray images. They initially approximated vertebra location and orientation through the generalized Hough transform edge detection method, and then subsequently utilized active appearance models (AAM) to deform a template toward the extracted edges in two stages. The first stage matches the intensity values to segment a vertebra and its two concurrent neighbors, and the second stage extracts a single vertebra by matching the weighted intensity values with the AAM template, giving priority to the vertebra edge where higher contrast is expected. Their 2-D contour extraction is limited to areas of high gradient values and high-contrast x-ray images.

Benjelloun et al.¹⁴ presented a semiautomatic cervical vertebra segmentation method in x-ray images using active shape models. They required two landmark placements at the beginning and end of the cervical spinal column for model initialization and applied Harris corner detector with additional filters to detect two corners of each cervical vertebra to localize the region of interest. A local vertebra template is deformed to segment each vertebra in the region of interest. They also investigate the influence of the initialization of the statistical contour, the influence of a number of landmarks placed on the vertebra boundary, with 20 landmarks used in their method, and training sample size, with 75 ideal shapes used. Their segmentation using the vertebra model resulted in a high success rate over 90% with a mean error of 0.6 mm; however, they do not provide specific details of their ASM construction and corner detection scheme.

Zhao et al.¹⁵ presented a modified gradient vector flow deformable snake with additional external forces as a method for segmenting spine MRI images. They proposed to segment vertebrae in the sagittal image plane and base their method on the observation that vertebra contours are similar to rectangles with concave edges in the 2-D plane. Therefore, external forces, which can be modified through weighted coefficients, are augmented to the snake equation for utilization in segmentation, producing results that converge faster, with marginally improved results. However, this simplistic shape assumption cannot be translated toward volumetric image vertebrae segmentation.

Recently, several groups have proposed two-dimensional machine learning segmentation and localization algorithms through convolutional neural networks, namely Lessmann, Whitehead, and Suzani along with their respective teams.^16–18 While encouraging, their two-dimensionality makes them less than readily applicable to surgery planning and simulation. In addition, the MICCAI Workshop on Computational Methods and Clinical Applications for Spine Imaging held a Challenge competition on IVD segmentation that featured an entry that used a 3-D deep learning approach,¹⁹ namely DeepSeg3D by Chen.²⁰ DeepSeg3D exploits 3-D convolutional kernels, which inputs 3-D volumetric data and directly outputs a 3-D prediction mask. The architecture of neural network contains two convolutional layers, two max-pooling layer for downsampling, and two unpooling layers for upsampling. Finally, two groups led by Chu et al.²¹ and Korez et al.²² demonstrated the application of machine learning approach to 3-D vertebra segmentation, albeit one restricted to vertebral bodies: Chu’s approach is based on random forests, whereas Korez employs a 3-D convolutional network. Neither method does not attempt to identify any of the processes. In machine learning approaches, it is not clear how the significant reliance on training data is compatible with the presence of pathologies, which are necessarily random in nature. Moreover, 3-D deep learning appears to be restricted to simple, smooth shapes such as the IVD, whereas vertebrae, with fine details such as processes and foramina, appear to be challenging for the current state of the art in deep learning. It can be argued that the data reduction involved in machine learning approaches, such as the max-pooling that simplifies a problem ultimately into one solvable by a fully connected layer, typically obliterates fine-level details such as processes and foramina while making it difficult to exploit a coarse-to-fine strategy.

1.3. Related Work in Simplex Deformable Surface Models

Broadly speaking, there are three categories of medical image segmentation techniques: voxel-based, boundary-based, and atlas-based approaches. The boundary-based approach can be advantageous in the presence of pathologies while also providing support for the consideration of shape priors. This statement is particularly true of the simplex deformable surface model, an efficient discrete deformable model introduced by Delingette²³ for 3-D shape reconstruction and segmentation. A $k$-simplex mesh embedded in Euclidean ${\mathbb{R}}^d$ space, where $k<d$, is a $k$-manifold discrete mesh with exactly $k$ + 1 distinct neighbors. A simplex mesh has the property of constant vertex connectivity. Simplex meshes can represent various objects depending on the connectivity $k$, where 1-simplex represents a curve, a 2-simplex represents a surface, and a 3-simplex represents a volume. Our research is focused on surface representation for image segmentation using 2-simplex meshes with constant 3-connectivity at each vertex, which is capable of representing arbitrary shapes of various values of genus and number of holes. This is especially relevant for this research, as the resulting intervertebral disc simplex mesh has genus = 0, whereas the vertebral simplex mesh has genus = 1.

Delingette²³ introduced topological operators to transform a simplex mesh, which in particular allows the addition or deletion of edges²⁴ improved on the former through topological macro-operators that ensured higher mesh quality through greater rotational symmetry, thereby potentiating the application of the simplex to a multiresolution model-to-image registration scheme. In general, a multiresolution surface model registration is preferable to a single-resolution approach as a result of the smaller number of false minima at a coarse level, which can robustly bring medium and fine-level meshes close to their respective global optima, which is a better option than finding that fine level’s optimum without any coarse-level cues. An illustration of two simplex mesh resolutions appears in Fig. 1(a).

Fig. 1.

Features of simplex surface model. (a) Example of multiresolution approach illustrated by coarse and fine overlapping meshes. (b) Duality of three-connected simplex mesh (black) with triangulated surface mesh (blue).¹⁵ (c) Definition of simplex angle, which results in a discrete approximation of surface curvature. (d) 2-D depiction of Gilles’ collision detection method based on medial axis models of the simplex surface boundaries (reproduced from Ref. 22).

Also important, simplex meshes are geometrically dual to triangulated surface meshes, as shown in Fig. 1(b). One can view the centroid of each simplex face as coincident with a vertex of a triangular mesh and vice versa. The two surface meshes are also topologically equivalent, in that a spherical simplex mesh surface (genus 0) leads to a spherical triangulated mesh, whereas a simplex toroid (genus 1) maps to a triangulated toroidal mesh. This duality entails that a simplex-based segmentation equates with a triangulated surface of similar density, which can serve as a hard boundary constraint for a tetrahedralization of the interior. This result is important to the tetrahedral meshing community as it leads to “high-fidelity tetrahedral meshes of controlled resolution,” provided that the interior mesh resolution mirrors the boundary mesh resolution, feasible with variational tetrahedralization.²⁵ In contrast, competing one-shot tetrahedralization methods such as octree-based approaches impose a large number of tiny tetrahedra near the borders to achieve a high-fidelity boundary,²⁶ making them inappropriate for haptics-driven simulation. Moreover, a surface meshing stage allows tetrahedral meshing to also benefit from efficient multisurface modeling and powerful statistical shape priors.

The simplex model also features a discretization of differential surface geometry, as shown in Fig. 1(c); the local geometry at a vertex $\mathit{P}$ is defined by its three neighbors ${\mathit{P}}_1$, ${\mathit{P}}_2$ and ${\mathit{P}}_3$, based on corresponding local parameters. ${ϵ}_i$ are the barycentric coordinates of the projection of vertex ${\mathit{P}}_{\perp }$ on the triangle (${\mathit{P}}_1{\mathit{P}}_2{\mathit{P}}_3$) such that ${ϵ}_1+{ϵ}_2+{ϵ}_3=1$. The orthogonal projection ${\mathit{P}}_{\perp }$ can be defined by the linear combination of the position of its neighbors ${\mathit{P}}_i$ with barycentric coordinates ${ϵ}_i$ within the triangle (${\mathit{P}}_1{\mathit{P}}_2{\mathit{P}}_3$). The variable $\varphi$ is the angle that determines the mean curvature of the mesh at that vertex $\mathit{P}$, as depicted in Fig. 1, which is denoted the simplex angle.²³ The neighborhood-based constraints of $P$ are thus uniquely governed by

$$\mathit{P}({ϵ}_1,{ϵ}_2,\varphi )={ϵ}_1{\mathit{P}}_1+{ϵ}_2{\mathit{P}}_2+(1-{ϵ}_1-{ϵ}_2){\mathit{P}}_3+h(\varphi )\mathit{n},\text{where}\mathit{n}=\frac{{\mathit{P}}_1{\mathit{P}}_3\wedge {\mathit{P}}_1{\mathit{P}}_2}{\Vert {\mathit{P}}_1{\mathit{P}}_3\wedge {\mathit{P}}_1{\mathit{P}}_2\Vert },$$ (1)

where $h(\varphi )$ represents a discrete approximation of mean curvature and $n$ is the surface normal, both estimated at the vertex of interest.

The deformation of the simplex model is governed by the position of a vertex with respect to its three neighbors. The dynamics of each vertex $\mathit{P}$ is governed by a Newtonian law of motion represented by

$$m\frac{{\mathrm{d}}^2{\mathit{P}}_i}{\mathrm{d}{t}^2}=-\gamma \frac{\mathrm{d}{\mathit{P}}_i}{\mathrm{d}t}+\alpha {\mathit{F}}_{\mathrm{int}}+\beta {\mathit{F}}_{\mathrm{ext}},$$ (2)

where $m$ is the vertex mass, $\gamma$ is the damping force, and $\alpha$ and $\beta$ are the weight factors of the internal and external forces, respectively. ${\mathit{F}}_{\mathrm{int}}$ is the sum of internal forces represented by an elastic force that enforces smoothness constraints and ${\mathit{F}}_{\mathrm{ext}}$ is the sum of external forces. This physically based deformable model is governed by forces to maintain internal stabilization through ${\mathit{F}}_{\mathrm{int}}$. ${\mathit{F}}_{\mathrm{ext}}$ is the sum of external forces comprised of image information and nonoverlap constraints. Image information, such as image edge and gradient intensity values, comprises of the similarity criteria to be maximized during simplex evolution along the normal direction.

During multiregion segmentation, collision handling ensures that object boundaries do not overlap or self-penetrate. Gilles²⁴ applied collision detection as an additional step after bone segmentation by the distance field method where the faces of the medial axis surface boundaries are stored into a bounding volume hierarchy in a preprocessing step. As bounding volumes are inflated, colliding regions result in overlapping bounding volumes. Figure 1(d) provides a 2-D example of the medial-axis based collision detection method. A collision vector ${\mathit{P}}_c$ between a colliding vertex $\mathit{P}$ and mesh face composed of ${\mathit{P}}_i$ vertices is defined as the linear combination ${\mathit{P}}_c=\sum _i{w}_i{\mathit{P}}_i-\mathit{P}$. Expected collisions are predetected by storing the indices and weights of the collision vector ${\mathit{P}}_c$ during initialization. This ${\mathit{P}}_c$ vector is updated during deformation as a collision response.

SSMs determine a mean shape and allowed variability within the model as well as construction of new shapes through a combination of the principal modes of variation within the expected shape. This SSM property can be combined with deformable models to constrain a deformation toward an expected shape during the segmentation process in the presence of image noise or artifacts that otherwise hinder object boundary detection. Tejos et al.²⁷ combined statistical knowledge with simplex meshes and snakes evolution to robustly segment knee ligaments despite low contrast and significant image noise. Schmid et al.²⁸ augmented simplex meshes with shape and appearance knowledge for segmentation of MRI musculoskeletal structures with limited field of view or presence of image artifacts. Although shape models can provide robust segmentation with the presence of image artifacts and low contrast, their performance is dependent on initialization. Our work improves on prior SSM-aware simplex models in its use of a fully automatic landmark placement that makes correspondences explicit through its precise, robust repeatability, which prevents any slippage that could occur otherwise.

1.4. Contribution of Proposed Approach

Our contribution is a coarse-to-fine deformable surface model that uniquely combines minimally supervised particle-system-based shape statistics with multisurface awareness, which leads to a general-purpose minimally supervised approach to segmentation, applied to the spine while enabling a lightweight process to support pathologies that would defy a purely statistical approach. It is also unique in supporting controlled-resolution multianatomy tetrahedralization, which is vital for the medical simulation community. We exploit controlled-resolution meshing conducive to a coarse-to-fine multiresolution approach to 3-D segmentation of both discs and vertebrae, as well as producing anatomical models with low element count and minimal spatial overlap for interactive simulation. This paper builds on our original deformable surface model presented in Refs. 29 and 30, as well as the shape statistics-aware deformable surface model presented in Ref. 31. The difference between the two deformable models is the emphasis on SSMs to achieve a strong shape prior in the new proposed method, in contrast to a weak shape prior encoded by a single typical shape, which did not arise from averaging several shapes as is the case in the proposed approach. As a result of building on SSMs, not only does the new approach leverage first-order shape statistics, but it can also rely on second-order statistics that encode variability in surface shape, which also makes it possible to place a greater weight on the shape prior. This greater reliability of shape priors is vital in enabling the application of the shape-aware deformable model to extract vertebrae and IVDs from the same tomographic image. Our first publication on an SSM-aware simplex model³¹ consisted of a preliminary study, which in particular did not integrate multiple surfaces along with SSMs as is proposed in this paper.

2. Materials and Methods

2.1. Overview: Combining Multisurface Multiresolution Simplex with Statistical Shape Models

This research proposes a minimally supervised multisurface spine segmentation method by incorporating SSM in discrete deformable models, namely multisurface simplex meshes, to identify healthy vertebrae and intervertebral disc structures of the spine, as well as user-assisted disc pathological regions. Our work uniquely combines the multisurface multiresolution simplex mesh with the SSM-aware simplex forces in order to segment lumber vertebrae and intervertebral discs. Moreover, this paper improves on existing work on SSM-aware discrete deformable models through its use of an information theory-based particle system for consistent landmark placement that automatically produces correspondences, in a manner that scales to multiple 3-D surfaces. As a result, the proposed approach constitutes a new, broadly applicable methodology for combining deformable surface models with collision detection-based multisurface representation and statistical shape priors.

Given that our approach emphasizes both deformable surface models and shape statistics, we view the initialization as an important component of this pipeline. Figure 2 depicts the overall image analysis pipeline. The initialization must feature a surface description that will reliably produce a good coarse-level agreement with the anatomical boundary in the CT or MR image. As a result, we strive for a surface template that can be initially registered unambiguously and deforms robustly to coarsely capture the desired shape. To that effect, we have developed some template surfaces that lead to good results, which was trivial for intervertebral discs but challenging for vertebrae, given their toroidal topology and complex shape in relation to their doughnut-like opening. The basic approach to producing this template is outline in Sec. 8. The next section emphasizes foundational concepts of SSMs with a particular emphasis on our approach to their construction and to their integration into deformable multisurface models.

Fig. 2.

Shape-aware simplex mesh methodology. The shape prior is either weak or SSM-based.

2.2. Shape Statistics-Aware Simplex Model-Based Segmentation

Discectomy planning and simulation require a patient-specific, robust 3-D representation of vertebral and intervertebral disc structures, including any pathology, of the lumbar spine. Although lumbar vertebral structures have high variability, the prominent features of the bone are consistent within a sample population. Moreover, healthy intervertebral discs are also consistent in shape. This consistency exhibited by both anatomical structures facilitates the incorporation of an SSM with expected variations into a volumetric image segmentation framework. Low image resolution and image artifacts, such as image noise, make biomedical volumetric image segmentation a challenge. Ambiguous image intensity results in incorrect, or even disconnected, boundary detection of the structure of interest. Prior knowledge, such as expected shape and variance within a sample population, can be incorporated through SSMs to optimize the image segmentation process.

The shape of an object is the geometrical information that remains after effects of translation, rotation, and scaling have been filtered.³² Although medical images consist of considerable variability by default, appearance and shape of the anatomical structure of interest to be identified are consistent across individuals. This information can be exploited by integrating statistical analysis in deformable models to optimize the segmentation process. Information to be utilized may be shape-based, such as points on a discrete mesh or curves in a continuous model. They may be appearance-based, such as global patterns of image intensity or gradient occurring regularly within a sample of images taken from a subject population.

Shape model construction consists of aligning a training dataset, applying principal component analysis (PCA), extracting principal modes of variation, and calculating an average shape model with expected shape variability. The calculated average shape and expected variations can be utilized to constraint the model deformation process. The training dataset used for model construction should be in point correspondence, which can be achieved by a variety of methods. Traditionally, in 2-D SSMs based on contours, point correspondences are achieved by manual identification of landmarks;³³ however, this method does not scale well to three dimensions, and this identification can also be undermined by locally ambiguous boundaries coinciding with low-gradient portions of a boundary. A related option, demonstrated by Tejos et al.,²⁷ involves the fitting and manual correction of a canonical surface mesh on a suite of patient images, with an emphasis on the consistent positioning of vertices over comparable anatomical structures (in this case, a suite of knee cartilage models). Alternately, a particle system such as described in this paper uses an energy formalism to consistently distribute a preselected number of landmarks on a series of surfaces extracted from binarized manually segmented anatomical structures.

A shape is invariant under similarity transformations of rotation, translation, and scaling in 2-D space. Alignment is the process of calculating the optimal $m\times m$ rotation matrix $\mathrm{\Gamma }$, $m\times 1$ translation vector $T$, and scale parameter $\beta$ to align all training shapes within a common coordinate space. Variations of shape within a training population can be modeled using PCA, also known as Karhunen–Lóeve expansion. Assuming that the training dataset covers a set of closely related shapes, correlation between shape points exists, which can be represented by a multivariate Gaussian distribution. PCA uses an orthogonal transformation to convert a set of observations of possibly correlated variables into linearly uncorrelated principal components. This transformation results in a first principal component representing as much variability in the data as possible and each succeeding component having the highest variance possible while being orthogonal to the preceding. As a very large number of shape points needs to be analyzed for statistical analysis, PCA is utilized to extract the principal modes, which represent data correlation along principal directions within the dataset, to reduce problem dimensionality. PCA is the process of determining the set of modes that captures the expected geometric variability within the training set (Fig. 3).

Fig. 3.

SSM computation methodology.

SSM construction through PCA requires point-to-point correspondence between a dataset. Correspondence is the process finding a set of points on one image or shape that can be mapped as the same set of points on another image, thereby ensuring that each structural boundary consists of an identical number of co-registered points. Salient points used for constructing a shape model are known as landmarks.³⁴ Cootes et al.³³ utilized manually identified landmarks to construct early SSMs in 2-D, however, this method does not scale well to 3-D, especially if there is any portion of the boundary that exhibits low contrast. Tejos used the simplex surface model itself, which was subsequently manually corrected, where landmark points were assumed to correspond to mesh vertices under a consistent template model alignment. We view this assumption as potentially difficult to enforce, with the result that the set of landmarks can easily slip from shape to shape if one proceeds in this manner. Our approach benefits from a fully automatic approach to homologous landmark placement, as described below.

2.2.1. Statistical shape model construction

This section describes a framework for the construction of SSMs of lumbar vertebrae and intervertebral discs from CT and MR images, respectively, of healthy subjects. The generated SSMs are utilized as a reference for knowledge-based priors to optimize segmentation of vertebrae and intervertebral discs in volumetric MR images. These shape models can be incorporated into a controlled-resolution deformable segmentation model of the lumbar spine. Incorporation of strong shape priors would facilitate quantification and analysis of shape variations across healthy subjects. It is intended as a tool for achieving minimally supervised spine segmentation results that can be utilized as part of an anatomical input to either to a spine surgery navigation system or to an interactive spine surgery training simulator for a discectomy procedure.²⁹

SSMs from 9 L1 vertebrae, 20 L2 and L3, and 20 L4 and L5 vertebrae as well as 40 L1 and L5 intervertebral discs have been generated to be utilized as shape priors during spine segmentation from volumetric MR images. Correspondence between instances within each model has been established using entropy-based point placement on the image surfaces,^35–37 which is independent of any reference bias or surface parameterization techniques.

2.2.2. Image dataset and preprocessing for SSM construction

Datasets provided by the SpineWeb Initiative have been utilized for generating shape models of an L1 vertebra and an L1 and L2 intervetebral disc. Volumetric CT scans of healthy subjects, along with binary masks, of 10 anonymized patients³⁸ were used for model construction of L1, coupled L2 and L3, and L4 and L5 vertebrae. The CT scans and binary masks had a resolution of $0.2\times 0.3\times 1{\mathrm{mm}}^3$. In addition, 40 expert interveterbral disc segmentations of 8 anonymized patients, with $2.0\times 1.25\times 1.25{\mathrm{mm}}^3$ resolution,³⁹ were preprocessed as input to the correspondence and shape model construction method.

These binary images were initially aligned along the first principal mode, and any aliasing artifacts were removed during image preprocessing. The method described in the following section was applied to generate distance maps of the binary images, which were used for 3-D surface reconstruction and establish correspondence between instances of both vertebra and disc shape models.

2.2.3. Correspondence establishment

Correspondence establishment is the process of finding a set of points on one 2-D contour or 3-D surface that can be mapped to the same set of points in another image. Anatomically, meaningful and correct correspondences are of utmost importance, as they ensure correct shape parametrization and shape representation. This can be achieved by co-registering manual landmarks onto the shape boundary in 2-D shape space but is challenging in 3-D space. Anatomical landmarks are points of correspondence on each shape that match within a sample population,³⁴ which may be manually or automatically placed. Correspondence landmarking may entail identifying matching parts between 3-D anatomical structures, which is challenging due to inherent variability within geometry or shape of the anatomical structure across a population.⁴⁰ Therefore, landmark placement to establish correspondence for robust statistical analysis is a significant task.

This research focuses on the application to shape-aware surface models of the correspondence technique introduced by Cates et al.,^35,36 which is independent of structure parameterization or a reference bias. This technique, depicted in Fig. 4, employs a two-stage framework, with soft correspondence establishment in the first stage, and correspondence optimization across all instances of the shape space in the second stage. Soft correspondence is established by automatically placing homologous points on the shape surface through an iterative, hierarchical splitting strategy of particles, beginning with a single particle. A 3-D surface can be sampled using a discrete set of $N$ points that are considered random variables $Z=(X_1,\dots ,X_N)$ drawn from a probability density function (PDF) $p(X)$.³⁵ Denoting a specific shape realization of this PDF as $z=(x_1,x_2,\dots ,x_N)$, the amount of information contained in each point is the differential entropy of the PDF function $p(x)$, which is estimated as the logarithm of its expectation $\log \{E[p(x)]\}$, $E(·)$ estimated by Parzen Windowing.³⁵ The cost function $C$ becomes

$$\begin{gathered} C\{x_1,\dots ,x_N\}=-H(P^i)=\sum _j\log \frac{1}{N(N-1)}\sum _{k\ne j}p(x_j) \\ =\sum _j\log \frac{1}{N(N-1)}\sum _{l\ne j}G(x_j-x_l,{\sigma }_j), \end{gathered}$$ (3)

where $G$ is an isotropic Gaussian kernel with standard deviation ${\sigma }_j$. These dynamic particles have repulsive forces that interact within their circle of influence, limited through the Gaussian kernel until a steady state is achieved, and are constrained to lie on shape surface through gradient descent in the tangent plane.

Fig. 4.

Particle system. (a) Naive depiction of Gaussian repulsion, for a simple 12-particle system (particles occluded from view depicted in pastel tones; nonoccluded particles shown as vivid). (b) System of 4038 particles on the IVD surfaces, based on three IVD models used to generate the IVD SSM: L1 and L2, L2 and L3, and L3 and L4. (c) System of 16,384 particles for L1 to L5 vertebrae used to generate shape priors for L1 vertebra, coupled L2 and L3 vertebrae and coupled L4 and L5 vertebrae.

These correspondences are further optimized by entropy-based energy minimization of particle distribution along gradient descent by balancing the negative entropy of a shape instance with the positive entropy of the entire shape space encompassing all instances (known as an ensemble).³⁵ Consider an ensemble $ϵ$ consisting of $M$ surfaces, such as $ϵ=(z_1,z_2,\dots ,z_M)$, where points are ordered according to correspondences between these surface pointsets. A surface $z_k$ can be modeled as an instance of a random variable $Z$, where the following cost function is minimized:

$$Q=H(Z)-\sum _{k}H(P_k).$$ (4)

The cost function $Q$ favors a compact representation of the ensemble and assumes a normal distribution of particles along the shape surface. Hence, $p(z)$ is modeled parametrically with a Gaussian distribution with covariance $\mathrm{\Sigma }$. This ensemble entropy term can be represented as

$$H(z)\approx \frac{1}{2}\log \Vert \mathrm{\Sigma }\Vert =\frac{1}{2}{\mathrm{\Sigma }}_k{\lambda }_k,$$ (5)

where ${\lambda }_k$ are the ensemble covariance eigenvalues. This process optimally repositions the particles of the shapes within the ensemble to generate robust shape representations with uniformly distributed particles.

ShapeWorks³⁵ was used to establish dense correspondences of 16,384 homologous points on 49 lumbar vertebral instances, and 4038 points on 40 L1 to L5 intervertebral disc instances. The ensemble shapes were, respectively, normalized according to centroid-referred coordinates and were further aligned during the correspondence optimization process through iterative Procrustes analysis.⁴¹

2.3. Statistical Shape-Based Simplex Mesh Evolution

Three SSMs of vertebrae were generated as follows:

• •an L1 verteba SSM comprising of nine vertebral shapes,
• •a coupled L2 and L3 vertebrae SSM comprising of 20 training shapes, and
• •a coupled L3 and L4 vertebae SSM comprising of 20 training shapes.

An intervetebral disc SSM representing shape variations of all five lumbar discs was constructed using 40 disc training shapes. These shape models are further evaluated to determine their statistical sufficiency and ensure that allowable shape variations within the dataset are efficiently represented. SSM-based forces were included as an additional term of the external force in the simplex deformable model.

A mean shape is initialized within the structure of interest through landmark-based affine registration. A shape closest to the structure boundary is determined by iteratively calculating the optimal transformation and shape variations. The simplex deformation is constrained by allowable variations within the PCA in the lower resolutions during the initial segmentation stage, leading to a more rigid deformation. This PCA-based shape influence is relaxed in the higher resolution, where the simplex mesh is closer to the image boundary, where the deformation is more influenced by the presence of image-based external forces. This results in robust segmentation of global and local variations within the population, leading to a more refined result capturing structural details present within the MR images.

2.4. Statistical Shape Model Evaluation Metrics

Shape model correspondences and the constructed statistical models may be evaluated through established metrics, such as model compactness, generalization ability, and specificity.⁴⁰ A robust statistical model should have low generalization ability, low specificity, and high compactness for the same number of modes. Compactness is the ability of the model to use a minimum number of parameters to faithfully capture shape variance within the dataset. This may be calculated as the cumulative variance captured by the first $m$ number of modes:

$$C(m)=\sum _{i=1}^m{\lambda }_i,$$ (6)

where ${\lambda }_i$ is the largest eigenvalue of the $i$’th mode.

The generalized ability of the statistical model to represent new, unseen instances of a new shape that are not present in the training dataset was evaluated by performing leave-one-out experiments. Vertebra and disc SSMs were generated using all training samples except one, which was considered the test sample. This test sample was then reconstructed using the SSM and the root-mean-square (RMS) distance, and Hausdorff distance errors were calculated between the reconstructed sample and the original test sample after rigid registration. This method was repeated over the entire vertebra and disc datasets, respectively, to calculate an average and worst measure of error for both statistical models. Generalization ability $G(m)$ and its associated standard error ${ϵ}_G(m)$ can be mathematically represented as

$$G(m)=\frac{1}{n}\sum _{i=1}^n{\mathbb{D}}_i(m),$$ (7)

$${ϵ}_{G(m)}=\frac{{\sigma }_{G(m)}}{\sqrt{n-1}},$$ (8)

where ${\mathbb{D}}_i(m)$ is the RMS or Hausdorff distance error between the test sample and the instantiated shape, $n$ is the number of shapes, and ${\sigma }_{G(m)}$ is the standard deviation of $G(m)$.

Model specificity is the measure of a model to only instantiate instances that are valid and similar to those in the training dataset. To measure our statistical models’ specificity, $(n-1)$ instances where randomly generated within ($-3\lambda$, $+3\lambda$) using our statistical models and compared to the closest shape in the training dataset. Specificity $S(m)$ and its standard error ${ϵ}_S(m)$ have been calculated as

$$S(m)=\frac{1}{n}\sum _{j=1}^n{\mathbb{D}}_j(m),$$ (9)

$${ϵ}_{S(m)}=\frac{{\sigma }_{S(m)}}{\sqrt{n-1}},$$ (10)

where $n$ is the number of samples, ${\mathbb{D}}_i(m)$ is the RMS distance error between a randomly generated instance and its nearest shape within the training dataset. $\sigma$ is the standard deviation of $S(m)$.

Evaluation results of the three constructed vertebrae SSMs and intervertebral disc SSM are presented in Sec. 4.1.

3. Results and Discussion

Lumbar intervertebral disc and vertebral segmentation results using inherent, weak-shape priors, and consequently strong-shape priors are presented. MeshValmet has been utilized for calculation of quantitative validation metrics. The mean absolute shape distance (MASD) (in mm) and absolute standard deviation of all errors (in mm), absolute mean square distance (MSD) (in mm), the Hausdorff distance (in mm), and DICE similarity coefficient comparison metrics have been calculated to compare the quality of our segmentation approach with ground truth. The Hausdorff distance is the maximum surface distance between two surface meshes and quantitatively represents a measure of the worst segmentation error. DICE similarity coefficient compares the similarity between the resulting segmentation and ground truth, and has been calculated as $s=(2|X\cap Y|)/(|X|+|Y|)$. MRI images used in these studies were $T_2$-weighted Spin Echo, acquired sagittally, on a 1.5 T scanner, with a spacing of $0.5\times 0.5\times 3{\mathrm{mm}}^3$.

Parameter values for the simplex model in the expression [Eq. (2)] were set as follows. The mass $m$ was set to 1. Some parameters varies over the evolution of the multiresolution approach. In simulations with weak shape priors, the value of the $\beta$ weighting the external force would operate in the 0.27 to 0.2 range for IVDs, and from 0.5 to 0.2 range for vertebrae. For SSM-aware simulation, $\beta$ would vary from 0.5 to 0.2 in IVDs and from 0.7 to 0.2 in vertebral models. The internal force $\alpha$ also varied over successive multiresolution iterations, from 0.45 to 0.3 for IVDs and from 0.6 to 0.4 for vertebrae. Finally, the dampening factor $\gamma$ varies from 0 to 0.5 over the course of 400 iterations for IVDs and from 0 to 0.7 through 400 iterations.

3.1. Segmentation Using Weak-Shape Priors

3.1.1. Healthy intervertebral disc segmentation

A statistical comparison of 16 automatic segmentations, using the multiresolution weak shape prior-aware simplex model, of healthy lumbar intervertebral discs with manually corrected segmentation results, considered ground truth, is represented in Table 1. The average absolute mean error of healthy disc segmentation approach is $0.32\mathrm{mm}\pm 0.46\mathrm{mm}$, with an average Hausdorff distance of 3.26 mm and average DICE score of 0.954. The maximum surface error was generally located at the lateral margins of the intervertebral disc, where the automatic segmentation approach failed to faithfully capture the image boundary due to image intensity ambiguity caused by surrounding spine tissues and ligaments. Figure 5 compares automatic segmentation of a healthy L5 and S1 disc with the semisupervised segmentation result, considered as ground truth. Maximum in error corresponds with maximum under-segmentation error and maximum out error represents the over-segmentation error. Our automatic weak-shape prior segmentation approach undersegmented the lateral margins with a maximum in error of $-2.45\mathrm{mm}$ and a mean absolute segmentation error of $0.19\mathrm{mm}\pm 0.29\mathrm{mm}$. This result is improved to $0.079\mathrm{mm}\pm 0.14\mathrm{mm}$ using strong shape-priors for healthy disc segmentation, as discussed in Sec. 4.2.

Table 1.

Average validation metrics comparing automatic segmentation results with corresponding semisupervised segmentation (ground truth) of 16 healthy lumbar intervertebral discs.

Validation metric	Healthy disc
MASD (mm)	0.321
Absolute std. dev. (mm)	0.455
MSD (mm)	0.342
Average Hausdorf distance (mm)	3.261
DICE coefficient	0.954

Fig. 5.

Comparison of an automatic L5 and S1 healthy disc segmentation result against its corresponding semisupervised segmentation (ground truth), with $-1.16\mathrm{mm}$ max. in, 2.45 mm max. out error.

3.1.2. Herniated disc segmentation

Average results of five herniated discs comparing semisupervised segmentation results against manual segmentation have been calculated. Evaluation results have been obtained by calculating the surface to mesh difference between the manual segmentation, considered ground truth, and the simplex model result from our approach of the corresponding intervertebral disc. Our approach demonstrates MASD of $0.61\mathrm{mm}\pm 0.52\mathrm{mm}$ of segmentation of five herniated intervertebral discs (Table 2). Our results are favorable in comparison with competing 2-D segmentation methods of herniated discs and 3-D segmentation methods of healthy discs, respectively. Michopoulou et al.⁴² reported a 2-D mean absolute distance of 0.61 mm, whereas Neubert et al.⁴³ achieved a 3-D segmented Hausdorff distance of 3.55 mm for healthy discs in high-resolution $0.34\times 0.34\times 1-1.2{\mathrm{mm}}^3$ MR images; in our case, the average Hausdorff distance is 3.26 mm using weak shape priors.

Table 2.

Average validation metrics comparing semisupervised segmentation results with corresponding manual segmentation of five herniated lumbar intervertebral discs.

Validation metric	Herniated disc
MASD (mm)	0.608
Absolute std. dev. (mm)	0.518
MSD (mm)	0.638
Average Hausdorf distance (mm)	3.485
DICE coefficient	0.917

Figure 6 shows the spatial distribution of error for a herniated disc; this error is defined in terms of initial automatic segmentation using the weak shape prior-aware simplex versus manual ground truth in (a), as well as the semisupervised segmentation result that combines the former approach with constraint-point guided model deformation against the same ground truth in (b). The weak prior-based multiresolution simplex model does a reasonably good job of segmenting the disc, with a maximum in error of $-3.61\mathrm{mm}$ near the disc pathology, although it exhibits undersegmentation at the pathology. This error is reduced to $-3.47\mathrm{mm}$ through semisupervised constraint point-guided segmentation of pathology. Figure 7 displays disc pathology with its corresponding segmentation using constraint points in a sagittal MRI slice. As herniated disc anatomy cannot be faithfully captured by prior shape or intensity features, weak shape prior influence is turned off locally and graceful degradation from these priors is allowed in a user-controlled manner, refining the segmentation result. It can be observed that maximum error in our semisupervised segmentation result is located at the lateral portion of the intervertebral disc. This is likely due to ambiguity in determining the intervertebral disc boundary at the lateral margins of the anatomy during manual segmentation.

Fig. 6.

Spatial segmentation error of an L5 and S1 herniated disc. (a) Comparison of weak-prior-based automatic (multiresolution simplex) segmentation against manual segmentation, considered ground truth ($-3.61\mathrm{mm}$ max. in and 2.60 mm max. out). (b) Comparison of semisupervised segmentation (multiresolution simplex with constraint points near the pathology) against its corresponding manual segmentation ($-3.47\mathrm{mm}$ max. in and 1.87 mm max. out).

Fig. 7.

Sagittal MRI slice of a herniated disc with corresponding segmentation and constraint points.

3.1.3. Inter- and intrarater variability

Robustness to variability in user supervision and landmark-placed template mesh initialization is demonstrated in a series of experiments where the same anatomist’s results are compared over several initializations, and where two anatomist results are also compared.

Table 3 displays the intrarater and interrater user variability during semisupervised segmentation of an L5 and S1 herniated disc. As demonstrated in Fig. 8, intrarater variability is present at the disc pathology where constraint points were required to correctly segment the herniated part of the anatomy. More variability exists between different anatomists, with a larger mean segmentation error of 0.25 mm, present at the lateral margins of the disc as well as the disc pathology, where manual interaction was required.

Table 3.

Validation metrics comparing two sets of semisupervised segmentations of a herniated intervertebral disc performed by the same anatomist and two different anatomists, demonstrating intrarater and interrater variability, respectively.

Validation metric	Intrarater	Interrater
MASD (mm)	0.050	0.254
Absolute std. dev. (mm)	0.062	0.323
Maximum out error (mm)	$-1.214$	$-2.593$
Maximum in error (mm)	0.905	2.290
Hausdorff distance (mm)	1.214	2.593

Fig. 8.

Maximum out (red) and maximum in (blue) segmentation error between two sets of herniated disc segmentations performed by (a) the same rater (0.91 mm max. in and 1.21 mm max. out) and (b) different raters (2.29 mm max. in and 2.59 mm max out). Over- and undersegmentation is present at the lateral margins and the pathology where constraint points were required to correct segmentation.

3.2. Vertebra Segmentation

Table 4 summarizes the segmentation results of L1 to L5 vertebral discs of the lumbar spine using inherent weak shape priors during segmentation pertaining to five patients. The absolute mean standard error is 0.42 mm, with an average Hausdorff distance, representing average worst error, is 2.86 mm. The average maximum and minimum segmentation errors observed for the remaining 24 vertebral segmentations were $-3.22$ to 2.34 mm, respectively, with an average DICE coefficient of 0.93. This maximum segmentation error was mostly located at the superior or interior processes, or the spinous processes where contiguous vertebral structure boundaries were present in the image volume. The presence of low image-to-noise ratio and low image contrast of the bones in T-2 weighted MR images likely caused image boundary ambiguity, resulting in over- or undersegmentation of these bone substructures.

Table 4.

Average validation metrics comparing 25 lumbar vertebrae automatic segmentation results, using weak shape priors, against corresponding manually corrected segmentation of patients in MR images.

Validation metric	Lumbar vertebrae
MASD (mm)	0.417
Absolute std. dev. (mm)	0.313
MSD (mm)	0.375
Avg. Hausdorff distance (mm)	2.863
DICE coefficient	0.932

Figure 9 depicts the worst vertebrae segmentation error encountered during an L3 segmentation, demonstrating the need for strong-shape priors to guide the segmentation result. It can be observed that the maximum in error, corresponding to undersegmentation, is located at the spinous processes. This is likely due to the ambiguity in vertebral boundary at that image subvolume, where the spinous process of an L3 vertebra is close to the spinous process of the L2 vertebra located above, with ligaments in between. The model was also unable to capture the lateral margins of the vertebral body, resulting in undersegmentation. The superior articulate process and the lamina were oversegmented, resulting in oversegmentation of maximum out error of 4.37 mm.

Fig. 9.

Comparison of an L3 vertebral segmentation using weak shape prior-aware simplex versus the manually identified ground truth, with $-3.56\mathrm{mm}$ max. in and 4.37 mm max out. This is the worst encountered vertebra segmentation error.

Segmentation of vertebral structures is a challenging task in MR images due to the low image contrast associated with bone in the image modality. Moreover, the thin ligaments surrounding the complex shape of the vertebral body, especially between the processes of the vertebrae, provide low image intensity and gradient change, resulting in image boundary ambiguity. Therefore, strong prior knowledge of the average vertebral shape, with allowed variations, has been incorporated within simplex models to guide and improve the segmentation of these complex structures. Vertebrae results using PCA-based segmentation are described in Sec. 4.3.

4. Segmentation Using an SSM-Aware Simplex Model

SSMs have been incorporated into simplex deformable models for segmentation refinement. This section discusses the evaluation results of the three vertebral SSMs and intervertebral disc SSM. Then the segmentation results of the strong shape-based simplex deformable models are presented. A proof of concept of a healthy disc compression is presented to demonstrate application of the multiresolution segmentation results within an FEM simulation. Validation studies employ an approach to ground truth inspired by Tejos,²⁷ who adopts a vertex-by-vertex manual correction of surface model results to achieve a perfect segmentation, as opposed to a slice-by-slice voxel-based identification of a tissue blob. In our case, we use a number of constraint points, which imposes a strong attraction to the surface model, in order to correct erroneous boundary surface sections.

4.1. Statistical Shape Model Evaluation

This section evaluates the three vertebrae SSMs and one intervertebral disc SSM incorporated within the segmentation framework by calculating model compactness, generalization ability, and specificity validation metrics as described in Sec. 2.4. Figure 10(a) illustrates the changes in the shapes along the first three principal modes of variation by $3\sigma$ for the constructed L1 vertebra SSM. The first mode of the shape model mainly captures scaling across the population. The maximum vertebral variability (16 mm) is observed at the inferior and superior articular processes and the spinous process. The second and third modes in the vertebral model capture variation and scaling in the transverse processes and foramen size, respectively.

Fig. 10.

Graphical representation of shape model variability (in mm) captured by the first three principal modes. (a) L1 vertebra SSM based on 10 shapes, seen from superior view. (b) Combined L2 and L3 vertebra SSM computed from 20 shapes, seen from inferior view. (c) Combined L4 and L5 vertebra SSM computed from 20 shapes, seen from superior view. (d) Combined intervertebral disc SSM computed from 40 shapes, shown from superior view. Red corresponds to the maximum outward signed distance (mm) from the mean shape, and blue corresponds to the maximum inward signed distance (mm) from the mean shape. The middle column corresponds to the average shape, and the left and right columns represent deviations of $-3\sigma$ and $+3\sigma$, respectively, with respect to the average shape.

In contrast, the first mode of the intervertebral disc model varies maximally by 7 mm. The second principal mode captured stretching in the lateral parts of the disc, and the third mode captured rotational effects in the lateral part of the disc, respectively.

Figure 10(b) illustrates the changes in the shapes along the first three principal modes of variation by $3\sigma$ for the combined L2 and L3 vertebra SSM. The first mode of the model mainly captures scaling of the vertebral body across the L2 and L3 vertebrae training shapes. The maximum vertebral variability (7 mm) is observed at the shape and size of the vertebral body, as well as the articular processes. Similar to the L1 vertebra SSM, the second and third modes in the vertebral model capture variation and scaling in the transverse processes and foramen size, respectively. SSMs combining two neighboring vertebrae were constructed to to exploit the similarity in shape between consecutive vertebrae, as well as to increase the training size.

Variation across the first three modes of variation in the combined L4 and L5 vertebral SSM is depicted in Fig. 10(c). The mean shape represents an average combined shape of the L4 and L5 vertebrae, with variation in the overall thickness of the vertebre captured by the first principal mode. The second mode captures variation in size, whereas the third mode captures the thickness of the vertebral processes.

The fourth constructed SSM capturing expected mean and variation in all five intervertebral discs of the spine from 40 shapes is depicted in Fig. 10(d). As expected, the first mode captures variation in disc size, specifically in the posterior shape. The second mode captures size variation and shape change along the anterior portion of the disc. The third mode captures changes in curvature of the disc.

Figure 11 graphically illustrates the compactness of the four statistical models as a function of the number of modes required to capture 100% of the variation across the population. Each principal mode represents a distinct shape variation among the shape population. The L1 vertebra shape model was able to capture variance within the first seven principal modes, with 39.45% variance of the captured by the first principal mode. The combined shape models showed some improvement in compactness, due to the increase in training dataset size. Both combined vertebrae SSMs were able to capture 95% variability within the first 12 modes. In contrast, the disc SSM performed much better due to a larger training dataset and captured 95% variability within the first 11 modes, with 42% variation captured by the first mode.

Fig. 11.

Compactness versus number of modes used for representation, featuring L1 vertebra SSM (red) combined L2 and L3 vertebral SSM (green), combined L4 and L5 vertebral SSM (violet), and intervertebral disc SSM (blue).

Figure 12 illustrates the Hausdorff-based generalization ability of the L1 vertebra, combined L2 and L3 vertebrae, combined L4 and L5 vertebrae, and the intervertebral disc shape models, respectively. The generalization ability is the ability of the model to represent unseen shapes, whereas specificity describes the robustness of the shape model in representing seen instances, such as from within the dataset. Results presented were calculated by performing leave-one-out analysis using their respective training datasets. Figure 13 demonstrates generalizability characteristics as assessed on the basis of RMS distance, whereas Fig. 14 depicts specificity based on RMS distance.

Fig. 12.

Generalization ability based on Hausdorff distance (mm) versus number of modes used for representation, featuring L1 vertebra SSM (red), combined L2 and L3 vertebral SSM (green), combined L4 and L5 vertebral SSM (violet), and intervertebral disc SSM (blue).

Fig. 13.

Generalization ability based on RMS error (mm) versus number of modes used for representation, featuring L1 vertebra SSM (red), combined L2 and L3 vertebral SSM (green), combined L4 and L5 vertebral SSM (violet), and intervertebral disc SSM (blue).

Fig. 14.

Specificity versus number of modes used for representation, featuring L1 vertebra SSM (red), combined L2 and L3 vertebral SSM (green), combined L4 and L5 vertebral SSM (violet), and intervertebral disc SSM (blue).

The results of the generalization ability of the constructed models can be further analyzed. For the first mode of variation, the average reconstruction error for an unseen instance is for the L1 vertebra model is 0.47 mm with a confidence interval of 0.03 mm, with an initial Hausdorff distance of 8.2 mm. This error converges to 0.4 mm with worst mean error of 7.6 mm. Our L1 vertebra models cumulative specificity error is 1.43 mm in seven principal modes with negligible standard error. The Hausdorff error for generalization ability of the L2 and L3 model using only the first mode of variation is 0.9 mm, which is reduced to 0.58 mm after 17 total modes. Similarly, the average Hausdorff distance for representing unseen shapes for the L4 and L5 vertebrae model is initially 8.2 mm, reducing to 6.7 mm over 17 modes of variation. It can also be noted that although the Generalization ability RMS error for the combined shape models is lower than that of the L1 vertebral model, the combined models are less compact due to higher variability within the training dataset, introduced not only due to larger dataset size but also because it represents variations between the the two consecutive vertebrae as well.

Our vertebra model results are similar to those in the literature. Vrtovec et al.’s⁴⁴ model is more compact, capturing 52% variability within the first principal mode. Rasoulian et al.¹⁰ captured $G(m)$ RMS error of 0.95 mm, with Hausdorff error 9 mm within the first principal mode, which is decreased to 0.8 mm RMS and 7.5 mm after seven modes. Their model is worse in generalization and specificity, but outperforms in model compactness (capturing 60% in the first mode). Kaus et al.⁴⁵ reported 1.66 mm mean error after 20 modes, with 30% first mode compactness, constructed with 32 (L1 to L4) vertebral training shapes.

Our intervertebral disc model is able to represent unseen instances with an initial RMS error of 1.4 mm, and Hausdorff distance of 4.08 mm, which converges to 0.18 mm RMS error and 1.5 mm worst error after 35 principal modes.

Overall, the compact model transitions coherently, with a tradeoff between compactness and the ability to faithfully represent new training shapes. Some outliers in the first principal mode can be noted in the variant vertebral shapes. These outliers may be reduced by increasing the size of the population dataset, as well as exploring probabilistic PCA instead of simple PCA, which may better account for any outliers in the model. Moreover, large variability exists between the vertebrae instances, leading to large variability in the shape models itself. An increase in the training dataset would lead to more robust and faithful vertebral shape models better able to represent variability within a population.

4.2. Intervertebral Disc Segmentation Using SSM

Table 5 shows the validation metrics comparing automatic strong shape-based segmentation results with manually corrected ground truth of 16 healthy intervertebral discs of the lumar spine. The average DICE coefficient achieved is 0.979, which is an improvement over 0.95 achieved without SSM incorporation. The absolute mean distance has reduced to $0.79\mathrm{mm}\pm 0.19\mathrm{mm}$, with average Hausdorff distance reduced to $<1\mathrm{mm}$.

Table 5.

Average validation metrics comparing automatic 16 lumbar disc segmentation results, using strong shape-based priors, against corresponding manually corrected segmentation of patients in MR images.

Validation metric	L1 and L2	L2 and L3	L3 and L4	L4 and L5	L5 and S1	Total
MASD (mm)	1.44	0.44	0.93	0.51	0.64	0.79
Absolute std. dev. (mm)	0.229	0.210	0.159	0.087	0.25	0.187
MSD (mm)	0.53	0.149	0.25	0.76	0.67	0.74
Avg. Hausdorff distance (mm)	1.63	1.12	1.09	0.984	0.982	0.979
DICE coefficient	0.77	0.981	0.971	0.984	0.982	0.979

Figure 15 displays the segmentation result of an L3 to L4 intervertebral disc within the image volume in sagittal, coronal, and axial view, respectively. The resulting segmentation is evaluated against a manually corrected segmentation considered as ground truth, which involves the application of a number of constraint points to correct any deviation of underlying boundary. These segmentation results are depicted in Fig. 16(a), which displays the signed maximum and minimum distance error. The error is located at the lateral margins of disc, with maximum over-segmentation error as 1.08 mm, and maximum undersegmentation (maximum in error) as $-0.73\mathrm{mm}$. PCA-based shape forces are relaxed very close to the image boundary using a high-resolution mesh for segmentation, and image gradient descent is allowed to guide the deformation along the normal direction so that local shape variation and details of the structure can be accurately captured. Image boundary ambiguity at the lateral margins of the disc may result in over or under estimation of the structure boundary. Figure 16(b) displays the segmentation result of an L4 to L5 intervertebral disc, with maximum oversegmentation as 0.88 mm and maximum undersegmentation of $-0.41\mathrm{mm}$. The maximum out error can be observed at the anterior margins of the disc. The statistical shape-based segmentation results show significant improvement of results, with reduced error observed at the lateral margins of the disc as compared to results of weak-shape prior segmentation. Our results are comparable with the state of the art, with mean absolute error as 0.79 mm.

Fig. 15.

Segmentation of an L3 and L4 intervertebral disc.

Fig. 16.

Segmentation evaluation of IVDs against their corresponding manually corrected segmentations, considered ground truth: (a) L3 and L4 intervertebral disc: the automatically segmented L3 and L4 IVD has a signed distance error of 1.08 mm oversegmentation and 0.73 mm undersegmentation. (b) L4 and L5 intervertebral disc: the L4 and L5 IVD exhibits a signed distance error of 0.88 mm oversegmentation and 0.41 mm undersegmentation.

4.3. Vertebral Segmentation Using an SSM-Aware Simplex Model

Strong shape-based segmentation results of lumbar vertebrae of five MR images are presented in Table 6. Automatic segmentation results of 25 L1 to L5 vertebrae have been validated against manually corrected segmentation, based on imposed constraint points, considered as ground truth. Our proposed method performs very well, with overall average Dice coefficient of 0.981, with average absolute mean error as $0.69\mathrm{mm}\pm 0.15\mathrm{mm}$. The average Hausdorff distance was observed to be 1.18 mm. Although the MASD has increased from 0.42 mm with SSM incorporation, the overall results demonstrate consistent improvement over results obtained from segmentation using weak-shape priors. The average Hausdorff distance, a measure of the worst error, has reduced from 2.86 mm to 1.18 mm, with an expected decrease in MSD error from 0.38 to 0.30 mm.

Table 6.

Average validation metrics comparing automatic 25 lumbar vertebrae segmentation results, using strong shape-based priors, with corresponding manually corrected segmentation of patients in MR images.

Validation metric	L1	L2	L3	L4	L5	Total
MASD (mm)	0.176	1.182	0.72	0.491	0.22	0.685
Absolute std. dev. (mm)	0.148	0.054	0.133	0.147	0.254	0.147
MSD (mm)	0.288	0.486	0.20	0.316	0.19	0.297
Avg. Hausdorff distance (mm)	1.068	1.162	1.629	1.189	0.863	1.182
DICE coefficient	0.984	0.989	0.978	0.976	0.981	0.982

Figures 17(a) and 17(b) display the segmentation results of L2 and L3 vertebrae, respectively. In the L2, our model has slightly oversegmented the structure, with maximum out error (0.48 mm) and maximum in error ($-1.01\mathrm{mm}$) observed at the superior and interior articular processes. In the L3, the maximum over-segmentation error (1.62 mm) can be observed at the superior articular process, with some oversegmentation of the spinous process; under-segmentation peaks at $-0.84\mathrm{mm}$. Similar to the weak prior shape-based segmentation results, most of the error lies at the articular and spinous processes of the vertebrae, where there may be image intensity ambiguity due to low image contrast between contiguous anatomical structures or ligaments surrounding the vertebral body.

Fig. 17.

Segmentation validation of L2 and L3 vertebrae with ground truth segmentation. (a) L2: maximum oversegmentation error is 1.62 mm, and maximum undersegmentation error is $-0.84\mathrm{mm}$. (b) L3: maximum oversegmentation error is 0.47 mm, and maximum undersegmentation error is $-1.01\mathrm{mm}$.

Contiguous vertebrae and disc structures were segmented with collision detection to mitigate any resulting mesh overlap. Figure 18 displays the segmentation of the lumbar vertebral and intervertebral structures of one of the testing MR datasets. Figure 19 displays segmentation in axial view of the vertebrae and intervertebral discs of the lumbar spine. It displays the segmentation of contiguous structures across a slice in the image volume, with a segmented L4 and L5 intervertebral disc, the superior articular process of L5 vertebra (red), and interior articular processes of the L4 vertebra located above (in blue).

Fig. 18.

Segmentation results of vertebrae and intervertebral discs of the lumbar spine.

Fig. 19.

Segmentation in axial view of the vertebrae and intervertebral discs. Contiguous structures segmented across a slice in the image volume, with a segmented L4 and L5 intervertebral disc, the superior articular process of L5 vertebra (red), and interior articular processes of the L4 vertebra located above (blue).

The segmented intervertebral disc surfaces were treated as hard bodies during vertebrae segmentation, such that the surface points of the intervertebral discs were considered as a repelling external force for the deforming vertebral simplex mesh. Collision handling forces were activated during the high-resolution segmentation scheme when the deforming vertebral mesh was close enough to the vertebral image boundary and the contiguous intervertebral disc boundaries. Figure 20 displays the signed surface distance between an L2 and L3 vertebra and an L2 vertebra without collision handling forces activated during segmentation and with collision detection. Meshes are considered to be contiguous and nonoverlapping at distance 0.0 mm. There was surface interpenetration, indicating oversegmentation, of 0.47 mm, which was reduced to $2.0\times {10}^{-5}$, resulting in nonoverlapping surface meshes.

Fig. 20.

Signed surface distance (a) between an L2 and L3 disc and an L2 vertebra before collision handling; there is maximum interpenetration of 0.47 mm. (b) From L2 vertebra to L2 and L3 disc after collision handling. (c) From L2 vertebra to L2 and L3 disc after collision handling.

Most vertebrae results without collision handling resulting in oversegmentation between 0.1 to 0.5 mm, with few under-segmented results. However, in case the surface overlap was over over 0.5 mm, collision detection at high resolution was less efficient. Figure 21 shows signed surface distance between an L4 and L5 intervertebral disc and an L4 vertebra, with the vertebral mesh segmented with and without collision handling. The vertebra is oversegmented by 0.8 mm, which is reduced to 0.36 mm after segmentation with collision detection. The location of the remaining surface overlap is indicated on the L4 and L5 intervertebral disc surface in Fig. 21(b) by identifying surface area where the signed distance map is below 0.0 mm.

Fig. 21.

(a) Signed surface distance between an L4 and L5 vertebra and an L4 vertebra before collision handling; there is maximum interpenetration of 0.8 mm. (b) Signed surface distance from L4 vertebra to L4 and L5 disc after collision handling, reduced to 0.36. Area with surface distance below 0, identifying remaining mesh overlap, is highlighted.

Incorporation of strong shape-based priors in simplex deformable models provided much accurate results, reducing the average Hausdorff distance error, which is a measure of the maximum error, to $<1.5\mathrm{mm}$ for both vertebrae as well as intervertebral disc segmentation. A significant improvement in the performance accuracy was achieved by utilizing SSMs for vertebrae segmentation, where the DICE coefficient increased from 0.93 to 0.98. This is an improvement to current segmentation techniques, where the lowest obtained DICE coefficient value of 0.935 is presented by Ibragimov.⁴⁶ However, segmentation results of the proposed method cannot be directly compared with current literature as they were validated on different training and testing datasets.

5. Intervertebral Disc Compression Simulation

Simulation Open Framework Architecture (SOFA)⁴⁷ is an open-source object-oriented software toolkit that is targeted toward real-time interactive medical simulations. Several components of a model can be combined in hierarchies through an easy-to-use scene file format to represent various model parameters such as material properties, deformable behavior, constraints, and boundary conditions, which makes SOFA a very powerful and efficient prototyping tool.

A healthy lumbar intervertebral disc has been modeled using SOFA to simulate the biomechanical and physiological changes of the disc under compression. The tetrahedral mesh of the healthy L2 and L3 disc has been generated from the segmented surface mesh using the isosurface stuffing method.⁴⁸ This volumetric mesh has been used to define the tetrahedral corotational FEM of the disc, depicted in Fig. 22(a), which corresponds to the behavior model^47,49 of the deformable object. The boundary conditions and external compression forces have been defined through the segmented surface mesh, which is linked to the underlying behavior model of the deformable object [Fig. 22(b)]. Following the actual anatomy of the simulated intervertebral disc, the bottom nodes that are in direct contact with the below rigid vertebral body have been constrained to be fixed to their initial locations, and a prescribed vertical pressure of $100\mathrm{N}/{\mathrm{cm}}^2$ has been applied to the top surface of the disc using SOFA’s TrianglePressureForceField component.

Fig. 22.

3-D simulation of a healthy intervertebral disc under pressure. (a) Tetrahedral FEM. (b) Behavioral model: the bottom nodes (red) are constrained to be fixed and the Neumann boundary condition is applied to the top surface (green) of the disc model. (c) Visual model: comparison of the disc model at rest (red) and deformed (green) configurations.

The tetrahedral models were built using a plugin based on Computational Geometry Algorithms Library (CGAL) found on SOFA,^50,51 where the resolution of the mesh could be specified with a FacetSize parameter, which relates to an edge length objective of each element and would typically equate with the resolution of the surface mesh. The CGAL tetrahedralization implementation is based on Alliez’ variational tetrahedralization algorithm,²⁵ which provides support for precise resolution control, in addition to alleviating the risk of sliver-like elements. The model depicted in Fig. 22(a) had a FacetSize $d$ of 5 mm, which is based on the formula for volume $\frac{d^3}{6\sqrt{2}}$, equates to a volume of $14.7{\mathrm{mm}}^3$. A typical disc volume is from 7300 to $10,000{\mathrm{mm}}^3$ in the lumbar region,⁵² with the L2 and L3 disc averaging roughly $8700{\mathrm{mm}}^3$, which works out to just under 600 tetrahedra. A typical FEM system such as this one can be solved on SOFA from 10 to 60 Hz on a CPU, depending on the choice of continuum model. For a larger simulation featuring all IVDs and with vertebrae modeled as rigid, we would advocate using a GPU-based continuum model, which is also supported by SOFA.

In the simulation phase, we have assumed a uniform isotropic material model for representing the intervertebral disc. Our interbertebral disc biomechanical properties are consistent with values published by Malandrino et al.⁵³ and Spilker.⁵⁴ Using these studies, we have chosen Poisson’s ratio to be 0.4 and Young’s modulus to be 15,800 Pa, representing the ratio of disc model expansion versus compression and the stiffness of the elastic model, respectively. The effect of the compression force on the disc has been captured in terms of the relative displacement of the surfaces of the original and deformed configurations [Fig. 22(c)], where the uncompressed disc is depicted in red and the compressed configuration in green. The simulation results in a slightly bulging disc.

This implementation is intended as a proof of concept to demonstrate use of segmentation results to initiate a patient-specific simulation in SOFA, such that an interactive response is feasible. Meanwhile, competing spine modeling methods emphasize dense tetrahedral decomposition and onerous finite-element computations that preclude an interactive response. In particular, our controlled-resolution modeling technique can produce a coarse triangular surface for constraining a coarse tetrahedralization for a behavior model,^47,49 a medium-resolution surface mesh for a collision model, and a fine-resolution surface mesh for a visual model, all running on SOFA and mapped to each other.

6. Discussion

Surgery and biomechanical simulations require patient-specific, high-fidelity, and robust 3-D segmentation of vertebral and intervetebral disc structures, and existing pathology, of the lumbar spine. This paper describes a framework for segmentation of lumbar vertebrae and discs from T2-weighted MR images of the spine. Our segmentation approach is based on simplex discrete deformable models.

This research initially exploits weak shape priors inherent in simplex deformable models for segmentation. An ellipsoid template mesh and a medial-axis-based template is initialized within the disc and vertebra volume image, respectively, using landmark-based affine registration. This template is allowed to deform according to simplex internal and external forces. In case the simplex mesh fails to capture image boundary in existence of disc pathology, weak shape priors are degraded gracefully and the user is allowed to guide mesh deformation by placing constraint points on the image volume. This manually corrected segmentation method has also utilized for generating ground truth used for validation of our test results. Segmentation results using weak shape priors pertaining to five patients yield DICE coefficients of 0.93 for vertebrae and 0.95 for intervertebral discs. Our method demonstrates the ability to successfully segment disc pathology, based on minimally supervised, spatially variable weighting of shape prior information. Vertebral segmentation in MR images posed a challenge due to low image contrast for bone in MR images, as well as the presence of image artifacts, thus requiring incorporation of strong shape priors in simplex models.

SSMs of vertebrae and discs were generated using training data of 10 and 8 patient datasets, respectively. Three SSMs of vertebrae: an L1 vertebral SSM, a coupled L2 and L3 vertebral SSM, and a coupled L4 and L5 vertebral SSM were constructed. An intervertebral disc SSM was generated using 40 training shapes. These vertebrae and disc SSMs were shown to faithfully capture variance within a population with few particle outliers, capturing 95% of variability within the first nine modes of variation.

Strong shape priors incorporated in our deformable model have been utilized for resegmentation of MR image testing dataset. PCA-based average shapes were initialized within the structure volume boundary through landmark-based affine registration using a multiresolution scheme. The shape model was set as a template mesh that was allowed to deform and capture the image boundary while constraining the mesh according to expected variation. The PCA shape influence was relaxed with increase in mesh resolution for result refinement. The proposed strong shape-based deformation method results in robust segmentation with DICE coefficient of 0.979 for intervertebral discs and 0.981 for vertebrae. We also exploit controlled-resolution meshing conducive to a multiresolution approach to segmentation as well as producing anatomical models with low element count for interactive simulation.

Evaluation of the proposed framework can be improved by increasing the size of the training dataset utilized for generation of vertebral and intervertebral SSMs. Images of diseased or degenerated vertebrae, such as compressed vertebrae or vertebral fractures that may occur due to osteoporosis, may be included during SSM construction to increase captured variation within the population. Incorporation of intensity-based features, such as statistical appearance models along with SSMs to classify intensity variation between a healthy and herniated disc image may assist with identification of disc pathology.

It should also be noted that the training data requirements for SSMs are not as onerous as the number of training samples needed for most voxel-based classifiers, especially in relation to deep learning methods that rely typically on thousands of data points. It can be argued that this lesser reliance on exhaustive training is partly as a result of the consideration of several forces at once, such as continuity-based internal forces and image gradient-based external forces, as well as a lesser-dimensionality problem in the form of a model-to-image registration, in comparison with the dimensionality of a low-level voxel-based image classification problem in general, especially in a deep learning framework. In addition, the consideration of a coarse-to-fine approach, especially if the process begins with a set of homologous point pairs, makes the optimization of an objective function more feasible using a small training set, in comparison with any single-resolution approach, and to our knowledge, this type of coarse-to-fine framework is not yet a hallmark of machine learning techniques. The analysis of compactness, generalization, and specificity suggests also that a relatively small subset of modes is needed to provide useful shape discrimination.

7. Conclusions

This paper presented a deformable multisurface model that embeds shape statistics force, with applications to lumbar spine segmentation for surgery planning and simulation. Our approach performs localization of the vertebrae and intervertebral discs within the volume image through landmark-based affine registration. This localization is dependent on manual input from the user through placement of landmarks on the initialized template, as well as the volume image. This user interaction can be eliminated by introducing an automated disc and vertebrae localization scheme that identifies the position of the vertebrae and intervertebral discs, which can be further used to initialize segmentation. Moreover, the proposed framework is limited by manual interaction for segmentation refinement in case disc pathology cannot be faithfully captured.

An alternate approach to generation of SSMs could be a hierarchical shape model approach, where one SSM of all lumbar vertebrae, as presented by Rasoulian et al.,⁵⁵ can be utilized to capture vertebral global pose and shape of the entire during low-resolution segmentation, and individual SSMs corresponding to each vertebrae can be used at higher resolutions to capture local shape variation. However, a large training dataset is required for such implementation that was not available at the time of the proposed framework.

Although there are similarities between our work and related research, our work features innovations essential to the development of an interactive spine surgery simulator, as well as a biomechanical FEM model. First, the proposed anatomical modeling enables a trade-off between shape priors and limited user supervision near the pathology of interest to the simulation. Second, our approach specifically emphasizes resolution control with the final simplex surface mesh, which leads to a controlled-resolution triangulated mesh by duality; moreover, the latter controlled-resolution triangulated mesh in turn leads to a like-resolution tetrahedral mesh bounded by it. Both aspects of the meshing are essential to the low element count needed for an interactive virtual tissue response.

8. Appendix A: Medial-Axis-Based Template Construction and Initialization

Medial-axis extraction is the process of reducing dimensionality of the structure without loss of topology. Traditional 3-D skeletonization methods, such as binary thinning⁵⁶ or level-set-based centerline extraction,⁵⁷ do not guarantee a connected component. This research modifies and augments the skeletonization approach of Hassouna and Farag⁵⁸ for generating a topologically consistent and connected vertebral 3-D medial axis from a binary image, which is subsequently converted into a triangulated and a simplex template mesh.

Hassouna et al. proposed the minimum cost path between two medial points as

$$F(x)=e^{\alpha \lambda (x)},\text{where}\alpha \ge 0,$$ (11)

$$\lambda (x)={\lambda }_1(x)+{\lambda }_2(x)+{\lambda }_3(x),$$ (12)

$$\lambda (x)=D(x)+\omega (\frac{1.0}{1.0+\Vert \nabla \Vert }+|\nabla .\nabla D(x)|),$$ (13)

where $\lambda (x)$ is a medial descriptor function controlling the propagation front of the fast marching method, $\omega$ is a weight $<1$, and $\nabla$ is the gradient operator. ${\lambda }_1(x)$ is the distance map of the 3-D image, describing the minimum distance from the structure boundary and provides a smooth transition during fast marching. ${\lambda }_2(x)$ is the medial descriptor function describing the signed, inverted gradient of the distance map. The gradient of the distance map is zero at local maximum, which is the maximum distance from the image boundary describing the medial points of the image. ${\lambda }_2(x)$ function is successfully able to identify strong medial points with small gradient values. ${\lambda }_3(x)$ is the outward flux medial descriptor function defining the gradient of the distance field. The centerline points of the image are the local image maximum, located inward toward the center of the image. This image traversal can be described as a fluid mechanics problem, with medial axis points having strong negative divergence, and boundary points having strong positive divergence. This research employs Siddiqi et al.’s⁵⁹ modification of divergence theorem as an outward flux from the image boundary to a medial point in the image volume center, making $\nabla$ differentiable.

Using a vertebral image [Fig. 23(a)] as input, the distance map ${\lambda }_1(x)$ [Fig. 23(b)], the gradient of the signed $D(x){\lambda }_2(x)$ [Fig. 23(c)], and outward flux ${\lambda }_3(x)$ [Fig. 23(d)] are calculated, respectively. The image is normalized from 0.0 to 1.0 after every medial descriptor function.

Fig. 23.

Vertebral template construction. (a) Input vertebra image volume: volume-rendering of binarized blob, (b) Euclidean distance map ${\lambda }_1(x)$, (c) normalized gradient of signed distance map ${\lambda }_2(x)$, (d) outward flux ${\lambda }_3(x)$, (e) resulting $\lambda (x)$, and (f) triangulated medial axis-based template mesh.

Using $\omega =0.2$ in Eq. (13), the resulting, combined medial descriptor $\lambda (x)$ of the image is depicted in Fig. 23(e). This signed distance map is then thresholded ($\text{threshold}=60$) to extract the medial axis subvolume represented by the maximum intensity values in the image. This subvolume is smoothed, and a triangulated surface is generated using the marching cubes method, as depicted in Fig. 23(f). This method generates a genus 1, toroidal template with smoothed edges and processes to ensure that template can be initialized completely within an arbitrary vertebra image subvolume. A more detailed template representing vertebral transverse processes could not be affinely registered completely within an arbitrary vertebral image volume.

The constructed template mesh is initialized within the vertebra MR image volume by placing nine homologous landmark points on the mesh surface as well as the MR image region of interest. These landmarks are placed on the high curvature points of the vertebra, which are the right and left transverse processes, the spinous process, the superior and anterior articular processes, and on the vertebral body itself. The template mesh is placed within the image volume through affine registration and allowed to deform according to simplex internal and external forces using a multiresolution scheme. Results of the vertebral segmentation using weak-shape priors are discussed in Sec. 3.2.

Although it can be argued that the results are dependent on the accuracy of the identification of these nine landmark points, in practice this potential vulnerability can be addressed a number of ways. One approach consists of exploiting surface curvature extrema and determining automatically on both the surface and the target image the nine local curvature extrema coinciding with sharply peaked features such as spinous and transverse processes, exploiting a putative affine transformation of the surface model to the target image. In the target image, this would involve determining corner points of anticipated image polarity (peaks comprised of bone pixels of high Hounsfield value pixels in CT surrounded by darker pixels),⁶⁰ based on a target image processed with a coarse Gaussian filter, or coarse Gaussian-like efficient recursive filter such as pioneered by Deriche.⁶¹ Alternately, the surface model can undergo several iterations of global affine transform with image forces of the simplex acting in concert, prior to enabling the simplex model to deform elastically. The two solutions, automatic curvature extremum detection in the target image and global affine transform iteration, can work hand-in-hand to stabilize the transformation. Nonetheless, although we have not pursued a formal analysis of the surface model to landmark point misalignment, as long as our template is designed to be fully contained within the anatomical boundary, which is true by design, and we proceed in a coarse-to-fine manner as described, starting from one initial global affine transformation, we obtain stable results in practice.

9. Appendix B: Statistical Shape Model Construction

9.1. Alignment

A shape is invariant under similarity transformations of rotation, translation and scaling in 2-D space. Alignment is the process of calculating the optimal $m\times m$ rotation matrix $\mathrm{\Gamma }$, $m\times 1$ translation vector $T$ and scale parameter $\beta$ to align all training shapes within a common coordinate space. Given a shape $X$ in $m$ dimensions (e.g., $m=2$) with $n$ points, vectorization of $X$ would be as follows:

$$X={[x_1,x_2,\dots ,x_n,y_1,y_2,\dots ,y_n]}^{\mathrm{T}}.$$ (14)

The most popular data alignment method is the Procrustes analysis that minimizes the Euclidean mean squared distance between shapes, known as the Procrustes shape distance. The full Procrustes ordinary sum of squares (OSS) distance between two shapes is calculated as

$$\mathrm{OSS}(X_1,X_2)={\Vert X_2-\beta X_1\mathrm{\Gamma }-1_k{T}^{\mathrm{T}}\Vert }^2.$$ (15)

Procrustes alignment is the minimization of $\mathrm{OSS}(X_1,X_2)$ by removing scaling, translation, and rotation effects in the training dataset. To remove the scale $\beta$ between shapes, the centroid size for each shape is calculated as

$$S(X)=\sum _{i=1}^n\sqrt{{(x_i-\overline{x})}^2+{(y_i-\overline{y})}^2},$$ (16)

where

$$(\overline{x},\overline{y})=(\frac{1}{n}\sum _{i=1}^n{x}_i,\frac{1}{n}\sum _{i=1}^n{y}_i).$$ (17)

Translation between two shapes can be removed by translating the centroid of one shape onto another. The $m\times m$ rotation matrix $\mathrm{\Gamma }$ can be represented as

$$\mathrm{\Gamma }=U{V}^{\mathrm{T}}.$$ (18)

$U$ and $V$ orthogonal matrices can be obtained by singular value decomposition (SVD) (SVD of $V\mathrm{\Lambda }{U}^{\mathrm{T}}$, where columns of $V$ are eigenvectors of $A{A}^{\mathrm{T}}$, columns of $U$ are eigenvectors of $A^{\mathrm{T}}A$, and $\mathrm{\Lambda }$ is the diagonal matrix of positive elements corresponding to the eigenvalues of covariance between shapes $X_1$ and $X_2$. $A$ is an $m\times n$ matrix of $\mathbb{R}$ or complex numbers. SVD identifies and orders the dimensions along which the shape points have maximum variability.) of the matrix

$$X_2^{\mathrm{T}}{X}_1=\Vert X_1\Vert \Vert X_2\Vert V\mathrm{\Lambda }{U}^{\mathrm{T}},$$ (19)

where $U$ and $V$ are the rotation matrices that superimpose $X_2$ onto $X_1$ and $\mathrm{\Lambda }$ is a diagonal matrix of positive values capturing the correlation between the two shapes. Rotation matrix $\mathrm{\Gamma }$ can then be decomposed as

$$\mathrm{\Gamma }=U{V}^{\mathrm{T}}=\left[\begin{array}{cc}\cos (\theta )& -\sin (\theta )\\ \sin (\theta )& \cos (\theta )\end{array}\right].$$ (20)

The optimal scaling factor $\beta$ can be calculated as

$$r\beta =\frac{\text{trace}(X_1^{\mathrm{T}}{X}_2\mathrm{\Gamma })}{\text{trace}(X_1^{\mathrm{T}}{X}_1)}.$$ (21)

Generalized procrustes analysis (GPA), developed by Gower⁴¹ iteratively minimizes the generalized sum of squared norms of pairwise differences for two or more training shapes represented by

$$G(X_i,X_2,\dots ,X_n)=\frac{1}{n}\sum _{i=1}^n\sum _{j=i+1}^n{\Vert X_i-X_j\Vert }^2,$$ (22)

where the average centroid size of all shapes is scaled to 1. Each shape $X_i$ has the Procrustes coordinates

$$X_i^P={\widehat{\beta }}_i{X}_i{\widehat{\mathrm{\Gamma }}}_i+1_k{\widehat{T}}_i^{\mathrm{T}},i=1,\dots ,n$$ (23)

represented by the minimizing parameters ${\widehat{\mathrm{\Gamma }}}_i$ as the rotation matrix, ${\widehat{\beta }}_i$ as the scaling factor, and ${\widehat{T}}_i^{\mathrm{T}}$ as the translation vector for shape $i$ calculated using the method described above. Thus the generalized mean shape can be calculated as

$$\overline{X}=\frac{1}{n}\sum _{i=1}^n{X}_i^P.$$ (24)

The GPA algorithm is as follows.

• 1.Set an arbitrary shape within a population of shapes as the mean shape $\overline{X}$.
• 2.Calculate the Procrustes coordinates $X_i^P$, where $i=1\cdots n-1$ for remaining shapes with respect to $\overline{X}$.
• 3.Set $\overline{X}$ as the Procrustes mean shape according to Eq. (24).
• 4.Repeat steps 2 and 3 until sum of squares according to Eq. (22) cannot be further minimized.

9.2. Shape Decomposition

Variations of shape within a training population can be modeled using PCA, also known as Karhunen–Lóeve expansion. Assuming that the training dataset covers a set of closely related shapes, correlation between shape points exists that can be represented by a multivariate Gaussian distribution. As a very large number of shape points needs to be analyzed for statistical analysis, PCA is utilized to extract the principal modes, which represent data correlation along principal directions within the dataset, to reduce problem dimensionality. PCA is the process of determining the set of modes that captures the expected geometric variability within the training set. A shape can be mapped onto another shape in a correlated dataset by a linear transformation. Given $N$ number of shapes represented by shape $X$ according to Eq. (14) with mean represented by Eq. (17), a linear transformation $Y$ of $X$ can be represented as

$$Y=MX,$$ (25)

where $M$ is an orthogonal transformation matrix and the shape covariance matrix of $X$ can be represented as

$${\mathrm{\Sigma }}_X=\frac{1}{N}\sum _{i=1}^N(X_i-\overline{X}){(X_i-\overline{X})}^{\mathrm{T}}.$$ (26)

Therefore, the mean of $Y$ can be represented as

$$\overline{Y}=\frac{1}{N}\sum _{i=1}^N(Y_i)=\frac{1}{N}\sum _{i=1}^{N}M{X}_i=M\overline{X}$$ (27)

and the covariance of $Y$ can be calculated as

$${\mathrm{\Sigma }}_Y=\frac{1}{N}\sum _{i=1}^N(Y_i-\overline{Y}){(Y_i-\overline{Y})}^{\mathrm{T}}=M{\mathrm{\Sigma }}_Y{M}^{\mathrm{T}}$$ (28)

substituting transformation matrix $M$ with eigenvectors $\mathrm{\Phi }$ and rearranging

$$M^{\mathrm{T}}{\mathrm{\Sigma }}_Y={\mathrm{\Sigma }}_X{M}^{\mathrm{T}},$$ (29)

$${\mathrm{\Sigma }}_X\mathrm{\Phi }=\mathrm{\Phi }{\mathrm{\Sigma }}_y.$$ (30)

Therefore, since the covariance matrix of the training shapes ${\mathrm{\Sigma }}_X$ is symmetric, if $\mathrm{\Phi }$ represents the eigenvectors of ${\mathrm{\Sigma }}_X$, then covariance of the transformed shapes ${\mathrm{\Sigma }}_Y$ represents the diagonal matrix of eigenvalues ${\lambda }_i$ belonging to the dataset.

Each eigenvector ${\varphi }_i$ represents the modes of variation within the training dataset, and the corresponding eigenvalue ${\lambda }_i$ captures the amplitude of variation along the corresponding eigenvector direction, with the largest $\lambda$ corresponding to the largest deformation in corresponding modes. The eigenvalues of $\mathrm{\Phi }$ are sorted in descending order such that ${\lambda }_i>{\lambda }_{i+1}$ and the largest $t$ eigenvalues and corresponding eigenvectors are kept so that

$${\mathrm{\Phi }}_t=({\varphi }_1,{\varphi }_2\dots {\varphi }_t).$$ (31)

A sample shape $X$ can be approximated as a linear combination of the mean shape and first $t$ modes of variation represented by

$$X=\overline{X}+b_t{\mathrm{\Phi }}_t,$$ (32)

where $b_t$ is a $t$-dimensional vector representing modes of variation. Assuming the mean shape $\overline{x}$ is located at the origin, 3 standard deviation of ${\lambda }_i$ usually captures expected shape variability with a 99.7% confidence interval.

The statistical-shape energy functional can be defined as

$$E_{\text{shape}}(S)=\frac{1}{2}{(S-\overline{S})}^{\mathrm{T}}{\mathrm{\Sigma }}_{\perp }^{-1}(S-\overline{S}),$$ (33)

where $S$ is the simplex model, $\overline{S}$ is the mean simplex shape, and ${\mathrm{\Sigma }}_{\perp }^{-1}$ is the inverse of the regularized covariance matrix.

Statistical shape modeling determines a mean shape and allowed variability within the model as well as construction of new shapes through a combination of the principal modes of variation within the expected shape. This SSM property can be combined with deformable models to constrain a deformation toward an expected shape during the segmentation process in the presence of image noise or artifacts that otherwise hinder object boundary detection. Tejos et al.²⁷ combined statistical knowledge with simplex meshes and snakes evolution to segment knee structures. Schmid et al.²⁸ augmented simplex meshes with shape and appearance knowledge for segmentation of MRI musculoskeletal structures with limited field of view or the presence of image artifacts. Although shape models can provide robust segmentation with the presence of image artifacts and low contrast, their performance is dependent on initialization. Moreover, they only allow deformation already captured within the modes of variation during model construction and do not consider any image information outside the scope of the estimated model shape during deformation.

Biographies

Rabia Haq received her PhD from Old Dominion University’s Department of Modeling, Simulation, and Visualization Engineering in December 2015, where her dissertation was entitled “Multi-Surface Simplex Spine Segmentation for Spine Surgery Simulation and Planning.” She is a postdoctoral fellow in the Department of Medical Physics at Memorial Sloan Ketting Cancer Center. Her current research interests include deep learning and atlas-based segmentation in medical images.

Jérôme Schmid has been working for many years in medical image processing and understanding, focusing on segmentation and registration. He specialized in physically based deformable models applied to the modeling of the human musculoskeletal system. He has also explored the combination of deformable models with machine learning, as well as the use of deep learning for computer-assisted diagnosis such as bone fracture recognition in wrist radiographs or Parkinson’s disease detection in single-photon emission computed tomography imaging.

Roderick Borgie attended the Medical School at the Uniformed Services University of Health Sciences, Bethesda, Maryland, USA, and completed a general surgery internship at Naval Medical Center San Diego (NMCSD), a residency in diagnostic radiology at NMCSD, and a two-year neuroradiology fellowship at Massachusetts General Hospital. He served as a neuroradiology chief at NMC, Portsmouth, Virginia, USA; a force surgeon at Commander Naval Air Force Reserve; and the chair at Medical Executive Committee for Commander, Naval Air Forces, U.S. Pacific Fleet.

Joshua Cates received his bachelor’s and master’s degrees from the University of Tennessee-Knoxville in 1996 and 2000, respectively, and his PhD in computational science from the University of Utah (UU) in 2010. He has been a director of research and product development at OrthoGrid Systems, Salt Lake City, since February 2017. He was a research scientist at the Scientific Computing and Imaging Institute of the UU from 2010 to 2017 and a director of Biomedical Image and Data Analysis Core at the UU from 2015 to 2017.

Michel A. Audette received his bachelor’s degree (BEng-EE) and PhD (BME) from McGill University and his master’s degree (MEng-EE) from the École Polytechnique. He is an associate professor in the Department of Modeling, Simulation, and Visualization Engineering at Old Dominion University. He did postdoctoral research in Japan and Germany. His career path includes flight simulation, welding automation, neuronavigation, and open-source software. His interests include medical image analysis, surgery planning, and medical simulation.

Disclosures

No conflicts of interest, financial or otherwise, are declared by the authors.