Prediction of Soft Tissue Deformations after CMF Surgery with Incremental Kernel Ridge Regression

Abstract

Facial soft tissue deformation following osteotomy is associated with the corresponding biomechanical characteristics of bone and soft tissues. However, none of the methods devised to predict soft tissue deformation after osteotomy incorporates population-based statistical data. The aim of this study is to establish a statistical model to describe the relationship between biomechanical characteristics and soft tissue deformation after osteotomy. We proposed an incremental kernel ridge regression (IKRR) model to accomplish this goal. The input of the model is the biomechanical information computed by the Finite Element Method (FEM). The output is the soft tissue deformation generated from the paired pre-operative and post-operative 3D images. The model is adjusted incrementally with each new patient’s biomechanical information. Therefore, the IKRR model enables us to predict potential soft tissue deformations for new patient by using both biomechanical and statistical information. The integration of these two types of data is critically important for accurate simulations of soft-tissue changes after surgery. The proposed method was evaluated by leave-one-out cross-validation using data from 11 patients. The average prediction error of our model (0.9103 mm) was lower than some state-of-the-art algorithms. This model is promising as a reliable way to prevent the risk of facial distortion after craniomaxillofacial surgery.

1. Introduction

Human facial appearance plays an important role in individuals’ quality of life. In many patients with craniomaxillofacial (CMF) deformities, both bones and facial soft tissues are involved, and patients undergo surgery to rectify such deformities. The success of CMF surgery depends not only on the technical aspects of the operation, but also on a precise presurgical plan [1–4]. Currently, surgeons can accurately plan osteotomies (surgical procedures on bone), but cannot accurately predict soft-tissue deformation after osteotomy despite multiple attempts at presurgical planning.

Facial soft tissue deformation following osteotomy is associated with biomechanical characteristics of bone and soft tissues [5, 6]. Currently, there are three main methods to simulate soft-tissue deformation utilizing biomechanics. The first is the mass spring modeling (MSM) method [7, 8]. This model represents the face as a collection of assembled mass-spring entities. This model has an easy architecture, which benefits computational speed. However, it is less biomechanically relevant because it does not incorporate the biomechanical characteristics [9]. The second is the finite element modeling (FEM) method [10–12]. This method is based on biomechanics to characterize the relationship between tissue deformations and biomechanical properties, and thus is more biomechanically relevant. FEM can be categorized into two classes: linear and nonlinear [13]. Linear FEM results from linear elasticity with isotropic, linear, and elastic material. When the materials are modeled as non-isotropic or non-elastic, a nonlinear FEM result occurs. The difference in the prediction of soft-tissue deformation between linear and nonlinear FEM is controversial. One study reported only that there was a difference between prediction results using linear and nonlinear FEM [13], while another reported that linear FEM outperformed the nonlinear FEM method [9]. However, FEM has the disadvantage of being computationally costly. The third method is the mass tensor modeling (MTM) method [14, 15], which is a mixture of the FEM and MSM approaches. It has the easy architecture of MSM, and at the same time keeps the biomechanical relevance of FEM. MTM can achieve accuracies comparable to those of linear FEM, while reducing the computational cost [9].

Unfortunately, none of the above methods includes population-based statistical information. Since we can collect patients’ preoperative and postoperative 3D images, it should be possible to establish a statistical model to yield statistical dependencies from the individualized soft tissue deformations. Meller et al. proposed the statistical deformable model (SDM) to capture the variety of preoperative facial morphologies in a group of patients, and their compared their corresponding postoperative deformations from 3D surface scans [16]. After fitting preoperative data for a new patient into the model, the postoperative morphology could be extracted. One drawback of this method was that it did To address this need, we reported a preliminary two-step algorithm, in which FEM was first performed to extract the nodal displacement features, and SDM was then used to learn the statistics of the nodal displacement over patients’ preoperative data [17]. However, this approach was an unsupervised method, in which real postoperative data were not used. This could be a source of inaccuracy.

There are different types of CMF deformities, and even within the same deformity (i.e. Angle’s Class III), many variations exist. An ideal set of training data would include every deformity and its variations to simulate soft tissue change. However, none of the currently available training data sets includes such variations. To remedy this limitation, deformities of new patients should be included into the statistical model. We conjecture that new patients’ biomechanical properties will help make prediction of postoperative soft tissue changes more accurate.

The goal of this paper is to establish a statistical model to describe the relationship between biomechanical characteristics and soft tissue deformations. We develop an incremental version of the kernel ridge regression (KRR) model, which not only builds nonlinear relations between biomechanical information and soft tissue deformations, but also is incrementally adjusted by incorporating the new patients’ biomechanical characteristics. The proposed model, called the incremental kernel ridge regression (IKRR) model, first trains a KRR model from a set of paired preoperative and postoperative 3D data, then adds biomechanical data from new patients into the KRR model. Prediction of IKRR is the convex combination ² of the predictions of KRR and FEM, where the combination coefficients are controlled by the trade-off parameter of KRR. Compared to [12], our model makes use of new patients’ information, and more importantly, also utilizes the supervised information (postoperative 3D data). The proposed method was validated in 11 patients. The IKRR model achieved lower prediction errors than other evaluated methods, which are Linear Finite Element Modeling (LFEM) [9], Statistical Deformable Model (SDM) [17], and Ridge Regression (RR). And it produced more faithful visualizations of the predicted images. Furthermore, the IKRR model was experimentally more efficient than the KRR model, and updated the model with new data.

The notations used in the paper were described. Vectors were denoted by bold lower case letters, and matrices by upper case ones. Vectors are denoted by bold lower-case letters, and matrices by upper-case ones. The transposes of a vector a and a matrix A are represented by a^T and A^T, respectively. The inverse and the determinant of a square matrix A are denoted by A⁻¹ and det A, respectively. We use I_n to denote the n by n identity matrix. The Euclidean norm of a vector a is denoted by $\Vert \mathbf{a}\Vert =\sqrt{{\mathbf{a}}^{\mathrm{T}}\mathbf{a}}$.

This paper completes our conference paper [18] by including more details of our methods and additional experiments.

2. Materials and methods

Fig. 1 showed the preoperative and postoperative surface scans of a patient. To improve his facial appearance and to reduce the prominence of his chin, this patient underwent surgery to set back the mandible (bilateral sagittal split osteotomies) and advance the maxilla (Le Fort I osteotomy). We left the osteosynthesis material. The postoperative surface scan was acquired 6 months after surgery to avoid surgical swelling.

Fig. 1.

(a) The preoperative surface scan; (b) The postoperative surface scan.

The FEM method was used to extract biomechanical information from the CT images. Then a regression model was used to establish the statistical relationships. As a new patient’s data arrived, the learned regression model was employed to predict the resulting soft tissue changes. The whole procedure was divided into two phases. In the first phase, named the training phase, we established regression models. The second phase, named the test phase, involved predictions of soft tissue deformations of new patients. Fig. 2 presents a flowchart of the two phases.

Fig. 2.

In the training phase, the statistical model was generated. During the test phase, the model was adjusted incrementally by incorporating biomechanical information from new patients. The predictions were then acquired from the adjusted model.

In the training phase, we collected a set of preoperative and postoperative 3D images. The features were extracted from the preoperative images with FEM. The details of feature extraction are shown in Fig. 3. First, an detailed anatomic template was generated to be applicable to all data. The template helped to automatically generate the anatomic detailed mesh for each patient, which substantially reduced the workload. The displacement boundary condition (surgical plan) could be determined from the paired preoperative and postoperative skulls. After obtaining the mesh and displacement boundary conditions, we employed FEM to extract biomechanical information of individuals. The extracted features were then imported as the input in the regression model. The output in the regression model represented the true displacements of the corresponding nodes in the preoperative and postoperative meshes. As data from a new patient became available, we adjusted the model to incorporate his/her biomechanical information. The displacements of mesh nodes of the new patient were predicted from the adjusted model. Predictions of postoperative appearance were visualized by using interpolation techniques.

Fig. 3.

Feature extraction with FEM methods in the training phase. The stress computed is utilized as the input of the KRR model.

2.1. Data Acquisition and Pre-processing

This study was approved by the Institutional Review Board of Wake Forest Baptist Medical Center (IRB00028345). All research data were based on existing image data for CMF surgery at Wake Forest Baptist Medical Center. Following data collection, patient-identifying information was destroyed, consistent with data validation and study design, producing an anonymous analytical data set. A pilot study designed for 11 random participants are followed prospectively and subsequent status evaluations with respect to the craniomaxillofacial deformity. As the study is conducted, outcome from participants is measured and relationships with specific characteristics determined in our model. We collected from 11 patients consisting of preoperative and postoperative CT scans and facial surface scans. The facial surface scans were acquired from a 3D surface camera operated by a physician. We use a calibrated laser surface scanner (Cyberware 3030 Head & Face Color Scanner; Cyberware, Monterey, CA) to capture both the 3D geometry and the absolute orientation of the human face. During the surface scanning, the patient’s facial expression was kept neutral. Facial surface scans were used to prevent any unintended soft tissue strain caused by CT scanning. In the following computation, the facial skin of the CT images was replaced with facial surface scans. All the images were rigidly registered to the preoperative CT images with Mimics software (Materialise, Leuven, Belgium). We segmented the bones of preoperative and postoperative CT images for further determination of surgical plans.

2.2. Feature Extraction

We used linear FEM (LFEM) methods to extract an individual’s biomechanical characteristics, including stress, strain and displacement. Stress can be calculated from strain and displacement with soft tissue material properties. Therefore, we used stress as a biomechanical feature in our work.

In the FEM procedures, deformation behaviors of soft tissues could be characterized with the mechanical equations of linear elasticity [9], described in Section 2.2.4. To solve the equations, LFEM was used. The 3D object was first discretized into small elements. The discretization of the object was called the mesh. In each element, the displacement was assumed in simple forms to obtain element equations. The equations obtained for each element were then assembled together with adjoining elements to form the global equation for the whole object. The displacement boundary condition was determined based on the skull structures of preoperative and postoperative CT images. Then the global equation was solved by incorporating the displacement boundary condition.

To implement the LFEM model, both the mesh and the displacement boundary conditions are required.

2.2.1 Generation of Detailed Anatomic Template

Next, the anatomic details were incorporated into the mesh. The facial muscles that contributed to facial soft tissue deformation were considered, i.e. Buccinator, Depressor anguli oris, Depressor labii, Levator anguli oris, Levator labii, Levator labii alaeque nasi, Masseter, Mentalis, Orbicularis oris, and Zygomaticus major and minor [19]. However, it is difficult to segment these muscles from patients’ CT data, and manual segmentation is onerous. We proposed a method to automatically locate muscles for each patient in the following paragraph and subsection 2.2.2.

We created a template of an anatomic detailed mesh using a Visible Human Female Dataset. The desired muscles were segmented from this dataset. Since the images of the dataset have multiple colors (24 bits of color) and are of high resolution (2048 pixels by 1216 pixels), muscles can be easily distinguished. Fig. 4 shows the segmented muscles. Apart from the bone and muscle, the remaining soft tissues were considered as a homogenous material. Thus, we classify the tissues into three kinds: bone, muscle, and other tissues. Each kind of tissues is considered as homogeneous and isotropic. We set the Young’s modulus to 1.2E+10 Pa and the Poisson ratio to 0.3 for bone. For muscle, the Young’s modulus and the Poisson ratio were set to 1.1E+5 Pa and 0.3, respectively. For other tissues, we set the Young’s modulus to 1.5E+4 Pa and the Poisson ratio to 0.49 [20]. All the segmented tissues were exported as stereolithography (STL) files and then imported into TrueGrid (XYZ Scientific Applications, Inc., Livermore, CA). An anatomic detailed mesh was generated as a hexahedral block mesh in TrueGrid. This mesh was further used to map anatomic structures into the CT images of patients.

Fig. 4.

The segmented muscles from the Visual Human Female Dataset.

2.2.2 Mesh Generation from the Template

Once the template was generated, detailed anatomic structures can be mapped from this model to data from patients. The deformation algorithm was created with the surface projection techniques implemented in TrueGrid. The 3D surface scan of each patient was imported into TrueGrid as a set facial geometries. The anatomic landmarks were manually labeled at first. Then the landmarks of the patient mesh were projected to those of the model mesh. Finally, the whole mesh was altered according to the projected landmarks. All anatomic details were mapped from the template to the patient. The muscles were then automatically located with the help of the landmarks.

Deformation algorithms can be used for both pre-operative and post-operative data without any changes in the numbers of finite elements or nodes. A natural correspondence between different mesh nodes was established, which was a precondition for statistical analysis. To limit the number of elements and to reduce computational complexity, we restricted the mesh to the zone below the eyes, with the zone above the nose was considered unchanged during surgery. The resulting mesh contained 29632 hexahedral elements and 33770 nodes, shown in Fig. 5.

Fig. 5.

Generated mesh from the template. The right subfigure shows the shape of one element in the left subfigure (cut the tip of nose, which is not deformation after surgery).

Only some specific areas undergo large deformations. These deformations distort the finite element mesh, and unable to provide accurate results. In such simulations it is necessary to use adaptive meshing tools to periodically minimize the distortion in the mesh. ABAQUS provides a very general and robust adaptive meshing capability for highly nonlinear problems. We refine the mesh template by using Element Distortion Control (EDC) in ABAQUS.

2.2.3 Determination of Displacement Boundary Condition

We classified the mesh nodes into boundary nodes and free nodes. Boundary nodes were located in skull parts which would be repositioned during surgery. In free nodes, displacement resulted from the displacement of boundary points. The displacement boundary condition consisted of the displacements of all the boundary nodes.

We first manually registered the post-operative skull to the pre-operative one based on an unaltered part, usually the part above the nose, using Mimics software. Afterwards, the pre-operative skull was cut into parts according to the post-operative one. Then, the skull parts were separately matched by manual alignment to the post-operative counterparts. The skull parts before and after displacements were exported as STL files which were further imported into Matlab (The MathWorks, Inc., Natick, Massachusetts). The Iterative Closest Point (ICP) algorithm [21] was used to compute the rotation transformation R and the translation transformation t between the pre-operative and post-operative skull parts. We assumed the coordinate of a boundary node was p, and its displacement was u. Then the displacement of this node was calculated as u = (R·p + t)− p. After the computation of all the boundary nodes, we derived the displacement boundary condition. Fig. 6 shows the skull structures before and after surgery.

Fig. 6.

(a) Skull structure before surgery, (b) Skull structure after surgery. The skull of (a) was cut into a number of parts, shown in different colors. The displaced counterparts were shown in (b) with the same color. Note that for this patient, all the parts were repositioned except the upper skull in white color.

2.2.4 Calculation of Stress with FEM

The displacement, body force, stress, and strain of each mesh node were respectively denoted by:

$$\{\begin{array}{cc}\text{displacement}:& \mathit{u}={\left(u,v,w\right)}^{\mathrm{T}}\\ \text{body}\text{force}:& \mathit{f}={(f_x,f_y,f_z)}^{\mathrm{T}}\\ \text{stress}:& \mathbf{\sigma }={({\sigma }_{xx},{\sigma }_{yy},{\sigma }_{zz},{\tau }_{xy},{\tau }_{xz},{\tau }_{yz})}^{\mathrm{T}}\\ \text{strain}:& \mathbf{\epsilon }={({\epsilon }_{xx},{\epsilon }_{yy},{\epsilon }_{zz},{\gamma }_{xy},{\gamma }_{xz},{\gamma }_{yz})}^{\mathrm{T}}\end{array}$$

The aim was to calculate the stress of the node. Linear elasticity was used to characterize the deformation behavior of soft tissues, which contained the following 15 equations and one displacement boundary condition [9].

The first 3 equations were the static equilibrium equations which stated that the sum of external forces and moments on a body was zero:

$$\{\begin{array}{l}\frac{\partial {\sigma }_{xx}}{\partial x}+\frac{\partial {\tau }_{xy}}{\partial y}+\frac{\partial {\tau }_{xz}}{\partial z}+f_x=0\hfill \\ \frac{\partial {\tau }_{xy}}{\partial x}+\frac{\partial {\sigma }_{yy}}{\partial y}+\frac{\partial {\tau }_{yz}}{\partial z}+f_y=0\hfill \\ \frac{\partial {\tau }_{xz}}{\partial x}+\frac{\partial {\tau }_{yz}}{\partial y}+\frac{\partial {\sigma }_{zz}}{\partial z}+f_z=0\hfill \end{array}.$$

The next 6 equations were the geometric equations which related the strain and displacement:

$$\begin{array}{l}{\epsilon }_{xx}=\frac{\partial u}{\partial x},{\epsilon }_{yy}=\frac{\partial v}{\partial y},{\epsilon }_{zz}=\frac{\partial w}{\partial z},\\ {\gamma }_{xy}=\frac{\partial u}{\partial y}+\frac{\partial v}{\partial x},{\gamma }_{xz}=\frac{\partial u}{\partial z}+\frac{\partial w}{\partial x},{\gamma }_{yz}=\frac{\partial v}{\partial z}+\frac{\partial w}{\partial y}.\end{array}$$

The last 6 equations were the constitutive equations which used Hooke’s law to describe the relation between stress and strain [20]:

$$\begin{array}{l}\left(\begin{array}{c}{\sigma }_{xx}\\ {\sigma }_{yy}\\ {\sigma }_{zz}\\ {\tau }_{xy}\\ {\tau }_{yz}\\ {\tau }_{xz}\end{array}\right)=\frac{E}{(1+\nu )(1-2\nu )}\left(\begin{array}{cccccc}1-\nu & \nu & \nu & 0& 0& 0\\ \nu & 1-\nu & \nu & 0& 0& 0\\ \nu & \nu & 1-\nu & 0& 0& 0\\ 0& 0& 0.5-\nu & 0& 0& 0\\ 0& 0& 0& 0& 0.5-\nu & 0\\ 0& 0& 0& 0& 0& 0.5-\nu \end{array}\right)\left(\begin{array}{c}{\epsilon }_{xx}\\ {\epsilon }_{yy}\\ {\epsilon }_{zz}\\ {\gamma }_{xy}\\ {\gamma }_{yz}\\ {\gamma }_{xz}\end{array}\right)\\ \iff \mathbf{\sigma }=\mathbf{D}\mathbf{\epsilon }\end{array}$$

with the material properties Young’s modulus E and Poisson ratio ν. The displacement boundary condition was the displacement calculated for the boundary nodes.

The FEM was employed to solve the above 15 equations associated with the displacement boundary condition. After determining the nodal displacement, we then can calculate the nodal stress.

To predict changes in facial appearance, we only need the nodes lying on the outer skin. There were a total of 2652 nodes, and each was associated with a stress vector. All the stress vectors were stacked together, resulting in a vector of length 2652*6=15912, denoted by σ_i for the ith patient, i = 1, 2, …, n. The vector σ_i was the feature extracted from the ith patient.

2.3. Training the Kernel Ridge Regression Model

A regression model was trained to determine the statistical relationships between biomechanical features and facial skin deformations. The input of the model was the feature computed in Section 2.2. The facial skin displacement was calculated from preoperative and postoperative meshes. Only 2652 nodes were needed for computation, as illustrated in Section 2.2. Then these 2652 nodal displacements were stacked together to form a vector u_i of length 7956, which served as the output. The statistical relationships were learned for the input and the output. Specifically, we used the input-output pairs (σ_i,u_i) ∈ ℜ¹⁵⁹¹²×ℜ⁷⁹⁵⁶, i = 1, …, n. The aim was to learn a function f such that f (σ_i) ≈ u_i for each i.

The ridge regression (RR) model is a widely used algorithm which models linear dependencies between inputs and outputs [22]. It assumed that the prediction function was of the form

$$f_{\text{RR}}(\mathbf{\sigma })={\mathbf{W}}^{\mathrm{T}}\mathbf{\sigma },$$

where W stood for called regression coefficients. The RR model gave an objective function consisting of a square loss term and a regularization term

$$L_{\text{RR}}(\mathbf{W})=\frac{1}{2}\sum _{i=1}^n{\Vert {\mathbf{u}}_i-{\mathbf{W}}^{\mathrm{T}}{\mathbf{\sigma }}_i\Vert }^2+\frac{\lambda }{2}\text{tr}({\mathbf{W}}^{\mathrm{T}}\mathbf{W}),$$ (1)

where the trade-off parameter λ>0 was added to control the overfitting of the model. We minimized the objective function to obtain regression coefficients. Then for any new input σ, we can predict its output by using f_RR. Details of the RR method have been published [22].

However, the relationship between stress and displacement was complex, and empirical results showed that RR did not fit well for our data. We thus used the nonlinear version of RR model [23]. To learn nonlinear relations, the input was first mapped into a higher dimensional space H via a nonlinear mapping ϕ. Then we performed RR in H rather than in the input space. Performing linear regression in H is equivalent to performing nonlinear regression in the input space.

The dimensionality of H could be very high, even infinite. Direct computations in H would be computationally infeasible. One strategy to avoid expensive computation is to employ the kernel function. The idea was to replace the inner product in H with the kernel function, which could be calculated much more efficiently. Given x and y in the input space and kernel function k, we have the relation [18]

$$k(\mathbf{x},\mathbf{y})={\varphi }^{\mathrm{T}}(\mathbf{x})·\varphi (\mathbf{y}).$$ (2)

Performing the RR model in H and replacing the inner product with the kernel function, we obtained the Kernel Ridge Regression (KRR) model. Fig. 7 shows the relationship between the RR and KRR models.

Fig. 7.

The RR approach performs linear regression in an input space, while the KRR approach creates a linear model in H.

Different kernel functions corresponded to different values for H, and thus lead to different KRR models. An empirical choice of kernel function was the widely used Gaussian kernel

$$k(\mathbf{x},\mathbf{y})=exp\left(-\alpha {\Vert \mathbf{x}-\mathbf{y}\Vert }^2\right)$$

with the width α>0 . This kernel has been extensively validated for many types of data.

The prediction function of KRR was assumed to be the following form

$$f_{\text{KRR}}(\mathbf{\sigma })={\mathbf{W}}^{\mathrm{T}}\varphi (\mathbf{\sigma }).$$

We then minimize the square loss with a regularization term:

$$L_{\text{KRR}}(\mathbf{W})=\frac{1}{2}\sum _{i=1}^n{\Vert {\mathbf{u}}_i-{\mathbf{W}}^{\mathrm{T}}\varphi ({\mathbf{\sigma }}_i)\Vert }^2+\frac{\lambda }{2}\text{tr}({\mathbf{W}}^{\mathrm{T}}\mathbf{W}),$$ (3)

where the trade-off parameter λ>0. Taking derivatives and equating them to zero gave the solution

$${\mathbf{W}}^{\ast }=\mathrm{\Phi }{({\mathrm{\Phi }}^{\mathrm{T}}\mathrm{\Phi }+\lambda {\mathbf{I}}_n)}^{-1}\mathbf{U},$$ (4)

where Φ = (ϕ(σ₁), …, ϕ(σ_n)) and U = (u₁, …, u_n)^T.

For any σ, the prediction of the KRR model could be expressed in terms of kernel function

$$\stackrel{\sim }{\mathbf{u}}={\left({\mathbf{W}}^{\ast }\right)}^{\mathrm{T}}\varphi (\mathbf{\sigma })={\mathbf{U}}^{\mathrm{T}}{(\mathbf{K}+\lambda {\mathbf{I}}_n)}^{-1}\mathbf{k}(\mathbf{\sigma }),$$ (5)

where k(σ) = (k(σ₁, σ), …, k(σ_n, σ))^T K = (k(σ_i, σ_j))_i,j, i, j = 1, …, n.

The crucial observation about (5) was that the prediction only involved kernel function. Therefore, it was not necessary to define the nonlinear mapping ϕ.

2.4. Prediction of Soft-Tissue Deformations with the Incremental KRR Model

The KRR model learned from the training data was viewed as a general relationship. We would like to incorporate individual information for new patients into the KRR model, making the model more appropriate for this new patient.

Thus, the KRR model was adjusted by making use of new patients’ biomechanical information. The feature σ̃ and the predicted displacement of LFEM ũ_FEM were computed from new patients’ data. We added the pair (σ̃,ũ_FEM) to the training set to update the KRR model. For efficient computation, the updated KRR model was computed incrementally by using the existing model. The result is called the incremental KRR (IKRR) model.

As shown in (5), the prediction of the updated KRR was computed as follows:

$$\begin{array}{l}\stackrel{\sim }{\mathbf{u}}={\left(\begin{array}{c}\mathbf{U}\\ {\stackrel{\sim }{\mathbf{u}}}_{\text{FEM}}^{\mathrm{T}}\end{array}\right)}^{\mathrm{T}}{\left(\begin{array}{cc}\mathbf{K}+\lambda {\mathbf{I}}_n& \mathbf{k}(\stackrel{\sim }{\mathbf{\sigma }})\\ {\mathbf{k}}^{\mathrm{T}}(\stackrel{\sim }{\mathbf{\sigma }})& k(\stackrel{\sim }{\mathbf{\sigma }},\stackrel{\sim }{\mathbf{\sigma }})+\lambda \end{array}\right)}^{-1}\left(\begin{array}{c}\mathbf{k}(\stackrel{\sim }{\mathbf{\sigma }})\\ k(\stackrel{\sim }{\mathbf{\sigma }},\stackrel{\sim }{\mathbf{\sigma }})\end{array}\right)\\ ={\left(\begin{array}{c}\mathbf{U}\\ {\stackrel{\sim }{\mathbf{u}}}_{\text{FEM}}^{\mathrm{T}}\end{array}\right)}^{\mathrm{T}}\left(\begin{array}{cc}\mathbf{A}& \mathbf{b}\\ \mathbf{c}& d\end{array}\right)\left(\begin{array}{c}\mathbf{k}(\stackrel{\sim }{\mathbf{\sigma }})\\ k(\stackrel{\sim }{\mathbf{\sigma }},\stackrel{\sim }{\mathbf{\sigma }})\end{array}\right)\\ =(\mathbf{ck}(\stackrel{\sim }{\mathbf{\sigma }})+dk(\stackrel{\sim }{\mathbf{\sigma }},\stackrel{\sim }{\mathbf{\sigma }})){\stackrel{\sim }{\mathbf{u}}}_{\text{FEM}}+{\mathbf{U}}^{\mathrm{T}}(\mathbf{Ak}(\stackrel{\sim }{\mathbf{\sigma }})+\mathbf{b}k(\stackrel{\sim }{\mathbf{\sigma }},\stackrel{\sim }{\mathbf{\sigma }})),\end{array}$$

where

$$\begin{array}{l}\mathbf{A}={(\mathbf{K}+\lambda {\mathbf{I}}_n)}^{-1}+d{(\mathbf{K}+\lambda {\mathbf{I}}_n)}^{-1}\mathbf{k}(\stackrel{\sim }{\mathbf{\sigma }}){\mathbf{k}}^{\mathrm{T}}(\stackrel{\sim }{\mathbf{\sigma }}){(\mathbf{K}+\lambda {\mathbf{I}}_n)}^{-1},\\ \mathbf{b}=-d{(\mathbf{K}+\lambda {\mathbf{I}}_n)}^{-1}\mathbf{k}(\stackrel{\sim }{\mathbf{\sigma }}),\\ \mathbf{c}=-d{\mathbf{k}}^{\mathrm{T}}(\stackrel{\sim }{\mathbf{\sigma }}){(\mathbf{K}+\lambda {\mathbf{I}}_n)}^{-1},\\ d={(k(\stackrel{\sim }{\mathbf{\sigma }},\stackrel{\sim }{\mathbf{\sigma }})+\lambda -{\mathbf{k}}^{\mathrm{T}}(\stackrel{\sim }{\mathbf{\sigma }}){(\mathbf{K}+\lambda {\mathbf{I}}_n)}^{-1}\mathbf{k}(\stackrel{\sim }{\mathbf{\sigma }}))}^{-1}.\end{array}$$

Making simplification for the above equation, we obtained

$$\stackrel{\sim }{\mathbf{u}}=t{\stackrel{\sim }{\mathbf{u}}}_{\text{FEM}}+(1-t){\stackrel{\sim }{\mathbf{u}}}_{\text{KRR}},$$ (6)

where ũ_KRR = U^T(K + λI_n)⁻¹k(σ̃) was the prediction of KRR, t = (e − λ)/e and e = d⁻¹. We would show that t ∈ [0,1) . Equivalently, we would prove e − λ>0 .

We introduced a matrix

$$\stackrel{\sim }{\mathbf{K}}=\left(\begin{array}{cc}\mathbf{K}+\lambda {\mathbf{I}}_6& \mathbf{k}(\stackrel{\sim }{\mathbf{\sigma }})\\ {\mathbf{k}}^{\mathrm{T}}(\stackrel{\sim }{\mathbf{\sigma }})& k(\stackrel{\sim }{\mathbf{\sigma }},\stackrel{\sim }{\mathbf{\sigma }})\end{array}\right).$$

The Schur complement of K + λI₆ in K̃ was e − λ. The following Lemma about the Schur complement stated [24]:

Lemma 1

Let G be a symmetric matrix partitioned as

$$\mathbf{G}=\left(\begin{array}{cc}{\mathbf{G}}_{11}& {\mathbf{G}}_{12}\\ {\mathbf{G}}_{12}^{\mathrm{T}}& {\mathbf{G}}_{22}\end{array}\right),$$

in which G₁₁ is square and nonsingular. Then G is positive definite if and only if both G₁₁ and ${\mathbf{G}}_{22}-{\mathbf{G}}_{12}^{\mathrm{T}}{\mathbf{G}}_{11}^{-1}{\mathbf{G}}_{12}$ are positive definite.

By using the positive semi-definiteness of the kernel matrix upon the data {σ₁, …, σ_n, σ̃} [25], we know that K̃ is positive definite. From Lemma 1, e − λ > 0.

Equation (6) shows that prediction of IKRR was a convex combination of the predictions of KRR and LFEM. IKRR included KRR as a special case by setting t=0. When t approached 1, the prediction of IKRR was similar to that of LFEM. Therefore, IKRR was more flexible. By adjusting t, we could control the contributions of KRR and LFEM. Equation (6) also revealed that IKRR was more efficient than KRR. Training n+1 data in KRR scaled O(n³) . In contrast, IKRR scaled O(n²) by updating the results of KRR.

2.5. Implementation Issues

A natural correspondence between all the meshes was established because the meshes were generated from the same template. Each feature was normalized to have zero mean and unit standard deviation in order to eliminate the scale effect. The experiments were implemented with Matlab on a 64 bit Windows PC with 1.6GHz CPU and 24GB RAM. Two parameters, namely the trade-off parameter λ and the width of Gaussian kernel α, were tuned via grid search. We selected the values of the parameters which produced the best performance.

3. Results

In this validation, we illustrated three points. The first point was that the prediction result of IKRR could be improved as more training data were available. The second was the usefulness of IKRR model in the prediction of soft-tissue deformations by comparing it to other algorithms, quantitatively and visually. The final point was to demonstrate that IKRR was efficient. It was achieved by comparing the running time of IKKR with that of KRR.

3.1. Predictions with Different Amounts of Training Data

The prediction accuracy of IKRR changed with different amounts of training data. Five IKRR models were generated using 6 to 10 training data sets, respectively. For each model, the prediction error was defined as

$$E=\frac{1}{2652}\sum _{i=1}^{2652}\Vert {\mathbf{d}}_i-{\stackrel{\sim }{\mathbf{d}}}_i\Vert ,$$ (7)

where d_i was the true displacement of the ith node, and d̃_i was the predicted displacement of the ith node. The prediction errors for different amounts of training data are shown in Table 1. Errors were reduced as more training data were utilized, showing that the statistical information learned from the training data was helpful for predicting soft tissue changes.

Table 1.

Prediction difference vs. number of training data

Number of training	6	7	8	9	10
E (mm)	0.7845	0.7817	0.7814	0.7811	0.7756

3.2. Empirical Comparisons

We applied leave-one-out cross-validation, which involved using data from 1 of the 11 patients as the validation data, and the remaining patients’ data as the training data. This procedure was repeated such that each patient’s data was used once as validation data. The evaluated algorithms were LFEM [9], SDM [17], KRR, and IKRR. The prediction errors, defined in (7), were recorded in Table 2.

Table 2.

B_max (mm) was the maximal skull displacement during surgery. E_LFEM, E_SDM, E_KRR and E_IKRR were the prediction differences of corresponding methods. Surgical plans were described in the second column: M represented mandible, X represented maxilla, A represented advance, B represented back, R represented right, L represented left.

Patient	Surgery	Bmax	ELFEM	ESDM	EKRR	EIKRR
1	MB	11.6847	1.3241	3.6352	2.9955	1.1963
2	MB+XB	6.1581	0.8054	1.8381	1.3840	0.7756
3	MB	7.3624	1.2456	5.0321	4.5722	1.0467
4	MB +XA	7.2343	0.7389	2.9817	2.4249	0.7082
5	MR	8.3622	0.8567	2.2543	1.6896	0.7870
6	MA	10.5866	0.9391	3.3946	2.6338	0.8739
7	MB+XB	13.4578	1.1490	3.8809	2.6418	0.9551
8	MB+XA	7.3898	0.8845	4.5982	3.4110	0.8090
9	ML	10.5032	0.9978	3.1190	2.3010	0.9783
10	MR	6.1608	0.9531	2.1845	2.1136	0.9209
11	MB+XA	13.6617	0.9821	3.3014	2.1602	0.9626
mean		9.3238	0.9888	3.2927	2.5752	0.9103

As stated in [17], the accuracy of SDM critically relied on the number of available data. The relative low accuracy of SDM may have resulted from the lack of training data. By using supervised information (i.e. the post-operative images), KRR improved performance compared with SDM. Without using new patients’ biomechanical data, KRR was less accurate than LFEM. SDM also did not take into account biomechanical data from new patients. In contrast, LFEM produced smaller prediction errors. IKRR was the most accurate method, since it achieved the smallest prediction errors for each patient. Incorporation of new patients’ biomechanical data into the statistical model was critical to prediction performance.

Fig. 8 summarizes the outcomes of statistical analysis for the four models. The standard deviation, the maximum and the median of the prediction difference are reported. These results also indicate that the IKRR model achieved the best performance.

Fig. 8.

Results of statistical analyses of the LFEM, SDM, KRR, and IKRR models.

We used inverse distance weighted interpolation [26] for visualization of 3D images. Fig. 9 depicts a preoperative image, a postoperative image, the prediction using LFEM, and the prediction using IKRR. Both predictions could provide reasonable visualizations, but the results using IKRR were more accurate. Both the lip and facial contours in the IKRR prediction were more faithful to the postoperative image.

Fig. 9.

(a) preoperative image, (b) postoperative image, (c) prediction using LFEM, (d) prediction using IKRR.

We also show the visual results for other patients in Fig. 10. IKRR predictions were faithful regarding the facial surface compared to the postoperative results.

Fig. 10.

Upper row: prediction of IKRR. Lower row: postoperative images.

3.3. Computation Time

We compared the running times of IKRR and KRR analyses. KRR re-trained the model as new data were available. The running time of KRR involved the computation of (5) by replacing n with n+1, whereas IKRR updated the existing results using (6). The elapsed time of IKRR was the time required for the computation of (6). Results are listed in Table 3. In the case of 1000 training data, IKRR was 96 times faster to compute than KRR.

Table 3.

Computation time (s) for repetitive KRR and IKRR. “Speedup” means the factor that IKRR gained in CPU time over KRR

Number of training	1000	2000	3000	4000	5000
KRR	0.8542	3.3293	7.6927	14.1571	22.6551
IKRR	0.0089	0.0203	0.0357	0.0477	0.0629
Speedup	96.0	164.0	215.5	296.8	360.2

When the amount of training data increases, the gains in computational cost for IKRR over KRR also increase. For 5000 sets of training data, IKRR is about 360 times faster than KRR. Therefore, IKRR required much less time than KRR for large amounts of training data.

4. Discussion and Conclusion

The success of CMF surgery depends not only on the surgical techniques, but also upon an accurate surgical plan. The simulation methods should be accurate and fast for surgical plan. Attaining both is difficult because these attributes are inversely related, the more accurate the model is, the longer it takes to prepare and run. Current approaches do not meet the clinical needs because they are either too inaccurate or too slow. To remove this critical barrier we are proposing to develop an innovative approach. The goal is to develop a model for clinicians to accurately simulate the facial soft-tissue changes that is resulted from virtual skeletal reconstruction. In current practice, doctors only plan osteotomies (improving hard tissue functions) and hope for the best for optimal facial soft tissue. By using our model, doctors will be able to accurately simulate soft tissue changes following the virtual osteotomies. Most importantly, instead of imagining soft-tissue-change “blindly” following the osteotomies, doctors will be able to accurately determine the amount of skeletal over- or under correction for camouflaging the soft-tissue defect. If any unwanted soft-tissue-change occurs during the planning, doctors will be able to revise the osteotomies and movements, or camouflage the deformities by adding bones (bone grafting) or trimming off bones (ostectomy), to ensure the ideal appearance of the soft tissue envelope. Patients will also be able to foresee their realistic facial appearance preoperatively. This is very important to patients that they always outweigh the facial soft tissue changes over the underlying skeletal correction. In this paper, an integrated biomechanical and statistical learning model was proposed to extract prior knowledge of soft tissue deformation from the training dataset. The results empirically showed our method can accurately simulate soft-tissue deformation.

The proposed model confirmed our belief that statistical information is important for prediction of soft-tissue deformation. We conjectured that the statistical model would be superior to the individual-based model, and that the prior knowledge learnt from the preoperative and postoperative data would be helpful in predicting soft-tissue deformation. However, other statistical models could be further investigated for more accurate simulation.

The IKRR approach is still in the experimental stage. In the future, we will include more preoperative and postoperative data in the training model. The acquisition of postoperative data is expensive. However, preoperative data can be collected relatively inexpensively. To this end, future studies will use a semi-supervised method well-suited to the training set, consisting of a large amount of preoperative data and a small amount of postoperative data.

Highlights.

• An anatomic detailed mesh template was developed based on a Visible Human Female Dataset.

• A three dimensional finite element method was constructed to extract biomechanical stress information based on bone displacement calculated from pre-operative and post-operative CT data.

• The incremental kernel ridge regression method was established to characterize the relationship between the biomechanical stress information and the soft tissue deformation.