Incremental Kernel Ridge Regression for the Prediction of Soft Tissue Deformations

Abstract

This paper proposes a nonlinear regression model to predict soft tissue deformation after maxillofacial surgery. The feature which served as input in the model is extracted with Finite Element Model (FEM). The output in the model is the facial deformation calculated from the preoperative and postoperative 3D data. After finding the relevance between feature and facial deformation by using the regression model, we establish a general relationship which can be applied to all the patients. As a new patient comes, we predict his/her facial deformation by combining the general relationship and the new patient’s biomechanical properties. Thus, our model is biomechanical relevant and statistical relevant. Validation on eleven patients demonstrates the effectiveness and efficiency of our method.

1 Introduction

Craniomaxillofacial (CMF) deformities affect human’s head and facial appearance. CMF surgery is designed to reconstruct such condition. This type of the surgery usually requires extensive presurgical planning. Currently we are able to accurately simulate osteotomies. However, soft-tissue-change simulation still remains a challenge. The most widely used method to simulate soft tissue change is biomechanical relevant Finite Element Model (FEM) [1] and its improvements [2–5]. However, a major disadvantage of FEM methods is that they are individually-based. Population-based statistical information was not considered. On the other hand, a statistical based method [6] is efficient but does not consider the biomechanical properties and thus it is less-than-accurate. To this end, we hypothesized that the soft tissue change could be accurately simulated if we could combine the FEM and statistical model into one model.

This integrated model should not only maintain the integrity of biomechanical information, but also be computational efficient. In this study, we developed an Incremental Kernel Ridge Regression (IKRR) model to effectively utilize the biomechanical information and statistical information. Kernel Ridge Regression (KRR) model was first established from the training data which consisted of a set of preoperative and postoperative 3D images. When a new patient arrived, the KRR model was adjusted incrementally to incorporate the new patient’s biomechanical information. Compared to [6], our method combined different information, the statistical information and biomechanical information, into one model. Eleven patients were used for validation. The average prediction error of IKRR was found to be lower than other evaluated algorithms. Comparison of running time revealed that IKRR was more efficient than KRR.

2 Methodology

2.1 Data Acquisition and Pre-processing

Eleven sets of patient’s preoperative and postoperative CT scans and facial surface scans, obtained from a 3D surface camera, were acquired. The only reason of using facial surface scans was to prevent any unintended soft tissue strain during the CT scanning. The 3D camera was operated by a doctor who ensured the patient’s facial expression was neutral. During the computation, the CT soft tissues were replaced with the 3D surface scans. Both preoperative and postoperative surface scans were rigidly registered to the preoperative CT images with the Mimics software (Materialise, Belgium). The bones of preoperative and postoperative CT images were segmented in Mimics which would be further used to determine surgical plan.

2.2 Feature Extraction

Biomechanical properties, including stress, strain and displacement, were computed from FEM. We used stress as a feature. In order to execute FEM, the following two components were utilized: the mesh and the surgical plan.

Mesh Generation

A Visible Human Female Dataset was used to generate an anatomic detailed mesh as a template. From the CT data, the following muscles contributed in facial soft tissue deformation were segmented from the dataset: Buccinator, Depressor anguli oris, Depressor labii, Levator anguli oris, Levator labii, Levator labii alaeque nasi, Mentalis, Orbicularis oris, Zygomaticus major, Zygomaticus minor and Masseter [7]. The remaining soft tissue tissues between the skin and mucosa were considered as a homogenous material. In order to generate a mesh structure applicable to all the patients, the segmented structures were then export as Stereolithography (STL) files and subsequently imported into TrueGrid (XYZ Scientific Applications, Inc., Livermore, CA). Finally, a hexahedral block mesh of this dataset was generated. It served as a template to map the detailed anatomic structures to real patients.

For real patient data, the segmented CT bones and registered 3D surface scan were imported into TrueGrid as facial geometries. The facial landmarks of each patient were manually marked. A surface projection technique of TrueGrid can change the template shape into patient shape by matching the corresponding landmarks.

Determination of Surgical Plan

The postoperative skull was firstly manually registered to the preoperative one based on an unaltered part at cranium. Afterwards, the preoperative skull was osteotomized into pieces according to the postoperative CT. Then, the bony segments were separately aligned to the postoperative counterparts. The Iterative Closest Point (ICP) algorithm [8] was used to compute the displacement between the preoperative and postoperative skull parts. After finding the displacement of all skull parts, we get surgical plan.

Calculation of Stress with FEM

For each node, it had the following quantity:

$$\{\begin{array}{cc}\hfill \text{displacement}:\hfill & \hfill \mathit{u}={(u,v,w)}^{\mathrm{T}}\hfill \\ \hfill \text{stress}:\hfill & \hfill \mathbf{\sigma }={({\sigma }_{xx},{\sigma }_{yy},{\sigma }_{zz},{\tau }_{xy},{\tau }_{xz},{\tau }_{yz})}^{\mathrm{T}}\hfill \\ \hfill \text{strain}:\hfill & \hfill \mathbf{\epsilon }={({\epsilon }_{xx},{\epsilon }_{yy},{\epsilon }_{zz},{\gamma }_{xy},{\gamma }_{xz},{\gamma }_{yz})}^{\mathrm{T}}\hfill \end{array}$$ (1)

Linear FEM (LFEM), based on linear elasticity to characterize the deformation behavior of soft tissues, was used to calculate the stress of the node [9]. Since we were interested in facial appearance, only the nodes lying on the outer skin were selected. There were totally 2652 nodes, and each node had a stress vector of length six. We stacked the stress of the selected 2652 nodes together to form a vector σ_i ∈ ℜ¹⁵⁹¹² for the ith patient, i = 1, …, n. We called σ_i the feature of the ith patient.

2.3 Training Kernel Ridge Regression Model

The feature was served as input in regression model. The true displacement of the selected 2652 nodes was calculated from the preoperative and postoperative meshes. These nodal displacements were stacked together to form a vector u_i of length 7956. Given input-output pairs (σ_i, u_i) ∈ ℜ¹⁵⁹¹² × ℜ⁷⁹⁵⁶, i = 1, …, n, we could learn a prediction function f such that f(σ_i) ≈ u_i for each i .

KRR model was adopted [10]. This was a nonlinear regression model. The input was first embedded into a higher dimensional space H via a nonlinear mapping ϕ. Space H induced a kernel function which characterized the inner product in H and was given by the relation k(x, y) = ϕ^T (x) · ϕ(y), where x and y were in the input space. The kernel function adopted here was the widely used Gaussian kernel

$$k(\mathbf{x},\mathbf{y})=\text{exp}\left(-\frac{\Vert \mathbf{x}-\mathbf{y}{\Vert }^2}{2{\omega }^2}\right)$$ (2)

with the width ω > 0. KRR performed linear regression in H which was equivalent to performing nonlinear regression in input space. KRR assumed that the prediction function was of the form f (σ) = W^Tϕ(σ), where W was the coefficients to be determined. By minimizing an objective function

$$L(\mathbf{W})=\frac{1}{2}\sum _{i=1}^n\Vert {\mathbf{u}}_i-{\mathbf{W}}^{\mathrm{T}}\varphi ({\mathbf{\sigma }}_i){\Vert }^2+\frac{\lambda }{2}\mathrm{t}\mathrm{r}({\mathbf{W}}^{\mathrm{T}}\mathbf{W}),$$ (3)

where λ > 0, we could find solution

$${\mathbf{W}}^{*}=\mathrm{\Phi }{({\mathrm{\Phi }}^{\mathrm{T}}\mathrm{\Phi }+\lambda {\mathbf{I}}_n)}^{-1}\mathbf{U}$$ (4)

with Φ = (ϕ(σ₁), …, ϕ (σ_n)) and U = (u₁, …, u_n)^T.

For any σ, the prediction of KRR model could be expressed as

$$\mathbf{ũ}={\left({\mathbf{W}}^{*}\right)}^{\mathrm{T}}\varphi (\mathbf{\sigma })={\mathbf{U}}^{\mathrm{T}}{(\mathbf{K}+\lambda {\mathbf{I}}_n)}^{-1}\mathbf{k}(\mathbf{\sigma }),$$ (5)

where k(σ) = (k(σ₁, σ), …, k(σ_n, σ))^T, K = (k(σ_i, σ_j))_i,j, i, j = 1, …, n. In (5), the kernel function was sufficient for calculating the prediction. Therefore, it was not necessary to know the nonlinear mapping ϕ. This could reduce the computational complexity since we could avoid the operations in high dimensional space.

2.4 Prediction of Soft-Tissue Deformations with Incremental KRR Model

We incrementally modified KRR model by adding pair (σ̃, ũ_FEM) to the training set, where σ̃ was the stress for the new patient and ũ_FEM was the displacement computed from linear FEM. Compared with KRR, our method predicted the output with biomechanical information ũ_FEM. We called it Incremental KRR (IKRR).

From (5), we could compute the prediction of IKRR as

$$\mathbf{ũ}={\left(\begin{array}{c}\hfill \mathbf{U}\hfill \\ \hfill {\mathbf{ũ}}_{\mathrm{F}\mathrm{E}\mathrm{M}}^{\mathrm{T}}\hfill \end{array}\right)}^{\mathrm{T}}{\left(\begin{array}{cc}\hfill \mathbf{K}+\lambda {\mathbf{I}}_n\hfill & \hfill \mathbf{k}(\mathbf{\sigma ̃})\hfill \\ \hfill {\mathbf{k}}^{\mathrm{T}}(\mathbf{\sigma ̃})\hfill & \hfill k(\mathbf{\sigma ̃},\mathbf{\sigma ̃})+\lambda \hfill \end{array}\right)}^{-1}\left(\begin{array}{c}\hfill \mathbf{k}(\mathbf{\sigma ̃})\hfill \\ \hfill k(\mathbf{\sigma ̃},\mathbf{\sigma ̃})\hfill \end{array}\right)=t{\mathbf{ũ}}_{\mathrm{F}\mathrm{E}\mathrm{M}}+(1-t){\mathbf{ũ}}_{\mathrm{K}\mathrm{R}\mathrm{R}},$$ (6)

where the prediction of KRR ũ_KRR = U^T (K + λI_n)⁻¹k(σ̃), t = (e − λ)/e, e = k(σ̃, σ̃) + λ − k^T (σ̃)(K + λI_n)⁻¹k(σ̃). By using the positive semi-definiteness of K, we could proof that t ∈ [0, 1).

Equation (6) showed that the prediction of IKRR was a convex combination of the prediction of KRR and prediction of FEM. There were three major advantages in IKRR. First, IKRR was more general. It contained KRR as a special case by setting t = 0. Second, IKRR was more flexible. It combined two parts together, one from the KRR, the other from the FEM. The contribution of each part could be tuned by changing t. Finally, IKRR was more efficient. It did not need repetitive training when adding new training data. The computational complexity of KRR for training n+1 data was O(n³) (See (5)). However, the complexity of IKKR reduced to O(n²) by updating the results of KRR (See (6)).

2.5 Implementation Issues

One key point in statistical model was the corresponding relationship amongst all the data. Since all the meshes were generated from the same template, a natural corresponding relationship was established. The input of statistical model was normalized to have a zero mean and one standard deviation for each feature. The computations of FEM and statistical analysis were implemented in Matlab on a 64 bit Windows PC with 1.6GHz CPU and 24GB RAM. The regularization parameter λ and the width of Gaussian kernel ω were selected via grid search. The best values of the parameters were those that gave the best performance.

3 Results

3.1 Predictions with Different Number of Training Data

We tested different number of training data, from 6 to 10, to generate IKRR models. The prediction accuracy of these five IKRR was recorded. The difference between the prediction and ground truth was calculated as

$$E=\frac{1}{2652}\sum _{i=1}^{2652}\Vert {\mathbf{d}}_i-{\mathbf{d̃}}_i\Vert ,$$ (7)

where d_i was the true displacement of the ith node, d̃_i was the predicted displacement of the ith node. Table 1 showed the results.

Table 1.

Prediction difference v.s. number of training data

Number of training	6	7	8	9	10
E (mm)	0.7845	0.7817	0.7814	0.7811	0.7756

The above table clearly showed that the prediction was improved when more training data were available. The statistical information gained from the added training data was beneficial to the prediction of soft-tissue deformations.

3.2 Empirical Comparisons

We carried out leave-one-out cross-validation using eleven patients’ datasets. The algorithms to be evaluated included LFEM [9], KRR and IKRR. Table 2 tabulated the prediction difference as defined in (7).

Table 2.

B_max (mm) is the maximal skull displacement during surgery. E_LFEM, E_KRR and E_IKRR are the prediction differences of corresponding methods. Surgical plans are described in the second column: M represents mandible, X represents maxilla, A represents advance, B represents back, R represents right, L represents left.

Patient	Surgery	Bmax	ELFEM	EKRR	EIKRR
1	MB	11.6847	1.3241	2.9955	1.1963
2	MB+XB	6.1581	0.8054	1.3840	0.7756
3	MB	7.3624	1.2456	4.5722	1.0467
4	MB +XA	7.2343	0.7389	2.4249	0.7082
5	MR	8.3622	0.8567	1.6896	0.7870
6	MA	10.5866	0.9391	2.6338	0.8739
7	MB+XB	13.4578	1.1490	2.6418	0.9551
8	MB+XA	7.3898	0.8845	3.4110	0.8090
9	ML	10.5032	0.9978	2.3010	0.9783
10	MR	6.1608	0.9531	2.1136	0.9209
11	MB+XA	13.6617	0.9821	2.1602	0.9626
mean		9.3238	0.9888	2.5752	0.9103

KRR underperformed LFEM because of its lack of the new patient’s biomechanical information. As a biomechanical based model, LFEM provided accurate predictions [9]. However, IKRR method outperformed all other algorithms by combining the statistical information learned from the training data and the test patient’s biomechanical information. The results indicated that the test patient’s biomechanical information was critical to the prediction performance.

The visualization was achieved by using inverse distance weighted interpolation [11] (Figure 1). This patient underwent a surgery to setback the mandible (bilateral sagittal split osteotomies) and advance the maxilla (Le Fort I osteotomy). As shown in Fig. 1, IKRR produced more accurate visualization than LFEM. The lower lip was prominent in LFEM prediction. While the lower lip was aligned with upper lip in IKRR prediction, which accorded with the postoperative image.

Fig. 1.

(a) preoperative image (b) postoperative image (c) prediction of LFEM (d) prediction of IKRR

3.3 Computation Time

We compared the computational time of IKRR and KRR side by side. Once an initial KRR model was generated, re-training process could be achieved by two methods. The first was to recompute KRR entirely when a new patient was added as shown in (5). The running time of repetitive KRR was the elapsed time for the computation of (5) by replacing n with n+1. The second method was IKRR approach in which it only incrementally updated the existing KRR model as shown in (6). The running time of IKRR was the elapsed time for the computation of (6). The larger the n is, the more meaningful the comparison is. The experimental results in Table 3 clearly showed that IKRR was much more efficient than KRR for large number of training data.

Table 3.

Computation time (s) for repetitive KRR and IKRR. “Speedup” means the factor that IKRR gained in CPU time over KRR

Number of training	1000	2000	3000	4000	5000
KRR	0.8542	3.3293	7.6927	14.1571	22.6551
IKRR	0.0089	0.0203	0.0357	0.0477	0.0629
Speedup	96.0	164.0	215.5	296.8	360.2

4 Conclusions and Discussions

We applied IKRR for the soft-tissue-change simulation after maxillofacial surgery. Unlike previous purly biomechanical based FEM [9] and statistical based model [6], our model integrated the statistical information and biomechanical information together. The results empirically showed our method outperformed the others.

Possible future work is discussed. The IKRR approach is still in the experimental stage. In the future, a varity of preoperative and postoperative data with different types of deformities should be included in the training model. The limitation for the application is the deficiency of postoperative images. In contrast, the preoperative images is easier to obtain. To this end, we will investigate a semi-supervised learning approach to use two types of data: paired pre- and postoperative data, and purely preoperatively data. The statistical model learnt from both types of the data will be compared with the model which is only learned with the first type of the data. We will finally determine whether the performance would be improved by adding the second type of data into the training model.