Cortical Surface Strain Estimation Using Stereovision

Abstract

We present a completely noninvasive technique to estimate soft tissue surface strain by differentiating three-dimensional displacements obtained from optical flow motion tracking using stereo images. The implementation of the strain estimation algorithm was verified with simulated data and its application was illustrated in three open cranial neurosurgical cases, where cortical surface strain induced by arterial blood pressure pulsation was evaluated. Local least squares smoothing was applied to the displacement field prior to strain estimation to reduce the effect of noise during differentiation. Maximum principal strains (ε ¹) of up to 7% were found in the exposed cortical area on average, and the largest strains (up to ~18%) occurred near the craniotomy rim with the majority of ε¹ perpendicular to the boundary, indicating relative stretching along this direction. The technique offers a new approach for soft tissue strain estimation for the purpose of biomechanical characterization.

1 Introduction

Tracking cortical surface motion noninvasively using stereovision is a practical and important intraoperative imaging technique to compensate for brain shift in image-guided neurosurgery. Typically, cortical surface displacement data is obtained through a rigid or nonrigid registration between the reconstructed stereo surface and preoperative magnetic resonance images [1] or between two stereo surfaces acquired at different surgical stages [2]. Because the parenchyma is constantly subjected to complex loading and boundary conditions throughout surgery (e.g., from surgical intervention, confinement by the craniotomy, and changes in intracranial pressure), the brain often deforms nonrigidly within the field of view. Although considerable attention has been focused on tracking cortical surface displacements, little effort has been devoted to investigating the non-uniformity of their distribution that directly determines the tissue surface strain.

Tissue surface strain is an important indictor of inherent tissue compliance and has been investigated in studies of the biomechanics of the eye where principal strain of the scleral shell detected from a single digital camera was found to correlate inversely with intraocular pressure [3]. Because a more compliant material undergoes a larger strain under the same applied force, tissue surface strains directly reflect the level of tissue compliance, which is an important indicator of physiological state, especially in the brain (e.g., [4]). A comprehensive understanding of the state of tissue strain may also be clinically valuable when the risk of mechanical damage exists (e.g., in cases of herniation, tissue retraction, and decompressive craniectomy).

In this paper, we present a computational framework to estimate surface strain from tracked tissue motion using stereovision and illustrate its application in the context of open cranial neurosurgery. Specifically, we estimate dynamic cortical surface strains induced by arterial blood pressure pulsation after dural opening, which is commonly observed but has not been previously quantified. We focus on the essential computational aspects of the approach in order to establish a foundation for the technique in anticipation of future studies relating cortical surface strain to clinical factors such as brain compliance and/or risk of tissue damage.

2 Material and Methods

2.1 Cortical Surface Reconstruction through Stereovision

Acquisition of stereo image pairs was externally triggered using two charge-coupled device (CCD) cameras (image size of 768 × 1024; pixel resolution of approximately 50–100 μm; images are in red, green, and blue (RGB)) rigidly attached to a surgical microscope. The stereo image acquisition frame rate varied for different patients (4–10 frames per second (fps)) but remained relatively constant for a given subject. Techniques for stereo image calibration and reconstruction based on a pinhole camera model and radial lens distortion correction can be found, e.g., in [1], and are briefly outlined here for completeness. A 3D point in world space (X, Y, Z) is transformed into the camera image coordinates (x, y) using a perspective projection matrix:

$$\left(\begin{array}{c}x\\ y\\ 1\end{array}\right)=\left(\begin{array}{cccc}{\alpha }_x& 0& C_x& 0\\ 0& {\alpha }_y& C_y& 0\\ 0& 0& 1& 0\end{array}\right)\times \mathbf{T}\times \left(\begin{array}{c}X\\ Y\\ Z\\ 1\end{array}\right),$$ (1)

where α_x and α_y incorporate the perspective projection from camera to sensor coordinates and the transformation from sensor to image coordinates, (C_x, C_y) is the image center, while T is a rigid body transformation describing the geometrical relationship between the two cameras. A total of 11 camera parameters (6 extrinsic: 3 rotation and 3 translation; and 5 intrinsic: focal length, f, lens distortion parameter, k₁, scale factor, S_x, and image center, (C_x, C_y)) are determined through calibration using a least squares approach.

Stereo matching was facilitated by constraining the search for correspondence points along an epipolar line (defined as the projection of the optical ray of one camera via the center of the other camera following a pinhole model) through image rectification and subsequent intensity correlation and smoothness criteria defined in [1]. Each pair of correspondence points was transformed into their respective 3D camera space using the intrinsic parameters, and then transformed into a common 3D space using the extrinsic parameters. Together with their respective camera centers in the common space, two optical rays were constructed with their intersection defining the 3D location of the correspondence point pair.

2.2 Motion Tracking through Optical Flow

Because radial image distortion is effectively compensated for during rectification, rectified images of the left camera taken at times t and (t+1) were used to track surface displacement. Similarly to [3], displacements were tracked from continuously acquired RGB images using an optical flow algorithm subject to constraint:

$$I_x{u}_x+I_y{u}_y+I_t=0,$$ (2)

where I_x, I_y and I_t are the derivatives of RGB image intensity at spatio-temporal location (x, y, t) in the corresponding directions, and (u_x, u_y) denotes displacement in the x and y directions. We employed an algorithm proposed in [5] based on [6] in which a gradient constancy assumption (Eqn. 3) is applied to allow small variations in image gray values to determine displacement vectors:

$$\nabla I(x,y,t)=\nabla I(x+u_x,y+u_y,t+1).$$ (3)

In addition, a discontinuity-preserving spatio-temporal smoothness constraint was further applied to generate a piecewise smooth flow field. A multiscale approach starting with a coarse, smoothed image set is also used to ensure global minimization [6]. The computational cost to achieve full-field displacements for each pair of rectified images was approximately 70 sec. The computational efficiency can be further improved by down-sampling the input images (e.g., by selecting one of every three pixels in both directions for both images, leading to a computational cost of approximately 12 sec), which did not result in a significant change in displacement (<~5%). The start and end points of each displacement vector in image space were then transformed into 3D space through stereovision, and the corresponding displacement vector was generated by subtracting the corresponding point locations.

2.3 In-Plane Strain Estimation

In-plane strain was calculated numerically by differentiating the displacement field. Because of noise in the measured displacements, smoothing prior to differentiation is typical [7], and a point-wise local least squares fitting scheme was employed for this purpose. For each 3D point of interest, a linear plane was used to approximate displacements (u_x, u_y, u_z) from the neighboring set of points of size (2m+1)×(2m+1):

$$\begin{array}{l}{u}_x(i,j)=a_0+a_1{x}_{i,j}+a_2{y}_{i,j}+a_3{z}_{i,j},\\ u_y(i,j)=b_0+b_1{x}_{i,j}+b_2{y}_{i,j}+b_3{z}_{i,j},\\ u_z(i,j)=c_0+c_1{x}_{i,j}+c_2{y}_{i,j}+c_3{z}_{i,j},\end{array}$$ (4)

where i and j span from −m to m for a set of grid points; (x_i,j, y_i,j, z_i,j) is the corresponding 3D point location; while (a, b, c) are the polynomial coefficients to be determined. A large m tends to eliminate local variations in strain, while a small m may not sufficiently smooth strain estimation in certain regions. In this work, m was empirically chosen to be 4 as a reasonable trade-off.

A set of discrete surface points and their displacement vectors generated from motion tracking, stereovision, and smoothing were used to calculate in-plane strains. The computations were facilitated by triangulating the points into a set of finite elements to approximate element strains using linear shape functions. Because the brain surface is not coplanar, a single 2D coordinate system is not sufficient for representation. Instead, local 2D coordinate systems were constructed for each triangular element with the x-axis parallel to one edge, and the in-plane nodal displacements (u_i, u_j) were transformed into these coordinates (Fig. 1).

Fig. 1.

Schematic of transforming a triangular element in 3D coordinates (X, Y, Z) into a 2D local coordinate system (x, y) with the x-axis parallel to one edge

The strain of each element, ε (defined as {ε _x ε _y 2γ_xy}^T), was determined from:

$$\mathbf{\epsilon }=\left\{\begin{array}{c}\partial u_x/\partial x\\ \partial u_y/\partial y\\ \partial u_x/\partial y+\partial u_y/\partial x\end{array}\right\}=\left[\begin{array}{cc}\partial /\partial x& 0\\ 0& \partial /\partial y\\ \partial /\partial y& \partial /\partial x\end{array}\right]\times \left[\begin{array}{cccccc}{N}_1& 0& N_3& 0& N_5& 0\\ 0& N_2& 0& N_4& 0& N_6\end{array}\right]\times \{\mathbf{u}\},$$ (5)

where N_i (i=1:6) are the element shape functions for the six in-plane displacement components (Fig. 1), and {u} are the nodal displacements [8]. Since the strain components depend on choice of coordinate system, we evaluated system-invariant in-plane principal strains by forming a strain tensor for each triangular element by rearranging the strain components (assuming an infinitesimal strain theory):

$$\mathbf{\epsilon }=\left[\begin{array}{cc}{\epsilon }_x& {\gamma }_{xy}\\ {\gamma }_{xy}& {\epsilon }_y\end{array}\right].$$ (6)

The maximum and minimum components of the principal strain (denoted as ε¹and ε², respectively) and their directions were then obtained by computing the Eigen values and Eigen vectors of the strain tensor [9].

2.4 Simulated Displacement Data and Clinical Cases

To verify algorithmic implementation for computing principal strains, two simulated displacement fields were generated where each grid point, (x, y), had a displacement, (u_x, u_y), governed by Eqns. 7 (for simple stretch) or 8 (for simple shear):

$$u_x=x/20,\text{and}{u}_y=0,$$ (7)

$$u_x=y/20,\text{and}{u}_y=0.$$ (8)

The numerically computed principal strains were compared with their ground-truth counterparts (ε^1,2=(0.05,0) and ε^1,2=(0.025,−0.025) for stretch and shear, respectively).

For clinical data, stereo image sequences captured from three patients (an 18 year-old male with epilepsy, a 65 year-old male with glioblastoma, and a 61 year-old female with meningioma, respectively) after dural opening were used to evaluate the dynamic surface displacement and strain. Patient arterial blood pressure synchronized with stereo image acquisition was also recorded (at 1 kHz). For illustration, the first stereo image pair in each case was selected as the “baseline” from which cortical surface deformation was estimated by comparing with subsequent stereo image acquisitions. All data analyses were performed on a Linux computer (2.6 GHz, 8 GB RAM) using MATLAB (R2010b, The Mathworks, Natick, MA).

3 Results

The computed principal strains were plotted at the centroid of each element for the two simulated displacement fields (Fig. 2). Both magnitudes and orientations of the computed principal strains matched exactly with their ground-truth counterparts, suggesting a correct implementation of the algorithm for calculating principal strains.

Fig. 2.

The computed ε¹ (red) and ε² (blue) for the two simulated displacement fields (left: simple stretch in the x direction; right: simple shear)

Using clinical data, the rectified image corresponding to the largest relative displacement (on average) was overlaid on the baseline for patient 1 (Fig. 3a), where relative movements in two representative regions are shown (insets). The corresponding composite distribution of displacement components in the x and y directions was generated on the same rectified baseline image (Fig. 3b). A heterogeneous distribution of displacement components was evident that appears to correlate spatially with cortical areas defined by major vessels. Artifact due to specularity (arrow in Fig. 3b) was confined locally, demonstrating the robustness of the motion-tracking algorithm in estimating the overall displacement distribution.

Fig. 3.

A rectified image corresponding to the maximum relative displacement on average (green) overlaid on the baseline (red; a), showing distinctive motion patterns in two representative regions (insets; dominant motion represented by thick arrows). The resulting distribution of displacement components in the cortical area is also shown (b; displacement vectors are shown at one of every 9 points and are magnified by 10 times).

Cortical surface principal strains were computed by first transforming the locally smoothed displacements into 3D space through stereopsis. Representative distributions of ε¹ for patients 1 (Fig. 4a) and 3 (Fig. 4b) are overlaid on their corresponding cortical surfaces reconstructed in 3D. For all patients, the largest strains occurred in areas near the craniotomy rim (up to 0.18; patient 2 not shown). The majority of ε¹ in this region was perpendicular to the craniotomy boundary (Fig. 4), indicating relative stretching along this direction. The maximum average ε¹ across the cortical area for the three patients was 0.067, 0.038, and 0.047, respectively.

Fig. 4.

Representative distributions of ε¹ for patient 1 (a; 0.067 on average) and 3 (b; 0.047 on average). Directions of ε¹ are shown at every 3^rd point (magnified by 10 times), and a representative few are magnified further to enhance visualization (thick arrows).

Typical average displacement and ε¹ as a function of time are shown for patient 3 in Fig. 5, clearly demonstrating that the cortical surface motion was in concert with blood pressure pulsation at a frequency of approximately 1 Hz. The peak values of displacement and ε¹ as well as the normalized arterial blood pressure also varied in time, likely due to patient respiration (at a frequency of approximately 0.125 Hz).

Fig. 5.

Typical average displacement and ε¹ as a function of time for patient 3, demonstrating that cortical surface motion was in concert with blood pressure pulsation (normalized for visualization) at approximately 1 Hz. Stereo images were captured at a frequency of ~9.3 fps.

4 Discussion and Conclusion

An accurate and comprehensive understanding of the state of soft tissue strain is important in determining tissue physiological and biomechanical properties that could provide new information for patient care and/or biomechanical modeling. Using stereovision, we have developed a completely noninvasive approach to estimate cortical surface strain from motion tracking. Our technique is based on spatially smooth full-field displacements that allow strain to be estimated over a complete 3D tissue surface determined from stereovision rather than from a single camera that is typically used in digital image correlation (DIC) which is popular in experimental mechanics [7]. In addition, the high-resolution images also allow a high sensitivity in displacement tracking (within 0.05–0.1 mm), which is important for high precision in strain estimation. The sensitivity of displacement tracking and precision in strain estimation can be enhanced further by either increasing the camera image resolution or decreasing the field of view (e.g., through image magnification).

The application of the technique was demonstrated in three surgical cases where cortical surface motion due to arterial blood pressure pulsation was evaluated. Local displacement smoothing was first performed prior to differentiation to reduce noise. In addition, in-plane principal strains were calculated to characterize the 3D surface strain because they are invariant to the coordinate system selected. Results suggest that the dominant cortical surface strain occurred near the craniotomy rim (ε¹ up to ~18%), likely because of brain-skull friction when the parenchyma distended in this vicinity. When the brain otherwise sagged resulting in brain-skull separation (e.g., as a result of gravity), the stretching near the craniotomy boundary was not evident (e.g., see upper boundaries in Fig. 4a and b). While an accuracy evaluation is nearly impossible because of the difficulties in determining ground-truth under clinical conditions, the dynamic evolution of displacement and strain over time with the synchronized blood pressure pulsation (Fig. 5) suggests our technique estimates strain with high fidelity (although the values are relative because they are based on a baseline that does not necessarily correspond to the true zero-state of strain). Nonetheless, our studies would benefit from additional quantitative assessments conducted in more controlled simulated and experimental environments in the future. Because of the relatively large strains especially around the craniotomy rim, finite strain theory which is applicable for arbitrarily large rotations and strains may also be more appropriate, and will be incorporated in the future (difference of ~10% at the 0.2 strain level is expected relative to the infinitesimal strain theory used in this study).

In summary, we have demonstrated the application of a completely noninvasive technique to estimate cortical surface strain from motion tracking during open cranial neurosurgery. The technique provides sensitive detection of surface displacement and tissue strain under physiological conditions as long as the surface is sufficiently rich in tracking features. With this technique, it may be possible to correlate the level of surface strain with tissue compliance [3], which is an important clinical indicator; or to provide tissue deformation feedback during surgical operations (e.g., tracking tissue strain around a retractor blade), which is important for patient care in practice.