Reversible Projection Technique for Colon Unfolding

Abstract

Colon unfolding provides an efficient way to navigate the colon in CT colonography. Most existing unfolding techniques only computed forward projections. When radiologists find abnormalities or conduct measurements on the unfolded view (which is often quicker and easier), it is difficult to locate the corresponding region on the 3D view for further examination (which is more accurate and reliable). To address this, we propose a reversible projection technique for colon unfolding. The method makes use of advanced algorithms including rotation-minimizing frames, recursive ring sets, mesh skinning and cylindrical projection. Both forward and reverse transformations are computed for points on the colon surface. Therefore, it allows for detecting and measuring polyps on the unfolded view and mapping them back to the 3D surface. We generated realistic colon simulation incorporating most colon characteristics such as curved centerline, variable distention, haustral folds, teniae coli and colonic polyps. Our method was tested on both the simulated data and 110 clinical CT colonography data. Comparison of polyp size measurements on the unfolded view and the 3D view clearly demonstrates the importance of our reversible projection technique.

1 Introduction

The colon is a convoluted tubular structure. The traditional way to navigate a colon surface along its centerline is not efficient due to the limited view angle and furthermore polyps behind the haustral folds often get obstructed. A more efficient way is to open and unfold the colon and view the surface from above. Although this is not possible in optical colonoscopy, it can be implemented in CT colonography (CTC) via computer graphics techniques. Several investigations [1-3] had showed that colon unfolding could increase the field of view and improve the sensitivity of polyp detection.

Many techniques have been proposed to unfold the colon. They can be put in three categories. The first category is based on raycasting [4]. Vilanova et al. [5] unfolded the colon locally using a local projection and then globally mapped the colon using a suitable parameterization. Wang et al. [6] proposed a technique using the electrical field of a charged centerline to transform the colon into a straight cylinder-like shape and uniformly sample the planar cross sections orthogonal to the centerline. They then applied a raytracing technique to generate the unfolded view. Sudarsky et al. [7] presented an efficient method based on skeletal subspace deformation and uniform raycasting orthogonal to the central path. The colon unfolding is conducted in two steps. First the colon is straightened using mesh skinning based on the centerline, then the dissection is computed using the ray-casting technique. However, as some papers in this category [7] pointed out, it was difficult to choose the right sampling rate in raycasting-based techniques and structures behind the first object encountered by the rays may be obstructed. The second category is based on conformal texture mapping [8]. Haker et al. [9] used an angle-preserving conformal mapping to map the entire colon surface onto a flat plane. Hong et al. [10] proposed a conformal mapping based on minimizing harmonic energy to achieve angle preserving and minimum distortion. The 3D surface was mapped to a 2D rectangular map. Huang et al. [11] adopted similar conformal mapping approach in their effort to visualize teniae coli on the unfolded view. They demonstrated that the three teniae coli are parallel and evenly distributed. This observation can not be easily appreciated on the original 3D colon view. The techniques in this category have to deal with texture distortion and surface parameterization. They often require high quality surfaces and are computationally expensive. The third category is based on local projection. Paik et al. [2] proposed various map projection techniques including cylindrical and planar projection for the fly-through of virtual colonoscopy. They increased the view angle up to 360 degree. In some sense, their technique essentially computed a local unfolded view. Vos et al. [12] projected six orthogonal images onto an unfolded cube to render the complete field of view. This was also a local unfolding but introduced extra distortion. Hoppe et al. [13] proposed a virtual colon dissection by placing a virtual camera oriented perpendicular to the centerline. The camera displays the inner colon surface with 90° field of view and rotates in 45° increments around the centerline. This technique has trouble seeing polyps behind a fold, and moreover different view panels stitched together create discontinuity.

All the above mentioned colon unfolding techniques only supported forward mapping. That is, only the projection from 3D colon surface to 2D unfolded plane is provided, not the reverse one. In the raycasting based techniques, multiple points may lay in the path of the same casting ray, which causes ambiguity in the reverse projection. In the conformal mapping technique, the reverse transformation is extremely difficult or not available due to the complex transformation. Projection based techniques are the only feasible ones supporting reverse mapping. Without the reverse mapping, when radiologists find abnormalities or conduct measurement on the unfolded view (which is often quicker and easier than on the 3D view), it is difficult to locate the corresponding region in the 3D view for further confirmation and examination (which is more accurate and reliable). To address this problem, we propose a reversible projection technique for efficient colon unfolding. The novelty of our technique and the main contributions are: 1) a recursive ring set method to resolve the centerline and surface correspondence problem (section 2.2); 2) reverse transformation from the unfolded view to the 3D view (section 2.4), and 3) validate the technique using realistic colon simulation and polyp measurement on clinical CTC data (section 2.5).

2 Methods

Our method is summarized as follows. Given a CT colonography data set, the 3D colon surface is first segmented using thresholding, region growing and level sets [14]. Then the centerline of the colon is extracted based on fast marching level set and topographical thinning [15]. Then rotation-minimizing frames (RMF) are established along the centerline. After that, a recursive ring set technique is applied to map vertices on the colon surface to their corresponding centerline points. Next, mesh skinning is employed to straighten the colon. After that, cylindrical projection is applied to unfold the colon. Finally, reverse transformation is computed for every vertex. The flow chart is illustrated in Figure 1.

Figure 1. Method overview.

2.1 Rotation-minimizing frame (RMF)

The colon centerline is a 3D spatial curve in the center of the colon. The centerline is represented as a sequence of 3D points from rectum to cecum. We first fit a 5^th degree B-spline curve to the centerline. The B-spline curve utilizes the original centerline as control points and has explicit forms to compute tangent, normal and binormal for every point along the curve [16]. We then discretize the centerline at 0.5 mm resolution (which is approximately the same resolution as the CT data set). The Frenet-Serret frame (FSF) is a common tool to study the differential geometry of spatial curves [17]. The FSF of a given curve c(u) is defined by the tangent t⃗(u), normal n⃗ (u) and binormal b⃗ (u) vectors of the curve,

$$\overrightarrow{t}(u)=\frac{\dot{c}(u)}{\Vert \dot{c}(u)\Vert },\overrightarrow{b}(u)=\frac{\dot{c}(u)\times \ddot{c}(u)}{\Vert \dot{c}(u)\times \ddot{c}(u)\Vert },\overrightarrow{n}(u)=\overrightarrow{b}(u)\times \overrightarrow{t}(u)$$ (1)

here c(u) is one point on the curve, ċ(u) is the first degree derivative and c̈(u) is the second degree derivative. However, the FSF is not the best choice for our application since it may exhibit strong rotation around the tangent vector. A so-called rotation-minimizing frame (RMF) is more appropriate [17]. The RMF is formed by the tangent vector t⃗(u) and two unit vectors f⃗₁(u) and f⃗₂ (u)spanning the normal plane at c(u). f⃗₁(u) and f⃗₂ (u) rotate as little as possible around t⃗(u) between neighboring frames. They can be derived from the FSF,

$$\begin{array}{c}{\overrightarrow{f}}_1(u)=sin\mathrm{\Omega }(u)\overrightarrow{b}(u)+cos\mathrm{\Omega }(u)\overrightarrow{n}(u)\\ {\overrightarrow{f}}_2(u)=cos\mathrm{\Omega }(u)\overrightarrow{b}(u)-sin\mathrm{\Omega }(u)\overrightarrow{n}(u)\end{array}$$ (2)

here Ω is the angle difference of two neighboring frames,

$$\mathrm{\Omega }(u)=-{\int }_0^u\tau (t)\Vert \dot{c}(t)\Vert \mathit{\text{dt}}$$ (3)

here τ = det{ċ,c̈,c⃛}/‖ċ×c̈‖² is the torsion of the curve. The proof that the RMF M(u) = {t⃗(u), f⃗₁(u), f⃗₂(u)} is rotation-minimizing along the centerline can be found in [17]. Figure 2 illustrates the FSF and RMF of one centerline. In the following description, we use c_j to represent the j^th point on the centerline and M_j as its RMF.

Figure 2. Frenet-Serret frame (FSF) and rotation-minimizing frame (RMF).

Left: FSF, blue: b⃗(u), green: n⃗(u)

Right: RMF, blue: f⃗₁(u), green: f⃗₂(u)

2.2 Recursive ring sets

Once the RMF is established, every vertex on the colon surface is associated to its corresponding centerline point and a local RMF. The methods in [5, 7] rely on distance map for this purpose. A distance map stores the distances of vertices to their nearest centerline points and is often computed by Euclidean distance transformation [18]. However, in 3D colon the distance map approach may fail to find the right correspondence in highly curved regions. For instance, in Figure 3a, c₁ is the topological corresponding centerline point of ν₁. However, c₂ is closer to ν₁ than c₁ in terms of Euclidean distance. Since c₁ and c₂ have different RMFs, the mis-correspondence will cause incorrect transformation in the unfolding process. To resolve this problem, we propose a recursive ring set technique that can establish a topology-preserving correspondence. The concept of ring set was also adopted in the centerline computation in [19], where two parallel planes perpendicular to the local centerline segment were used to generate the a ring set. However, the method in [19] is computational expensive since each vertex needs to be tested for all ring sets. We propose a ring set technique based on recursive refinement to obtain topology-preserving correspondence between the colon surface and its centerline.

Figure 3. Recursive ring sets.

a) Method scheme (solid line: colon wall, thick dashed line: centerline, thin dashed line: ring sets), b) ring sets associated with the centerline (color strips).

The recursive ring set technique is summarized as follows. It is a four-step procedure. In the first step, for every surface vertex ν_i, the closest centerline point c_j based on Euclidean distance is recorded as its initial corresponding centerline point, i.e.

$$\mathit{\text{CL}}({\nu }_i)=\underset{c_j}{arg\; min}(\Vert {\nu }_i-c_j\Vert )$$ (4)

here CL(ν_i) represents the corresponding centerline point for ν_i. A BSP (binary space partition) tree [20] of the centerline is created to accelerate the searching. The searching time is therefore reduced from O(N) to O(logN), where N is the number of points on the centerline.

In the second step, a ring set R_j is created for each centerline point c_j. The initial set for R_j includes all vertices {ν_i} whose closest centerline point is c_j, i.e.,

$$\forall {\nu }_i,\mathit{\text{if}}\mathit{\text{CL}}({\nu }_i)=c_j,\mathit{\text{then}}{\nu }_i\in R_j,$$ (5)

In the third step, the ring sets are recursively updated along the centerline. A connected component analysis is conducted for each ring set R_j. Two vertices are considered connected if there is an edge joining them. The largest connected surface patch remains in the ring set, and the rest are removed from R_j and put in a refinement set Q.

In the fourth step, for every point ν_k in set Q, its closest neighbor ν_m not in set Q is retrieved, i.e.,

$${\nu }_m=\underset{{\nu }_m,{\nu }_m\notin Q}{arg\; min}(\Vert {\nu }_k-{\nu }_m\Vert )$$ (6)

and ν_m′s corresponding centerline point is assigned to ν_k, i.e. CL(ν_k)=CL(ν_m). The process is repeated until Q is empty. Step 3 and 4 are repeated until there is only one ring set for every centerline point.

The scheme is illustrated in Figure 3a. Here ν₁ is in the ring set of c₁, so CL(ν₁)=c₁. Figure 3b shows the recursive ring sets associated with the centerline (each color strip indicates one ring set).

2.3 Mesh skinning

Mesh skinning [21] is a skeleton-driven deformation technique widely used in computer animation. We apply this technique to straighten the colon using its centerline as the skeleton and the colon surface as the skin. Here we use a similar technique as the one in [7].

This is a two-step procedure. In the first step, the centerline is deformed to form a straight line along the z axis. The straightened centerline is represented as,

$$\begin{array}{c}{c_j}^{\prime }.x={c_j}^{\prime }.y=0\hfill \\ {c_j}^{\prime }.z={c_{j-1}}^{\prime }.z+\Vert c_j-c_{j-1}\Vert \hfill \end{array}$$ (7)

here c_j and c_j′ are the coordinates in the original and straightened centerlines respectively. In the second step, the vertices on the colon surface are transformed using the RMF associated with their corresponding centerline point CL(ν_i). The transformation is written as,

$${{\nu }_i}^{\prime }={c_j}^{\prime }+M_j^{-1}({\nu }_i-c_j)$$ (8)

here c_j = CL(ν_i) is the corresponding centerline point of ν_i, M_j is the RMF of c_j. In order to achieve a smooth transformation, we adopt the strategy in [7] to include a neighborhood of c_j in the calculation. The new transformation is then updated as,

$${{\nu }_i}^{\prime }=\sum _{k=j-\delta }^{k=j+\delta }{w}_{i,k}({C_k}^{\prime }+M_k^{-1}({\nu }_i-c_k))$$ (9)

here δ is the neighborhood range (default value is 3), w_i,k is the weight which is defined to be inversely proportional to the distances d_i,k between vertex ν_i and centerline point c_k, i.e.,

$$\begin{array}{c}{w}_{i,k}=\frac{{d_{i,k}}^{-1}}{\sum _{j=k-\delta }^{j=k+\delta }{d_{i,k}}^{-1}}\hfill \\ d_{i,k}=\Vert {\nu }_i-c_k\Vert \hfill \end{array}$$ (10)

Figure 4 shows the straightened colon using mesh skinning and one close-up view.

Figure 4. Colon straightening using mesh skinning.

a) Original 3D colon, b) straightened colon, c) close-up view of the straightened colon.

2.4 Reversible projection and colon unfolding

Cylindrical projection is widely used in geographical map projection to unfold the earth onto a 2D map [22]. In the cylindrical map projection, the meridians are mapped to equally-spaced vertical lines and circles of latitude are mapped to horizontal lines. One trait of cylindrical projection is its ability to preserve distance on the surface. Similarly, we can unfold the straightened colon using the cylindrical projection. In analogy to geographic map, the colon centerline is the analogue of the Earth's axis of rotation.

The cylindrical projection maps every point on the straightened colon onto a plane at y=0. We preserve the distance-to-centerline function for every vertex by mapping it as the height elevation above the plane. The projection and unfolding are written as,

$$\begin{array}{c}{\stackrel{\sim }{\nu }}_i.z={{\nu }_i}^{\prime }.z\hfill \\ {\stackrel{\sim }{\nu }}_i.y=\sqrt{{({{\nu }_i}^{\prime }.x)}^2+{({{\nu }_i}^{\prime }.y)}^2}\hfill \\ {\stackrel{\sim }{\nu }}_i.x=arctan({{\nu }_i}^{\prime }.y,{{\nu }_i}^{\prime }.x)\times {\stackrel{\sim }{\nu }}_i.y\hfill \end{array}$$ (11)

here ν˜_i is the coordinate of vertex ν_i on the unfolded colon, ν˜_i.z is the distance along the centerline, ν˜_i.y is the distance to the centerline, and ν˜_i.x is the arc length around the centerline. We cut open the colon at 0° meridian when unfolding the colon. An edge (ν_i, ν_j) is removed from the unfolded colon if it is across 0° meridian, i.e. arctan(ν_i′.y, ν_i′.x)*arctan(ν_j′.y, ν_j′.x)<0. All other edges are retained. Therefore the unfolded colon is essentially a 3D surface. Since there is one-to-one correspondence between the 3D colon and the unfolded one, we not only preserve the topology of the original colon surface but also the vertex resolution. Figure 5 shows the unfolded colon and two close-up views. Our method can handle totally collapsed colon segments.

Figure 5. Unfolded colon.

a) Entire unfolded colon, b,c) close-up views. One section near the rectum is collapsed (arrow).

Furthermore, the projection from the 3D colon to the unfolded colon is reversible. The reverse projection of Eq. 11 can be computed in two steps (Eq. 12 and Eq. 13). Given a coordinate ν˜_i in the unfolded colon, the coordinate in the straightened colon ν_i′ can be obtained as,

$$\begin{array}{c}{{\nu }_i}^{\prime }.x={\stackrel{\sim }{\nu }}_i.y\times cos(\frac{{\stackrel{\sim }{\nu }}_i.x}{{\stackrel{\sim }{\nu }}_i.y})\hfill \\ {{\nu }_i}^{\prime }.y={\stackrel{\sim }{\nu }}_i.y\times sin(\frac{{\stackrel{\sim }{\nu }}_i.x}{{\stackrel{\sim }{\nu }}_i.y})\hfill \\ {{\nu }_i}^{\prime }.z={\stackrel{\sim }{\nu }}_i.z\hfill \end{array}$$ (12)

We can further compute the reverse transformation from the straightened colon to the original 3D colon using the following equation,

$${\nu }_i=c_j+M_j({{\nu }_i}^{\prime }-{c_j}^{\prime })$$ (13)

Combining (12) and (13), a point ν˜_i on the unfolded colon can be reversely projected back to a point ν_i on the original 3D colon. The reverse transformation allows a user to locate and measure lesions on the unfolded colon and then map them back to the 3D colon for confirmation. This provides an easier and quicker way than performing the measurement task directly on the 3D colon.

2.5 Algorithm complexity

Our technique is computational efficient. We conduct a complexity analysis by breaking down to the three main parts of the algorithm: recursive ring sets, mesh skinning and reversible projection.

In the following analysis, N is the number of centerline points and V is the number of colon vertices. V is much greater than N in our analysis.

In the recursive ring sets stage, step 1 requires every colon vertex to search for its closest centerline point, the time complexity is O(V logN). Step 2 needs to locate the ring set for every vertex (O(V)). Step 3 involves connected component analysis for every ring set. The complexity is O(N * K logK), where K is the average number of vertices in a ring set and K=V/N. Therefore the complexity in step 3 can be further simplified as O(V log (V/N)). Step 4 is the rearrangement of vertices in set Q, the complexity is O(Q), where Q≪V. Therefore is the complexity for the recursive ring set is O(V logN)+O(V)+O(V log(V/N)+O(Q) = O(V logN).

In the mesh skinning stage, step 1 is the straighten of the colon centerline, the complexity is O(N). Step 2 is the mesh skinning of colon vertices, the complexity is O(V). The complexity for mesh skinning is then O(N)+O(V)=O(V).

In reversible projection stage, every vertex on the colon is projected to a flat plane. The complexity is O(V).

Therefore, the overall complexity of the entire algorithm is

$$\mathrm{O}(VlogN)+\mathrm{O}(V)+\mathrm{O}(\nu )\mathrm{V}\mathrm{O}(VlogN)$$ (14)

2.6 Colon simulation and CTC data

Our colon unfolding technique is evaluated using both simulated colon data and clinical CTC data.

We created a realistic colon simulation using the mesh skinning technique in three steps. First, a digital phantom of a cylindrical tube model with haustral folds and teniae coli is built (Figure 6a). Haustral folds are added in the form of indentations onto the tube. The folds are modeled as a series of trisected torus with elliptic cross-sections. The trisections are partitioned equally in circumferential location at 0, 120 and 240 degrees on the cross-section plane on the digital phantom. The spaces between the trisection are in accordance with the appearance of teniae coli. Second, the centerline from a clinical CTC is computed and employed. Every point on the centerline is equipped with a rotation-minimizing frame (Section 2.1) and its associated distention value computed from the CTC data. Third, mesh skinning (Section 2.3) is applied to warp the tube phantom around the centerline and generate realistic colon simulation. Simulation of colonic polyps, modeled as semi-ellipsoids, can also be added to the phantom. Figure 6b shows the surface of the colon simulation. Figure 6c shows the surface of a real colon whose centerline and distention data is employed to generate the simulation in Figure 6b. The simulated colon highly resembles the real one. To our best knowledge, this is the first colon simulation that incorporates most colon characteristics in one model, including curved centerline, variable distention, haustral folds, teniae coli and colonic polyps.

Figure 6. Colon simulation.

a) digital phantom of a cylindrical tube with haustral folds and teniae coli, b) surface of a simulated colon, c) surface of a real colon whose centerline and distention are used in b).

The clinical CTC data are acquired in three institutions. Each patient was scanned in both supine and prone positions. CT scanning parameters included 1.25- to 2.5-mm section collimation and 1-mm reconstruction interval. Each data set has more than 400 slices on average. To evaluate the quality of the unfolded colon, an expert visually examined every case. An unfolded colon is judged as satisfactory if the colon features such as haustral folds and polyps are preserved and can be easily identified. We also recorded the computation time for the unfolding process.

2.7 Polyp size measurement on unfolded colon

Estimating the size of a polyp is essential in CTC diagnosis since large polyps are more likely to become malignant and cancerous. To explore the reverse transformation brought by our technique, we conduct and compare the polyp measurement on both the 3D view and the unfolded view. Here the polyp size is defined as the largest diameter across the polyp determined by the operator. In this experiment, the operator first conducted the measurement on the unfolded surface and recorded the two endpoints. The two endpoints were then mapped to the 3D surface for additional measurement. Figure 7 illustrates a polyp on both the 3D view and the unfolded view.

Figure 7. Polyps in 3D and unfolded view.

a) 3d view, b) unfolded view, Green: polyp, red: a neighborhood

3 Results

We ran tests on the simulated colon to validate the capability of our unfolding technique in preserving the location, orientation and size information. Haustral folds were used as the landmarks. Figure 8 shows the unfolded view of the colon simulation and the ground truth of the unfolded view. We located the two endpoints of the middle folds on both the unfolded view and the ground truth view. We then compared the difference of the fold location (midpoint of two endpoints), orientation (direction of two endpoints) and size (distance between two endpoints). Table 1 summarizes the statistics of the comparison for 49 folds. We were able to achieve sub-voxel accuracy (voxel size is 1mm³)

Figure 8. Unfolded view of simulated colon and ground truth.

Table 1. Fold comparison between unfolded view (UV) and ground truth (GT) for the simulated colons (N=49).

|UV-GT|	Location	Orientation	Length
Mean	0.42 mm	0.45°	0.43 mm
Stdev	0.29 mm	0.47°	0.07 mm

Our method was then tested on 110 clinical CTC studies. All unfolded colons were determined visually acceptable. The average computation time on a 3.0 GHz Pentium processor with 4GB RAM was 14.9 ± 3.8 seconds. Currently our algorithm is not optimized for speed. Figure 5 shows one example of the unfolded colon. Our method succeeded in unfolding the collapsed section near the rectum.

We conducted dimensional measurements for 103 colonic polyps on both the 3D colon surface and the unfolded surface. Figure 9 shows the scatter plot of the two sets of measurements on the 3D vs. unfolded views. The two measurements show high-degree correlation. The equation for the linear regression is y=1.07x, which indicates the unfolded view tends to give larger measurements when the colon is straightened. Our reversible projection provides an efficient and reliable polyp measurement tool. The operator can conduct the measurement on the unfolded view (which is often quicker and easier to do), and then reversely project the endpoints to the 3D view for verification and generation of the 3D measurement.

Figure 9. Polyp size measurement on 3D surface and unfolded view.

4 Discussion and Conclusion

Both our method and the one described in [7] used the mesh skinning technique to straighten the colon and were much more efficient than other prevalent methods. However, our method shows several improvements over the one discussed in [7]. First and foremost, we developed a novel recursive ring set technique to find the topology-preserving correspondence between the centerline and vertices on the colon surface. It is more robust and does not rely on the distance map computed from the original CT data. Secondly, we do not use raycasting. We use a cylindrical projection instead to map the vertices to the rotation-minimizing frames along the centerline so that it preserves the resolution and topology of the 3D surface and avoids the obstruction problem. Thirdly, our method supports reverse transformation from the unfolded colon to the original 3D colon, which allows us to conduct the polyp measurement and detection on the unfolded view. Finally, our technique maps each vertex onto the unfolded plane instead of sampling the centerline and surface, so that we can avoid the undersampling (missing polyps) and oversampling (double polyps) problems that occur in some other methods [5]. Though Paik et al. [2] adopted a similar cylindrical projection in their virtual colon fly-through to increase the view angle; the technique was limited to the local view field and their projection was not reversible.

In conclusion, we presented a computationally efficient reversible projection technique for colon unfolding based on recursive ring set formulation, mesh skinning and cylindrical projection. The above techniques have their origins in diverse areas like computer animation and geographical mapping. The proposed method is quite robust and works for all colon segments including poorly distended ones. The method has been tested on realistic colon simulation and 110 clinical data sets. It achieved sub-millimeter level accuracy in the simulation data and generated satisfying results in the clinical data. The technique supports polyp measurement on the unfolded view, an essential feature for virtual colonoscopy.

In the future work, we are investigating effective ways to integrate our technique in colon navigation and computer aided detection. We also plan to apply the realistic colon simulation in other research areas, such as supine and prone colon registration, tenia coli detection and colon segmentation.