Anatomic Surface Reconstruction from Sampled Point Cloud Data and Prior Models

Abstract

In this paper, we propose an approach for reconstruction of an anatomic surface model from point cloud data using the Screened Poisson Surface Reconstruction algorithm, which requires a collection of points and their normal vectors. Various algorithms exist for estimating normal vectors for point cloud data; however, in this work we describe a novel approach to estimating the normal vectors from a high-resolution prior model. In many medical applications, a preoperative high-resolution scan is acquired for diagnostic and planning purposes, whereas intraoperative, lower fidelity imaging is utilized during the procedure. This approach assumes an already existing registration between intra-operatively acquired data and the preoperative model. We conducted simulation experiments to evaluate the effect of registration error, point sampling rate, and noise levels on the acquired point cloud data samples. In addition, we evaluated the effect of using both the closest point, as well as a neighborhood of closest points on the prior model for estimating the normal. Our results showed that surface reconstruction error increases with higher registration error; however, acceptable performance was achieved with clinically-acceptable registration error. In addition, the best reconstruction was obtained when estimating the normal using only the closest point on the prior model, as opposed to utilizing a neighborhood of points. When combining the effect of all factors (Gaussian sampling noise of zero mean and σ =1.8mm; Gaussian translational error of zero mean and σ=2.0mm; and Gaussian rotational error of zero mean and σ=3°) the overall RMS reconstruction error was 0.88±0.03mm.

Introduction

In many medical applications, a high-resolution CT or MRI scan is acquired as a baseline scan prior to treatment or intervention and real-time, lower fidelity imaging is utilized during the treatment or intraoperative procedure. For example, in multi-time point studies, a high-resolution scan may be initially obtained and then lower resolution scans are acquired throughout the course of the therapy. In image guidance applications, a pre-operative scan is often acquired prior to treatment, and real-time, lower resolution images such as fluoroscopy, ultrasound, or cone-beam CT are used to guide treatment. It is often desirable, however, to have access to a high fidelity anatomical model during treatment for planning integration, assessment or treatment guidance. Moreover, it is beneficial to merge the pre- and intra-operative data to create fused, high spatial and temporal resolution models. In this paper, we propose an approach for reconstructing a surface from sampled point cloud data using information from a prior high-fidelity surface model. This technology could be used in any application which entails anatomic reconstruction from limited-resolution intra-operative for which a prior model exists.

Surface reconstruction from a point cloud data is a classic research topic in the field of computer graphics. Here, we utilize the Screened Poisson Surface Reconstruction (SPSR) technique as it enables fast, high-quality surface reconstruction with relatively low memory usage [1,2]. SPSR assumes both the position of the points and the normal vectors of the reconstructed object surface are known. However, in real-world clinical applications, often the normal vectors of the scanned object are unknown. Various algorithms have been proposed for estimating the normal vector directly from a point cloud [3,4]; however, this still remains a challenging problem, as inaccurate or inconsistent normal vectors (Figure 1a) have a major impact on the quality of the reconstruction (Figure 1b).

Figure 1.

Inconsistent normal vectors and their effect on reconstruction. a) The blue lines indicate the estimated normal vectors of each point which are shown as white dots. A collection of points (marked by the red rectangle) have inconsistent normal vectors (pointing outward). b) Inconsistent normal vectors resulted in a swollen and manifold structure in the reconstructed surface (marked by red rectangle). c) The prior surface model built from preoperational CT data (human left atrium).

To address the problem of normal vector estimation, we propose an alternative approach which uses prior model data, with the assumption that the prior model and the sampled points are in registration. If a prior model exists and it is already registered to the point cloud data, the normal vectors can be estimated more rapidly and more accurately by leveraging the prior model rather than relying on the low resolution point cloud. However, registration errors are common in clinical applications, point cloud sampling may be noisy, and some of the sampled points may be sparse. To assess the robustness of the proposed algorithm across these variables, simulation experiments are performed.

Methods & Materials

The two main steps of the surface reconstruction algorithm are as follows: (1) estimation of the normal vector from the prior model, and (2) surface reconstruction using SPSR. Since the vertex connectivity of the prior model is known, the computation of the normal vectors for the prior surface is straightforward. Figure 2 is the schematic diagram illustrating how the normal vectors of the sampled point cloud are estimated from the prior model in a simplified 2D case. For each point of the point cloud, its k-th nearest neighbors in the prior model are searched using a k-dimensional tree structure, with the average computation complexity defined as O (log n), where n is the number of vertices in the prior model. The normal vector of each vertex in the sampled point cloud is estimated by averaging the normal vectors of its k nearest neighbors from the prior model. The surface model is then reconstructed via the SPSR algorithm, using the estimated normal vectors and the coordinates of sampled points.

Figure 2.

Schematic diagram of normal vector estimation approach. S₁ denotes the prior model with known connectivity and normal vectors for each vertex. S₂ denotes the point cloud. For each point p from S₂, K nearest neighbors q₁, q₂, …, q_K from S₁ were detected. The normal vector of p was estimated by averaging normal vectors of q₁, q₂, …, q_K.

Due to factors such as sampling rate, sampling noise, and registration errors (both translational and rotational errors), there will likely be shape and position differences between the sampled point cloud and prior model. In addition, the chosen k value may also affect the estimated normal vectors and, in turn, the surface reconstruction quality. To study these effects, we conducted simulation experiments using volumetric CT left atrial data to build the prior surface model (Figure 1c). This model contains 10578 vertices and 21184 faces, with surface area of 7595.70 mm² and vertex density of 1.39 vertices/mm². The normal vector for each surface point was computed using the Meshlab [5] software. Using the high-resolution model as reference, several simulated datasets were generated.

• The prior model was randomly sub-sampled at a sample rate from 95% to 5%, with a step of 10%. We also tested the reconstruction error using all vertices of the prior model. Since the sampled point cloud was exactly part of the prior model, it was reasonable to set k = 1. After this experiment, we chose one sub-sampled data set at 35% sampling rate, as shown in the “Results and Discussions” section.

• To test the effect of sampling noise, we added Gaussian noise with zero mean and different standard deviations (0.5mm, 1.0mm, and 2.1mm, which is equivalent to 0.5%, 1%, and 2% of the bounding box diagonal length of the prior model respectively) to the test point cloud data set and then measured the reconstruction errors.

• To test the effect of translational registration error, we moved the point set along X axis for different increments (−2.1, −1.0, −0.5, 0.5, 1.0, 2.1 mm, which is equivalent to 2%, −1%, −0.5%, 0.5%, 1%, and 2% of the bounding box diagonal length of the prior model respectively) and then measured reconstruction errors.

• To test the effect of rotational registration error, we rotated the point set around the center of mass of the prior model along the X axis for different angles (−15°, −10°, −5°, −3°, −1°, 1°, 3°, 5°, 10°, and 15°) and then measured the reconstruction errors.

To evaluate the effect of the k value, each of the above experiments (except the first one) was carried out four times for k = {1, 4, 8, 12}. Finally, the effect of the combined factors was tested. Figure 3 shows the flow chart of the combined factors experimental design. First, random noise (zero mean Gaussian noise, σ=1.8mm) was added to the test data set. Next, the test dataset was randomly translated (zero mean Gaussian noise, σ=2.0mm), rotated (zero mean Gaussian noise, σ=3.0°), and finally estimated with different k values (k = 1, 4, 8, 12). This test was repeated 20 times to obtain a statistical measurement of reconstruction errors.

Figure 3.

Flow chart of simulation experiment to test effect of combined factors on surface reconstruction quality.

The surface reconstruction error was measured using the Metro tool [4], which adopts an approximate approach based on surface sampling and the computation of point-surface distances. The surface of the prior model was sampled with a sufficiently fine step size of 0.1% of the bounding box diagonal. For each sampled point, the reconstructed error was computed as its distance to the surface reconstructed from the point cloud. The maximum error and mean RMS error indicate the quality of surface reconstruction.

Results & Discussions

Figure 4 show the RMS mean reconstruction errors of each single factor experiments. As demonstrated in Figure 4(a), there is a negative relationship between the reconstruction error and sampling rate. In particular, when the sampling rate is smaller than 35%, there is a steep increase in the reconstruction error. For this reason, we selected the 35% subsampled point set as the test data for the other factors. The vertex density of the 35% sub-sampled point set was 0.49 vertices/mm².

Figure 4.

Reconstruction errors. a) Errors caused by sample rate. b) Errors caused by sample noise. c) Errors caused by translational error. d) Error caused by rotational error.

As shown in Figure 4(b), as noise levels increase, the RMS mean reconstruction error also increases. However, increasing the value of k did not improve reconstruction quality. The best reconstruction was obtained for k = 1, i.e. when using the nearest neighbor’s normal vector in the prior model as the estimation of normal vector of the sampled point cloud. Figure 4(c) and 4(d) demonstrate that as translation and rotation angles increase, reconstruction errors also increase. In addition, if we translate or rotate around the X axis in the opposite direction, the reconstruction errors were very similar as we translate or rotate in the positive X direction. Again, the best reconstruction is obtained when K equals one.

Table 1 shows the results of the combined factors experiment. Using a 35% sub-sample rate, zero mean, σ=1.8 mm Gaussian sampling error in X, Y, Z coordinates, zero mean, σ=2.0 mm Gaussian translational error in X, Y, Z directions, zero mean, σ =3° Gaussian rotational error along X, Y, Z axes, the RMS mean reconstruction error was 0.88±0.03 mm (K=1). As seen in Figure 5, the largest errors are in the branch-like vascular structures rather than the left atrial body. Moreover, the reconstructed vascular surface is enlarged, indicating that the fine structures are more subject to the sampling noise and registration errors. This effect can be seen in Figure 6, which shows the distribution of reconstruction errors. The largest errors (colored in red) appear along the vascular structures. We speculate two reasons for this effect: (1) as the distance to the mass center increases, the difference from the prior model caused by rotational error also increases, (2) the vascular structures are much thinner than the left atrial body, thus their shape changes more rapidly and are subject to greater registration error.

Table 1.

Reconstruction errors of combined factors test (unit: mm)

	K=1	K=4	K=8	K=12
RMS	0.88 ± 0.03a	0.98 ± 0.02b	1.04 ± 0.02c	1.05 ± 0.02c
Max	3.72 ± 0.13a	3.87 ± 0.08a	4.59 ± 0.07b	4.74 ± 0.10b

Note: Values are mean±standard error. Pairwise comparisons are within rows: means that do not share letters are significantly different (p<0.05).

Figure 5.

Comparison of reconstructed models and the prior model. Prior model is shown in red and reconstructed models are shown in green. The reconstruction parameters for a) ~ d) are K=1& RMS=0.88 mm, K=4 & RMS=1.02 mm, K=8 & RMS=1.05 mm, K=12 & RMS=1.06 mm respectively.

Figure 6.

Visualization of reconstruction error on the surface of the prior model. The color map was scaled from 0 mm (blue) to 3.6 mm (red) error and thresholded at 3.6 mm. The reconstruction parameters for a–d are K=1&RMS=0.88mm, K=4&RMS=1.02 mm, K=8&RMS=1.05 mm, K=12&RMS=1.06 mm, respectively.

Table 1 also gives the statistical comparison among the reconstruction results of different k values. Since the standard errors of different k values are very close, a two-sample Student’s t test was used in the analysis. According to the results, as k increases from 1 to 8, the RMS mean error significantly increases. Moreover, the maximum error of larger K values (k ≥ 8) is significantly larger than that of smaller K values (k ≤ 4), although the statistical tests showed no significant difference between smaller (k ≤ 4) and larger K values (k ≥ 8). These trends can also be observed in Figure 5 and 6. In Figure 5, as k increases, the reconstructed model becomes increasingly enlarged, especially in the fine vascular structures. In Figure 6, as k increases, larger reconstruction errors are observed, as denoted by the dark red regions.

Conclusions

In this work, we propose a novel surface reconstruction approach that makes use of the normal vector information from a prior model to construct a 3D surface model from a point cloud dataset sampled from the same anatomy. We conducted a series of simulation experiments to assess the effect of sampling rate, sampling noise, and registration errors. When assessing the impact of these factors in a series of single factor simulation tests, the errors were small, with overall RMS error less than 1 mm. In addition, we evaluated the effect of the neighborhood size on the prior model for estimation of the surface normal, finding the best reconstruction results occurred for k = 1. When combining all of the above factors, the RMS mean reconstruction error was still less than 1 mm, and the maximum error was about 3.73 ± 0.13mm. Those findings suggest that this method will produce robust, clinically useful results.

In future work, we plan to investigate the utility of this method for reconstructing anatomy using data from a limited field of view. This could potentially provide surface models for image guidance after scanning only a small region of the anatomy. In addition, we will utilize this algorithm to reconstruct surfaces from physical phantom models scanned with real-time, tracked ultrasound images.