Three-Dimensional Reconstruction of Blood Vessels of the Human Retina by Fractal Interpolation

Abstract

In this work, data from two-dimensional (2D) images of the human retina were taken as a case study. First, the characteristic data points had been removed using the Douglas–Peucker (DP) method, and subsequently, more data points were added using random fractal interpolation approach, to reconstruct a three-dimensional (3D) model of the blood vessel. By visualizing the result, we can see that all the small blood vessels in the human retina are more visible and detailed. This algorithm of 3D reconstruction has the advantage of being fast with calculation time less than 40 s and also can reduce the 3D image storage level on a disk with a reduction ratio between 78% and 96.65%.

Introduction

Today, with the advent and development of new technologies for 3D imaging and high calculation capacity of computers, the observation of internal anatomical structures can be performed virtually, without alteration of the specimen. But, the downside of 3D data, they are very memory intensive. The aims of procedural generation is to create algorithms to generate with very little information at the beginning a large amount of data if possible with the presence of many details with less errors. The applications are in the creation of virtual model. In this paper, we will use the fractal technique for 3D model generation.

There are many useful methods for reconstructing 3D models from a single image [1]. Sturm and Maybank [2] presented an approach for 3D reconstruction of objects from a single image, their method is based on user-provided coplanarity, parallelism, and perpendicularity constraints. Perpendicularity of constraints is used to calibrate the image, and the other two constraints are used to perform the 3D reconstruction. Colombo et al. [3] used reconstruction process and the texture of the 3D acquisition metric of surfaces of revolution from a single uncalibrated picture. Horry et al. [4] proposed a technique called tour in the image, to rebuild part models wise to plan for paintings or photographs. Zhang et al. [5] proposed a method that combines a sparse set of constraints specified by the user, such as the position of the surface normal and folds, to generate a 3D surface that behaves well to satisfy these constraints. All the above techniques can create realistic models. Inspired by these techniques, we proposed a fractal approach to reconstruct models of the blood vessels of the human retina in 3D from a single image with less user interaction.

In this paper, we have studied and developed the fractal geometric modeling as an extension of classical forms geometric [6]. We start from the iterated function system (IFS) model [7], which provides access to an extremely varied range of shapes while being simple to use. The classical fractal geometric forms take as parameters a set of control points. These control points are the entry parameters of the IFS algorithm, which uses predefined matrices to calculate the new points [8]. If one starts with a classical geometric form and that this operator is applied repeatedly, it converges to the invariant [9].

The first stage of work is the acquisition of the retina image [10] and implementing 2D filters to overcome the different noises in the image [11–15]. The next step is to determine the characteristic points from the curve of the vessels [16–19] and modulate them with 3D circles [20,21]. The crucial step in the work is to generate connections between the characteristic points with the 3D fractal interpolation [22–25].

The paper is structured as follows: In Sec. 2, we introduce the algorithm used in this paper. Moreover, we present two algorithms for generating fractals in 3D. In Sec. 3, we describe the image segmentation. In Sec. 4, detailed explanation of 3D fractal interpolation is provided. Furthermore, in Sec. 5 we present experimental results and some examples presenting our algorithm. Finally, in Sec. 6 we present our conclusion.

General Algorithm of 3D Reconstruction by the Fractal Method

There are many fractal 3D reconstruction algorithms, for example, Sun [20,21] introduced the mathematical model of fractal interpolation from a surface and gave its matlab program, and Chen and Bi [22] explored the use of 3D IFS to create fascinating scenes fractals. Algorithm 1 illustrates the algorithm followed in this work. The implementation of this algorithm is relatively complex. In the following, the different steps will be detailed.

Algorithm1.: 3D fractal reconstruction

Start

 • Read the image.

 • Binarization image.

 • Skeletonization image.

 • Determining the different types of pixels.

 • Determining the characteristic points.

 • The construction of the 3D circle and using the characteristic points as center, and extracting the coordinates (x, y, z).

 • Calculate matrices a_i, b_i, c_i, d_i, e_i, f_i, s_i, o_i.

 • Implement IFS.

 • Plot the coordinates (Xnew, Ynew, Znew)

End

Pretreatments 2D

Image Segmentation.

The separation of blood vessels in the retina image is of great interest. We propose an automated method for the identification and separation of retinal vessels in a tree image. The skeleton of the segmented blood vessel tree is extracted to reduce this complexity by finding the optimum curve of the blood vessel segments. This technique does not take into account the width of the blood vessel, but could measure other qualities, such as orientation, the vessel segments length, also the presence of gaps, and detecting connection points and crossover points.

We have adopted a methodology using information from 2D shapes to detect the centerline of blood vessels [11,12]. Then, we have tried to determine a relationship between a point O (i, j) that belongs to the centerline and the vector P (x, y, z) (the set of points of the 3D circle located at the distance of the point O). The 3D circle is defined by its center O and radius R.

The first step toward this aim would be to generate the detailed skeleton of the human retina (Fig. 1(a)): the complete set of vessels constitutes the internal support structure that gives shape to the retina (Fig. 1(b)). Then, we classify the pixels of the skeleton into three classes [13,14]: endpoints, bifurcation points, and centerline points. This classification depends on the number of their neighboring pixels which could be one, two, or three to eight, respectively (Fig. 1(c)). In this figure, the circle symbol represents the endpoint, the rectangle pixels are bifurcation points [15], and others are centerline points. Some redundant pixels of the skeleton should be removed because vessel segments become very short and create false branches. The false branches pixels are deleted when the distance between the endpoint and bifurcation points is less than a threshold value (for example, the length of six pixels).

Fig. 1.

(a) Original image, (b) the skeleton of image, and (c) pixel classification

Determining Characteristics Points.

Simplification is an operation that reduces the details of a curve. Many algorithms exist in computational geometry simplification. White [24] conducted a study of three algorithms of simplification, based on the work of Marino [25] on the characteristic points. It showed that the method of DP was the best among the three methods, the result obtained shows that the simplifications produced by Douglas algorithm are believed to be the best examples of perceptual representations of original lines in 86% of all the tests.

DP Algorithm.

DP algorithm is to reduce the number of points (vertices) of which it is composed, the polyline, trying to keep as close to the original. The algorithm takes as input a polyline or a curve (an orderly continuation of points) and a distance threshold also called tolerance ɛ [16,17]. The operation is as follows:

(1) Connect the two endpoints of a polyline (the first and last points) with the line segment $\overline{{\mathrm{P}}_0{\mathrm{P}}_1}$ as shown in Fig. 2.

Fig. 2.

The different stages of DP algorithm

(2) At each stage, it traverses all points between terminals and calculates the orthogonal distance D relative to the segment.

(3) Find the point farthest from the line $\overline{{\mathrm{P}}_0{\mathrm{P}}_1}$.

If D > ɛ

(4) The polyline is divided by the farthest point of the segment, which has the effect of creating two new polylines $\overline{{\mathrm{P}}_0{\mathrm{P}}_2}$ and $\overline{{\mathrm{P}}_2{\mathrm{P}}_1}$.

(5) Repeat the same work on the new polylines.

(6) Continue the process until all the orthogonal distance is below the tolerance ɛ.

Else

Return.

End if.

Three-Dimensional Circle Reconstruction.

This methods use the natural shape of the blood vessel: a vessel has a cylindrical shape which may be represented as a combination of an axis and a surface represented as a stack of contours. To create a cylinder, a set of successive 3D circles are required [18].

Taking a classic circle parameterization (Fig. 3), the center of circles O (x0, y0) and radius r lead to the following parametric equation [19]:

Fig. 3.

Three-dimensional circle

$$\{\begin{array}{c}X(t)=x0+r*\hspace{0.17em}\text{cos}\hspace{0.17em}(\theta )\\ Y(t)=y0+r*\hspace{0.17em}\text{sin}\hspace{0.17em}(\theta )\\ Z(t)=r*\hspace{0.17em}\text{sin}\hspace{0.17em}(\theta )\end{array}\hspace{1em}\text{with}\hspace{1em}\theta \in [0:2\pi ]$$ (1)

IFS Theory

The IFS model was introduced initially in a set-formulation directly translating intuitive property of self-similarity. IFS is a finite set of transformations Contracting {W_i, i ∈ Σ} [6,7]. These transformations or operators are defined on a complete metric space (X, d) (usually provided R² or R³ of the Euclidean distance) [8]. The geometric shape modeled by IFS is the nonempty X compact checking A = ∪i∈ Σ W_i (A) [9]. A sufficient condition for the existence of this transformation is that each compacted W_i contracts.

Building the 3D Model With IFS.

Consider a set {P_i (x_i, y_i, z_i)} i = 0. N points that one seeks to interpolate. IFS theory can be used to interpolate. For this, we use N affine applications W_i (although the affinity is not required). The N applications realize a partition of the interval [x₀, x_N] to [x₀, x₁] ∪ [x₁, x₂] ∪. ∪ [x_n−1, x_n]

$$w(P)=w_1(P)\cup w_2(P)\cup w_3(P)\cup \dots \mathrm{..}{w}_n(P)$$

The affine transformation W_i component IFS is defined as follows:

$$w_i\left(\begin{array}{c}x\\ y\\ z\end{array}\right)=\left(\begin{array}{c}{a}_i\\ c_i\\ 0\end{array}\begin{array}{c}{b}_i\\ d_i\\ 0\end{array}\begin{array}{c}0\\ 0\\ s_i\end{array}\right)\left(\begin{array}{c}x\\ y\\ z\end{array}\right)+\left(\begin{array}{c}{e}_i\\ f_i\\ o_i\end{array}\right)$$ (2)

W_i includes two transformations and can be written as ${\gamma }_i({\phi }_i(A\cap P_i))$ with ${\phi }_i$ as defined by the coefficients a_i, b_i, c_i, d_i, e_i, and f_i, and ${\gamma }_i$ by the coefficients s_i and o_i. ${\phi }_i$ is an affine spatial transformation [7,8].

In order to obtain a homogeneous coordinate system, the terms of the matrix 3 × 3 represents the scaling and rotation of the 3D point (x, y, z) by the parameter a_i, b_i, c_i, d_i, and s_i. Then, a translation by the matrix [e_i, f_i, o_i].

Each W_i transformation can generate from a P_i pixel of image to a set of pixels P_j+1, this operation is referred to as bonding. The spatial transformation, ${\phi }_i$, contracts, if the determining module $\left|\begin{array}{cc}{a}_i& b_i\\ c_i& d_i\end{array}\right|$ is less than 1. Let y and z axis has no rotation term, i.e., parameter b = 0, S_i = 1, and O_i = 0 in formula (1). When 0 < d_i < 1, IFS has a single attractor and this attractor must be the diagram of certain continuous functions and crossing original data point.

With the coefficients a_i, c_i, e_i, and f_i

$$a_i=\frac{x_i-x_{i-1}}{x_N-x_1}$$ (3)

$$c_i=\frac{y_i-y_{i-1}}{x_N-x_1}-d_i*\frac{y_N-y_1}{x_N-x_1}$$ (4)

$$e_i=\frac{x_N{x}_{i-1}-x_1{x}_i}{x_N-x_1}$$ (5)

$$f_i=\frac{x_n{y}_{i-1}-x_1{y}_i}{x_N-x_1}-d_i*\frac{x_N{y}_1-x_1{y}_N}{x_N-x_1}$$ (6)

Dimension of IFS Attractor.

Fractal dimension (FD) in IFS is associated with the perpendicular scaling factor d_i. It can be learned from the theorem proposed in Refs. [2,3] that if the total number of data points is n, a_i (i = 2,3,., n) follows the above-mentioned affine transformation IFS, when 0 < d_i < 1 and ${\sum }_{i=2}^N{d}_i>1$ [6,9], suppose interpolation data points are not collinear, then attractor FD is the only real solution of the following equation:

$$\sum _{i=2}^N{d}_i{a}_i^{\text{DF}-1}$$ (7)

When x axis components of data points are equal-interval distributed, or x_i − x_i−1 = constant, then a_i = 1/(N − 1). Let perpendicular scaling component di = d be fixed. Then, the equation above can be simplified as follows [26]:

$$d={(N-1)}^{\text{DF}-2}$$ (8)

Generation of Blood Vessel Models by IFS.

Our method is based on the use of characteristic points determined by the DP algorithm. These points are considered the centers of 3D circles, and Eq. (1) is used to determine their coordinates. Fractal interpolation is applied to generate the intermediate points between these circles. This algorithm therefore takes as argument the previously defined function in Eqs. (3)–(7) to determine the Wi matrices (2) got from the initial form P0 (3D circle), and the IFS is applied for a number of well-defined iterations to have the image of blood vessels in 3D. This method will then create a list of W_i successive images in the following form:

$$\{P_0,W0(P_0),W1(W_0(P_0)),.W_i^j(P_0)\}$$

Finally, it remains to show each point of this list obtained.

Experimental Result

We have used the stare database images [10] (Fig. 4) to show the example of a raw image obtained from this database, the reconstruction time takes into account the time required for image segmentation and the time needed for the interpolation fractal. All these results were obtained on a workstation equipped with an Intel Pentium B960 central processing unit at 2.20 GHz and 4 GB of RAM processor.

Fig. 4.

Example of a raw image in jpeg

DP Algorithm.

The tolerance parameter (ɛ) is the minimum distance to keep a point in the process between a peak and the reference line. It is difficult to find the appropriate tolerance value. We made the choice to do a sensitivity analysis and calculate the rate simplification dark count rate (DCR) for each value of these tolerances parameters for the image used in Fig. 1

With DCR = ((N − n)/N) × 100

And the number of pixels in the image N = 1077

Table 1 shows the results obtained after the simulation of the DP algorithm for different tolerance values ε. Six tolerance values ε described in this table are selected to analyze the rates simplification. For ε = 2 pixels, there are 35 feature points extracted from 1077 point of the image, which means that 96.65% of them have been reduced in the representation of blood vessels curve. Then, for ε = 0.5, the feature points are reduced to 238 points, which indicate that 78% of the curve points are rejected. As the results show that when ε value is large, the rates simplification is at a relatively high level. Conversely, if ε value is smaller, then the rates simplification decreases.

Table 1.

Number of pixels features and DCR% for different values of ε

Tolerance, ɛ	Number of characteristic points (n)	Rate simplification DCR (%)
0.5	238	78
0.8	113	89.5
1	68	93.68
1.5	53	95
1.8	44	95.91
2	36	96.65

The DP algorithm has been used by Chen et al. [26] to reduce data of the geographic profiles. His research is collected from the digital elevation model data of the island of Taiwan, with a resolution of 40 m. They selected two profiles East–West and North–South, each profile has 1000 points. The study result indicates that the reduction rate of East–West profile data can amount to 96% and North–South to 95%.

In terms of reduction rates by the DP method to calculate the characteristic points of retina image, it is found, from comparing the results obtained by Chen et al. [26] and the test result in Table 1 (the reduction rate achieved may reach 96%), that the results are almost identical. In terms of skill of DP method, they are not only fast in reduction but also high in reduction rate.

Figure 5 shows the result obtained by the DP algorithm for the tolerance ε = 1.

Fig. 5.

Douglas-Peucker (DP) algorithm for ε = 1 (the rectangle symbol represents the: characteristic point and white pixel: blood vessel)

Simulation of the 3D Reconstruction Algorithm.

Figure 6 shows a measurement data group on an example of the blood vessel. The interval of data points is 2130, and the numbers of interpolated points are 210001.

Fig. 6.

Three-dimensional fractal interpolation example

We presented in Fig. 7 the behavior of the 3D reconstruction algorithm with fractal interpolation applied to the human retina image of N = 1076 points in 2D images and 3D object of a synthetic P = 32,280 points.

Fig. 7.

The different stages of reconstruction 3D with fractal interpolation

By visually comparing the constructed model, we see that all the folds of blood vessels in the human retina are very visible and detailed. But if we want very precise measurements on blood vessels, this model cannot be used in a medical procedure, such as diagnostics or surgery, our virtual model is not really suitable.

The advantage of this model is that it allows us to preserve the topology of the vascular network and calculates the geometric quantities, such as the local orientation or curvature of the structure, and it also determines the different geometric parameters of the blood vessels, such as the length and diameter.

For the simulation of the algorithm proposed, the following principle is used: for each value of the iteration number, calculate the corresponding value of interpolated point and thereafter varying the value of the number of iterations until being the minimum error.

According to the results in Table 2 and after the change in a number of iterations, we notice that error increases with decreasing the number of iteration. However, we find an error decrease with the increase in the number of iteration, this is justified by increasing the number of interpolated point and the scanning of all set of image.

Table 2.

Performance of our reconstruction method for various point clouds (calculation time in seconds)

Number of iterations	50	100	200	300	500	700	1000
Number of interpolated point (×106)	3.46	6.93	13.86	20.79	34.65	48.51	69.3
Execution time (s)	4.04	5.88	9.09	13.3	21.5	27.6	39.2
Number of points that do not appear	2545	1879	1284	956	662	311	255
Error %	7.88	5.82	3.977	2.96	2.05	0.96	0789

This method has many advantages. First, the algorithm starts with reduced number of points, which allows reduction of the memory consumption of the 3D image, which reduces costs and equivalently reduce the transmission time. Then, the execution time of the algorithm is short since they do not exceed 40 s. Also, this method uses all the information provided by the 2D image segmentation, it combines the strengths of 2D and 3D to ensure the best view the image in 3D. Finally, we find that the built 3D fractal interpolation can produce data points with a higher resolution than initially observed and reconstructed profile gets more natural and real details.

Despite all these advantages, the method has drawbacks. The first one is that it requires specialized hardware, because this algorithm needs high computing power. We even noticed that this type of compression does not standardize the use and not yet fully democratized since we have problems in terms of determining the iterations number required and its stop points.

Conclusion

In this paper, we propose an approach to reconstruct the blood vessels of the human retina in 3D from a single image by fractal interpolation, this approach is within the framework of 2D/3D techniques. In the first part, we have quickly presented a background of imaging techniques and processing a 2D image, it is very important to find graphical topology of the skeleton of the human retina and classify the different types of the pixel. Then, we determine the characteristic points of the vascular networks by DP algorithm. The second part is devoted to a detailed description of the 3D reconstruction, 3D reconstruction starts by stacking 3D segmentation results 2D. IFS is applied to determine the parameters of the 3D fractal interpolation to generate new points and model the blood vessels in 3D. By visually comparing the result, we see that all the folds of blood vessels in the human retina are very visible and detailed; this algorithm has the advantage of being fast and also can reduce the 3D image storage level.

Nomenclature

Σ =

finite set of indices