Neural network algorithm for image reconstruction using the “grid-friendly” projections

Abstract

The presented paper describes a development of original approach to the reconstruction problem using a recurrent neural network. Particularly, the “grid-friendly” angles of performed projections are selected according to the discrete Radon transform (DRT) concept to decrease the number of projections required. The methodology of our approach is consistent with analytical reconstruction algorithms. Reconstruction problem is reformulated in our approach to optimization problem. This problem is solved in present concept using method based on the maximum likelihood methodology. The reconstruction algorithm proposed in this work is consequently adapted for more practical discrete fan beam projections. Computer simulation results show that the neural network reconstruction algorithm designed to work in this way improves obtained results and outperforms conventional methods in reconstructed image quality.

Introduction

X-ray computerized tomography (CT) remains the most popular and the most widespread tomography method used in medicine. The tomographic images are obtained by applying a method of projection acquisition and an appropriate image reconstruction algorithm. The key problem arising in computerized tomography is image reconstruction from projections which are received from an X-ray scanner of a given geometry. There are several well-known reconstruction methods to solve this problem. The most popular reconstruction algorithms are methods using convolution and back-projection [1–3] and the algebraic reconstruction technique (ART) [4–7]. Besides those methods, there exist some alternative reconstruction techniques. The most worthy of emphasis seem to be neural network-based algorithms. Neural networks are used in different implementations, for example, in image processing [8–10], in particular in computerized tomography. Reconstruction algorithms based on supervised neural networks have been presented in various papers, for example [11–14]. Other structures representing the so-called algebraic approach to image reconstruction from projections and based on recurrent neural networks have been studied by several authors [15–17]. Their approach can be characterized as a unidimensional signal reconstruction problem. In this case, the main disadvantage is the extremely large number of variables arising during calculations. The computational complexity of the reconstruction process is proportional in that approach to the square of the image size.

In this paper, an original approach to the reconstruction problem will be developed, based on original transformation methodology [18, 19]. The most important improvement of our reconstruction method, in comparison to the previous publication, is an adaptation of the discrete Radon transform (DRT) concept (see e.g. [20, 21]) in the fully original way. This methodology provides the so-called “grid-friendly” angles at which the projections are performed. Because this concept limits the number of the performed during investigation projections, we develop our approach and provide a new idea—the modified (extended) “grid-friendly” methodology which lifts that limitation. In this paper, we present a comparison of the equiangular interval sampling procedure with the “grid-friendly” and the modified “grid-friendly” methodologies for specifying the projection angles. We can decrease in this way the artifacts in reconstructed image without any cost: geometry of tomographic scanner do not need to be changed and reconstruction algorithm needs only some small reformulations. It is worth to emphasise, that these reformulations do not cause the algorithm to become more computationally complex.

In our approach, a recurrent neural network [22] is proposed to design the reconstruction algorithm. Owing to the 2D methodology of the image processing in our approach, we significantly decreased the complexity of the tomographic reconstruction problem. The applied recurrent neural network proposed to solve the reconstruction problem is designed in a fully analytical way. We show how all parameters of this network can be obtained, in particular the weights of the network, and what roles these parameters play. The calculations of these weights will be carried out only once before the principal part of the reconstruction process is started. Additionally, because the number of neurons in the network does not depend on the resolution of the projections performed earlier, we can quite freely modulate the number of projections carried out.

The reconstruction method presented in this paper, originally formulated by the author, can consequently be applied to the fan-beam scanner geometry of the tomography device, as is described later in this work.

The paper is organized as follows. The reconstruction method is presented in “Neural network reconstruction algorithm” section. The acquisition of the fan-beam projections (“Parallel beam collection” section), the rebinning procedure (“Back-projection operation” section) and the neural network reconstruction algorithm (“Reconstruction using a recurrent neural network” section) will be depicted in subsequent subsections. “Fan-beam reconstruction algorithm” section describes the performance of the computer simulations and presents the most important results. “Experimental results” section gives some conclusions.

Neural network reconstruction algorithm

The image processing procedure in our reconstruction method resembles one of the transformation algorithms—the ρ-filtered layergram method [2]. In our approach, instead of 2D filtering, we implemented a recurrent neural network. This network performs the function of an “energy pump”, which carries out the reconstruction process from the blurred image obtained after the back-projection operation. The principal idea of the presented reconstruction method using the recurrent neural network is shown in Fig. 1, where the rather theoretical parallel beam geometry of the scanner is taken into consideration.

Fig. 1.

Neural network image reconstruction algorithm using parallel beams

Parallel beam collection

Only a limited number of parallel projection values are chosen for further processing. Firstly, we determine the values of the angles . Let denote discrete values of parallel projections taken at angles indexed by the variable ψ. In our approach, according to the concept of the discrete Radon transform (DRT) [20, 21], we propose only grid “friendly” angles of parallel projections. The motivation for this approach is the better adjustment of the rays in the parallel beam crossing the discrete image to the grid of pixels in this image, if at every angle of projection every ray crosses at least one pixel. In this case, we propose where is the number of projections. Considering the above condition, the discrete values of parameter α ^p are as follows:

1

The proposed distribution of the projection angles is approximately equiangular in the range of , as is depicted clearly in Fig. 2 for the case of .

Fig. 2.

The choice of “grid-friendly” parallel projection angles

The number of “grid-friendly” projection angles is strictly limited and is equal to 256 for a half rotation around the investigated object and 512 for the full rotation. One can introduce a certain modification of the above approach to avoid this limitation, by multiplying the value 256 (512) by k. We evolve Eq. 1 into the following expanded form

2

Alternatively, as a comparative case, we can choose the equiangular set of parallel projections taken at angles indexed by variable ψ _e, where , where Ψ _e is the number of projections. In this simplified case, the discrete angles of projections are given by the following relationship

3

where is the angle, given in radians, by which the tube-screen pair is rotated after each projection.

The topological differences between both concepts of projection angle determination are depicted in Fig. 3 for the case of a reconstructed image having a resolution 5 × 5 (only the rays lying on the symmetry lines of given projections are depicted).

Fig. 3.

Topology of “grid-friendly” parallel projection angles (a) and equiangular positioning of the parallel beam scanner (b)

Now we determine a uniform sampling on the screen at points , where L is an odd number of virtual detectors, from the projection obtained at angle α ^p_ψ. It is easy to calculate the distance between each parallel ray from the origin in the (x,y) space if these detectors are symmetrically placed on the screen. The distance is given by

4

where is the sample interval of the virtual projections on the screen. Taking into consideration the sample of parameters s and α ^p of the parallel projections, we can write

5

In this way, we obtained all the imaginary parallel projections given on the grid (or alternatively ), which will be used in the following steps of the reconstruction procedure.

Back-projection operation

After the next step of our reconstruction algorithm for parallel beams, namely the back-projection operation [1, 2], we obtain a blurred image which can be expressed by the following formula

6

Because we have only a limited number of the virtual parallel projection values, it is necessary to apply interpolation. In this case, a projection value mapped to a certain point (x,y) of the reconstructed image is given by the equation

7

where is an interpolation function and .

In the presented method, we consider the discrete forms of the images and . That means these continuous functions of the images will be substituted by their discrete equivalents and , respectively, where ; ; I and J are the numbers of pixels in the horizontal and vertical directions. Thus, the discrete approximation of Eq. 7 is given by the expression

8

which is convenient from a computational point of view. In (8), is an interpolation function, . If we use the linear interpolation function [2]

9

Eq. 8 has only two terms and can be reformulated as [1]

10

where is the highest integer value less than the value of variable s _ij, .

In practice only a limited number of projections are performed. In particular, if we use “grid-friendly” methodology, at angles , where (I—size of the processed image), then we can approximate the integration over the angle by a finite sum. In consequence, Eq. 6 takes the following form

11

where , . It is a very similar case if we use the modified “grid-friendly” set of projection angles specified by Eq. 2, that is

12

Alternatively, in the case of the equiangular approach, we perform projections at angles , where and we can approximate the integration in Eq. 6 over the angle as follows

13

The discrete image obtained after the back-projection operation includes information about the original image blurred by a geometrical term. Our task is to reconstruct the original image from the given form of using a recurrent neural network [22]. Before we start the design process of this network, it is necessary to formulate the discrete reconstruction problem, and in particular to calculate the coefficients representing the geometrical term distorting the original image. In our approach, we take into consideration the interpolation function used during the back-projection operation.

Reconstruction using a recurrent neural network

Due to relationships (6), (7) and the definition of the Radon transform it is possible to define the image, obtained after the back-projection operation, in the following way

14

where . After some reformulations of the Eq. 14, approximation of the integrations by a finite sums, we obtain relationship the following relation (see e.g. [18, 19]),

15

where

16

Since the interpolation function is even, we can write

17

Therefore, we are able to formulate a very convenient relationship between the original image and the image obtained after the back-projection operation, in the form of

18

where

19

for the “grid-friendly” choice of projection angles (see Eq. 1) or

20

for the modified “grid-friendly” projection angles (see Eq. 2).

Alternatively, in the case of the equiangular approach to determining the projection angles, we obtain the following equivalent of Eq. 19

21

As one can see from Eq. 18, the original image of a given cross-section of the object, obtained in the way described above, is equal to the amalgamation of this image with a geometrical distortion element expressed by formulas (19), (20) or (21). The number of coefficients is greatly reduced and the values of these coefficients are easily calculated. The h _Δi,Δj coefficients are used to determine the weights in the recurrent neural network.

The recursive neural network structure for 1D signal reconstruction was proposed for the first time in [23] and later in [15, 24]. The network realizes the image reconstruction from projections by the deconvolution of relationship (22). The problem of deconvolution can be reformulated to the following optimisation problem, basing on the maximum likelihood (ML) methodology:

22

where —the optimal image (reconstructed image), —the matrix with elements from image being reconstructed, —the activation function, and

23

If the value of the coefficient v tends to infinity or is suitably large, then the solution of the optimisation problem (22) tends to the optimal one. Our research has shown that the following activation function yields the always stable reconstruction process (other possible forms of this function are presented in [24]):

24

where λ is a slope coefficient, v is a suitable large positive acceleration coefficient.

In our experiments we have never observed any divergent iterative reconstruction process using activation function (24) (at suitably chosen in experimental way parameters v and λ). That means the iterative realisation of the neural reconstruction algorithm is robust even if we change the reconstructed image. The main motivation to use this form of activation function was a property of its derivation used in reconstruction process. This derivation takes the following form:

25

Thanks to the saturation effect of the function (25) outside the range e _ij , it is possible to avoid instabilities in the reconstruction process when there is a drastic increase in the value of any of the variables used in the calculations.

Now we will formulate the energy function which will be minimized by the constructed neural network. Simultaneously, we will realise the task of deconvolution (see Eq. 18). The energy function is given by

26

In order to find the minimum of function (26) we determine the derivative

27

If we let (see [18, 19])

28

then Eq. 27 takes the form of

29

One can see that the values of Eq. 29 are always less than or equal to zero, that is . Therefore, if then it means that and the minimum of E is obtained. Our calculation tends to this state and when we can stop the reconstruction process.

The neural network performing the minimization task consists of two layers with the same topology of neurons. The structure is shown in Fig. 4.

Fig. 4.

Structure of the recurrent neural network: a topology of the neurons in the net; b scheme of connections in the net

Fan-beam reconstruction algorithm

The principal idea of the presented reconstruction method using the recurrent neural network is shown in Fig. 5, where the target fan-beam geometry of the collected projections is taken into consideration.

Fig. 5.

Neural network image reconstruction algorithm using fan-beams

The first step in the reconstruction procedure described is the collection of all the fan-beam projections using a scanner, as depicted in Fig. 6.

Fig. 6.

Fan-beam geometry of the scanner

A given ray from a fan-beam is involved in obtaining a particular projection value , where the projection value is obtained at angle and β is the angle of divergence of the ray from the symmetry-line of the fan-beam. In real scanners, only samples of the projections are measured, where usually are equiangular rays, are indexes of these rays, α ^f_γ = γ · Δ^f_α are particular angles of the X-ray source at which the projections are obtained, are the indexes of these angles. For simplicity, we can define the discrete values of the projections as .

In the next step of our reconstruction algorithm, we perform the rebinning operation, which re-sorts the fan-beam projection values obtained in the previous step into equivalent parallel projection data [1]. Referring to Fig. 7, we can find the relationships between the parameters in both of the scanner geometries considered, as

30

Fig. 7.

Geometric relationship between parallel and fan X-ray beams

After defining the parameters of the virtual parallel projections (see Eqs. 1, 4) we can start the rebinning operation. Unfortunately, in a lot of cases there is a lack of equivalences for parallel rays in the set of fan-beam projections. As a remedy we use an interpolation, in the simplest way—bilinear interpolation. In this case an estimation of the parallel projection can begin by identifying the neighbourhood of the fan-beam projection given by

31

The neighbourhood is determined based on four real measures from a whole set of fan-beam projections: where is the highest integer value less than

32

is the highest integer value less than

33

γ ^↑ = γ ^↓ + 1. In order to calculate a linear interpolated value the following expression is used

34

Having all the required parallel projection values, we can then perform the reconstruction procedure for parallel beams. In our case, this is a method using a recurrent neural network, as was explained in “Neural network reconstruction algorithm” section.

Experimental results

It is very useful, for various reasons, to simulate projection data. Idealized projection measurements obtained in this way allow us to develop and evaluate the reconstruction algorithms we have designed. One of the most widespread of this kind of simulation method is the use of a head phantom model, the so-called Shepp–Logan mathematical phantom [6, 1]. In our experiments, we used the Shepp–Logan model extended in an original way to 3D space, similar to the approach presented in [25]. Our 3D phantom consists of ellipsoids, whose parameters are described in Table 1.

Table 1.

Parameters of the ellipsoids used to construct our mathematical phantom

No.	Coordinates of the centre			a (semi-axis x)	b (semi-axis y)	c (semi-axis z)	Inclination	Density μ const
	x 0	y 0	z 0
I	0.000	0.000	0.000	0.6900	0.9200	0.9000	0.0	2.000
II	0.000	0.000	0.000	0.6624	0.8740	0.8800	0.0	−0.980
III	−0.220	0.000	−0.250	0.4100	0.1600	0.2100	108.0	−0.020
IV	0.220	0.000	−0.250	0.3100	0.1100	0.2200	72.0	−0.020
V	0.000	0.330	−0.250	0.2200	0.2200	0.3700	0.0	0.010
VI	0.000	0.100	−0.250	0.0460	0.0460	0.0460	0.0	0.020
VII	−0.060	−0.650	−0.250	0.0460	0.0230	0.0200	0.0	0.010
VIII	0.060	−0.650	−0.250	0.0460	0.0230	0.0200	90.0	0.010
IX	0.060	−0.105	0.625	0.0560	0.0400	0.1000	90.0	0.020
X	0.000	0.100	0.625	0.0560	0.0560	0.1000	0.0	−0.020

A view of the mathematical model of a skull phantom is depicted in Fig. 7—the size of the processed image was fixed at pixels. Such a resolution of the image seems to be a good choice, taking into account the balance between the reconstructed image quality and the real time of calculation during the computer simulations.

Figure 8b, c show two cross-sections of the 3D mathematical phantom. These images will be used in our experiments to evaluate the designed neural reconstruction algorithm both for parallel projections and for fan-beam projections.

Fig. 8.

Mathematical model of the phantom given in Table 1: a a view in the x–z plane; b cross-section in the plane A; c cross-section in the plane B

It is quite easy to reformulate the above model for fan-beam projections using the following relationship

35

During the simulations, we established 170 measurement points (detectors) on the screen as virtual parallel projections. We chose the number of these projections to be 512 rotation angles because this number is suitable for the approach with “grid-friendly” projection angles. In other experiments, the number of projections was modified.

Before we start the reconstruction process, it is necessary to evaluate the coefficients . This is only done once, for all the possible further processing approaches: with equiangular rotation, with only “grid-friendly” projection angles and the expanded “grid-friendly” technique. Using the linear interpolation functions from Eqs. 19, 20 and 21, the values of these coefficients are presented in Fig. 9. Because of the very fine differences between the three approaches analysed, we only present one chart showing a general view of the coefficients h _Δi,Δj and an enlargement showing details of the chart around the origin. In the cases of the equiangular sample and the modified approach with “grid-friendly” methodology, we used 7200 projection angles to calculate the coefficients h _Δi,Δj and in the case of the “grid-friendly” approach only 512 angles. A more in-depth discussion of the number of necessary projections performed during calculation of the coefficients h _Δi,Δj is presented below.

Fig. 9.

Values of coefficients : a the general view; b values around origin, where Δj = 0

Having obtained the coefficients h _Δi,Δj, we can start the next step of the reconstruction procedure and perform the back-projection operation using relationships (11), (12) or (13) to get a blurred image of the X-ray attenuation coefficient distribution in a given cross-section of the investigated object (see Fig. 10). (We must use the same interpolation function as in the calculation of the coefficients h _Δi,Δj, for example, the linear interpolation given by Eq. 9).

Fig. 10.

Distorted image of the mathematical model obtained after the back-projection operation

The image obtained in this way was next subjected to a process of reconstruction using a neural network, whose structure was explained in the previous section. To do this we adopted the discrete Eq. 23 taking into consideration the time-varying values of the pixels in the reconstructed image. Thus

36

Euler’s method was used to approximate linear Eq. 27 in the following form [17]

37

where is an appropriate small time step.

It is very subjective to evaluate a reconstruction procedure based only on a view of the reconstructed image. That is why the quality of the reconstructed image has been evaluated by an error measure defined as follows

38

where is the original image of the Shepp–Logan mathematical phantom.

Additionally, during the experiments, we used the following error measure [17], which is more relevant to subjective observation of reconstructed image

39

where and are the original image of the mathematical phantom, the reconstructed image and the mean value of the original image, respectively. All images are transformed by the so-called window determined by parameters C (centre) and W (width):

40

The measure described by Eq. 39 allows us to evaluate the subjective impression of an observer viewing the reconstructed image on a real screen.

As was mentioned earlier, we evaluate the coefficients h _Δi,Δj in the first step of the reconstruction procedure. It is crucial to choose the minimum number of projections necessary to calculate these parameters objectively. In this experiment, we use the most intuitive approach with equiangular projections and the extended “grid-friendly” methodology, in both cases fixing the number of projections during the initial acquisition process starting the actual reconstruction algorithm at 256 (the “grid-friendly” methodology is a special case with 512 projection angles). In the experiment, the value of coefficient v was selected at v = 2.5 × 10¹⁰, and the slope coefficient at . The objective results of these investigations are depicted in Fig. 11 and views of the reconstructed images of the mathematical phantom in the cross-section in plane A after 30,000 iterations are presented in Fig. 12.

Fig. 11.

Results of the reconstruction process, dependent on the number of projections during the calculation of the coefficients, evaluated by: a the MSE measure (see Eq. 38); b the Error measure (see Eq. 39)

Fig. 12.

View of the reconstructed image, dependent on the number of projections during the calculation of the coefficients (window: C = 1.0, W = 0.1—see Eq. 40)

Based on the plots in Fig. 11 and the views in Fig. 12, we can say that using the “grid-friendly” and the extended “grid-friendly” methodologies of projection performance, we obtain a reconstructed image more quickly and with better quality.

In the next step of our investigations, we carried out some experiments incorporating the fan-beam reconstruction method described in “Parallel beam collection” section. At this stage, we used the neural network reconstruction algorithm for parallel projections as depicted in Fig. 5 with the extended “grid-friendly” method of calculating the h _Δi,Δj coefficients (the number of projections was fixed at 7200). Projection acquisition for the fan-beam reconstruction algorithm can be performed at angles (exactly 512 measurement samples) specified by pure “grid-friendly” methodology without any loss of reconstruction image quality. However, for the extended “grid-friendly” approach, the experiments were carried out with different numbers of performed projections. Results of these simulations are shown in Fig. 13 for cross-sections in planes A and B after 100,000 iterations of the neural network algorithm. For comparison, the standard convolution/back-projection method with rebinning and the Shepp–Logan kernel is also considered. In all cases, we used the following geometrical parameters of the fan-beam scanner: , where .

Fig. 13.

View of the images (window: C = 1.02, W = 0.11): a original image; b reconstructed image using the algorithm described in this paper, after 100 000 iterations; c reconstructed image using the standard convolution/back-projection method with rebinning and the Shepp–Logan kernel

Conclusions

In this paper, we propose an original neural network image reconstruction from a projection algorithm based on the “grid-friendly” methodology of projection acquisition. Our experiments showed objectively that the “grid-friendly” method of specifying the projection angles gave better results than the more intuitive equiangular scheme of projection angle sampling, for parallel beam scanner geometry. This phenomenon may follow from the fact that the parallel rays used for projection acquisition in the “grid-friendly” approach are closer to the pixels in the reconstructed image, which is assumed to be a discrete function in our method (see the discrete reconstruction problem formulation considered in “Reconstruction using a recurrent neural network” section, Eq. 22).

Based on the results obtained for parallel beams, we extended the above conclusion to the problem of reconstruction from fan-beam projections, using in these further simulations only the “grid-friendly” methodology both for the calculation of the h _Δi,Δj coefficients and in the projection acquisition used for the actual reconstruction process. The simulations showed the superb quality of the reconstructed image of the cross-section of the investigated mathematical model, with respect to quality measures (38) and (39), when compared to the standard reconstruction method, in the case of fan-beam scanner geometry. Therefore, we are entitled to state that our method outperforms algorithms used recently in commercial CT scanners and it can be in easy way extended to helical geometry of scanner. The simulations also show that sequential realization of the proposed reconstruction algorithm is very time consuming. On the other hand, parallel hardware implementation of our neural network structure, for example, by effective implementation of VLSI or nanotechnologies, e.g. core–shell systems, could give incomparably better results than the previous methods of image reconstruction from projections, as far as the time to process the reconstruction is concerned. In this case, the time complexity of our neural algorithm is proportional to the number of iterations this algorithm performs (for a parallel geometry of scanner). For comparison, in the case of the standard convolution/back-projection method, the computational time depends on 2I²Ψ additions and multiplications, where I is a dimension of the processed image and Ψ is the number of projections. The rebinning operation and back-projection (interpolation) are identical in both cases.