Multi-Objective Evolutionary Algorithm for PET Image Reconstruction: Concept

Abstract

In many diagnostic imaging settings, including positron emission tomography (PET), images are typically used for multiple tasks such as detecting disease and quantifying disease. Unlike conventional image reconstruction that optimizes a single objective, this work proposes a multi-objective optimization algorithm for PET image reconstruction to identify a set of images that are optimal for more than one task. This work is reliant on a genetic algorithm to evolve a set of solutions that satisfies two distinct objectives. In this paper, we defined the objectives as the commonly used Poisson log-likelihood function, typically reflective of quantitative accuracy, and a variant of the generalized scan-statistic model, to reflect detection performance. The genetic algorithm uses new mutation and crossover operations at each iteration. After each iteration, the child population is selected with non-dominated sorting to identify the set of solutions along the dominant front or fronts. After multiple iterations, these fronts approach a single non-dominated optimal front, defined as the set of PET images for which none the objective function values can be improved without reducing the opposing objective function. This method was applied to simulated 2D PET data of the heart and liver with hot features. We compared this approach to conventional, single-objective approaches for trading off performance: maximum likelihood estimation with increasing explicit regularization and maximum a posteriori estimation with varying penalty strength. Results demonstrate that the proposed method generates solutions with comparable to improved objective function values compared to the conventional approaches for trading off performance amongst different tasks. In addition, this approach identifies a diverse set of solutions in the multi-objective function space which can be challenging to estimate with single-objective formulations.

I. Introduction

THE goal of this work is to perform tomographic image reconstruction of a set of solutions that are optimal for more than one objective. This is motivated by the reality that medical images are typically used for more than one task, such as detection and estimation tasks. In positron emission tomography (PET) imaging, oncologic F18-fluorodeoxyglucose (FDG) studies are commonly used for the detection of hot features [1] and the quantification of the uptake in those features [2]. A single image is rarely optimal for both of these goals [3].

Iterative image reconstruction for tomography typically seeks to estimate a single image that is optimal for a single objective function. This objective function could have many terms to achieve various goals such as data fidelity, noise regularization [4], edge enhancement [5], and texture properties [6]. Despite this multitude of goals, the objective function is formulated as a single function with predefined weights (hyper-parameters) leading to a single optimal solution. If a different noise versus resolution tradeoff (or other tradeoffs) is desired, weights need to be changed and a new optimization needs to be performed. Alternatively, earlier methods performed post-reconstruction regularization in which a final optimized estimate was filtered to achieve different performance tradeoffs. For these approaches, the optimal regularization strength to achieve a certain task is generally not known in advance. In practice, for any given data set, multiple reconstructions would need to be performed to map out the tradeoff of task performance as a function of hyper-parameter selections.

In contrast, this work proposes setting multiple distinct objectives and estimating the set of solutions that are optimal for these objectives. In essence, we want to identify the set of solutions as close as possible to the Pareto optimal front in a multi-objective space. This front is defined as the set of solutions for which none of the objective function values can be improved without reducing the opposing objective function. With this framework, it is generally not possible to identify solutions with optimizers that rely on closed-form update equations. Therefore, we propose a genetic algorithm-based approach that evolves a front of solutions towards this Pareto optimal goal.

Multi-objective evolutionary (MOE) optimization algorithms aim to find a set of Pareto-optimal solutions in a single optimization run instead of obtaining a single solution in each optimization [7]. Evolutionary multi-objective optimization approaches seek to obtain a set of 1) well-convergent solutions close to and 2) well-divergent solutions spread along the efficient Pareto front.

The idea of population-based multi-objective optimization is presented in Figure 1. At each iteration, a set of solutions is generated by an MOE algorithm with the goal to obtain subsequent sets of solutions closer to the Pareto front with high diversity along the front. This procedure continues until solutions stop evolving towards the Pareto front. Previously proposed MOE algorithms for solving multiand many-objective optimization problems include NSGA-II [8], MOEA/D [9], NSGA-III [10], and BNSGA-III [11]. The majority of applications of these optimization algorithms were to problems with relatively few unknowns [7]. This work represents one of the first to apply the method to tomographic image reconstruction, an inverse problem with on the order of 10⁴–10⁶ unknown voxel values. We develop a MOE algorithm for PET and term this approach as multi-objective evolutionary algorithm for PET (MOEAP).

Fig. 1:

A set-based multi-objective optimization algorithm in which solutions (represented by points) evolve to maximize multiple objectives.

In general, genetic optimization tends to be more computationally intensive than optimizations with closed-form update operations. Our use of genetic optimization is motivated by the goal to optimize objective functions that may not have calculable gradients. The growing computational resources present in local and cloud based instances are allowing what used to be computationally impractical algorithms to be performed in a reasonable time. Under the assumption of sufficient computing and storage resources, genetic algorithms may provide viable alternatives to conventional optimization. In addition, computation for genetic optimization can be substantially reduced through algorithmic enhancements such as directed mutation and parallelization.

The rest of the paper is organised as follows. Section II summarizes related evolutionary algorithm applications to tomography. Section III presents the proposed multi-objective optimization algorithm and its application to PET image reconstruction. Section IV evaluates results obtained for image reconstruction with simulated data. Section V and VI discuss results and future directions.

II. Related Work

The innovations proposed here include 1) the use of an evolutionary algorithm with data-informed, directed updates and 2) the use of a multi-objective formulation for PET image reconstruction.

Several efforts have applied genetic algorithms for image reconstruction in electrical impedance tomography (EIT). Unlike CT and PET, the boundary measurements in EIT are not linearly related to underlying image (distribution of electrical resistivity/conductivity) making inversion even more challenging. Genetic algorithms offer the potential benefits of not needing derivatives of the objective function and not requiring function continuity. Olmi et al. proposed an approach using the reciprocal of the reconstruction error as the fitness function [12]. Another study integrated a genetic algorithm for the solution of the static EIT inverse problem via a two-step genetic algorithm [13]. The first step seeks to minimize the square error of specific target resistivities, and the second step minimizes the reciprocal of reconstruction error for the remaining resistivities. While the proposed GA approaches yielded superior image quality for EIT, they are relatively expensive in terms of computing time and resources.

A few prior efforts have applied evolutionary algorithms to other tomographic image reconstruction applications. For example, Kodali et al. developed an approach for ultrasound-computerized tomography [14]. They explore different methods for evolving image solutions based on grouping pixels and find that block mutations (updating whole neighborhoods to a shared value rather than single pixel updates) is beneficial. Ouaddah and Boughaci proposed variations of meta-heuristic algorithms for general inverse tomography [15]. They use harmony search algorithms and rely on fully-random updates of solutions to identify new candidate populations. Batenburg presented an evolutionary algorithm for a discrete tomography problem, finding a binary image that satisfies a sparse set of projections [16]. Both of these papers report that single fully-random image updates were suboptimal and proposed memetic algorithm alternatives [17], that incorporate a local, stochastic hillclimb operation after each crossover or mutation operation. In short, each time a population evolves (through random operations), another local evolution is performed to move the solutions towards optimality. This memetic approach is different than our proposed single-step, directed mutation evolution.

Very few efforts have used multiple distinct objective functions in tomography. Li et al. proposed a multi-component objective function with terms for square error, image smoothness, and maximum entropy [18]. Their algorithm takes a weighted combination of the three components to form a single objective function. To reconstruct a single image solution, they formulate an algorithm combining genetic operations with a local search approach. Fernandes et al. presented a multi-objective optimization using simulated annealing to solve the regularized EIT reconstruction problem. The objective functions are the Euclidean distance (between the potential values measured in the cross-section of the body and the calculated values) and a regularization component [19], [20]. A somewhat related approach has been applied to MRI image reconstruction from sparse signals by Yue et al. [21]. They proposed a multi-objective particle swarm optimization algorithm reliant on two objective functions: measurement error and sparsity. These later two methods are similar to our proposed work in that they lead to a set of solutions that are optimal for more than one objective.

III. Proposed Method

A. Elitist Non-dominated Sorting Genetic Algorithm (NSGA-II)

We developed a modified version of the non-dominated sorting genetic algorithm (NSGA-II [8]) for the PET image reconstruction problem. The proposed algorithm utilizes the following features:

• an elitist approach,

• crowd distancing approach to ensure good diversity, and

• the principle of a non-dominated solution.

For all multi-objective optimization problems, a solution dominates another solution if the former solution is better than the latter solution in all objectives. In a population of N solutions, the solutions that are not dominated by any other population member are called non-dominated solutions of the population. The Pareto-optimal solutions are those that are not dominated by any other feasible solution in the entire search space [17]. The set of all non-dominated optimal solutions are called the Pareto optimal front. Pareto optimal fronts can take many shapes such as convex, concave, connected, disconnected, and there may exist many local fronts.

In general, evolutionary multi-objective optimization algorithms find multiple non-dominated solutions. Distinction amongst these approaches is commonly related to different elitism operators to enhance the convergence and diversity aspects. In NSGA-II diversity is promoted through a crowd distance operation applied to acceptable solutions from the lowest ranked front as discussed below [22]–[25]. The NSGA-II algorithm uses a crowded-comparison approach to maintain diversity among population members, as discussed below. This approach does not require any user-defined parameter for maintaining diversity and requires less computational costs than some other diversity promoting methods. Additional details on NSGA-II are available in [8].

Figure 2 graphically presents the NSGA-II procedure applied for the PET image reconstruction. At any iteration (generation) t, the algorithm begins with a parent population P_t, representing a set of N solutions). Initially, this population is generated by creating N images with pixels values drawn from a uniform distribution plus a initial filtered backprojection reconstruction of the data. Then from the parent population, we create the offspring population Q_t of size N with the following operations:

1. random selection,

   For random selection, we use in our proposed algorithm tournament selection operator [7]. Tournament selection is an approach to choose a member from a population in a genetic algorithm. Tournament selection involves running several ”tournaments” among a few members chosen at random from the population. The winner of each tournament (the one with the best fitness) is selected for the crossover operator.

2. simulated binary crossover (explained in subsection III-B) [26],

3. directed mutation (explained in subsection III-C),

Fig. 2:

Non-dominated sorting genetic algorithm.

Next, we combine the parent and offspring populations to form an intermediate population R_t of size 2N. Then, we evaluate the proposed objective functions for quantification and detection explained in subsections III-D and III-E for each member in R(t). The non-dominated sorting procedure is applied for all solutions in the combined population to rank and divide them into different non-dominated fronts.

The new parent population P_t+1 is created by selecting individuals from the non-dominated fronts, sequentially. All of the individuals in the best-ranked front are selected first then, the next best front and, we continue until we form a population of size N members from the N highest ranked solutions. In cases where there are not exactly N members from the best-ranked fronts, we select a portion of the members from the lowest ranked acceptable front using a crowded distance approach to ensure diversity of membership. For example, say N = 20 and the highest performing fronts from best to worst are F₁, F₂, and F₃. For this example, F₁ contains 10 members, F₂ contains 5 members, F₃ contains 12 members. In order to have 20 optimal members, we would select all of the members from F₁ and F₂ and 5 of the members from F₃. The members from F₃ would be selected with maximal distance along the F₃ front. The remaining N lowest ranked solutions are discarded.

B. Simulated binary crossover for PET Reconstruction

A crossover is a reproduction approach that merges two parent images to obtain two new offspring images. The basic idea in the crossover is that the new images may be better than both of the parents if they inherit superior characteristics. In 1995, Deb and Agrawal [26] developed a simulated binary crossover operator. The procedure for calculating the offspring pixel value $q_j^{(1,t+1)}$ and $q_j^{(2,t+1)}$ from the parent solutions $p_j^{(1,t)}$ and $p_j^{(2,t)}$ at location j is explained as follows. The spread factor, β_j, is the ratio of the spread of offspring points to that of the parent points incorporated as:

$$q_j^{(1,t+1)}=0.5\left[\left(1+{\beta }_j\right)p_j^{(1,t)}+\left(1-{\beta }_j\right)p_j^{(2,t)}\right]$$ (1)

$$q_j^{(2,t+1)}=0.5\left[\left(1-{\beta }_j\right)p_j^{(1,t)}+\left(1+{\beta }_j\right)p_j^{(2,t)}\right]$$ (2)

For each pixel location, a uniformly-distributed random number u_j ∈ [0, 1) sets the β_j according to a function defined as:

$${\beta }_j^{*}=\{\begin{array}{cc}{\left(2u_j\right)}^{\frac{1}{{\eta }_c+1}}& u_j\le 0.5;\\ {\left(\frac{1}{2\left(1-u_j\right)}\right)}^{\frac{1}{{\eta }_c+1}}& \text{otherwise,}\end{array}$$ (3)

where η_c is a non-negative real number called the distribution index. Finally, we include a probability of crossover, P_c, where P_c of the entries experience crossover (β_j = β_j^∗) and 1 − P_c of pixels are the same as the parent points (β_j = 1).

To offer intuition, these parameters offer the following trends. The spread factor term contributes to:

• contracting crossover β_j < 1 (The offspring values are enclosed by the parent values),

• expanding crossover β_j > 1 (The offspring values enclose the parent values), and

• stationary crossover β_j = 1 (The offspring values are the same as parent values).

For the distribution index,

• large η_c tends to generate offspring closer to the parents, and

• small η_c allows the offspring to be far from the parents.

Finally, we enforce non-negativity by simply setting any negative offspring pixels, q_∗, to 0.

C. Mutation operator for PET Reconstruction

For PET image reconstruction, the goal is to reconstruct an emission distribution x from recorded events y from emitted photons. Suppose that x_j denotes the mean number of emissions from the jth voxel and y_i the measured number of events in the ith measurement bin. PET measurements can be modeled as independent Poisson random variables related to an unknown image as:

$$y~Poisson(\overline{y})s\text{.}t.\overline{y}=cPx+r,$$ (4)

where P is the system matrix defining the probability of the detection of an event originating at each voxel in x being detected by each element in y. Multiplicative factors (such as attenuation) are represented with c and additive components (such as random and scattering events) with r.

The proposed directed mutation contains two steps: 1) an expectation maximization (EM) update and 2) post-smoothing. The EM algorithm [27] is arguably the most common approach for PET image reconstruction defined by

$$x^{n+1}=\frac{x^n}{P^{\prime }cI}{P}^{\prime }c\left[\frac{y}{cP{x}^n+r}\right].$$ (5)

After a single update from expectation maximization, we perform smoothing (2D Gaussian filtering) on the image, where the full width at half maximum of the 2D Gaussian kernel is randomly selected from a uniform distribution between 0 and 2cm. For the mutation step, we use the crossover results as the initial image.

D. Quantification accuracy objective function (conventional Poisson log-likelihood function)

In order to achieve quantitatively accurate values in the reconstructed image, we would like the data to fit the statistical model given the image as accurately as possible. As noted above, PET data consisting of M measurements when conditioned on the underlying emission distribution is well-modeled as Poisson distributed with a likelihood function of the form:

$$L(x)=\prod _{i=1}^M\frac{e^{-{\overline{y}}_i}{\left({\overline{y}}_i\right)}^{y_i}}{y_i!}$$ (6)

and a log-likelihood function, ignoring the constant term, given by:

$${\mathrm{\Phi }}_{\text{quantification}}(x)=\sum _{i=1}^M\left(-{\overline{y}}_i+y_i\log \left({\overline{y}}_i\right)\right).$$ (7)

This is a commonly used definition for data fidelity in iterative PET image reconstruction and the objective function we use for quantitative accuracy.

E. Detection objective function (generalized scan-statistic model)

The second goal for our images is to have high detection task performance. For this paper, we adapted a method to determine an image’s goodness at detecting 1 cm hot features in a cold background, representative of a common detection task of finding FDG avid lesions in oncologic PET imaging. This method follows the generalized scan-statistic approach originally proposed by Popescu et al [28], [29] with the shared goal of evaluation of detection task performance in the context of an unknown signal location.

For this paper, we seek detection of objects, or features, defined as small regions with a higher activity than the background activity. To detect such small objects, we scan the image by sliding a scanning window over the area/volume of interest, and for each position compute pertinent local statistics of the image elements inside the scanning window. We use a disc scanning window with a fixed radius and the statistic is simply the mean of the values of the pixels inside the disc. In this approach, an image is searched for multiple candidate signal present and signal absent locations. Scan window locations with the highest mean value are deemed signal present and with the lowest mean value signal absent. These multiple, non-overlapping locations provide estimates to calculate the average signal to noise ratio defined as:

$${\mathrm{\Phi }}_{\text{detection}}(x)=\frac{1}{L}\sum _{l=1}^L\frac{{\overline{x}}_{s,l}-{\overline{x}}_{b,l}}{\sqrt{\frac{1}{2}\left({\sigma }_{s,l}^2+{\sigma }_{b,l}^2\right)}}$$ (8)

where ${\overline{x}}_{s,l}$ and ${\overline{x}}_{b,l}$ are the mean values in the signal and background for the lth candidate, and ${\sigma }_{s,l}^2$ and ${\sigma }_{b,l}^2$ are the corresponding variances. The total number of signal present+absent candidate regions searched for is L and is predefined. Equation 8 is the objective function for detection task performance. In this work, the signal present and background candidate regions are identified in each image by sliding a scanning window (disc with fixed 1 cm diameter) over the area/volume of interest. This operation is accomplished rapidly with two dimensional convolution.

F. Application to simulated PET data

The proposed algorithm was applied to simulated PET data generated from a heart and liver slice of the XCAT phantom [30] shown in Figure 3. The data was analytically simulated with attenuation, randoms, and scatter contributions. In short, high resolution versions of the original emission and attenuation XCAT images were forward projected [31]. Poisson noise was added to mimic 4 × 10⁵ events for the heart slice and 5 × 10⁵ events for the liver slice. In all our experiments, sinograms with dimensions 180 × 180 were reconstructed to 128 × 128 images (3.5 mm² pixel). The additive contribution from random and scatter events was applied uniformly to the whole sinogram accounting for 10% of the measured events.

Fig. 3:

True images of the heart (A) and liver slice (B) used for simulation of PET data.

Images were reconstructed with the proposed MOEAP approach. For the evolutionary algorithm, we use the simulated binary crossover operator [26] with the probability of crossover, P_c = 0.95, and η_c = 20. We evolved a population of 100 PET images for 200 generations to determine a non-dominated front approaching the Pareto optimal front. For the detection objective function, we searched for L = 2 candidate signal present+absent locations in each image to offer a more realistic scenario in which the number of suspicious regions is not known in advance. Furthermore, the search region in each image was limited to the lungs in the heart slice and the liver in the liver slice. These regions could be found in real patient data with a simple pre-segmentation of target organs.

We compared the proposed approach with conventional approaches for trading off performance: maximum likelihood (ML) estimation with increasing explicit regularization through post-reconstruction filtering and maximum a posteriori estimation with varying regularization strength (MAP). For the ML approach, we used expectation maximization (EM) updated for 200 iterations and then applied post-reconstruction smoothing (2D Gaussian filtering) with FWHM varying from 5 to 10 mm in increments of 0.5 mm. For the penalized likelihood approach (MAP), we used iterative coordinate descent with a conventional quadratic penalty [32] and varied the penalty strength parameter from 0.1 to 50 (a.u.). To ensure convergence with MAP, we stopped iterating when the percent absolute error between the images from the current and previous iteration was less than 0.001.

We employed two reference tests on the final reconstructed images to evaluate if the proposed objective functions, applied without known feature locations, mirrored actual quantification and detection performance. For quantification evaluation, we calculated the mean error (ME) of the pixel values in the tumor. For detection evaluation, we used the signal location-known SNR, using equation 8 but with signal location known exactly information and with L = 1 for the one known tumor in the image. Images with low mean error should perform well for the quantification objective function. Likewise, the SNR of the known tumor location should be high for images that perform well for the detection objective function.

IV. Results

A. Heart Slice

Figure 4 presents the objective function values for different generations of populations. Each point in these graphs represents a single image. The value for the detection objective function is on the y-axis and the quantification objective function is on the x-axis (both have arbitrary units). The genetic algorithm sought to estimate a set of solutions that maximizes both of these objectives simultaneously providing a solution set that increases in the x and y direction for increasing evolution from the initial set.

Fig. 4:

Objective function values for the initial population of images (A) and population after 10 generations (B) applied to heart slice data.

Figure 5 shows the comparison between the final solutions obtained from the MOEAP approach (blue points), the conventional approach solutions (EM with varying degrees of post-smoothing, blue points), and the penalized likelihood solutions (MAP solutions, black points). This figure shows that the genetic algorithm provides a set of solutions that are generally more favorable in both of the proposed objectives compared to the EM with post smoothing and penalized likelihood approaches. For a few solutions, the penalized likelihood approach outperforms the MOEAP algorithm, but with increasing the penalty parameter, the detection function decreases.

Fig. 5:

Objective function values for the final generation of the MOEAP solutions, EM+post smoothing image reconstruction and the penalized likelihood solutions for the heart slice.

Figure 6 displays some of the images from the set of solutions. Image A is the solution from point A in figure 5 and has the maximum value for the detection objective function. Point B represents the compromise between the two objective functions. Point C is the image with the maximal value of quantification objective. In general, the visual assessment of these images follows the intuition for common PET imaging applications that smoother images with less noise are generally better for detection and less-smooth, high-frequency content images are generally better for quantification. Image D is from the conventional EM approach with post-smoothing equal to 6mm offering a compromise solution between two objective functions. Also, image E with post smoothing equal 8.5mm represents the maximum detection function for the EM+post smoothing approach. Image F represents a compromise between maximum quantification and maximum detection objective functions obtained from penalized likelihood solutions with strength parameter 3.8 a.u..

Fig. 6:

Three images from the set of solutions from the proposed multi-objective genetic algorithm, two images from EM+post smoothing approach and one image from the penalized likelihood approach for the heart data slice. Image with maximum value for the detection objective function (A), image with a compromise of high quantification and high detection (B), image with the maximum value for the quantification objective function (C), image from EM+post smoothing regularization approach (D), image from the maximum value for detection function using EM+post smoothing approach (E), and image from the solutions with a compromise of high quantification and high detection using the penalized approach (F).Image labels match solutions presented in figure 5.

Figure 7 presents images from five generations of the proposed MOEAP. Images from A to E are solutions at first, 5th, 10th, 20th, and 200 generations, respectively. Each image represents the single solution with quantification and detection objective values closest to the mean of all the solutions at that generation. Visual inspection of the images demonstrates improving quantification and detection performance.

Fig. 7:

Five images from the set of solutions from the proposed multi-objective genetic algorithm for the heart data slice present a compromise of high quantification and high detection at the first, 5th, 10th, 20th, and 200 generations, respectively.

To evaluate the quality of the objective functions for performing quantification and detection tasks, we compare objective function values to signal known-exactly mean error and SNR. Figure 8 presents feature mean error vs the quantification objective function value showing the trend of a reduction in error as the quantification objective function increases. A fraction of the solutions, drawn from the highest performers for the quantification metric, have the reverse trend of slightly higher mean error. There is a stronger relationship between the SNR and the detection objective function. Both figures indicate that the proposed objective functions trend with surrogates of task performance for quantification and detection.

Fig. 8:

Reference tests versus objective function values for the set of solutions from the final generation 200 for the heart slice data. Mean error versus quantification objective (top) and signal known exactly SNR versus detection objective (bottom).

Table I compares summary metrics for the background and tumor for the truth, A, B, C, D, E, and F images shown in Figure 6 for the heart slice. As anticipated, image C with the maximum quantification objective function value also has the lowest mean error in the tumor. Image A with the maximum value for detection function obtained from the MOEAP algorithm has the highest signal known exactly SNR suggesting that it is the best image to detect the tumor.

TABLE I:

Summary of reference tests for images from heart slice. Includes mean and standard deviation for known exactly features and background regions, mean error, and SNR.

Images	Background (g/mL)	Tumor (g/mL)	Mean error (%)	SNR
Truth	1.5 ± 0.0	5 ± 0.0	0.0	9.2
A, MOEAP Maximum Detection	1.5± 0.2	3.7± 0.4	−21.9	6.4
B, MOEAP Compromise	1.5± 0.3	4.4± 1.2	−5.8	2.8
C, MOEAP Maximum Quantification	1.4± 0.8	4.7± 2.6	1.5	1.6
D, EM Compromise	1.5± 0.4	4.2± 1.3	−9.2	2.5
E, EM Maximum Detection	1.5± 0.2	3.8± 0.5	−18.6	4.6
F, MAP Maximum	1.5± 0.2	3.5± 0.2	−21.2	7.5

B. Liver Slice

Along with the different distribution of activity and attenuation, we simulated the liver slice with more counts than the heart slice emission. Figure 9 presents the objective function values for generation 200. Similar to the heart slice, the genetic algorithm successfully evolved a set of solutions to improve performance in terms of the quantification and detection objectives. The front of MOEAP solutions is better than solutions obtained from both expectation maximization with post smoothing and penalized likelihood approaches.

Fig. 9:

Objective function values for the final generation of the MOEAP solutions, EM+post smoothing approach and the penalized likelihood solutions for the liver slice.

Figure 10 displays representative images from the liver slice. From the 100 PET images in the front obtained from MOEAP we choose three images, the first image represent the maximum in detection function (point A), the second solution represent equal priority’s to both objective functions (point B), and the third image represents the maximum Poisson log-likelihood function (Point C). Image D is the compromise between two objectives obtained from the maximum likelihood solution with increasing explicit regularization with the amount of smoothing equal to 6 mm. Solution E represents the maximum value for detection function obtained with EM+post smoothing approach with an amount of smoothing 7.5 mm. Image F represents equal priorities to both objective functions using a penalized likelihood approach.

Fig. 10:

Images reconstructed from the liver slice. Presents image with the maximum value for the detection objective function (A), image from with a compromise of high quantification and high detection (B), image with the maximum value for the quantification objective function (C), image from EM+post smoothing regularization approach (D), image from the maximum value for detection function using EM+post smoothing approach (E), and image from the solutions with a compromise of high quantification and high detection using the MAP approach (F). These image labels match solutions presented in figure 9.

Figure 11 shows the relation between mean error and SNR for the feature on the y-axis versus the quantification and detection objective function on the x-axis. Both figures demonstrate that the proposed objective functions correctly trend with objective measures of quantification and detection. The correlations of these measures are higher for this liver slice example compared to the heart slice likely due to the fact that there are more counts (less noise) in the liver simulated data leading to less error in the estimates of performance.

Fig. 11:

Reference tests versus objective function values for the set of solutions from liver slice reconstructions. Mean error versus quantification objective (left) and signal known exactly SNR versus detection objective (right).

C. Convergence performance

We performed empirical testing to explore convergence properties for a different population size and different initializations. Figure 12 presents objective function values obtained solutions from the MOEAP at generations 10, 50, 100, and 200 using 40 members in the population. As with our previous results using 100 members, the solutions converge to a non-dominated front in roughly 100 generations for this problem. Figure 12 plots the mean distance of the front from the origin versus generation using four different random initializations. The mean distance increases with generations for each run until convergence to approximately the same set of solutions, indicating, at least for these proof-of-concept tests, that this approach is robust to different initializations. Future work could perform more thorough evaluation of the impact of population size and initialization along with additional goals of a stopping criterion for convergence and diversity aspects.

Fig. 12:

Top: Objective function values for generations 10, 50, 100, and 200 obtained from the MOEAP solutions for the heart slice. Bottom: Mean distance from the origin for the set of solutions versus generations for the MOEAP solutions using four different random initializations.

V. Discussion

We demonstrate through proof-of-concept application to simulated PET data that the proposed MOEAP can reconstruct a set of solutions along a shared front with high detection and quantification objective function values. And, in this proof-of-concept application, the set of solutions lies on a front that is superior compared to conventional EM reconstruction with post-filtering and the majority of that front is superior to one variation of MAP reconstruction. Visual inspection suggests that the images along the front offer the anticipated tradeoff with images with visually apparent/detectable lesions (fig 6.A and fig 10.A) having higher detection objective values and those with more accurate, although noisier, values having higher quantification values (fig 6.C and fig 10.C). Likewise, images from the middle of the front of solutions offer a tradeoff in performance (fig 6.B and fig 10.B). We also demonstrate that the objective functions, which offer surrogate measures of potential task performance, lead to values that are highly correlated with measures of quantification (mean error of features) and detection (signal to noise of feature to background).

This work is among the first to apply genetic optimization algorithms to tomographic reconstruction and to formulate the image reconstruction problem as having multiple distinct objectives, requiring optimization of a set of solutions. The quantification objective function follows the commonly used PET data fidelity term, the Poisson log-likelihood. It should be stressed that this objective will promote unbiased images, but they may suffer from high variance. Considering both bias and variance contribute to quantification performance, this objective does not capture all the goals of quantification. This objective was selected to ground the proposed exploratory method in arguably the most common objective function in photon counting image reconstruction. Future work could explore more appropriate functions for low bias and variance images.

For the detection objective function, we adapted a scan statistic approach requiring a search for candidate feature present and feature absent locations. The search relies on ad hoc settings, such as a fixed search size and number of candidate regions. While this search worked well in our simulated example with a fixed feature size, future work would benefit from exploring more generalizable detection objective functions. One of the benefits of the proposed genetic optimization is that it does not necessarily require differentiable objective functions. This flexibility would permit new models offering blind estimation of detection performance, perhaps from approaches based on radiomics or deep learned models.

We present a bi-objective function problem. The proposed method of MOEAP could also scale to more than two objectives. For example, optimal images may be needed to perform more than two tasks. For example, one might benefit from a set of solutions optimal for quantification and detection of both small and larger lesions or for both hot and cold features. By increasing the number of objectives to more than two objective functions, the Pareto optimal front will be a surface. Also, increasing objectives to many objectives requires more computation directly proportional to the time required to calculate the objective function values. If the objective functions are very light-weight (requiring minimal computation such as our proposed detection objective function), then adding multiple objectives may not be too cumbersome. The ranking and crowd distance operations are minimal. That said, if more objectives are explored there may be a need for evolving larger populations. There is an extension for the NSGA-II algorithm denoted as NSGA-III to solve many-objective functions (i.e. number of objectives greater than or equal to three). In the NSGA-III algorithm [10], there is a suggested formula to obtain the population size (for example, for 3-obj: N=92, 5-obj: N=212, and 10-obj: N=276).

In essence, objective function values need to be calculated for each candidate solution and this is the dominant computational expense for MOEAP. Given this, there may be value to using light-weight, minimal computation objective functions, such as our proposed detection objective function. For example, adding a third objective that requires 10% the computational cost of the first objective will increase the overall computation of this approach by roughly 10%.

This work proposes a directed mutation operator based on a single EM image update plus a random amount of post smoothing with a fixed kernel shape. Undirected mutation of images would be based on some random image update step and likely require many more generations to converge to a front. That said, an undirected mutation update would be faster than the EM update. Future work could explore the computation and convergence properties of other mutation operators.

Table II compares the computational time between the conventional EM and the proposed MOEAP. Both algorithms were implemented on a consumer-grade laptop (Intel i7; 16Gb of RAM). For both the conventional EM and MOEAP, single image updates are essentially identical considering the MOEAP uses a directed mutation operation that is essentially the same as an EM update (with the addition of a rapid, random amount of smoothing). The evolutionary algorithm in MOEAP requires the evolution of an entire population of solutions and therefore requires more computation than solving for a single image. Likewise, in general the semirandom evolution of solutions requires more image updates (generations/iterations) than conventional approaches. Future work is warranted to explore the necessary population size and faster convergence approaches to achieve optimal solutions. Considering this approach is computationally intensive, this method could be used to refine images reconstructed with conventional methods, in order to present an interpreter with a set of solutions optimal for multiple desired tasks.

TABLE II:

Computation time (in seconds) for conventional and proposed approach applied to simulated data in this study

Approach	Single Update	Final Image
For single solution:
Conventional EM	0.05	2.3 for ~40 it
MOEAP	0.05	10.2 for 200 gen
For population of N = 100 solutions:
MOEAP	2.5	508.0 for 200 gen

The main concern for any multi-objective evolutionary algorithm when applied to real applications is that no one can guarantee solutions on the ideal, efficient Pareto optimal front. Recently, the first author in cooperation suggested a theoretical measure, Karush-Kuhn-Tucker proximity measure (KKTPM), [33], [34] to determine if a solution is a Pareto optimal point or not. Also, these measures attempt to estimate how close a solution is to the efficient front without any prior information about the efficient front. Future work will need to be done to incorporate these measures to see if the obtained PET images from MOEAP are optimal or not.

In addition to the comments above, our work has several limitations. First, we only applied the method to two proof-of-concept simulation studies based on 2D PET data. Future efforts would be warranted to evaluate performance with real data across a range of feature sizes and contrast levels and in the context of fully 3D PET data. Furthermore, we implemented a previously proposed approach for the crossover operation (simulated binary crossover). This approach is very flexible but is reliant on several hyperparameters that may have minimal impact of the convergence properties. Future work is needed to optimize these operations for computation speed and performance.

Finally, current clinical practice relies on the interpretation of a single image. This proposed approach could potentially have value in future practice if different clinical tasks are executed with different images optimized for each task. For example, the human observer could interpret the image optimized for visual assessment, then automatic segmentation could be performed on the image optimized for segmentation, and then quantification performed on the images optimized for bias and then variance. The proposed method offers one potential approach for identifying a set of images optimizing multiple objectives that could be expressive of performance on these different tasks.

VI. Conclusion

We developed an multi-objective evolutionary algorithm for PET (MOEAP) image reconstruction. The objective functions for the proposed algorithm are the Poisson log-likelihood function, used for quantification, and a generalized scan-statistic model, to reflect detection performance. This algorithm uses selection, crossover, and directed mutation operators. This proposed approach was applied to simulated PET data of the heart and liver with a single nodule. To test the proposed surrogate functions for quantification and detection, we used two metrics: feature mean error and feature-to-background SNR. Results for mean error and SNR show a correlation between the actual and surrogate measures. The proposed multi-objective algorithm could potentially enable future interpretation approaches where images are optimized for specific objectives and used to perform different clinical tasks.