Skip to main content
PMC home page
Scientific Reports logoLink to Scientific Reports
. 2017 Feb 9;7:42236. doi: 10.1038/srep42236

Facing the phase problem in Coherent Diffractive Imaging via Memetic Algorithms

Alessandro Colombo 1, Davide Emilio Galli 1,a, Liberato De Caro 2, Francesco Scattarella 3, Elvio Carlino 3
PMCID: PMC5299835  PMID: 28181592

Abstract

Coherent Diffractive Imaging is a lensless technique that allows imaging of matter at a spatial resolution not limited by lens aberrations. This technique exploits the measured diffraction pattern of a coherent beam scattered by periodic and non–periodic objects to retrieve spatial information. The diffracted intensity, for weak–scattering objects, is proportional to the modulus of the Fourier Transform of the object scattering function. Any phase information, needed to retrieve its scattering function, has to be retrieved by means of suitable algorithms. Here we present a new approach, based on a memetic algorithm, i.e. a hybrid genetic algorithm, to face the phase problem, which exploits the synergy of deterministic and stochastic optimization methods. The new approach has been tested on simulated data and applied to the phasing of transmission electron microscopy coherent electron diffraction data of a SrTiO3 sample. We have been able to quantitatively retrieve the projected atomic potential, and also image the oxygen columns, which are not directly visible in the relevant high-resolution transmission electron microscopy images. Our approach proves to be a new powerful tool for the study of matter at atomic resolution and opens new perspectives in those applications in which effective phase retrieval is necessary.

Full-field and scanning microscopes can be either lens-based or lensless imaging systems. Coherent Diffractive Imaging (CDI) is a lensless technique that permits imaging matter at a spatial resolution not limited by lens aberrations. The seminal idea of CDI was due to David Sayre in 19521 but it was only experimentally demonstrated for X-rays in 19992 and, more recently, also for electrons, using a Transmission Electron Microscope (TEM), giving rise to the Electron Diffractive Imaging (EDI)3,4,5.

The goal is to retrieve a qualitative/quantitative image of a scattering function related to a physical property of the scattering object, such as the electron density (X-ray CDI) or the atomic potential (EDI). High Resolution TEM (HRTEM) images of the projected atomic potential are phase-contrast images limited by the high-order aberrations of the objective lens, which distort the phase of the scattered wave function, giving rise to images of the sample, which in general are not immediately interpretable in terms of its atomic structure6. Instead, diffraction patterns of scattering objects are not affected by these aberrations. Therefore they contain, in principle, undistorted information on the scattering function at a better spatial resolution with respect to lens-based imaging systems3,4,5. The diffracted intensity, for weak-scattering objects, is proportional only to the modulus of the Fourier Transform (FT) of the scattering function. Any phase information, which is experimentally lost (phase problem1), has to be retrieved by means of suitable algorithms. The lensless image of the scattering function, obtained by means of an inverse FT of the diffraction pattern once that the correct phase has been retrieved, is characterized by a final resolution experimentally limited only by the Numerical Aperture (NAdiff) corresponding to the highest spatial frequency contained in the diffraction pattern that can be related to the atomic structure of the investigated sample5. Wavelength, noise, radiation damage, thermal and mechanical stability of the experimental setup, dynamics of the detector, etc.7 could limit the spatial resolution achievable.

In order to find a unique solution to the phase problem, phase retrieval algorithms need a-priori constraints, such as fixing a region around the sample characterized by zero scattering8. The extension of the zero-scattering region has to be large enough regarding the object support S, containing the scattering object. This requirement is the so-called oversampling of the diffraction pattern, needed to satisfy the Nyquist sampling requirements1.

For extended samples, alternative approaches have been developed to have enough a-priori information to make the phasing problem overdetermined. In particular, Abbey et al.9 for X-rays CDI experiments substituted the region of zero scattering with a region of zero illumination using a confined probe with a technique named keyhole CDI. The possibility to perform Keyhole coherent diffraction experiments has been demonstrated also for electrons in EDI, obtaining the KEDI approach10.

So far, several phase retrieval algorithms have been developed11, which are mostly evolutions of Fienup’s modification8 of the Gerchberg-Saxton’s algorithm12 working with a dual-space strategy. The common approach of many phasing algorithms is to impose constraints both in real space (the prior information on the zero scattering/illuminating region) and in Fourier space (the amplitudes are adapted to the experimental values). The imposition of a constraint in one space always causes the violation of the constraint in the other. Consequently, the standard strategy is to use iterative schemes but, in this way, the global minimum of the reconstruction errors is often reached with difficulty13. Indeed, the support S of the scattering function is either unknown (X-ray CDI) or known at a worse spatial resolution. This is the case of EDI, for example, where the support is obtained by means of a lens-based image of the scattering function (projected atomic potential), i.e. by the HRTEM image.

In case of unknown complex scattering functions the knowledge of the support S (non-zero illumination region) at the same spatial resolution of the measured diffraction pattern seems to be mandatory for the success of the phasing, especially when phasing algorithms are not suitably structured to escape stagnation in local minima5,10. This aspect would limit the advantages of indirect imaging based on lensless systems with respect to conventional lens-based set-ups.

Indeed, standard phasing approaches, such as the Hybrid Input-Output (HIO), are mainly deterministic iterative algorithms, which, going back and forth from the real to the Fourier space, try to optimize a specific error functional8. For this reason they are highly efficient in finding local minima, but they suffer from stagnation mainly due to the incomplete knowledge of the support S. Furthermore, the final result is highly dependent on the initial conditions13. In order to overcome these limitations, phasing procedures usually involve a lot of parallel and independent retrieval processes, with different initial conditions, choosing the scattering function with the lowest reconstruction errors as a possible solution. A first step toward a smarter use of information coming from multiple phase retrieval processes is the Guided Hybrid Input-Output algorithm14.

A different approach to face the phase problem in CDI could make use of pure stochastic optimization methods. However, a major limitation of these methods is their inefficency once the number of unknowns is huge, so that, in typical CDI applications, such an approach is doomed to fail even by exploiting actual super–computing facilities.

In this article we make a step further proposing a new hybrid stochastic approach to better explore the phase solution space through a smart use of Genetic Algorithms (GAs)15. GAs have been already applied to the phase problem in different fields16,17,18,19. The novelty of our new approach consists in the development of a Memetic Algorithm (MA)20 in the context of phase retrieval applied to CDI; this scheme represents a natural choice for a smart merging of stochastic and deterministic optimization methods: the algorithm has been developed hybridizing a GA, which guarantees a wide exploration of the configuration space, with local optimization algorithms like Hybrid Input-Output and Error Reduction.

We have shown on simulated data that the MA phasing approach is able to retrieve the correct scattering function, when it is real, imposing a very loose support constraint S in the direct space. Moreover, still on simulated data, we have shown that the new MA phasing approach is able to retrieve the correct scattering function, even if it is complex, starting from a knowledge of the support S at a resolution four times worse than the one corresponding to NAdiff, even worse than normally observed in EDI/KEDI real experiments. Our new approach shows convergence performances towards the global minimum that go well beyond those achievable by standard phasing deterministic algorithms.

Finally, we have applied the GA-based phasing approach to a KEDI experiment realized on a SrTiO3 sample in a [100] axis orientation. The image obtained after the phase reconstruction is a detailed structural map of the specimen atomic potential projected along the [100] direction at a sub-Ångström spatial resolution corresponding to the highest frequency measured in the experimental diffraction pattern. The intensity distribution enables one to distinguish between atomic sites containing different chemical species. Also the oxygen signals can be detected, despite the presence of heavy atoms in the crystal cell, whereas they are not visible in the relevant HRTEM image.

These results pave the way to the highest spatial resolution, accuracy and reliability achievable in lensless imaging and represent a new powerful tool for the study of matter.

Results and Discussion

The Memetic Phase Retrieval approach

A GA15 is a stochastic optimization method that imitates the survival–to–fitness typical of the natural evolution of a population. In general, this is obtained by elaborating the genetic information via three genetic operators: Selection, Crossover and Mutation. In our algorithm we induce the genetic dynamics on a set of initial densities Inline graphic (also called population) which represents Np possible solutions to the phase problem. In standard GAs Mutation and Crossover induce a stochastic shift on every element of the population in the space of configurations; this improves the ability of exploring the space, but makes the GA efficient only when Np increases with the number of unknowns21. In the phase retrieval problem this condition makes standard GA impracticable due to the computational cost.

A practised way to overcome this issue consists in implementing hybrid GAs, known as Memetic Algorithms (MAs)22,23,24,25, where global optimization is boosted by local optimization procedures. This operation, in the framework of Memetic Algorithms, is known as Self (or Local) Improvement.

Using this method, we introduce local iterative deterministic phase retrieval procedures in our algorithm as Self Improvement operation. For this reason, the new proposed phasing approach is called Memetic Phase Retrieval (MPR). Figure 1 shows an overview on the procedure, while a more detailed description of the method is reported in the Methods section. It is worth noting that the standard phasing approach can be seen as MPR without the genetic operators Selection, Crossover and Mutation. MPR has to be considered as a smart framework to take advantage and improve the performances of any current iterative phase retrieval approach; it is clear, in fact, that any phase retrieval algorithm can be implemented in MPR. In this work we use the Error Reduction and Hybrid Input-Output algorithms as Self Improvement operations because they are simple, well known and well-characterized methods. Moreover, as we will show in the following, the sole inclusion of HIO and ER inside MPR is enough to build a very powerful phase retrieval algorithm. MPR actually includes also methods for the retrieval of the optimal support function, like the Shrinkwrap algorithm26; this feature makes MPR a Co-evolving Memetic Algorithm27, where the Self Improvement co-evolves along with candidate solutions. This peculiar feature will be discussed in future works because it has not been used in the present application on KEDI data, where sufficient information about the support function was available, making Shrinkwrap procedure unnecessary.

Figure 1. Comparison between the MPR algorithm and the standard phasing approach.

Figure 1

Open in a new tab

Testing MPR on simulated data

In the Supplementary Information we discuss in detail several numerical tests performed to verify the potentialities of the new proposed approach to retrieve in a reliable way phase information in comparison to standard deterministic phasing approaches. Here, we reassume the main results obtained by simulations before discussing the application of MPR on true experimental data of a KEDI experiment.

In the comparison of the performances between MPR and standard phasing algorithms, applied on simulated data, we will focus on the best phase reconstructed by both methods and not on a statistical analysis of the results obtained from the set of phase retrieval processes started in parallel with different initial conditions. This is necessary because the genetic dynamics has the distinctive feature to mix and share parallel information during the stochastic evolution; this makes smarter the stochastic search for a better phase, but also tends to push the population near to the best candidate solution. A statistical analysis would be thus biased in favor of MPR. Moreover, we are going to compare MPR and standard phasing algorithms under equal conditions: the same number of candidate solutions in the population and the same kind of iterative phase retrieval algorithms. The first test of comparison between MPR and the standard phasing approach is a phase retrieval of a positive and real-valued two-dimensional (2D) scattering function, as shown in Fig. 2a. In the test the provided support function S (Fig. 2b) is a square four times smaller than the total area of the direct space, which gives a constraint ratio28 Ω = 2. With this regard it is useful to remember that Ω ≥ 1 is the mathematical condition to assure the existence of a unique solution to the phase problem. Ω is defined as the ratio between the support of the autocorrelation of the scattering object and two times the object support, which are areas for 2D phase problem, volumes for 3D ones. In this example, the support function is not updated during the phase retrieval.

Figure 2. Real-valued data.

Figure 2

Open in a new tab

(a) Unknown test function (Lena, adapted from the picture 4.2.04 in the USC–SIPI image database29), (b) support function.

Figure 3 shows the obtained results for a population Np = 16384. In particular, Fig. 2b shows the support S, never updated during the phasing process. Figure 3a shows the retrieved unknown function obtained by standard phasing algorithms (see Supplementary Information for further details). The poor quality of the retrieved phase is due to the weak constraint in real space, as the support constraint S has not been updated. Figure 3c shows the retrieved unknown function obtained by MPR (see Phase retrieval of real-valued data in Supplementary Information for further details).

Figure 3. Real-valued phasing test.

Figure 3

Open in a new tab

Retrieved function (a) and a zoom (b) for the standard approach; retrieved function (c) and a zoom (d) for MPR.

The performances of the new proposed stochastic approach are much better than the classic deterministic phasing methods. MPR works accurately even without a tight support constraint.

In the case of simulated data one can evaluate the true error defined as the normalized absolute difference for every pixel of the retrieved 2D function with respect to the solution (Lena image, adapted from the picture 4.2.04 in the USC–SIPI image database29) (see Supplementary Information for more details). The true error for the deterministic phasing approach is larger than 24%. Instead, MPR leads to a true error less than 1%.

A second numerical test of MPR concerns the reconstruction of a complex-valued scattering function. In particular, we have considered a situation typically encountered in EDI, in which HRTEM allows us to obtain a lens-based image of the sample under investigation characterized by a worse spatial resolution with respect to that corresponding to the NAdiff of the measured diffraction pattern. In order to simulate this experimental situation, we have binned with a factor 4 the module of the scattering function to be retrieved (Fig. 4a) to obtain a rough estimation of its support, thresholding the binned image as shown in Fig. 4c, whereas Fig. 4b shows the phase in direct space that has also to be retrieved.

Figure 4. Complex-valued data.

Figure 4

Open in a new tab

(a) Unknown modulus of the complex-valued function (modulus of the solution); (b) unknown phase of the complex-valued function (phase of the solution), with range 0–360°; (c) initial support function; (d) representation of the complex solution (adapted from the image at http://openwalls.com/image?id=17210, Copyright Creative Commons Attribution 3.0 Unported), assigning to the brightness the values of the modulus and to the hue the values of phases, following the Hue-Saturation-Value (HSV) color system.

This data can be represented as depicted in Fig. 5a via the Hue-Saturation-Value (HSV) color system, where the information on the phase is stored in the hue, the modulus corresponds to the value and the saturation level is set to the maximum. Even in this case, the standard phasing approach is far from recovering the correct complex scattering function, reported in Fig. 5b. Instead, MPR is able to correctly retrieve both module and phase of the complex unknown scattering function (Fig. 5c) (see Supplementary Information for further details). The final true error for this test is about 10% for the standard approach and 1.5% for MPR.

Figure 5. Complex-valued phasing test.

Figure 5

Open in a new tab

(a) Detail of the solution (adapted from the image at http://openwalls.com/image?id=17210, Copyright Creative Commons Attribution 3.0 Unported); (b) Retrieved scattering function for the standard approach; (c) Retrieved scattering function for MPR. The values of the modulus are assigned to the brightness while the values of phases are assigned to the hue, following the Hue-Saturation-Value (HSV) color system.

Application of MPR on experimental data for Keyhole Electron Diffractive Imaging

KEDI experiments10 are challenging tests for phase retrieval algorithms as the scattering function to be reconstructed is complex. SrTiO3 was considered as a case study for the great importance of this oxide from both an applicative and a fundamental point of view. The role of the oxygen sub-lattice is of particular importance in the studies of two dimensional electron gases formed at the interface between two insulating oxides and has recently attracted great attention30. The capability to image the lattice of complex oxides at atomic resolution is necessary to understand the intriguing properties of this class of material. Moreover imaging of a light chemical element, such as oxygen, in a matrix of heavier atoms, like titanium and strontium, is not straightforward31,32. The samples were prepared for KEDI experiments in a [100] zone axis as this configuration enables the imaging of different atomic species in the crystal sub-lattice (see Methods section).

KEDI requires an HRTEM image (Fig. 6a) and a nano-diffraction pattern acquired from the same sample area with the same electron optical conditions10. The HRTEM experiment enables us to image the sample (Fig. 6a,b), to complement the diffraction pattern at the lower spatial frequency (Fig. 6c) and to estimate the support (Fig. 6d) at the resolution allowed by the experimental conditions and by the electron objective lens aberrations10. In the case of the electron-optical set-up used for these experiments the relevant spatial resolution in the HRTEM image at optimum defocus is 0.19 nm33. The HRTEM image in Fig. 6b has been successfully simulated in the framework of full dynamical Bloch-wave approach34 for a thickness of 25 nm and an underfocus value of 41.3 nm (see Supplementary Information). It should be noted that the phase contrast in the HRTEM image of Fig. 6b does not show any evident clues that could be correlated to the presence of the oxygen atomic columns which should be seen in the [100] projection of the SrTiO3 atomic potential, as evidenced in the simulation (see Fig. S7 of Supplementary Information). Indeed, it is worthwhile to remark that the HRTEM image is, in general, an interference pattern of the waves scattered by the atomic potentials in the specimen, and the positions of the maxima and minima in the image cannot be straightforwardly interpreted as structural features and therefore the comparison with the simulated images is needed6 (see Fig. S7 of Supplementary Information).

Figure 6. KEDI data.

Figure 6

Open in a new tab

(a) Experimental HRTEM image in [100] zone axis and (b) zoom; (c) KEDI diffraction pattern used as input for the phasing; (d) initial support function extracted from (a).

The KEDI diffraction pattern, shown in Fig. 6c has been obtained by combining the measured diffraction pattern with the modulus of the HRTEM image FT, after a suitable matching procedure requiring its rotation and scaling10. The pattern in Fig. 6c is the starting point for the phase retrieval process. It is worth noting that the MPR phasing process has been carried out without any a–priori information about the phases, an information which is, instead, needed by standard phasing procedures applied to KEDI5. Further details on MPR applied to experimental data have been reported in the Supplementary Information.

Figure 7 shows the retrieved scattering function obtained by using MPR, where the brightness corresponds to the modulus and the hue to the phase of the retrieved real-space complex-valued scattering function. The long range phase variation is due to the phase variation of the illumination nano-probe35.

Figure 7. Recovered image.

Figure 7

Open in a new tab

Brightness is proportional to the retrieved modulus and colors indicate the retrieved phase.

Figure 8a shows the phase retrieved amplitude for the structure of the SrTiO3 seen in a [100] projection. Figure 8d has been obtained by subtracting the contribution of the TEM illumination function to quantify the SrTiO3 projected potential. The first important point that should be emphasised, is that in the phase recovered image the positions of the maxima are correctly in correspondence with the expected positions of the atomic columns seen in the [100] projection. In other words, the phase reconstructed image is a structural image of the specimen. This is a fundamental issue that paves the way for quantitative structural imaging at atomic resolution. Indeed, as shown in Fig. 8d, the Sr and Ti + O columns are precisely seated on the relevant square sublattice of the SrTiO3 in the [100] projection (see Fig. 8b,c). Approximately in the center of the sublattice there is a lower signal which corresponds to the oxygen columns. A second point concerns the retrieval of quantitative information about the atomic potential. By comparing data shown in Fig. 8d with the expected projected atomic potential (Fig. 8c) we found that the ratios between the intensities of Sr, Ti + O and O columns are correctly retrieved, providing truly quantitative information on the specimen. In particular, the expected intensity ratios are IO/ISr = 0.35 and ITi+O/ISr = 0.96 while the experimental retrieved data give IO/ISr = 0.35 ± 0.05 and ITi+O/ISr = 0.89 ± 0.10. The possibility to do KEDI experiments by MPR reconstruction paves the way for a detailed structural characterization of the investigated samples and opens up new possibilities for the understanding of the properties of the matter at sub-Ångström resolution.

Figure 8. Results of MPR on experimental KEDI data.

Figure 8

Open in a new tab

(a) Modulus of the retrieved scattering function (relevant to 6a); (b) SrTiO3 unit cell; (c) simulation of the SrTiO3 projected potential in [100] zone axis and (d) experimental data extracted from the phased map.

Conclusions

In this work we have discussed some of the potentialities of a new phasing approach, stochastic in the exploration of the space of solutions, based on a memetic algorithm, applicable both to X-ray and electron coherent diffraction imaging. We have tested the new phasing algorithm, named Memetic Phase Retrieval, on simulated data and we have obtained the result that the knowledge of the support - which defines the boundaries in the direct space of the unknown scattering function that one wants to retrieve - is less binding than previously reported. The more efficient exploration of the space of solution, possible thanks to the stochastic genetic procedures implemented in MPR, should be the higher gear of the new proposed phasing method with respect to those already available. Indeed, both the possibility to correctly retrieve a real-valued scattering function by its diffraction pattern without imposing any tight support and to reconstruct a complex-valued scattering function by using a low-resolution estimate of the support, are examples of the great capabilities of MPR to face the phase problem. Our tests on simulated data demonstrate the superior capabilities of the MPR for accurate phase retrievals. Indeed, by using the same computational resources, the MPR approach has proved to be much more powerful than deterministic phasing procedures in facing the phase problem. The application to an experimental case of Keyhole Electron Diffraction Imaging has shown that the atomic potentials of SrTiO3 can be quantitatively imaged, representing a relevant improvement for the study of the matter. We believe that the Memetic Phase Retrieval approach could be of interest in all the fields that require accurate phase retrievals.

Methods

The phase problem as an optimization problem

The ideal solution to the phase problem is a function ρs(x), representing the spatial distribution of the sample, whose Fourier Transform (FT), Inline graphic, has a square modulus equal to I(q), which is proportional to the experimental diffraction pattern intensity. ρs(x) is also assumed to be zero outside a well-defined region of the real space, the so-called support S, in order to satisfy the oversampling condition, which assures the necessary information to retrieve ρs(x)1. Here, x = (x, y), with x and y the cartesian components of the position vector x with respect to the reference system. Analogously, q = (u, v), where u and v are the spatial frequencies components with respect to the reference axes.

In principle, this solution can be represented as an intersection of sets. Inline graphic can be defined as the set of all functions ρ(x) compatible with experimental data I(q), i.e.:

graphic file with name srep42236-m4.jpg

Inline graphic is, instead, the set of all functions ρs(x) satisfying the oversampling condition, which is defined by a binary function Π(x) representing the object support S. So, the set Inline graphic is described by:

graphic file with name srep42236-m7.jpg

Thanks to (1) and (2), the ideal solution is:

graphic file with name srep42236-m8.jpg

The main issue concerning experimental measurements is the presence of noise and lack of data; this, in general, implies that Inline graphic and Inline graphic do not intersect:

graphic file with name srep42236-m11.jpg

Due to the condition (4) a different way to define what we mean by “solution” is needed. It is useful, at this point, to introduce two projection operators, which act on the function ρ(x):

graphic file with name srep42236-m12.jpg
graphic file with name srep42236-m13.jpg

It’s trivial to prove that Inline graphic and Inline graphic are projectors on sets Inline graphic and Inline graphic, previously defined in (1) and (2). Thanks to these operators, it’s now possible to give a new definition of solution in place of the one defined in the eq. (3):

graphic file with name srep42236-m18.jpg

where the functional D[A, B] represents the metric of the space. Hereafter, we will refer to the eq. (7) whenever we will talk about the “solution” of the problem, ρs(x).

It’s now clear that, in this framework, finding a solution to the phase problem means minimizing the distance between sets Inline graphic and Inline graphic: the phase problem becomes an optimization problem for the quantity Inline graphic, which can be reinterpreted as the error of the recovered density ρ(x). Different definitions of the metric imply different definitions of the error assigned to a given ρ(x) and, as consequence, different optimization targets. We can define the error functional E[ρ] as

graphic file with name srep42236-m22.jpg

such that the eq. (7) turns into

graphic file with name srep42236-m23.jpg

Standard approaches to the phase problem are mainly deterministic iterative algorithms which, going back and forth from the real to the Fourier space, try to minimize a specific error functional8. These methods are highly efficient in finding local minima, but they suffer from stagnation and the final result is highly dependent on the initial conditions13. In order to overcome these issues, phasing procedures usually involve a lot of parallel and independent retrieval processes with different initial conditions and then selecting the one with the lowest error.

The founding idea of the new proposed phasing method is to better perform this parallel exploration of the space, through the use of a Memetic Algorithm.

Selection as a Rigged Roulette

The Selection process is a delicate step in the Evolution process. A Selection strongly favoring only the better elements in Inline graphic (i.e., elements with the better fitness value) will improve the convergence speed, but the algorithm will suffer with stagnation in local minima. On the other side, a selection process that weakly favors those elements will have, instead, an unstable convergence and will require an excessive length of time to find the solution.

There are several ways to select elements depending on their fitness value. The one chosen in this work is the so-called “rigged roulette”. Once an error value Ei is assigned to every ρi(x) in Inline graphic according to the eq. (14), the set Inline graphic is ordered by increasing values of Ei (which is equivalent to a decreasing values of the fitness). Whenever the algorithm has to select a ρi(x) in Inline graphic for the Crossover operation, an index is extracted through the relation

graphic file with name srep42236-m28.jpg

where r ≥ 1 is related to the “strength” of the selection process. Usual values of r range from 1.5 to 2.5.

Eq. (10) maps a flat distribution in Inline graphic to an unbalanced distribution in Inline graphic, where the higher the value is the greater the probability is of getting a lower index and, therefore, selecting a better element.

Differential Crossover

In the Natural Evolution process, the Crossover operation is the mixing of the parents’ genetic pool. In our implementation chromosomes are represented by every single (complex) value of Inline graphic. This means that, given two parent functions ρ1(x) and ρ2(x) selected according to their fitness, the son function ρson(x) is created according to

graphic file with name srep42236-m32.jpg

where rand [0, 1) is a random number with flat distribution in [0, 1) and C is a balancing coefficient between 0 and 1.

An improvement in performances can be obtained using the so called Differential Crossover36 where, instead of selecting two parents, four parents, ρ1(x) ρ2(x) ρ3(x) and ρ4(x), are chosen. The differential crossover acts as follows:

graphic file with name srep42236-m33.jpg

where Dc is called differential coefficient with typical values between 0.5 and 1.5.

The population of sons can be, in general, smaller than the whole population. This means that if the population of sons has Ns = G · Np elements, a fraction of Np − Ns parents, chosen randomly, will survive to the next generation.

The parameter G, which can be called genetic fraction, has values between 0 and 1: G = 1 means that all of the parent population is replaced by the sons, while G = 0 means that no sons are created, the genetic operators are switched off and we get a situation equivalent to the standard deterministic approach, as depicted in Fig. 1.

Mutation

Every element in the population Inline graphic may be subjected to a stochastic modification. In this work, the mutation operation has been switched off because it does not introduce a remarkable improvement in the performance of MPR on treated data. Different implementations of the mutation operator are under study and will be topics of future works.

Self improvement via deterministic optimization

Optimization algorithms such as Error Reduction (ER) and Hybrid Input-Output (HIO) are efficient methods to find local minima or, more precisely, minima bounded to a region of the configuration space near the starting point. These algorithms are strictly bounded to the metric D[A(x), B(x)] defined as:

graphic file with name srep42236-m35.jpg

This implies that the local optimization target is the functional E[ρ] defined as:

graphic file with name srep42236-m36.jpg

In our algorithm this local optimization is carried on by elaborating every ρi(x) with NHIO iterations of the Hybrid Input-Output algorithm and NER iterations of the Error Reduction algorithm. In this work the global optimization target, i.e., the fitness of MPR, coincides with the local optimization target of ER and HIO algorithms just shown in (14). This is not to be taken for granted because, in general, we can define any arbitrary global optimization target different from the local one (13). We are testing different fitness definitions for the global optimization, like the Csiszar’s Information Divergence37, and different local optimization algorithms.

The choice of the initial guess

The phase retrieval process can be divided into two main steps. The first one concerns the choice of the initial population of densities Inline graphic. In the second step, we have to choose the parameters both of the genetic and the local optimization algorithms.

Standard approaches like Hybrid Input Output and Error Reduction need a single initial guess, which represent the first estimation of the solution.

MPR approach requires, instead, a set of initial guesses. This set is produced from a single guess, simply randomly shifting every phase. This means that, given an initial guess ρinit(x), every ρi(x) in Inline graphic is created via the relation

graphic file with name srep42236-m39.jpg

The parameter Rc, which has values between 0 and 1, depends on the accuracy of the initial guess ρinit(x). If ρinit(x) is already a good estimation of the solution, it will be useful to set a low value (usually near to 0) for coefficient Rc in order to well explore the space near ρinit(x). If, instead, ρinit(x) is considered to be far from the solution, it is useful to set a value of Rc near to 1, in order to explore also areas of the space far from ρinit(x).

KEDI experiment

A KEDI experiment was performed following the procedure reported in ref. 10, which enables one to deliver a low dose of electrons to the specimen. The experiment requires the acquisition of an HRTEM image and a diffraction pattern by using the same electron optical set up. The experiments were performed by using a JEOL JEM 2010 F UHR operated at 200 kV. The cathode is a high coherence Shottky type. The microscope has an objective lens with low spherical aberration coefficient Cs = 0.47 ± 0.01 mm and a relevant resolution at optimum defocus in HRTEM of 190 pm. The environment around the microscope is thermally and mechanically very stable allowing us to achieve in the scanning TEM (STEM) high angle annular dark field (HAADF) mode a resolution of 126 pm, which is the theoretical limit for the used electron optical set up38.

In a KEDI experiment the optical setup produces an electron nano-beam. The latter defines the mathematical support of the scattering function for the illuminated nanometric region of the extended crystal. As in a microscope the field of view is proportional to the inverse of magnification, the size of the illumination function (beam size) is somehow related to the spatial resolution. The electron beam size S (which defines the support) is directly related to the final resolution to be achieved and to the size of the detector used to record HRTEM image and n-ED pattern. In fact, if the highest frequency of the diffraction signal recorded in the reciprocal space is ρ−1 pm−1, we should have ρ at least two or three times the pixel size Δmap of the phased map to have an electron projected potential two-dimensional map calculated with a sufficient number of points to be plotted continuously.

For example, if we reached a final resolution – after the phase retrieval process – of ρ = 70 pm we should have Δmap ~ 25 ÷ 30 pm which, multiplied by the detector pixel number along a line, N = 1024 in our experimental case, would lead to a spatial region O (scattering region plus non-illuminated surrounding region) of ~25 ÷ 30 nm in size. Moreover, for the Nyquist theorem’s requirement, the illuminated beam size S (the support) has to be less than Inline graphic, i.e., at maximum ~17 ÷ 20 nm in size. Hence, in order to properly run the phase retrieval algorithms, the illuminated region of the sample in the direct HRTEM image has to be properly chosen with respect to the whole detector area to satisfy the above KEDI oversampling condition.

Here, the cathode emission condition and the electron optical illumination system of the microscope has been experimentally set up to increase the probe coherence on the smallest illuminated area achievable10. The microscope has an illumination system composed by three magneto-static lenses. These lenses were operated independently, together with the electrostatic lens of the emitter, to produce the smallest-sized probe on the focal plane of the pre-field of the objective lens and hence the smallest-sized coherent parallel beam on the specimen. The emission conditions of the microscope cathode were chosen to increase the coherence of the electron probe by decreasing the temperature of the emitting tip. We used a heating current for the filament that halves the emission current with respect to the standard operation, decreasing at the same time the electron dose delivered to the specimen. The current density on the specimen was below the detection limit of the amperometer connected to the phosphorus screen of the microscope (<0.1 pA cm−1), allowing us to acquire the relevant diffraction pattern on the 1024 × 1024 Charge-Coupled Device (CCD) camera without using the beam stopper for the direct beam. Thus all the diffracted intensities were available for the phasing process and a very small dose is delivered to the specimen. The small electron probe, without any changes, was used to acquire both HRTEM image and diffraction from the same area of the specimen. Figure 6a shows the HRTEM image. The illuminated area is 10 ± 2 nm. The interference pattern of the phase contrast HRTEM image formed in the image plane of the objective lens is shown at a higher magnification in Fig. 6b. In Fig. 6c the diffraction pattern formed in the back focal plane of the objective lens is shown. The central part of the pattern has been replaced, after proper scaling and rotation, by the FFT of the HRTEM image in Fig. 6a, as established in EDI method3,4,5. The highest Miller’s index spot measurable in the pattern is the (5, 5, 0), which corresponds to a spacing of 55 pm. Thus, the expected gain in resolution of the maximum spatial frequency contained in the diffraction pattern (~NAdiff−1) with respect to that corresponding to the FT of the HRTEM image is about four times.

Additional Information

How to cite this article: Colombo, A. et al. Facing the phase problem in Coherent Diffractive Imaging via Memetic Algorithms. Sci. Rep. 7, 42236; doi: 10.1038/srep42236 (2017).

Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Material

Supplementary Information
srep42236-s1.pdf (1MB, pdf)

Acknowledgments

This work was supported by the NOXSS PRIN (2012Z3N9R9) project and Progetto premiale MIUR 2013 USCEF. We acknowledge the CINECA and Regione Lombardia LISA award LI05p-PUMAS, for the availability of high-performance computing resources and support.

Footnotes

The authors declare no competing financial interests.

Author Contributions A.C. and D.E.G. conceived the algorithm. A.C., supervised by D.E.G. and L.D., developed, implemented and tested the algorithm, and carried out the reconstructions. E.C. designed and performed the TEM EDI experiments, contributing to the optimization of the relevant data reduction and phasing. L.D. and F.S. made data reduction. All authors have equally contributed to the preparation and the revision of the text.

References

  1. Sayre D. Some implications of a theorem due to Shannon. Acta Crystallographica 5, 843 (1952). [Google Scholar]
  2. Miao J., Charalambous P., Kirz J. & Sayre D. Extending the methodology of x-ray crystallography to allow imaging of micrometre-sized non-crystalline specimens. Nature 400, 342–344, doi: 10.1038/22498 (1999). [DOI] [Google Scholar]
  3. Zuo J., Vartanyants I., Gao M., Zhang R. & Nagahara L. Atomic resolution imaging of a carbon nanotube from diffraction intensities. Science 300, 1419–1421 (2003). [DOI] [PubMed] [Google Scholar]
  4. Huang W., Zuo J., Jiang B., Kwon K. & Shim M. Sub-ångström-resolution diffractive imaging of single nanocrystals. Nature Physics 5, 129–133 (2009). [Google Scholar]
  5. De Caro L., Carlino E., Caputo G., Cozzoli P. D. & Giannini C. Electron diffractive imaging of oxygen atoms in nanocrystals at sub-angstrom resolution. Nat Nano 5, 360–365, doi: 10.1038/nnano.2010.55 (2010). [DOI] [PubMed] [Google Scholar]
  6. Spence J. C. Experimental high-resolution electron microscopy (Oxford University Press, 1988). [Google Scholar]
  7. De Caro L., Carlino E., Siliqi D. & Giannini C. Coherent diffractive imaging: From nanometric down to picometric resolution. In Handbook of Coherent-Domain Optical Methods, 291–314 (Springer, 2013). [Google Scholar]
  8. Fienup J. R. Phase retrieval algorithms: a comparison. Appl. Opt. 21, 2758–2769 (1982). [DOI] [PubMed] [Google Scholar]
  9. Abbey B. et al. Keyhole coherent diffractive imaging. Nature Physics 4, 394–398 (2008). [Google Scholar]
  10. De Caro L., Carlino E., Vittoria F. A., Siliqi D. & Giannini C. Keyhole electron diffractive imaging (KEDI). Acta Crystallographica Section A 68, 687–702, doi: 10.1107/S0108767312031832 (2012). [DOI] [PubMed] [Google Scholar]
  11. Shechtman Y. et al. Phase retrieval with application to optical imaging: a contemporary overview. IEEE signal processing magazine 32, 87–109 (2015). [Google Scholar]
  12. Gerchberg R. & Saxton W. A practical algorithm for the determination of the phase from image and diffraction plane pictures. Optik (Jena) 35, 237 (1972). [Google Scholar]
  13. Marchesini S. Invited article: A unified evaluation of iterative projection algorithms for phase retrieval. Review of Scientific Instruments 78, –, doi: 10.1063/1.2403783 (2007). [DOI] [PubMed] [Google Scholar]
  14. Chen C.-C., Miao J., Wang C. & Lee T. Application of optimization technique to noncrystalline x-ray diffraction microscopy: Guided hybrid input-output method. Physical Review B 76, 064113 (2007). [Google Scholar]
  15. Goldberg D. E. Genetic Algorithms in Search, Optimization and Machine Learning, 1st edn. (Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 1989). [Google Scholar]
  16. Thust A., Lentzen M. & Urban K. The use of stochastic algorithms for phase retrieval in high resolution transmission electron microscopy. Scanning Microscopy 11, 437–454 (1997). [Google Scholar]
  17. Nicholson J., Omenetto F., Funk D. & Taylor A. Evolving FROGS: phase retrieval from frequency-resolved optical gating measurements by use of genetic algorithms. Optics letters 24, 490–492 (1999). [DOI] [PubMed] [Google Scholar]
  18. Taylor J. R., King B. A. III, Steincamp J. & Rakoczy J. Genetic algorithm phase retrieval for the systematic image-based optical alignment test bed. Publications of the Astronomical Society of the Pacific 118, 319 (2006). [Google Scholar]
  19. Li N., Gao P., Lu Y., Yu W. & Yu B. Phase retrieval for hard x-ray in-line phase contrast imaging based on a parallel hybrid genetic algorithm. In Computational Sciences and Optimization (CSO), 2011 Fourth International Joint Conference on, 66–70 (IEEE, 2011). [Google Scholar]
  20. Ong Y.-S., Lim M. H. & Chen X. Research frontier-memetic computation–past, present & future. IEEE Computational Intelligence Magazine 5, 24 (2010). [Google Scholar]
  21. Goldberg D. E. Sizing populations for serial and parallel genetic algorithms. In Proceedings of the 3rd international conference on genetic algorithms, 70–79 (Morgan Kaufmann Publishers Inc., 1989). [Google Scholar]
  22. Moscato P. et al. On evolution, search, optimization, genetic algorithms and martial arts: Towards memetic algorithms. Caltech concurrent computation program, C3P Report 826 (1989). [Google Scholar]
  23. Renders J.-M. & Flasse S. P. Hybrid methods using genetic algorithms for global optimization. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 26, 243–258 (1996). [DOI] [PubMed] [Google Scholar]
  24. El-Mihoub T. A., Hopgood A. A., Nolle L. & Battersby A. Hybrid genetic algorithms: A review. Engineering Letters 13, 124–137 (2006). [Google Scholar]
  25. Neri F. & Cotta C. Memetic algorithms and memetic computing optimization: A literature review. Swarm and Evolutionary Computation 2, 1–14 (2012). [Google Scholar]
  26. Marchesini S. et al. X-ray image reconstruction from a diffraction pattern alone. Phys. Rev. B 68, 140101, doi: 10.1103/PhysRevB.68.140101 (2003). [DOI] [Google Scholar]
  27. Smith J. E. Coevolving memetic algorithms: a review and progress report. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 37, 6–17 (2007). [DOI] [PubMed] [Google Scholar]
  28. Millane R. P. & Arnal R. D. Uniqueness of the macromolecular crystallographic phase problem. Acta Crystallographica Section A: Foundations and Advances 71 (2015). [DOI] [PubMed] [Google Scholar]
  29. Weber A. G. The USC-SIPI image database version 5. USC-SIPI Rep. 315, 1–24 (1997). [Google Scholar]
  30. Banerjee H., Banerjee S., Randeria M. & Saha-Dasgupta T. Electronic structure of oxide interfaces: A comparative analysis of GdTiO3/SrTiO3 and LaAlO3/SrTiO3 interfaces. Scientific reports 5 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Varela M. et al. Atomic scale characterization of complex oxide interfaces. Journal of materials science 41, 4389–4393 (2006). [Google Scholar]
  32. Muller D. A., Nakagawa N., Ohtomo A., Grazul J. L. & Hwang H. Y. Atomic-scale imaging of nanoengineered oxygen vacancy profiles in SrTiO3. Nature 430, 657–661 (2004). [DOI] [PubMed] [Google Scholar]
  33. Carlino E. Tem for characterization of semiconductor nanomaterials. In Transmission Electron Microscopy Characterization of Nanomaterials, 89–138 (Springer, 2014). [Google Scholar]
  34. Spence J. C. High-resolution electron microscopy (OUP Oxford, 2013). [Google Scholar]
  35. De Caro L., Scattarella F. & Carlino E. Determination of the projected atomic potential by deconvolution of auto-correlation function of TEM electron nano-diffraction patterns. Crystals. 6, 141 (2016). [Google Scholar]
  36. Storn R. & Price K. Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces. Journal of global optimization 11, 341–359 (1997). [Google Scholar]
  37. Csiszar I. Why least squares and maximum entropy? An axiomatic approach to inference for linear inverse problems. Ann. Statist. 19, 2032–2066, doi: 10.1214/aos/1176348385 (1991). [DOI] [Google Scholar]
  38. Carlino E. & Grillo V. In Proceedings MCEM VII Portoroze (Si), 159 (2005). [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Information
srep42236-s1.pdf (1MB, pdf)

Articles from Scientific Reports are provided here courtesy of Nature Publishing Group

RESOURCES