Deep learning analysis on transmission electron microscope imaging of atomic defects in two-dimensional materials

Summary

Defects are prevalent in two-dimensional (2D) materials due to thermal equilibrium and processing kinetics. The presence of various defect types can affect material properties significantly. With the development of the advanced transmission electron microscopy (TEM), the property-related structures could be investigated in multiple dimensions. It produces TEM datasets containing a large amount of information. Traditional data analysis is influenced by the subjectivity of researchers, and manual analysis is inefficient and imprecise. Recent developments in deep learning provide robust methods for the quantitative identification of defects in 2D materials efficiently and precisely. Taking advantage of big data, it breaks the limitations of TEM as a local characterization tool, making TEM an intelligent macroscopic analysis method. In this review, the recent developments in the TEM data analysis of defects in 2D materials using deep learning technology are summarized. Initially, an in-depth examination of the distinctions between TEM and natural images is presented. Subsequently, a comprehensive exploration of TEM data analysis ensues, encompassing denoising, point defects, line defects, planar defects, quantitative analysis, and applications. Furthermore, an exhaustive assessment of the significant obstacles encountered in the accurate identification of distinct structures is also provided.

Graphical abstract

Computer science; Materials science

Introduction

The two-dimensional (2D) materials with atomic thickness, high flexibility, and unique physical and chemical properties are promising for applications in electronics.^1,2,3 As a result of the second law of thermodynamics, defects have commonly existed in 2D materials. The presence of various additional structures such as point defects, line defects, and planar defects in 2D layered materials can strongly influence its properties.^4,5,6,7,8 The complex and small defect structure makes it difficult to visualize and analyze.^9,10,11 Transmission electron microscopy (TEM) is a powerful tool to characterize morphology, crystal structure, and chemical composition with atomic resolution.^12,13,14,15 Because the first TEM machine was successfully manufactured, scientists have been working hard to improve its spatial, temporal, and energy resolution.^16,17 The spatial resolution of the TEM is related to the wavelength of the electron beam. Increasing the accelerating voltage of the electron source can reduce the wavelength of the electron beam, thereby increasing the resolution. However, the increased accelerating voltage makes the electrons have high energy, which will cause damage to the material during the characterization process, such as knock-on defects and radiolysis damage. 2D materials with atomic thickness are susceptible to damage. It is important to reduce the accelerating voltage of the TEM to reduce the damage of two-dimensional materials during the characterization. With the advent of spherical aberration correctors, TEM can maintain high resolution at low accelerating voltage. Recently, combining spherical aberration correction technology and other technologies such as monochromators, the spatial resolution of TEM can reach 39 pm at low acceleration voltages (80 keV).¹⁸ Such techniques are critical for characterization of radiation-sensitive 2D materials.

With the advancement of thin-film growth processing technology and microelectromechanical systems (MEMS) technology, it is possible to introduce various external fields in the narrow pole piece space (in situ TEM technology).^19,20 In addition to static characterization, in situ TEM technology can provide an intuitive characterization of the relationship between structure evolution and property under the action of an external field.^{3,21,22,23,24,25} Further, fast imaging techniques, such as high-speed cameras and direct electron detectors, have been used to improve the time resolution of TEM. However, with the improvement of time resolution and spatial resolution, TEM results are changed from a single image to a series of images (video), and a single image contains more information. This results in a large dataset that is beyond the data processing ability of researchers using the traditional method.²⁶ Traditional TEM image analysis relies on skilled expertise and is time-consuming. Tedious manual work limits the breadth and diversity of TEM image datasets, making it insufficient for high-throughput data analysis. The resurgence of deep learning provides an excellent opportunity for automated TEM data analysis.^{27,28,29,30,31,32,33,34} The development of deep learning and its integration with TEM technology is shown in Figure 1.^{35,36,37,38,39,40} The concepts of deep learning can be briefly summarized. Deep learning is to train an efficient model to bridge input data and correct output labels. The role of this model is to decompose the complex recognition process into simple but non-linear mathematical operations. The calculated information can directly go through all layers or loop by feeding current and previous information to the current layer in the network. More and more abstract and structural information can be achieved by multi-step calculation processes.^41,42,43 In essence, deep learning is to build a machine learning model containing multiple hidden layers. It uses multiple layers of nonlinear information processing and abstraction for supervised, unsupervised, semi-supervised, self-supervised, weakly supervised feature learning, representation, classification, regression, and pattern recognition.^44,45 And the purpose of feature learning is achieved through deep learning model. Convolutional neural network (CNN) and its variants have been applied to TEM data analysis and demonstrate their excellent performance.^46,47,48 With components such as convolution operations, pooling operations, and activation functions, it is able to extract features efficiently and train and optimize them through backpropagation algorithms.⁴⁹ CNNs are increasing in size and complexity with the development of deep learning and advances in computing hardware. As deep learning develops and computing hardware advances, convolutional neural networks are increasing in size and complexity. For example, introducing deeper network structures (such as ResNet, Inception, etc.) and larger training datasets can further improve model performance. And large training datasets are exactly what TEM images have. Combining the advantages of TEM and deep learning, many interesting and complex works, including accurate identification, quantitative analysis, prediction, and mechanism exploration, can be completed automatically (Figure 2).^{49,50,51,52,53,54,55,56} In this review, the recent developments in the TEM data analysis of defects in 2D materials using deep learning technology are summarized. We first present the technical evolution of the TEM data analysis. Then, the peculiarities of the TEM images are discussed in detail. Then, TEM data analysis is discussed in detail, such as denoising, point defects, line defects, planar defects, quantitative analysis, and applications. The key challenges in identifying different structures are also discussed in detail. Finally, the prospects of deep-learning-assisted TEM data analysis are discussed.

Figure 1.

Historical perspective in TEM imaging and machine learning

(Left) Selected milestones in the timeline of TEM imaging of TEM technology. Reproduced with the permission from Levin et al. (2021) and Huhle et al. (2015). Copyright 2021, IOP science. Copyright 2015, Springer. (Right) Selected developments in the timeline of machine learning and its application. Reproduced with the permission from Misra et al. (2020), Lecun et al. (1998), and Rahman et al. (2020). Copyright 2020, MDPI. Copyright 1998, IEEE, Copyright 2020, MDPI. (Middle) The application of machine learning used in 2D materials science. Reproduced with the permission from Sainju et al. (2022), Treder et al. (2022), and Lee et al. (2020). Copyright 2022, Springer. Copyright 2022, Oxford University Press, Copyright 2020, American Chemical Society.

Figure 2.

The characteristics and capabilities of TEM technology and machine learning in 2D material defects analysis

Combining the advantages of both, the application of artificial intelligence in TEM image recognition is worth looking forward to. Reproduced with the permission from Yang et al. (2021), Lee et al. (2021), Madsen et al. (2018), Wang et al. (2021), Sadre et al. (2021), Ziatdinov et al. (2019), Lee et al. (2022), and Maxim et al. (2020). Copyright 2021, Wiley-VCH. Copyright 2021, American Chemical Society. Copyright 2018, Wiley-VCH. Copyright 2021, IOP science. Copyright 2021, Cambridge University Press. Copyright 2019, American Association for the Advancement of Science. Copyright 2022, American Chemical Society. Copyright 2020, IOP science.

Machine learning analysis of TEM images

Difference between TEM images and natural images

Machine learning has achieved great success in natural image processing. Unlike natural images, TEM images have featured characters, and the scale of the microscopy data is larger due to the high resolution. It makes the tasks of TEM image analysis more complex than natural image data; this requires exploring additional methods for TEM image analysis. Experimental TEM images exhibit high-level noise and complicated distortions. The quality and information of TEM imaging are affected by many factors. It has strong associations with physical and chemical information contained in materials and experimental environments, such as spherical aberration, defocus, e-beam dose, accelerating voltage, the thickness of the sample, environment factors, and even TEM model and operator. It is more severe in high-resolution TEM (HRTEM) images due to the phase contrast of HRTEM, which is based on a non-linear contrast transfer function; this makes the TEM images more complex and confusing. It is crucial to remove the interference factors in the TEM image and obtain the intrinsic information of the material.

Further, the TEM images are gray-scaled, whereas natural scene images are mostly color-scaled and have natural textures. The contrast of TEM images is based on the imaging mechanism. This contrast is the result of the interaction of electrons with matter. For example, the contrast in STEM images reflects the atomic number of the material. In HRTEM images, the contrast reflects the local thickness variation, atomic number, defocus, periodicity, and so on. Unlike natural images with clear and definite knowledge, the prior information on the structures of various atomic configurations and defects is limited or absent. Furthermore, numerous phenomena within materials science emerge unexpectedly or elude anticipation, necessitating scientists to undertake additional investigations to uncover their underlying causes and mechanisms. It makes the ground truth label hard-achievable.⁴¹ Commonly, establishing the ground truth involves the expertise of human investigators. This process demands substantial human intervention and is frequently time-intensive, rendering it insufficient for high-throughput data analysis requirements. In addition, the training of deep learning models requires large amounts of accurately labeled data. In order to increase the number and diversity of datasets, it is essential to perform augmentation on limited data.^52,57,58 Based on the aforementioned difference, exploring deep learning methods applicable to TEM image analysis is urgent. The following section will discuss deep learning applications on TEM images of different defects in 2D materials. Commonly used models for defect identification in 2D materials are summarized in Table 1.

Table 1.

Deep learning models used in TEM imaging of atomic defects in 2D materials

Types	Network models	Addressed problems
Noise	CNN50,61	Image quality and precision limited by SNR
Point defect	CNN,52,74,77 FCN38,50,87	Locate and classify point defects (atomic coordinates, bonding, species, and periodicity)
Line defect	CNN,52 FCN53,78	Local atomic configurations, defect population, and transformation networks.
Planar defects	CNN,88 DCNN,86 Bayesian NN,75 FCN,55,89 GAN,90 VGG91	Classify a large number of structural classes while being robust with respect to theoretical or experimental sources of inaccuracy and physically driven deviations from ideal crystal symmetry.

Point defects identification

The second law of thermodynamics indicates that there is a certain degree of disorder in 2D materials.⁵⁹ Impurities and defects may also be introduced during the material preparation process. Point defects are the simplest and most abundant defects in 2D materials.⁵ The presence of point defects can tune various properties of 2D materials with atomic thickness such as optical and electric properties. The Z-contrast STEM could give the structure and chemical composition of a sample with atomic resolution. That is to say, the species and position of atoms can be characterized clearly. Take MoS₂ as an example, the monosulfide and disulfide vacancies, as well as antisite defects and regular lattice sites can be unambiguously distinguished by STEM technology.⁶⁰ TEM with sub-angstrom spatial resolution and flexible stimulus could provide new sight for atomic structure characterization and evolutionary dynamics. The difficulties such as large datasets, complex information extraction, and low signal-to-noise ratio (SNR) limit the quantitative analysis of TEM images. In this section, recent advances in the quantitative study of point defects using deep learning are discussed in detail.

Noise in TEM images

Precise and robust atomic column localization and segmentation within atomic-resolution TEM images are imperative.⁶¹ Among them, noise reduction and deblurring is a challenging task. Low signal-to-noise ratio will introduce complex interference and reduce the accuracy of the image, making quantitative analysis difficult.⁶² The accurate recognition and categorization of local structures stand as a pivotal initial stride toward quantitative analysis of TEM images. It is also the critical cornerstone for transforming from pixel-based to chemical-structure-based classification (it will be discussed in the quantitative analysis section). Traditional coordinate positioning is based on identifying atomic positions by searching for local intensity extremes or comparing them directly with existing templates.^63,64 This method is similar to the method of natural image processing. However, due to the low SNR of TEM images, this method does not achieve good results in TEM images. The compromise between electron irradiation damage and image SNR during TEM image capture requires careful consideration.^65,66,67 High radiation doses and accelerating voltage could provide high-precision measurements of the position of single atoms. However, when the high-energy electron beam interacts with the matter, the energy carried by the electron beam will be transferred to the matter, resulting in the damage of the intrinsic structure of the matter, such as diffusion, knock displacement, radiolysis, charging, and heating. The damage alters the structure of materials, severely limiting the increase of SNR and destroying the intrinsic structure. The irradiation damage is worse in radiation-sensitive 2D materials due to the monoatomic layer thickness. To maintain the intrinsic structure and reduce the radiation damage, the accelerating voltage and electron dose should be reduced (e.g., below 10⁸ e⁻/nm² at 80 kV).^68,69 Reduced accelerating voltage and electron beam dose limit the SNR. Further, with the development of direct electron imaging technology, the increased temporal resolution also reduces the SNR. Developing a reliable denoising process is important for atom identification and precise positioning in 2D materials. Deep learning network has great potential for high-precision automatic TEM image recognition of point defects in 2D materials (Figure 3A).⁵⁰ The computing units in the designed network are symmetric. They could extract local and global features at multiple scales. Based on the denoising model, the noise background can be quantitatively analyzed and extracted. The average deviation between the ideal atomic position and the actual atomic occupancy can also be analyzed statistically in the TEM images with low SNR (Figures 3B–3E).⁵⁴ The deep learning model after training could exhibit a surprising effect. The deep learning neural network can recognize the physical system, such as the lattice period, under multiple studies. It can even accurately identify atomic occupancy that is nearly indistinguishable to the human eye. For example, the deviation below 10 pm or even sub-pm can be identified unambiguously (Figures 3E–3G).³⁸ Deep-learning-based TEM image recognition exhibits small errors in recognizing TEM images with low SNR, which is within the instrumental error and is almost negligible.

Figure 3.

Restoration of noisy experimental TEM image by denoising process based on deep learning

(A) A typical deep learning network model constructed for the denoising. Yang et al. (2021). Copyright 2021, Wiley-VCH.

(B) The deviation of the predicted positions from the ground truth positions of atoms for different combinations of noise (λ′) and blurring (σ′).

(C) Simulated image of area marked in (B).

(D) Atom position identification by deep learning.

(E) Statistical histogram of atomic displacements. Ziatdinov et al. (2019). Copyright 2019, American Association for the Advancement of Science.

(F) Statistical histogram of projected W−W separations in raw images.

(G) Statistical histogram of projected W−W separations in images after deep learning recognition. Lee et al. (2020). Copyright 2020, American Chemical Society.

Point defect structure identification

After solving the noise problem, the atoms in the TEM image can be identified and transferred to the coordinate system to complete the identification of the point defect structure. In most cases, defect detection is a labor-intensive and time-consuming work. This has typically involved manual inspection or basic image processing techniques like Fourier filtering or intensity thresholding.^70,71,72,73 The advent of neural networks presents a prospect for automating defect identification, enabling the streamlined discovery of numerous defects and systematic generation of class averages with minimal human intervention (Figure 4A).⁷⁴ The network is trained to output a pixel-wise classification map. From the output data, the probability of each pixel belonging to a particular type of defect can be obtained. It is found that the network can distinguish disorder from the nondefective part at atomic scale. Then it returns the location of the detected defects and generalize to previously unseen defect structures. Using graphene as an example (Figure 4B), the images contain a substantial amount of noise.⁷⁵ The atom occupancy and atomic configuration are difficult to recognize by the naked eye. As 2D graphene in Figure 4B has only one layer, there is no overlap in the z direction. The x and y coordinates of C atoms can reflect its actual structure from TEM images. Approximate atomic positions from TEM images can be obtained after the deep learning procedure (Figure 4B). For multi-layer 2D materials, by feeding a neural network with TEM images of same area with various shooting parameters (a focal series), the column heights (z) could also be extracted in HRTEM images (Figure 4C).⁵² When the atomic positions are identified, combining specific information and macroscopic properties of the material, defect configurations can be further derived. Coupled with the advantages discussed earlier, the individual defect configurations with high SNR can be precisely identified and reconstructed (Figure 4D).³⁸

Figure 4.

Training a deep learning network to recognize point defects

(A) A typical schematics of a deep learning network with an encoder-decoder type of structure. Maksov et al. (2019). Copyright 2019, Springer.

(B) HAADF images of graphene structures with defects. The atomic positions are reconstructed by a deep learning network. Leitherer et al. (2021). Copyright 2021, Springer.

(C) Deep learning network is used to retrieve the structural information of column heights. Madsen et al. (2018). Copyright 2018, Wiley-VCH.

(D) The identification and classification of point defects by deep learning. Lee et al. (2020). Copyright 2020, American Chemical Society.

Quantitative analysis

According to the deep learning based on pixel-wise maps discussed earlier, point defects can be found in 2D materials. However, the fine structure classification is still incomplete due to the lack of quantitative chemical environment analysis (e.g., bond length, bond angle, coordination number). For example, the difference between SiC₄ and SiC₃+vacancy in Si-dopped graphene can be hard to identify and categorize into the same category.⁵⁴ Combined with suitable mathematical operations such as the Laplacian of Gaussian and the physics constraints behind the information, a deep learning network can extract relevant structural/chemical parameters for further defect classification. Based on the partial priori knowledge of the materials, the translation from pixels in TEM image to materials-specific information can be assisted. To quantitatively analyze structural parameters, some crystallographic and chemical constraints should be introduced, such as minimum/maximum coordination number and minimum/maximum length of chemical bond. The introduction of these constraints can better help the identification and classification of similar defects. For compound defects, whether there is a physical connection between different parts of the defects needs additional consideration.^54,76 Ziatdinov et al. determined the possibility of bond formation of neighboring atoms of each identified atom and counted the change of bond length by deep learning (Figure 5A).⁵⁶ All of the atomic distances and bond lengths are confirmed and calculated automatically. The spatial variations are analyzed over time by studying the bond variation in TEM images of Si-dopped graphene. Combining the statistics of bond angles, the Si defects with various chemical environments can be precisely determined, categorized, and counted (Figures 5B–5D).⁵⁴

Figure 5.

Training a deep learning network for quantitative analysis of point defects

(A) Mapping atomic distortions in STEM images by deep learning. Maxim et al. (2020). Copyright 2020, IOP science.

(B) Distribution of bond lengths.

(C) Distribution of bond angles.

(D) Accurate identification of point defects with different structures. Ziatdinov et al. (2019). Copyright 2019, American Association for the Advancement of Science.

(E) Structure statistics of the most often found defects. Trentino et al. (2021). Copyright 2021, American Chemical Society.

This method can realize the quantitative calculation of chemical bonds, and the effect is similar to theoretical calculation (such as first-principles calculation), but principles are different. This demonstrates that deep learning has great potential. Different from single atom identification based on local analysis, this deep learning method not only completes local atom identification but also constructs the connection between global atoms. It represents the transition from pixel-based defect classification to chemical-structure-based classification. The physicochemical information behind the image is included in the results. For example, when we want to distinguish between 3-fold and 4-fold coordinated Si atoms, pixel-wise classification methods often classify them into one category. On the contrary, when we introduce physicochemical constraints and use chemistry-based classification, they can be precisely identified.⁷⁶ Furthermore, the defects of different structures in the 2D materials can be accurately identified and counted (Figure 5E).⁷⁷

Application

Deep learning can quickly and accurately identify the structure, species, and distributions (fluctuations) in an automated manner. This provides opportunities for processing large amounts of data and extracting rich information in 2D materials. Based on these characteristics, there are many applications for big data real-time processing and quantitative analysis of physical mechanisms. For example, video processing is generated by in situ TEM experiment. In the previous discussion, TEM with high spatial resolution was primarily used as a tool for static defect characterization. With the development of the TEM, the property-related structures could be investigated under various external stimuli (e.g., mechanical, thermal, environmental, electrical). It results in a large dataset. Using deep learning, Ziatdinov et al. extend deep learning method to track silicon defects over time (Figures 6A–6C).⁷⁶ They explored transformation between 3-fold and 4-fold coordination. The defect class was classified based on its chemical structure instead of pixels. The evolution of the defect’s structure is analyzed for each frame. In addition to big data processing, deep learning can also be used to explore physical mechanisms quantitatively. For example, Lee et al. measured the displacement and strain fields for defects with different types in 2D WSe_2−2xTe_2x (Figure 6D).³⁸ By mapping 2D displacement vectors overlaid on TEM images of local areas, the distribution of 2D strain tensor components for each defect type can be analyzed. The construction of local structures and properties is conducive to the in-depth exploration of physical mechanisms. Further, the deep learning method can also be extended to explore more complex structures and rich properties with large areas (Figure 6E).⁵²

Figure 6.

Applications of deep learning in quantitative processing of TEM images

(A) STEM image of a silicon atom changing over time.

(B) Classification of the Si defect type for each frame in (A).

(C) Localization of the defect within each frame. Ziatdinov et al. (2017). Copyright 2017, American Chemical Society.

(D) Displacement and strain fields for chalcogen site defects. Lee et al. (2020). Copyright 2020, American Chemical Society.

(E) The strain mapping for complex structures with larger areas. Madsen et al. (2018). Copyright 2018, Wiley-VCH.

Line defects identification

TEM could characterize line defects (boundary and edge) and crystal orientation at the same time. This property is often used to study interfacial structure and the resulting property changes compared with the crystal without line defects (e.g., heterojunction). The size span of boundaries and edges is large, from nanometer to micrometer and even larger. There are various configurations of interface structures in the boundaries and edges, which affect the properties of 2D materials, such as electrical properties, mechanical properties, etc. Modulating the interfacial structures, mismatch, and size morphology of line defects is important for electronic device performance. Line defects encompass various categories of point defects, embodying the amalgamation and topological lattice reconfigurations of multiple individual point defects, which contributes to their heightened complexity. The task of detecting defects positioned along the boundary or edge is more intricate compared with those embedded within the bulk. This entails the establishment of precise bonding geometries and identification of chemical species, while also considering the inherent discrete rotational symmetry of the host lattice. This comprehensive approach subsequently facilitates the illumination of defect distributions and elucidation of chemical transformation networks through the meticulous analysis of atomically resolved TEM data. Wang et al. present the border structure evolution of 1T phase 2D material (Figure 7A).⁵³ Using the deep learning method, the edge morphology evolutions are automatically identified. The crystal faces of the edge can also be analyzed. Furthermore, Kalinin et al. not only study the evolution of the boundary but also extract and classify different defect structures formed during the evolution (Figures 7B and 7C).⁷⁸ They provide ideas for improving deep learning in recognition accuracy and dynamic structure recognition analysis. The deep learning method is developed by combination of mathematical algorithm, such as multivariate statistics, and Markov analysis. Therefore, transient interfacial defect structures (such as metastable structures) under interface configuration evolution can be captured and identified. Deep learning research lays the foundation for quantitative analysis of fundamental mechanisms of transformations and chemical reactions directly from experimental data.

Figure 7.

Training a deep learning network to recognize line defects

(A) The evolution of the nanopore borders identified by deep learning. Wang et al. (2021). Copyright 2021, IOP science.

(B) Experimental STEM and recognized data on graphene nanohole with Si impurities.

(C) Statistical analysis of silicon defects at the interface. Ziatdinov et al. (2019). Copyright 2019, Wiley-VCH.

Planar defects identification

For 2D crystals, besides the presence of point defects and line defects, the existence of various multiphase structures strongly influences their properties.^4,79,80 Typically, 2D materials with planar defects contain point and line defects, making the structure more complex. This multiphase structure includes crystalline heterogeneous multiphase, homogeneous multiphase, and crystalline-amorphous multiphase (surface contamination). The premise of quantitative analysis of planar defects is to distinguish different phases. Planar defects contain more complex point defects and line defects, and their spans are larger. This not only requires local analysis but global analysis is indispensable. A possible solution is to convert some physical information into descriptors that can be extracted and computed by neural networks, such as coordinates, chemical species, and periodicity. Appropriate descriptors can significantly increase deep learning models' quality and generalization ability.^81,82,83,84 Belianinov et al. provide an excellent method to describe phases in machine language.⁸⁵ They define the atom’s chemical neighborhood via the nearest neighbors' number and identity. The types of atoms with statistically different chemical neighborhoods and their spatial distribution define the chemical composition, that is, phases. For multiphase in 2D materials, Leitherer et al. use smooth-overlap-of-atomic-positions as the descriptor.⁷⁵ Combining Bayesian deep learning, a comprehensive and diverse polycrystalline system is correctly classified (Figures 8A and 8B). Lee et al. extend the utility of deep learning for analyzing a large volume of STEM images, including images with complex contaminants.⁵⁵ The experimental STEM images with various stacking polymorphs can be identified (Figure 8C). It exhibits high-precision phase recognition capability in multiphase systems (over two phases). It can also provide the phase information along the z-direction for different stacks. For large-area phase identification with more phases (over three phases), operating in the reciprocal space (vectors) is a good scheme. Working in k-space to explore structural information (e.g., bond length, bond angle, periodicity) is considered a classical approach. However, it usually contains complex periodic-related signals, for example, multiple diffraction orders and coincidence of diffraction points. Noise and slight distortions will always exist that can make handcrafting particular rules more difficult in practice. Vasudevan et al. used deep learning to solve the difficulty of manually identifying diffraction points. Five different 2D crystal lattice types at the mesoscopic scale can be distinguished in k-space. This method also allows for automated prediction of the Bravais lattice type (Figure 8D).⁸⁶ It provides experiment-based descriptors for the structure of 2D materials that can be used in large-area multiphase recognition.

Figure 8.

Training a deep learning network to recognize planar defects

(A) Polycrystal classification framework strided pattern matching for slab-like systems.

(B) Segmenting surface contaminants, phases, and lattices. Leitherer et al. (2021). Copyright 2021, Springer.

(C) Polymorph identification from experimental STEM images of MoS₂. Lee et al. (2022). Copyright 2022, American Chemical Society.

(D) Polymorph identification from mesoscopic 2D fast Fourier transforms of images in reciprocal space. Vasudevan et al. (2018). Copyright 2018, Springer.

Perspectives

With the development of temporal and spatial resolution, the amount of data in electron microscopy images is enormous. However, while studying TEM images, scientists will study one of the problems with specific objectives. TEM images are often discarded after a specific question has been studied. This leaves much missing information in the electron microscope images and causes a waste of resources. Moreover, it hinders the rapid adaptation of the neural network trained by specific datasets to new tasks. It will be better to build a globally shared TEM database. Rich datasets with various types can help improve the accuracy of deep learning and help fully use TEM data to explore more information. Further, the different TEM pictures will help the established deep learning methods to solve interference factors such as machine models, external environments, operators, etc. Finding suitable descriptors to convert physical information into computer-understandable information can better explore the physical mechanism behind the phenomenon and the relationship between the structure and performance. In addition, the current deep learning is mainly used to recognize the TEM image after the shooting is completed. TEM imaging is a complex process, and the final imaging quality is related to various parameters during the imaging process. Deep learning can be used for parameter optimization during the shooting process to obtain high-quality TEM images. For example, intensity is often used for imaging. Phase information is often overlooked due to the complex operations and reconstruction. Deep learning will help realize image processing to reconstruct the exit wave from a focal series of TEM images. According to different imaging mechanisms, using the advantages of deep learning to optimize the parameters in the imaging process will be a feasible way to improve image quality in the future. Moreover, deep learning can be the “brain” of the TEM to manipulate “eyes” (TEM). It can identify pictures in real time during the operation. According to the identified results, deep learning operates the electron microscope to complete specific research and finally realize full automation.