Optical multilayer thin film structure inverse design: From optimization to deep learning

Summary

Optical multilayer thin film structures have been widely used in numerous photonic domains and applications. The key component to enable these applications is the inverse design. Different from other photonic structures such as metasurface or waveguide, multilayer thin film is a one-dimensional structure, which deserves its own treatment for the design process. Optimization has always been the standard design algorithm for decades. Recent years have witnessed a rapid development of integrating different deep learning algorithms to tackle the inverse design problems. A natural question to ask is: how do these algorithms differ from each other? Why do we need to develop so many algorithms and what type of challenges do they solve? What is the state-of-the-art algorithm in this domain? Here, we review recent progress and provide a guide tour through this research area, starting from traditional optimization to recent deep learning approaches. Challenges and future perspectives are also discussed.

Graphical abstract

Natural sciences; Physics; Applied sciences

Introduction

Optical multilayer thin film structure^1,2,3 (shortened as multilayer structure) is a type of photonic platform that consists of multiple layers of material stacked together, with thickness typically in the range of tens of nanometers to several hundred nanometers. Because of their ease of fabrication, they have been widely used in many different photonic applications. For example, most photovoltaics^4,5 devices are based on thin film structures to achieve high power conversion efficiency in large-scale manufacturing. Multilayer thin film-based transmissive/reflective filters^6,7 are widely used as functional coatings and protective layers for optical lenses and glass pieces. The light interference phenomenon inside multilayer structures also provides an ideal platform to make structural color coatings^8,9 as an eco-friendly and advanced decoration with high aesthetic expression. Other important applications include displays,^10,11 touch screens,¹² radiative cooling devices,¹³ to name a few.

For a given multilayer structure, obtaining its optical responses is easy and straightforward, through well-developed electromagnetic simulation algorithms, such as Transfer Matrix Methods¹⁴ (TMM). However, the inverse problem, to design a multilayer structure with desired optical responses is non-trivial and usually does not have a closed-form. The inverse design requires determining the total number of layers and identifying the best combination of materials at each layer as well as their corresponding thickness, which is a combinatorial optimization problem. At the early stage, many designs are based on researchers’ physics intuition or optical knowledge. However, this method is only suitable for some simple and conventional structures. As design complexity increases, optimization-based methods have been widely adapted to automatically identify optimal structures, including needle optimization,^15,16 particle swarm optimization,¹⁷ genetic algorithms,¹⁸ and so forth. To speed up the design process, in recent several years, researchers have started to use deep learning as an alternative approach for inverse design.^{19,20,21,22,23} After learning from the training dataset, these deep learning models can establish a direct mapping from optical responses to the multilayer structure and, therefore, can finish the design process within a fraction of a second, with a speed much faster than the optimization approach. Many different deep learning models have already emerged and demonstrated excellent performance in diverse applications, including tandem networks,^19,24,25 Variational Auto-Encoders (VAE),²⁶ Generative Adversarial Networks (GAN),²⁷ Mixture Density Networks (MDN),^20,21 and so forth. There are also many nice benchmarking works that compare different inverse design methods.^28,29,30

With such significant progress in multilayer structure inverse design, a comprehensive review of these inverse design algorithms could benefit the relevant research community by providing a high-level perspective on the development and future directions. Though there are many excellent reviews that summarize deep learning-based inverse design for photonic and nano-photonic,^{31,32,33,34,35,36,37,38} there is essentially no review that focuses on the multilayer structure inverse design. Different from other photonic structures such as metasurface or waveguide, multilayer thin film is a one-dimensional structure, which requires special treatment and considerations during the design process. For example, recently many researchers started to treat multilayer structures as a sequence,^23,39,40 which is a special type of data that has not been considered in other types of photonic structures.

In this review, we will fill this gap by providing a perspective of multilayer structure inverse design and looking through the research roadmap. In the first part, we will provide an overview of multilayer structure inverse design by dissecting the representations of both multilayer structures and optical responses and present the overall challenges. Then we will go through different mainstream inverse design methods and summarize their achievements from a historical perspective: starting from the traditional optimization-based methods over decades ago, then the deep learning using vectorized representation starting around several years ago, the recent combined methods using both optimization and deep learning, and finally the deep learning using sequential representation, which was just proposed in recent two years. We hope this review could provide insights into current research development and promote the exploration of new research frontiers.

Multilayer structure inverse design

In this section, we will provide a general introduction to multilayer inverse design fundamentals. Inverse design usually deals with finding the suitable structures that can satisfy the desired optical response. Therefore, there are two aspects that researchers are interested in: multilayer structures and optical responses. We will first describe core concepts that we need to figure out when designing a multilayer structure, as well as three different methods when representing a multilayer structure. This is the part where readers can tell the difference between multilayer design and other nanophotonic inverse designs. Following this, several common types of optical responses with general interest and wide applications will be discussed. In the last part of this section, we will summarize several challenges during inverse design and provide an illustrative table to show how existing inverse design methods can tackle these challenges in different aspects. Details of these methods will be discussed in the following sections.

Design parameters for multilayer structure

As the name suggests, a multilayer structure consists of multiple layers of materials stacking on top of each other. Usually, each layer can be made from a different material with different thickness, and these different layers contribute together to control light propagation and interaction inside the medium, and collectively determine the multilayer structure’s optical responses. For a multilayer structure, there are three sets of parameters to be determined during the design process: the total number of layers, material combinations at each layer, and their corresponding thickness.

The total number of layers usually depends on the type of optical responses. For example, only 3–4 layers of suitable materials can already make a nice structural color filter.^8,41,42 However, in some other cases, such as designing a perfect reflective or transmissive filter, the total number of layers can easily exceed 100. Generally, there is no specific criterion for how many layers should be used for a given application; however, it is always desirable to use structures with fewer layers considering the fabrication complexity and cost. Material combinations are another important design parameter. In optical frequency, different materials usually have different refractive indices, which uniquely determine how light will pass through and interact with the layers. Based on their absorption ability in optical frequency, most materials can be divided into two types: dielectric, or metallic. Dielectric materials such as silicon dioxide (SiO₂) or titanium dioxide (TiO₂) typically have low optical absorption and allow a significant portion of light to pass through in the visible range, while metallic materials such as silver (Ag) or gold (Au) have high absorption and are usually opaque in optical frequency. Note that we also include semiconductor materials in the dielectric category. Identifying the correct material selections at each layer can effectively determine how light is propagating inside the layers as well as light refraction at each interface, and thus, the optical responses in the layered system. However, this is not a trivial task as materials are usually dispersive, i.e., the refractive index is dependent on the incident light wavelength. The last important design parameter is the thickness at each layer. The major usage of thickness is to introduce a desirable propagation phase while light passes through each layered medium. In some special cases, this can create constructive or destructive interference phenomenon. One example is the Distributed Bragg Reflector (DBR)^43,44,45 that consists of alternating materials with high/low refractive index profiles. When each layer’s thickness is around one-quarter of the effective wavelength, strong interference in the reflection side can lead to a perfect reflector that is close to 100%.

As an example, in Figure 1A, we illustrate a simple five-layer structure (left side) and use a table to illustrate these three design parameters (right side). This structure is made on a glass substrate which is not treated as a design parameter. As a notation, we count the number of layers from top to bottom, i.e., the first layer is closer to the air and the fifth layer is closer to the substrate. We will use this notation for the rest of this review. Ideally, we hope that an inverse design algorithm can identify all three design parameters based on the provided target optical responses. However, there are significant difficulties when trying to incorporate them into a computer-based inverse design algorithm. For example, how can we let an algorithm to understand the number of layers, which is supposed to be a varying number? In addition, how to represent different materials in the algorithm? Remember that materials usually have dispersion, which hinders us from using a single refractive index to distinguish them.

Figure 1.

Visualization of multilayer structures as well as three common types of representation

(A) An example of a five-layer thin film structure (left) as well as the three design parameters presented inside a table (right), including the total number of layers, the material combinations, and their thickness at each layer.

(B–D) Shows three widely used representation for a multilayer structure in inverse design algorithms. (B) is the vectorized representation, where material and total number of layers are fixed and only continuous thickness will be designed. (C) is the combined representation, where total number of layers are fixed and we need to design both material and thickness. Usually, material is a discrete variable that is selected from a pre-defined material database. (D) is the sequential representation, where all three parameters will be designed in a layer-by-layer way.

Perhaps the easiest design parameter to deal with is the thickness, which is a continuous variable that aligns well with computational considerations. Currently, there have been three different ways of representing a multilayer structure inside the inverse design algorithm and the first type is the vectorized representation that only relates to the thickness parameter. We illustrate this idea in Figure 1B. Within this representation, one needs to first determine how many layers to use as well as material choices at each layer based on their prior knowledge or experience, and only use the algorithm to find out the thickness. Common algorithms include some optimization-based methods such as Particle Swarm Optimization¹⁷ (PSO), and many machine learning based methods such as tandem networks.¹⁹

The major limitation of this vectorized representation comes from the fact that its design performance is heavily determined by the pre-selected materials. Considering the large variety of materials in nature, it is possible that these selected materials may not lead to a good performance for any thickness choices,⁹ which can leave the design algorithm running in vain. An inverse design algorithm that can automatically determine the material at each layer would be preferable. This leads to the second type of combined representation (see Figure 1C). Here, the material is represented by a discrete variable and thickness is a continuous variable. Usually, there are several candidate materials to be selected and they are denoted by different integers. For example, when there are two materials, we can use 0 to represent one material and 1 for the other material. In some special cases when all candidate materials have low dispersion, a single refractive index value can be used to represent the material. Many global optimization methods use this type of representation, such as genetic algorithm¹⁸ and memetic algorithm.⁴⁶

This combined representation has been popular for a long time. Only very recently, with the development of sequential machine learning models such as Recurrent Neural Networks^47,48 (RNN) and transformers,⁴⁹ researchers started to establish the third idea of sequential representation, describing a multilayer structure with a sequence (see Figure 1D). Instead of splitting material and thickness separately, this sequential representation treats both material and thickness on an equal footing, and assembles a multilayer structure in a layer-by-layer manner, just like in actual fabrication. Two recent models, the sequential decision network called OMLPPO,^23,40 and the conditional sequence generation network called OptoGPT,^39,50 are adapting this representation. Both methods have been demonstrated to automatically determine the optimal number of layers not known beforehand and terminate the design process when a target is reached.

Optical responses

In the previous section we mainly talk about the output of the inverse design, namely, the design parameters. Here, we would like to briefly summarize the input of inverse design: the targeted optical responses for the designed structure. In optical frequency, multilayer structures have been widely used as transmissive/reflective filters, structural color coatings, absorbers, radiative cooling devices, and so forth. These applications can be attributed to the following types of optical responses.

• (1)Transmission or Reflection Spectrum: They are mostly used as design targets for many special filters. For example, a band-pass/band-notch filter requires a transmission window/block region at the desired frequency. Designing DBR is looking for a near 100% reflection spectrum within the expected wavelength region. On the other hand, a near-zero reflection spectrum is needed for designing anti-reflection coatings.
• (2)Absorption Spectrum: Light absorptance is very important for many energy-harvesting and detection applications. In solar-thermal harvesting devices, a high absorption over a broadband wavelength that covers the whole solar spectrum is always desirable. Many detection devices such as spectrometers require a perfect absorption spectrum to completely remove potential crosstalk and improve the measurement accuracy.
• (3)Structural Color: Compared to traditional color, structural color has unique advantages of higher spatial resolution, better stability, more environmental friendliness, and so forth. They can be well described using a color space such as RGB, xyY, or Lab values.

There are also many other important optical responses not listed here, e.g., combined spectrum and angular responses, which present higher level difficulties. Different from multilayer structures that require special representations in the design algorithm, these optical responses (i.e., spectra) are usually continuous variables and can be easily represented as a high-dimension vector when integrated into the design algorithm. Therefore, we will not provide more description on this.

Challenges

An ideal inverse design algorithm should be able to quickly output multilayer structures given any type of response as the input. However, this can lead to significant difficulties and challenges. The first challenge is how to design the multilayer structure globally (Global Design). By global we mean the algorithm can automatically design for all three sets of design parameters, including the total number of layers, the material sequence, and their thickness at each layer. As a comparison, local design only deals with thickness design, where the performance is heavily constrained by the fixed material selections. Global design is non-trivial as the design space is extremely large. For example, considering a five-layer structure where each layer has ten different material choices and the thickness also only has ten different discrete values, the design space can expand to ${10}^{10}$ different combinations of structures. It is impossible to search all potential designs exhaustively, therefore, advanced algorithms are required to navigate through this large design space and identify the optimal structure.

The second challenge is how to finish a design quickly (Efficient Design). Traditional optimization algorithms are based on iterative evaluations and use fitness functions to update structures to reach user-defined goals. Although many numerical methods such as TMM can finish simulating a multilayer structure and obtaining its optical characteristics within 1 s, the iterative evaluation process often requires thousands or even more simulations, resulting in a significantly long processing time. The situation can be worse when multiple design targets are required. Speeding up the design process and making it more efficient would benefit everyone in this field.

In addition to these two major challenges related to the algorithm itself, researchers and engineers are also interested in leveraging inverse design algorithms to assist the practical fabrication process. Based on this, the third challenge is how to obtain multiple different designed structures with minimal effort so that researchers can select the best one for their fabrication needs based on the availability of materials and deposition methods (Diverse Design). In principle, this is feasible because of the one-to-many mapping feature in the inverse design, meaning multiple structures can lead to similar optical properties. However, most existing algorithms cannot tackle this challenge because they can only output one definite design. An algorithm with probabilistic output can potentially solve this difficulty. The fourth challenge is how to integrate practical constraints into the design process (Flexible Design). With this challenge solved, researchers may restrict the material selections and thickness range at any desired layer based on their fabrication or design needs. For example, some may require the top layer to be dielectric material to protect the next metallic layers from oxidation. The latter two challenges have not yet been extensively explored.

In this review, we will focus on these four challenges and summarize how researchers came up with innovative ideas and advanced algorithms to tackle each of them. Decades ago, some global optimizations, including genetic algorithms¹⁸ and memetic algorithms,⁴⁶ have been proposed to simultaneously design materials and thickness from a global perspective. Later around 2017, with the great success of machine learning, many different neural network-based methods were utilized to speed up the inverse design process. However, both methods have their own disadvantages: optimization-based algorithms are not efficient, while deep learning based algorithms cannot design structures globally. Therefore, later around 2020, the combination of optimization and deep learning provides a solution to alleviate these problems to design in the global space while still making the algorithms fast. However, the iterative process still takes quite a long time. Finally, in recent two or three years, with the development of sequential learning, researchers started to treat the multilayer structure as a sequence and eventually developed OptoGPT that can solve all four challenges, declaring the victory of the long-lasting design issues in multilayer structure. In Table 1, we summarize all these impactful algorithms and list their accomplishments.

Table 1.

A comparison of different inverse design algorithm for multilayer structure

Methods	Reference	Year	Multilayer Representation	Global Design?	Efficient Design?	Diverse Design?	Flexible Design?
Needle Optimization	Tikhonravov et al.16	2007	Combined	Yes	No	Yes	No
Genetic Algorithm	Schubert et al.18	2008	Combined	Yes	No	No	No
PSO	Rabady and Ababneh17	2014	Vectorized	No	No	No	No
Memetic Algorithm	Shi et al.46	2018	Combined	Yes	No	No	No
Tandem Networks	Liu et al.19	2018	Vectorized	No	Yes	No	No
MDN	Unni et al.20	2020	Vectorized	No	Yes	Yes	No
VAE	Zandehshahvar et al.26	2021	Vectorized	No	Yes	No	No
GAN	Dai et al.51	2022	Vectorized	No	Yes	Yes	No
GLOnet	Jiang and Fan52	2021	Combined	Yes	No	Yes	No
NEUTRON	Wang and Guo53	2022	Combined	Yes	No	Yes	No
OMLPPO	Wang et al.23	2021	Sequential	Yes	No	Yes	No
OptoGPT	Ma et al.39	2024	Sequential	Yes	Yes	Yes	Yes

Optimization-based methods

Since decades ago, heuristic optimization-based methods have become essential tools in tackling inverse design challenges, offering robust and flexible approaches to explore the vast solution space and find near-optimal designs. Although dealing with inverse design, these methods are mainly based on iterative forward simulation and follow a similar pipeline shown in Figure 2. When given a design target, the algorithm will first start from a random structure and obtain its simulated optical responses. Then, the difference between the simulated responses and target responses will be evaluated and used to calculate the fitness function. If the fitness function does not meet the termination criteria, the algorithm will update the initial structure and keep running this iterative process until reaching satisfaction. After sufficient iterations, a structure with desired optical responses can be found. Different optimization methods, including needle optimization, particle swarm optimization, genetic algorithms, differ from each other in the way how they update the structure in each iteration, with illustrations can be found in Figure 2. A summary and benchmark of optimization-based methods on photonic applications can be found in Schneider et al.³⁰ and Campbell et al.³⁸

Figure 2.

A general pipeline of optimization-based methods for inverse design

Optimization-based methods mainly deal with thickness or material design. In the beginning, researchers use the vectorized representation to only optimize the thickness. One such example is Particle swarm optimization (PSO), an evolutionary algorithm inspired by the social behavior of birds. PSO maintains a set of particles that move through the solution space and are influenced by their own best-known position and the best-known positions of the entire particles. In this case, each particle is a vector of layered thickness that can adjust its trajectories based on its own experience and the experience of neighboring particles, balancing the exploration and exploitation. Rabady et al.¹⁷ used PSO to design band-pass filters and demonstrated good performance (see Figure 3A). PSO has also been applied in various real-case multilayer thin film applications, including transmissive spectrum design,⁵⁴ inverse extraction of thickness and optical constants,⁵⁵ transmissive filters for image differentiation,⁵⁶ and so forth.

Figure 3.

Several design examples for optimization-based methods

(A) Design performance of Particle swarm optimization (PSO) algorithm for a band-pass optical interference filter in the range of 1000–2000 nm wavelengths. Reprinted from Rabady and Ababneh¹⁷ Copyright @ 2014, Elsevier.

(B) A demonstration of versatile design using Needle Optimization. Reprinted from Tikhonravov et al.¹⁶ Copyright @ 2007, Optica Publishing Group.

(C) Genetic Algorithm designing a rejection filter of 90% over the 0.55–0.65 μm region. Reprinted from Schubert et al.¹⁸ Copyright @ 2008, Optica Publishing Group.

(D) Iterative refinement of memetic algorithm with mixed-integer programming (MIP) to design better generations. Reprinted from Shi et al.⁴⁶ Copyright @ 2018, ACS Publications.

To further explore the global design of both material and thickness, Tikhonravov et al.¹⁵ introduced needle optimization for optical coating design. Needle Optimization iteratively adds or adjusts layers in a thin film design to improve its optical properties toward the design target by modifying the thickness or refractive index of the material configuration (see Figure 3A). The needle optimization evolves gradually, allowing for the generation of multiple solutions with nearly the same performance metrics, providing the designers with a range of options to select the most practical and manufacturable design. Another example is the application of genetic algorithms (GAs) to the design of optical multilayer structures by Martin et al.¹⁸ Genetic algorithms are stochastic optimization methods inspired by the principles of natural selection and genetics. They operate by generating an initial population of potential solutions and iteratively evolving these solutions through selection, crossover, and mutation operations to maximize a fitness function, which is usually taken as the merit function commonly used in traditional approaches. In each iteration, the individuals with the highest fitness score are more likely to be selected for reproduction, ensuring that their genetic material is passed on to the next generation. Crossover combines parts of two parent solutions to create offspring with potentially better performance, while mutation introduces random changes to individual solutions to maintain diversity within the population and prevent premature convergence to local optima. The authors demonstrated the effectiveness of this algorithm by designing the refractive index and thickness of each structure layer for three different optical filters: antireflection coating, beam splitter, and rejection filter (see Figure 3C). This idea has also been applied to the design of broadband reflectors,⁵⁷ edge filters,⁵⁸ and so forth.

However, using the refractive index to represent materials restricts the design target to be a narrow band because of materials’ dispersion. To design for a broader wavelength, Shi et al.⁴⁶ introduced a mixed-integer programming (MIP) with a genetic algorithm solution to optimize the design of multilayer thin films. MIP can handle both discrete and continuous variables in the optimization process. In this context, the discrete variables represent different dielectric materials, while continuous variables correspond to the thickness of each material layer. This algorithm improves upon traditional genetic algorithms by introducing discrete materials into the crossover and mutation steps (see Figure 3C). The authors demonstrated state-of-the-art performance using the algorithm on two design tasks with a broad wavelength range: radiative cooling devices and incandescent light bulb filters. The GA with the MIP algorithm’s ability to design a variety of solutions with distinct material and thickness permutations discovered innovative designs for multilayer thin films at a global solution space for wide wavelengths, materials, and thicknesses.

Deep learning methods with vectorized representation

The fundamental unit inside neural networks is the artificial neuron. As shown in Figure 4A, a single neuron first takes one or multiple inputs, then sums up all these inputs multiplied by learnable weights and outputs results through a nonlinear activation function. In this way, information can be passed in, get processed, and propagated to the next level. Usually, a single neural network (NN) can contain thousands or millions of neurons, leading to millions or billions of learnable weights. After training on a large dataset, these learnable weights are updated to capture the general mapping between the inputs and outputs and process extremely complicated information such as image classification,^59,60,61 speech recognition,^62,63,64 sequence decision,^65,66,67 language understanding,^49,68,69,70 and so forth. Different types of neural networks differ in the way how the neurons are connected, and therefore, are suitable for processing different types of data. For example, Convolutional Neural Networks (CNN)⁵⁹ use the convolutional layers to capture spatial correlations, thus widely used for processing and analyzing image-type data. Recurrent Neural Networks (RNN)⁴⁷ employ recurrent connections to obtain temporal dependencies, making them well-suited for sequential data.

Figure 4.

Illustration of different types of neural networks

(A–F) Visualizations of (A) a single neuron, (B) Multilayer-Perceptron (MLP), (C) tandem networks, (D) Variational Auto-Encoders (VAE), (E) Generative Adversarial Networks (GAN), and (F) Mixture Density Networks (MDN).

In the previous discussion, we concluded that most of these optical responses (inputs) can be denoted as vectors while multilayer structures (outputs) can be denoted using either vectorized, combined, or sequential representations. However, combined representations require a discretized output for material design, which is non-trivial for NN since their outputs are continuous variables. Also, it was not until recent two or three years ago that researchers figured out the sequential representation. Therefore, most of the early works that used deep learning for inverse design seek to solve the vector-to-vector mapping. A general working process is to first generate a training dataset containing multilayer structures and corresponding optical responses using simulation (e.g., TMM), then train the NN model to fit the inverse mapping function from optical responses to multilayer structures. Once trained, the model can be used to design multilayer structures for different optical responses instantaneously, providing a much faster alternative than optimizations, which usually take minutes or hours.

In terms of the deep learning models for vector-to-vector mapping, perhaps the most straightforward way is the Multilayer-Layer Perceptron (MLP), where neurons are stacked layer-by-layer and all neurons are fully connected to other neurons in nearby layers (see Figure 4B). However, directly using MLP to inverse design will give inaccurate results. Because of the one-to-many mapping issue, i.e., there are multiple possible structures for a given optical response, it is difficult to minimize the loss function (the difference between model outputs and target outputs) and have the NN model converged during training. Eventually, the model will learn to output an averaged structure, which usually does not have the desired optical responses.³⁴ Therefore, special buildup of neural networks is required to handle this one-to-many mapping issue. Many different types of neural networks have been proposed and demonstrated effectiveness in inverse design. Based on the type of outputs, they can be classified into two different categories: (1) deterministic NN, mainly including tandem networks, and (2) stochastic NN (or generative NN), including Generative Adversarial Networks (GAN), Variational Auto-Encoders (VAE), and Mixture Density Networks (MDN). In the remaining section, we are going to discuss each model and summarize its variants and applications.

Deterministic NN

In deterministic NN, the model always maps the input into a single output. The most popular one is the tandem network. As the name “tandem” suggests, tandem networks connect two NN in series (see Figure 4C): the Forward Neural Networks (FNN) and the Inverse Neural Networks (INN). The FNN takes in the structural parameters and outputs the predictions of their corresponding optical responses. Once trained, we can use FNN to quickly predict the optical responses for a given structure with high accuracy. On the other hand, INN takes in the target optical responses and outputs predicted structures. The training of tandem networks involves a two-step training procedure for each individual network. In the first step, we need to train FNN separately to obtain a fast surrogate predictor for optical responses. The second step is to connect the output of INN to this pre-trained FNN and use FNN to supervise the learning of INN. In this way, we can resolve the one-to-many mapping issue by only predicting one structure suggested by FNN.

Tandem networks is first introduced in Liu et al.¹⁹ to design transmission spectra for multilayer structures. The multilayer system in consideration is a twenty-layer structure with SiO₂/Si₃N₄ alternating material combinations. Using this method, the authors have demonstrated excellent design performance for arbitrary spectra, as well as the special Gaussian shaped spectra (see Figure 5A). In addition, it only takes a fraction of a second to finish a design, which is much faster than optimization-based methods. Later, extensive works have been dedicated to utilizing tandem networks to design different multilayer structures and tackle various design problems efficiently, including designing a Fabry-Perot-cavity-based color filter,⁷¹ organic light-emitting diode,⁷² and so forth. In addition to multilayer structures, tandem networks are also widely used in other types of photonic structures and target responses.^73,74,75,76 For example, Gao et al.⁷³ used tandem networks to design reflective structural color for silicon nanorod metasurface; Ma et al.⁷⁴ combined tandem networks to design a reflection spectrum for three-dimensional chiral metamaterials; He et al.⁷⁵ utilized tandem networks to design the far-field radiation profile for plasmonic nanoparticles.

Figure 5.

Deterministic NN for multilayer structure inverse design

(A) Tandem network visualization in Liu et al.¹⁹ as well as their design performance on gaussian-shaped spectrum. Reprinted from Liu et al.¹⁹ Copyright @ 2018, ACS Publications.

(B) Performance comparison of original tandem network with modified tandem network using ReLU activation function with intermediate structural loss. Reprinted from Xu et al.²⁴ Copyright @ 2021, Elsevier.

(C) A modified tandem network: meta-material spectrum transformer (MST), which uses transformer as the building block for both forward and inverse networks. The model achieves good performance on solar metamaterial absorber (SMA). Reprinted from Chen et al.²² Copyright @ 2023, Wiley.

There are also many variants to improve the design performance of tandem networks. For example, Xu et al.²⁴ modified and benchmarked different activation functions and loss functions used in tandem networks to find the optimal schema. They found that using ReLU activation function⁷⁷ together with intermediate structural loss can significantly increase the design accuracy (see Figure 5B). On the other hand, many researchers seek to find alternate neural networks to replace the MLP architecture used in INN and FNN.^22,78 For example, Chen et al.²² proposed to use transformer architecture for both INN and FNN by leveraging the self-attention mechanism to effectively extract and process information. In this work, they proposed a meta-material spectrum transformer (MST) to design a high-performance broadband solar metamaterial absorber that exhibits a high average absorptance of 94% in a broad solar spectrum (see Figure 5C). In addition, considering that the original tandem network can only output one designed structure, Yuan et al.²⁵ came up with the idea of connecting the INN to multiple different FNNs, which they called multi-head tandem networks, to expand its capability of outputting multiple structures.

Stochastic NN

Different from deterministic NN, stochastic NN (also called generative NN) can output multiple different predictions given the same input. This is because these models take in one latent variable randomly sampled from a normal distribution as an extra input, which enables them to generate different outputs conditioned on this extra input. Therefore, they can intrinsically solve the one-to-many mapping issue. There are three major types of generative NN that are widely used in inverse design: VAE,⁷⁹ GAN,⁸⁰ and MDN.⁸¹ We illustrate their model architectures in Figure 4D−4F, respectively. There are already many reviews^32,33,35 and benchmark^28,29,82,83 that discuss these models, but toward a more general audience on the photonic inverse designs. Here, we mainly summarize the work done for multilayer structure designs.

Figure 4D shows the model architecture for VAE, which originates from information theory: it first uses an NN (called encoders) to encode the high-dimensional input information into a low-dimension hidden representation, usually with a normal distribution, then uses another NN (decoders) to decode hidden representations back into the corresponding output. During inverse design, each input (target optical responses) will be encoded to different normal distributions with different means and variances. Sampling from this distribution and sending it through the decoder will give the designed structure. Since this sampling process is random, VAE can output multiple structures when decoding from different latent variables, thus solving the one-to-many mapping issue naturally. For example, Zandehshahvar et al.²⁶ used the VAE to design multilayer structures composed of consecutive layers of SiO₂ and TiO₂. In their work, they demonstrated that when given a single transmission spectrum as input, their model can output three different structures (Figure 6A left side). The simulation shows that their simulated spectra are close to the target spectrum (Figure 6A right side). VAE has also demonstrated successful applications in other types of photonic structures, including Fano-type resonant nanopillars,⁸⁴ hybrid plasmonic/phase-change material metasurface,⁸⁵ a periodic array of Au nanoribbons,⁸⁶ and so forth.

Figure 6.

Stochastic NN for inverse design with multiple outputs

(A) VAE can output multiple designs that satisfy the desired transmission spectrum. Reprinted from Zandehshahvar et al.²⁶ Copyright @ 2021, Optica Publishing Group.

(B) A three-layer structural color designed by GAN showing different structures. Experimental results also verified the design performance. Reprinted from Dai et al.⁵¹ Copyright @ 2022, Elsevier.

(C) The output probability from MDN showing a multi-modality distribution which corresponds to different designs. All designs show desired performance in the transmission spectrum. Reprinted from Unni et al.²⁰ Copyright @ 2020, ACS Publications.

Another type of stochastic NN is GAN (Figure 4E). It also contains two neural networks: the generator NN that generates designed structures based on the random latent variables and the input optical responses, and the critic NN that learns to distinguish if the input structure comes from a model or not. Different from VAE, GAN originated from game theory and these two NN learn to generate by combating with each other: the generator tries to generate outputs that are distributed as close to the training dataset as possible to fool the critic, while the critic tries to distinguish the generated outputs from the ground truth in the training dataset. A successful application is demonstrated in Dai’s work,⁵¹ where they used GAN to design a transmissive Fabry-Perot-cavity-based color filter using the three-layer Ag-SiO₂-Ag structure and identified an average number of 3.58 solutions for each color. Experimental results in Figure 6B also validated the effectiveness of these designs.

MDN is the third type of stochastic NN (see Figure 4F). Different from VAE and GAN which utilize single-modal normal distribution, MDN models the design parameters as a multimodal probability distribution. Here, each distribution in this multi-modality is parameterized by a different normal distribution with unique mean, variance, and weight. The total number of distributions is a hyperparameter that is usually pre-determined and can be tens or hundreds. After training, the designed structure will be obtained after sampling from this predicted multimodal distribution, which inherently solves the one-to-many mapping issue. As an example, Unni et al.²⁰ utilized MDNs with 16 mixtures to design 10 layers of alternating SiO₂ and TiO₂ structure. Their results in Figure 6C demonstrated that the MDN can give multiple structures that satisfy the desired transmission spectrum.

All these four different neural networks, including tandem networks, GAN, VAE, and MDN have been exploited for multilayer inverse design with vectorized representation. However, each model still has its own limitations. For example, tandem networks are simple to understand and easy to train, but they can only output one designed structure for one input target, which can lead to difficulty when multiple structures are desired. On the other hand, all the other three models are stochastic models and can output diverse structures. Both GAN and VAE samples from a single-modal distribution, but GAN usually has higher diversity than VAE, while VAE is more robust than GAN. With a multi-modality distribution, MDN can learn a more diverse solution than both GAN and VAE. However, MDN is harder to train because of the increased number of modalities and parameters. A delicate adjustment of a number of modalities based on the specific dataset is required to obtain better performance. For an exhaustive discussion, we recommend some benchmark works that compare these models in a fair setting.^28,29,83

Optimization-combined deep learning

Although optimization and deep learning based methods are widely used for inverse design, both have their own strengths and limitations. Optimization has much better flexibility to incorporate the material into design consideration. Designing in this global space with both materials and thicknesses usually leads to better performance than deep learning-based methods. However, optimization requires iterative evaluations to minimize the fitness function, which can be time-consuming. In addition, one needs to restart the optimization process from scratch whenever encountering a new design target, making it task-specific. On the other hand, because deep learning based methods usually use neural networks to learn a general mapping from the space of optical responses to the space of multilayer structure, they can finish a design task much faster than optimization and can quickly adapt to a different design target. The probability output for stochastic NN also enables multiple designs to solve one-to-many mapping issues. However, these neural networks usually have vectorized outputs, making it difficult to incorporate the discrete materials into outputs, thus, resulting in limited design performance. A natural question to ask is if it is possible to combine both methods together and obtain greater benefits. It is encouraging to see that many researchers have already started to do so. Based on how these two methods are integrated together, we can classify them into three categories and we will summarize each of them later in discussion.

Neural networks as a fast surrogate simulator

Neural networks are not only fast for inverse design, but also a fast surrogate simulator to predict the optical responses given the structure.^19,50,87 Considering that optimization requires an iterative evaluation process, it is straightforward to use NN as the surrogate simulator to speed up the optimization process. For example, Unni et al.²¹ proposed a mixture-density-based tandem optimization network to design high reflectors. As shown in Figure 7A, they first pre-trained a forward simulator network to instantly evaluate any designs, then ran optimization algorithms to quickly identify the optimal structure. To further speed up the design process, they trained a mixture-density network to give some initial starting structures for optimization. Using this method, they have successfully designed an ultra-broad high reflector using a 20-layer structure. In another work, Ma et al.⁸⁸ combined deep learning and multi-objective double annealing algorithms to simultaneously maximize solar spectrum absorption and minimize infrared radiation. As shown in Figure 7B, they first trained an MLP for fast evaluation, then used the numerical optimization to find the multilayer structure that leads to the best performance. Compared to traditional optimization, this deep learning-aided optimization can drastically reduce simulation time and make optimization several orders of magnitude faster.

Figure 7.

Multiple ways to combine optimization and deep learning together

(A) A mixture-density-based tandem optimization network. Reprinted from Unni et al.²¹ Copyright @ 2021, Elsevier.

(B) Deep learning combined with multi-objective double annealing algorithms. Reprinted from Qu et al.⁸⁷ Copyright @ 2023, Elsevier.

(C) Global optimization networks (GLOnets) and (D) the design performance comparison using GLOnets and traditional optimization. Reprinted from Jiang and Fan.⁵² Copyright @ 2021, Elsevier.

(E) NEUTRON: Neural particle swarm optimization for material-aware inverse deign and its design performance combined to NN only (F). Reprinted from Wang and Guo.⁵³ Copyright @ 2022, Cell Press.

Neural network as surrogate gradient update

In addition to being fast, another advantage of neural networks is that they are differentiable. After learning from the dataset, NN can build up a continuous and analytical mapping between input and output spaces, even if the relationship in the original data is too complicated to describe with closed-form equations. Therefore, we can use NN to calculate the gradient information and update the input structures directly. Motivated by this idea, Fouchier et al.⁸⁹ first trained a forward simulator, then fixed the parameter in NN and used backpropagation to calculate the gradient and update input structure parameters. They successfully used this method to design coatings with different light scattering properties. Another typical work is called global optimization networks (GLOnets).^52,90,91 As shown in Figure 7C, GLOnets combine generative NN with a TMM simulator to perform population-based optimization. During the design process, the generative NN will output thickness information and refractive index profile using a probability matrix. The combined multilayer structure output will go through TMM solvers and obtain a loss function, which will be used to calculate the backward gradient and update weights in generative NN. After optimization, the generative NN will map the input optical responses to a designed structure. Using this combined method, they have identified an anti-reflective coating with 1.81% averaged reflection, which is lower than traditional optimization methods (See Figure 7D). In a later work, Zhang et al.⁹² used GLOnets to design the optical transfer function of a multilayer film at wavelengths of 532 nm and 633 nm and successfully achieved incoherent differentiation with a high resolution of 6.2 um, opening the door for high-speed imaging processing, object tracking, and disease diagnosis.

Two-step design process

Different from previous two methods that use NN as a surrogate model, the third type splits the design process into two steps: first use NN to design material combinations, then use optimization to identify the suitable thickness. An example is the algorithm of Neural Particle Swarm Optimization (NEUTRON), which was introduced by Wang et al.⁵³ to inverse design structural colors. The idea was illustrated in Figure 7E. Here, instead of design material and thickness at the same time, NEUTRON first uses a classifier to predict the most suitable materials at each layer for a given layered structure, then uses MDN to give the initial thickness guess in a probability distribution form and combine PSO to refine and optimize the thickness and achieve optimal performance. Using this two-step process, they demonstrated a much more accurate structural color design compared to single MDN, and used it to output structures for a large number of colors in a painting. We give one design example in Figure 7F.

Deep learning methods with sequential representation

Sequence modeling and understanding is an important research field in machine learning as most of the data in our world can be treated as sequences, including natural language,⁴⁹ human voice,⁹³ time-series,⁹⁴ video stream,⁹⁵ and so forth. Different from vectorized data with pre-determined shapes, sequential data usually have variable length and require special architecture in NN, e.g., the recurrent connection inside RNN,^47,48 to handle the serialized information. Recently, researchers have observed that the varying numbers of layers in a multilayer structure is analogous to the varying lengths in sequential data. Therefore, they have started to treat multilayer structures as a sequence and use sequential NN to deal with inverse design. In this way, researchers can completely automate the inverse design process for the first time: NN can design the material and thickness simultaneously at each layer, and can also design the total number of layers by smartly terminating the design process whenever NN thinks the current design is good enough. This points to a bright future for multilayer inverse design. Currently, there are two different sequential learning methods used for inverse design: sequential decision process and conditional sequence generation.

Sequential decision process

The first type of sequential learning model is based on the sequential decision process,⁶⁶ where the model needs to decide what to do next based on the current situation. It has been widely used in robot manipulations,⁹⁶ game playing,^65,67 self-driving cars,^97,98 and so forth. In 2021, Wang et al.²³ proposed to use a sequential decision process for multilayer design and introduced the algorithm of OMLPPO based on the reinforcement learning algorithms.^66,99 In this work, OMLPPO designs and finalizes the multilayer design in a layer-by-layer manner: after figuring out the material and thickness for the first few layers, it will decide what type of material to use for the next layer as well as its corresponding thickness. This process will keep going until the designed structure reaches the desired performance, or the sequence length (namely, the number of layers) equals the maximum allowable number of layers. The proposed OMLPPO algorithm learns to make sequential decisions in a trial-and-error way and uses iterative evaluation to update designs and approach the ideal structure. The complete pipeline is shown in Figure 8A. Although this iterative process looks similar to heuristic optimizations, OMLPPO can learn multilayer structure design domain knowledge during training and leverage it to improve the design performance. For example, the algorithm automatically designed a five-layer ultra-wideband absorber with 97.64% average absorption, which is ∼2% more than the one designed by human experts (Figure 8B). They also used the algorithm to design incandescent light bulb filters and achieved an 8.5% higher visible light enhancement factor than the structure designed by optimization (Figure 8C). Because of its high performance, OMLPPO has also been adapted to solve other non-trivial design tasks. For example, Saha et al.⁴⁰ identified two multilayer structures that have a similar visible reflection spectrum as decorative Cr, serving as an environmentally friendly replacement for the harmful and toxic chemical finishing process (Figure 8D).

Figure 8.

Sequential decision process for multilayer design

(A) The working pipeline of OMLPPO algorithm.

(B) The perfect absorption spectrum designed by OMLPPO, which is 2% higher than demonstrated human designs.

(C) The incandescent light bulb filters designed by OMLPPO, which exhibits 8% performance improvement than optimization methods.

(A–C) are reprinted from Wang et al.²³ Copyright @ 2021, IOP Publishing.

(D) The decorative Chrome-looking reflective coating designed by OMLPPO. Reprinted from Saha et al.⁴⁰ Copyright @ 2023, ACS Publications.

Conditional sequence generation

Another type of sequential learning model is conditional sequence generation, where the model needs to generate serialized output based on the input. One important example is the language modeling¹⁰⁰: to predict the probability of a sequence of words conditioned on previous words and construct coherent and contextually relevant texts. It is the backbone for a wide range of natural language processing (NLP) applications, including machine translation,¹⁰¹ question answering,¹⁰² text generation,¹⁰³ and so forth. The recent introduction of GPT-type models^{69,103,104,105} which use the decoder part of the Transformer⁴⁹ model, not only improves the learning efficiency of language modeling but also enhances the model’s ability to generate more coherent texts by scaling up the model size and training dataset, leading to the development of powerful language models, including ChatGPT,¹⁰⁶ Llama,¹⁰⁷ BART,⁷⁰ and so forth. The success of the GPT model in language modeling also intrigues researchers to solve other sequential generation problems in scientific and engineering applications, including DNA sequence generation,¹⁰⁸ protein prediction,¹⁰⁹ molecule modeling,^110,111 and so forth.

Inspired by these successful GPT models, in 2024, Ma et al.³⁹ proposed the OptoGPT model and completely tackled these four challenges for multilayer design. Figure 9A and 9B show the analogy between GPT models and the proposed OptoGPT model. For GPT models, given some language texts as input, e.g., a question or task description, they can generate a sequence of words that are related to the inputs. Similarly, given the input of optical responses (in their case, is reflection and transmission spectrum), the OptoGPT model can output the serialized representation of multilayer structures directly, without any iterative process.

Figure 9.

Idea of conditional sequence generation using OptoGPT model

(A and B) Shows the diagram of general GPT models in NLP and the OptoGPT model in multilayer inverse design, respectively.

(C) Different types of input to the OptoGPT model as the design target.

(D) One example of the “structure tokenization” and “structure serialization” for an N-layer structure on the glass substrate. All figures are reprinted from Ma et al.³⁹ Copyright @ 2024, OE Journals.

To make OptoGPT work, we first introduce the “structure serialization” technique to convert the material and thickness information at each layer into a single token by simply concatenating them together. As shown in Figure 9D, a token is a human-readable notation similar to a word or a phrase in language. Then, we proposed the “structure serialization” technique and added these tokens one by one to form the sequence representation of a multilayer structure. In this way, the OptoGPT model can effectively design for both material and thickness information, and automatically determine the optimal number of layers. The input to the OptoGPT model is optical responses. To make OptoGPT well suited for different types of optical responses, we propose to design for both transmission and reflection spectrum and use multiple techniques to extend the input to diverse responses, including absorption spectrum, reflective and transmissive structural color, and so forth. Examples can be found in Figure 9C.

OptoGPT is a foundation model¹¹² that once trained it can be used to solve extensive downstream tasks and applications to design multilayer thin film structures. OptoGPT demonstrated a unified inverse design toward a range of applications, including transmissive/reflective filter, high reflection filter, perfect absorber, arbitrary absorber, reflective/transmissive structural color, and so forth. Figure 10 gives some inverse design examples. Notably, each design task can be completed within 0.1 s. In addition, because the output is based on probability sampling, therefore, OptoGPT can give different designs by running inference multiple times, which solves the diversity challenges. To solve the flexible design challenge, we further proposed “probability resampling” to design structures that can satisfy arbitrary constraints during fabrication or other practical requirements. Using finetuning and “mixed sampling,” OptoGPT can even design structures for different incident angles and polarization states, as well as simultaneous design for multiple angles requirements. In this way, OptoGPT solves all previously mentioned challenges in multilayer inverse design, significantly pushing the research boundaries forward.

Figure 10.

Examples of different types of optical designs that OptoGPT can design with

(A) Design for a band-notch filter at 550 nm.

(B) Design for high reflection in near-infrared.

(C) Design for perfect absorber.

(D) Design for arbitrary absorber.

(E–F) Design for reflective/transmissive type structural color. All figures are reprinted from Ma et al.³⁹ Copyright @ 2024, OE Journals.

Discussion and outlook

In conclusion, this review summarized the approaches of different types of inverse design algorithms specifically for multilayer thin film structures, starting from traditional optimization-based methods, to the recent popular deep learning based algorithms. Optimization-based methods require iterative simulations of multilayer structures as well as evaluations of the fitness function, which can be time-consuming for complicated structures and multiple designs. On the other hand, deep learning based methods learn a general mapping from some established datasets, therefore, they are much faster than optimization. However, considering that collecting training datasets and training modes could take much time, it would be wise to first estimate the time consumption as compared with the task to be accomplished, and then decide which method to use. In addition, optimization-based methods are more flexible when dealing with new materials from the lab, or some custom multilayer structures with different surrounding mediums and substrates. In both cases, one would need to collect a new dataset and retrain deep learning models.

Compared to other types of photonic structures, multilayer structures are special as they can be represented using three different methods. Each of them has its unique advantages and drawbacks, and corresponds to a specific type of inverse design algorithms. It was only until recently that researchers have identified the sequential representation, a unique data type that is specific to layered structure. This new method has shown promising potential to reshape the inverse design.

Despite the great success deep learning-based methods have achieved, there are still remaining questions to be solved. Perhaps the biggest question too deep learning is the “black-box problem,” i.e., the predictions from NN cannot be properly explained due to the high-level complexity and nonlinearity inside NN. In inverse design, this explainability is very important to understand why certain structures are preferred during the design process, e.g., if the designed structure is coming from some existing physical behaviors or potentially includes a new physical mechanism that has not been discovered before. However, uncovering the black-box problem has not been extensively explored yet. New ideas and approaches are needed to tackle this issue and reveal hidden physical principles.

Another issue is that many deep learning based methods require a large dataset. Although after training, NN is extremely fast for inverse design, generating the training dataset itself can take a long time. This leads to the dilemma of whether the speed-up in the inverse design process is worthwhile. In addition to the data-based deep learning, currently another type of neural operator-based learning has been popular in solving complicated partial differential equations, such as PINN^113,114 and DeepONet.^115,116 By integrating the physics rules and mathematical equations into NN, these models can directly learn the complicated mapping from input to output without any data. More efforts are needed in order to incorporate these methods into inverse design.

Finally, there are still several issues that require special attention for both methods. For example, it is important to consider the physical conditions for the size of the incoming light beams during the design process, especially for the applications where the input light spot size is very small, such as functional filters for laser beams. If these conditions are not properly incorporated into design considerations, the multilayer structure may not perform as expected at some higher incidence angle. Currently, these issues have not been extensively explored for both optimization and deep learning-based methods. New methods and techniques are required to solve these challenges. One possible direction to take is to treat the specialized cases as “out-of-distribution” datasets, where we have recently proposed a method to handle some of these situations, based on the pre-trained transformer and incorporating perturbations and genetic optimization.¹¹⁷

Acknowledgments

We thank the National Science Foundation FET-2309403 for the support of this work.

Author contributions

T. M. and L. G conceived the review framework. All authors contribute to the writing.

Declaration of interests

The authors declare no competing interests.