Ab initio generalized Langevin equation

Significance

Recent developments of neural network–based atomistic models enable large scale molecular dynamics based on quantum mechanical theory. However, there is still a gap between modeling microscale molecular dynamics and modeling mesoscale coarse-grained (CG) dynamics with the same level of accuracy. The ab initio generalized Langevin equation (AIGLE) model is proposed to bridge the gap, by learning accurately an effective equation of motion for multidimensional CG variables from molecular dynamics data. The capability of AIGLE is demonstrated by simulating domain wall dynamics and extensive dipolar motion in the ferroelectric crystal lead titanate. This work is a step toward bottom–up modeling of mesoscale transformations in materials.

Abstract

We introduce a machine learning–based approach called ab initio generalized Langevin equation (AIGLE) to model the dynamics of slow collective variables (CVs) in materials and molecules. In this scheme, the parameters are learned from atomistic simulations based on ab initio quantum mechanical models. Force field, memory kernel, and noise generator are constructed in the context of the Mori–Zwanzig formalism, under the constraint of the fluctuation–dissipation theorem. Combined with deep potential molecular dynamics and electronic density functional theory, this approach opens the way to multiscale modeling in a variety of situations. Here, we demonstrate this capability with a study of two mesoscale processes in crystalline lead titanate, namely the field-driven dynamics of a planar ferroelectric domain wall, and the dynamics of an extensive lattice of coarse-grained electric dipoles. In the first case, AIGLE extends the reach of ab initio simulations to a regime of noise-driven motions not accessible to molecular dynamics. In the second case, AIGLE deals with an extensive set of CVs by adopting a local approximation for the memory kernel and retaining only short-range noise correlations. The scheme is computationally more efficient than molecular dynamics by several orders of magnitude and mimics the microscopic dynamics at low frequencies where it reproduces accurately the dominant far-infrared absorption frequency.

Developing accurate and reliable mesoscale physical models is a long-standing problem (1–3). In this context, the Mori–Zwanzig formalism (4) stands out as a general methodology for constructing effective coarse-grained (CG) models for any set of collective variables (CVs) defined in terms of microscopic degrees of freedom, such as the atomic coordinates. The idea is to project the dynamics of the microscopic variables on the space of the CVs. Finding an approximate surrogate model for the formal projective dynamics requires knowledge of the free energy as a function of the CVs, and brings in two important effects: memory, because CV dynamics is generally non-Markovian, and noise, associated with the initial condition for the variables eliminated in the projection process. These effects are difficult to model. As a consequence, one often resorts to simpler approximations for the effective dynamics, such as the Markovian Langevin equation (LE).

Combined with Landau free energy models (5–7), LE has been a popular tool for describing mesoscale dynamical processes. Well-known examples include the Landau–Lifshitz equation for the evolution of the magnetization in materials (8), the Allen-Cahn and the Cahn–Hilliard equations for the dynamics of phase transitions and separations (9, 10), and, more generally, the phase field models (11) and the phase-field-crystal models (12, 13) for a variety of problems in materials science. Landau-based LEs provide invaluable physical insight but may lack the flexibility required to quantitatively model the CG dynamics of real systems. A main issue is the insufficient separation of time scales between the CVs and the noise. In realistic systems, noise may originate from vibrational modes that are not significantly faster than the CVs. In this scenario, the non-Markovian generalized Langevin equation (GLE) is a much better approximation. It can be rigorously derived within the Mori–Zwanzig formalism for Hamiltonians that depend quadratically on the microscopic degrees of freedom (4). In the presence of anharmonicity, the GLE is not exact but can be a flexible enough tool for connecting micro- and mesoscale dynamics, similar in spirit to the way in which semilocal density functional theory (DFT) bridges electronic quantum mechanics and atomistic models (14). So far, efforts to develop quantitatively accurate GLE models have been limited by difficulties in the parameterization of the memory and noise terms (15–17). In the context of bottom–up multiscale modeling, these difficulties lie in the lack of microscopic data, on the one hand, and of robust algorithms to parameterize the GLE, on the other.

In recent years, machine learning has emerged as a powerful tool in the study of static and dynamic statistical properties of molecular systems (18–23), enabling ab initio simulations of unprecedented scale (25, 26). Today, massive amounts of data can be generated by all-atom molecular dynamics (MD) trajectories with ab initio accuracy. As we will demonstrate below, machine learning can also address the second difficulty mentioned above.

In this paper, we introduce a machine learning–based method for constructing accurate CG GLE models from fine-grained/microscopic Hamiltonians. We illustrate the approach with atomistic models derived from DFT, but the methodology can also be applied to microscopic models derived phenomenologically. In our scheme, memory is of finite length and translationally invariant in time, and the noise satisfies the constraint imposed by the second fluctuation–dissipation theorem (2FDT) (27), which connects the memory kernel to the autocorrelation function (ACF) of the noise. The 2FDT is essential to describe the dynamics of near-equilibrium physical systems. We call the schemes constructed in this way ab initio generalized Langevin equation (AIGLE) models, because they are trained on data generated with an ab initio microscopic model. The LE model derived from AIGLE by taking the Markovian limit in the memory kernel and the noise will be called ab initio Langevin equation (AILE) model.

Previous works have studied data-driven parameterizations of the GLE (28–43), the LE (44–46), the far-from-equilibrium GLE (47, 48), and generic stochastic processes (49, 50). For GLE restricted by 2FDT, the noise generator is usually constructed from a predetermined memory kernel or from the ACF of the noise. For instance, in refs. 29 and 30, the noise generator is a Yule–Walker linear autoregressive (AR) model fitted to the ACF of the noise. The resulting model does not guarantee the stationarity of the noise. That can be imposed by adjusting the roots of the characteristic equation, but this may lead to uncontrolled errors. Another approach, reported in refs. 31 and 32, assumes that the noise generator is a Fourier series with random coefficients sampled from a distribution defined by the memory kernel. In practice, the Fourier series is of finite length, and the generated noise and its ACF become periodic. Recently, ref. 41 proposes to convert a GLE into Markovian equations of motion for fictitious degrees of freedom, by using a finite order Padé approximant for the memory kernel. The dynamics of the fictitious degrees of freedom is constructed according to the memory kernel while retaining the 2FDT constraint on the noise. In general, approaches that use an average property like the ACF to fix the noise are “mean-field” approximations, aiming at consistency with data on 2FDT rather than on presumably less relevant features like higher-order correlations or kurtosis. Concerns have been raised that in some of these approaches, the statistical error of the correlation functions may be amplified in an uncontrolled way (43).

Alternatively, one can go beyond “mean-field” by adopting a regression approach. For example, ref. 49 introduced a nonlinear autoregressive model for generic stochastic processes not constrained by the 2FDT. Refs. 50 and 51 used recurrent neural networks for learning generic dynamical systems, as the nonlinear nature of general regression tasks may require sophisticated deep neural network models. However, in specialized but important cases such as near-equilibrium systems, knowledge of the free energy surface (FES) and the 2FDT facilitate the task, making it possible to reproduce the time series with relatively shallow and simple neural network regressors. Then, accuracy, stationarity, and efficiency can be achieved simultaneously. In AIGLE, we strive to optimize these three qualities, while “mean-field” approaches essentially compromise accuracy. We constrain the memory kernel with the 2FDT and model the noise with a neural network-based generalized autoregressive (GAR) scheme that can deal with insufficient time-scale separation and anharmonic coupling of the modes. These complications are common in real materials but are often overlooked in toy models. To have an efficient noise generator suited for long-time simulation, we keep the neural network as simple as a compact feed-forward neural network. While most data-driven GLEs assume prior knowledge of the FES, in AIGLE not only the noise but also the FES and the couplings to the driving fields can be parameterized. Moreover, while most existing literature uses a one-dimensional GLE, we introduce a multidimensional version of AIGLE, based on a local kernel approximation and a consistent GAR model, which can reproduce not only the one-body but also local two-body correlations. The adopted approximation balances efficiency and accuracy, making it possible to deal with infinite-dimensional, homogeneous CVs. To our knowledge, multidimensional GLEs have only been used so far to study the few-body dynamics of low-dimensional CVs (52–54).

In this paper, we expose the details of AIGLE and demonstrate its effectiveness in an ab initio multiscale study of ferroelectric lead titanate (PbTi${\text{O}}_3$). The scheme is not limited to ferroelectric problems and its mathematical structure can be used in reduced models of general crystalline materials. In the present application, the order parameters, i.e., the CVs, depend on the local electric dipole moments associated with the crystalline lattice. These local moments act like lattice spins in ferromagnets, and, as the latter, can be coarse grained to scalar or vector fields in the continuum limit. Unlike lattice spins of fixed magnitude, the local dipoles fluctuate in both direction and magnitude. The sum of the local dipoles defines the polarization of the system, which is a typical example of a global order parameter in Landau’s theory of symmetry breaking. Damped vibrational modes associated with polar phonons are embedded in the dynamics of the local dipoles, inducing oscillating correlations among the dipoles, a behavior that differs significantly from the diffusive dynamics of Brownian particles, whose velocity ACF decays exponentially with time. Hence, the difficulties encountered in CG dipole dynamics are similar to those encountered in realistic multiscale models of materials and macromolecules.

Specifically, we consider two examples of mesoscale dynamics in crystalline materials: the field-driven dynamics of a planar ferroelectric domain wall treated as a virtual particle in a non-Markovian bath and the dynamics of extensive local order parameters with translationally invariant interactions. In both cases, AIGLE is trained with atomic trajectories generated at room temperature with the Deep Potential (DP) scheme (21), a deep learning approach that closely reproduces the quantum mechanical potential energy surface at the DFT level of theory. The microscopic lattice dipoles, rigorously defined in the theory of the electric polarization within DFT (55, 56), are represented by an equivariant generalization of the DP model (57). In the first example, we study the glassy dynamics of a planar 180° domain wall induced by a weak electric field $\mathit{E}=E\widehat{z}$ parallel to the polarization of one of the domains. We find that the domain wall shifts by a succession of rare events. At low fields, the domain velocity $v_D$ predicted by AIGLE gradually deviates from its AILE counterpart and from the phenomenological scaling law of Merz (58), according to which $v_D\propto e^{-E_a/E}$, with constant $E_a$. Merz’s law can be derived from the theory of elastic interface motion (59, 60) that describes the dynamics with an overdamped Langevin equation. Our results suggest that inertia and memory effects captured by AIGLE play a role in the glassy motion of elastic interfaces under weak applied fields. In the second example, we consider the dynamics of a CG lattice of dipoles in the bulk of a compressively strained PbTi${\text{O}}_3$ crystal. AIGLE reproduces well self- and close-neighbor correlations of the dipoles and captures approximately the ACF of the time derivative of the polarization, whose Fourier transform yields the far-infrared optical spectrum. AILE fails in this task but still models correctly the relaxation pattern of a domain structure, when this is driven by surface tension and memory is not important. A CG lattice dynamics of extensive CVs like that provided here by AIGLE or AILE would be useful, in general, in studies of the dynamics of extended crystal defects and of epitaxial growth of materials.

The paper is organized as follows. In Section 1, we introduce the AIGLE formalism. In Section 2, we use AIGLE for ab initio multiscale modeling of PbTi${\text{O}}_3$. Specifically, we report in Section A a model for the field-driven motion of a planar domain wall in epitaxial PbTi${\text{O}}_3$. In Section B, we report an extensive model of CG lattice dynamics. Details of one-dimensional AIGLE are in the Material and Methods section. Details of multidimensional AIGLE are in SI Appendix, which also includes the microscopic models for PbTi${\text{O}}_3$ and other technical details.

1. The AIGLE Model

The starting point is a microscopic model of molecular dynamics. CVs, obtained by coarse-graining the microscopic degrees of freedom when constructing the FES, form a column vector $\mathit{x}$. The aim is to eliminate the remaining degrees of freedom, and obtain an accurate dynamic model for the CVs, using the GLE ansatz:

$$\begin{array}{c}\hfill \mathrm{M}\frac{d^2\mathit{x}(t)}{d{t}^2}=-\mathrm{\nabla }G(\mathit{x})+\mathit{F}(t)+{\int }_0^{t}dsMK(s)\frac{d\mathit{x}(t-s)}{\mathit{dt}}+\mathit{R}(t).\end{array}$$ [1]

Here, $M$ is the effective mass matrix, $G(\mathit{x})$ is the FES, the vector $\mathit{F}$ comprises the external driving forces, $K$ is the memory kernel matrix, and the vector $\mathit{R}$ represents the noise. We define $\mathit{v}=\frac{d\mathit{x}}{\mathit{dt}}$, $\mathit{a}=\frac{d^2\mathit{x}}{d{t}^2}$, $\mathcal{F}=-\mathrm{\nabla }G(\mathit{x})+\mathit{F}$, and use the subscript $T$ for the transpose of a vector or a matrix. We shall use the brackets $⟨\cdots ⟩$ to indicate an average over the equilibrium ensemble at $t=0$. We require $⟨\mathit{R}(t)⟩=0$, and the orthogonality condition $⟨\mathit{R}(t){\mathit{v}}^T(0)⟩=0$, from which the 2FDT can be derived (27). The 2FDT prescribes that, at equilibrium, $⟨\mathit{v}(0){\mathit{v}}^T(0)⟩K^T(s)=-⟨({\mathit{M}}^{\mathbf{-}\mathbf{1}}\mathit{R})(0){({\mathit{M}}^{\mathbf{-}\mathbf{1}}\mathit{R})}^T(s)⟩$, relating the memory kernel to the ACF of the noise. In addition, although the noise should not be strictly stationary, $⟨\mathit{R}(t_0+t){\mathit{R}}^T(t_0)⟩$ should be independent of $t_0$ for sufficiently large $t_0$.

In AIGLE, Eq. 1 is learned from the trajectories of $\mathit{x}$. The scheme can use, but does not require, a predetermined FES (61–69), as all the terms in Eq. 1 can be learned from adequate trajectory data. The resulting GLE satisfies numerically the 2FDT for the equilibrium ensemble, and the model can be extended to near-equilibrium dynamics. Extensions to general nonlinear dynamics beyond the 2FDT would be possible, but this paper is limited to near-equilibrium situations. We show that the scheme can be constructed from large-scale MD simulations of realistic materials models. First, we formulate AIGLE for a one-dimensional CV and then we generalize it to infinite-dimensional lattice CVs.

To learn from time series data generated by MD, it is convenient to transform the integro-differential equation [1] into discrete form. We assume that the memory kernel is homogeneous, i.e., independent of position (70) and time origin (71), and use $\mathrm{\Delta }t$ for the time step of the discretized GLE. Setting $t=0$ for the arbitrary starting time, the current time is $t=n\mathrm{\Delta }t$, and we use the notation $f_{(n)}$ to indicate a time-dependent function $f(n\mathrm{\Delta }t)$. Then, the discretized form of Eq. 1 for a one-dimensional CV reads

$$\begin{array}{c}\hfill {ma}_{(n)}=-\mathrm{\nabla }G(x_{(n)})+F_{(n)}+{\displaystyle {\displaystyle \sum _{s=0}^{n-1}}}m{K}_{(s+\frac{1}{2})}{v}_{(n-s-\frac{1}{2})}\mathrm{\Delta }t+R_{(n)}.\end{array}$$ [2]

Eq. 2 is propagated with the leapfrog algorithm:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill v_{(n+\frac{1}{2})}& =v_{(n-\frac{1}{2})}+a_{(n)}\mathrm{\Delta }t,\hfill \\ \hfill x_{(n+1)}& =x_{(n)}+v_{(n+\frac{1}{2})}\mathrm{\Delta }t.\hfill \end{array}\end{array}$$ [3]

This setup allows synchronization with MD data when $\mathrm{\Delta }t$ equals an integer multiple of $\delta t$, the integration time step of MD. More accurate multistep schemes for integrating stochastic dynamics exist (72, 73), but here, we adopt the simple leapfrog scheme because the autoregressive noise model of AIGLE benefits from a simple discretization scheme. Moreover, in consideration of the errors in the free energy calculations, the errors in autoregression, and the lack of a conservation law for stochastic dynamics, the numerical integration error is a minor issue as long as $\mathrm{\Delta }t$ is sufficiently small relative to the shortest vibrational period of the CVs.

The free energy $G$ in Eq. 2 can be parameterized empirically, e.g., using a polynomial ansatz, or, more generally, it can be represented by a neural network ansatz when dealing with high-dimensional CVs. The noise term $\{R_{(n)}\}$ in Eq. 2 is modeled by a GAR model

$$\begin{array}{c}\hfill R_{(n)}={\displaystyle {\displaystyle \sum _{k=1}^{m_A}}}{\varphi }_{(k)}{R}_{(n-k)}+{\mu }_{(n)}+{\sigma }_{(n)}{w}_{(n)},\end{array}$$ [4]

where $\{{\varphi }_{(k)}\}$ are Yule–Walker linear autoregressive parameters (74). ${\mu }_{(n)}$ and ${\sigma }_{(n)}$ are nonlinear functions that depend on the history of the noise. In our approach, ${\mu }_{(n)}$ and ${\sigma }_{(n)}$ are the outputs of a deep neural network whose arguments are $R_{(n-1)},\cdots ,R_{(n-m_A)}$. The residual noise $w_{(n)}$ represents the uncorrelated part of the noise on the scale of $\mathrm{\Delta }t$, and should be close to Gaussian white noise for the scheme to be successful. The GAR model becomes a standard AR($m_A$) model (75) for constant ${\mu }_{(n)}$ and ${\sigma }_{(n)}$. When the time dependence of ${\mu }_{(n)}$ and ${\sigma }_{(n)}$ cannot be ignored, GAR outperforms AR in reducing $w_{(n)}$ to an almost ideal white noise upon training with MD data, a crucial property for the time correlation functions of CG dynamics to agree with the data.

Eqs. 2–4 constitute the AIGLE model. The corresponding AILE model is $m{a}_{(n)}=-\mathrm{\nabla }G(x_{(n)})+F_{(n)}+m\vartheta v_{(n-\frac{1}{2})}+w_{(n)}$, where the friction satisfies $\vartheta =\sum _{s=0}^{n-1}{K}_{(s+\frac{1}{2})}\mathrm{\Delta }t$, and the white noise $w_{(n)}$ is fixed by the Markovian 2FDT. In AIGLE, the parameters defining $G$ and $F$, the memory kernel, the Yule–Walker model, and the neural networks in the GAR model, are learned from MD trajectories kept near thermal equilibrium by a stochastic environment that mimics a heat bath. This is a common situation in realistic finite-temperature systems. Memory in these open systems should extend over a finite time interval, specified by the integer $m_K$, i.e., $K_{(s+\frac{1}{2})}=0$ for $s>m_K-1$. $m_K$ and $m_A$ are a priori parameters of the same order of the relaxation time of the ACF of the velocities of the CVs. At the beginning of the learning protocol, it is convenient to set $m_A=m_K$ to be several times larger than the relaxation time. Upon fine tuning, the final values of $m_A$ and $m_K$ get close to the relaxation time of the velocity ACF, and we find empirically that a good choice corresponds to $m_A<m_K$. The mass $m$ is fixed by the equipartition theorem.

The recommended learning procedure involves the three actions outlined below. More details are in the Methods section. In the first action, the models for $G$ and the memory kernel $\{K_{(s+\frac{1}{2})}|s\in [0,m_K-1]\}$ in Eq. 2 are trained on equilibrated MD data. Static and conservative forces $\{F_{(n)}\}$ are absorbed into $G$. Two steps are iterated to self-consistency. In the first step, $G$ is optimized while keeping the memory kernel fixed. The loss function is the mean squared deviation from MD of the model prediction for the force on the CV without including noise effects. This procedure is equivalent to minimizing the noise. In the second step, the memory kernel is optimized, while keeping $G$ fixed, by imposing orthogonality of velocity and noise in the least squared sense. Self-consistency typically requires a few thousand iterations. At the completion of the first action, the noise $\{R_{(n)}\}$ is defined by subtracting the gradient force, $-\mathrm{\nabla }G$, and the memory-dependent friction force, predicted by the model, from the true force acting on the CV in the MD data. Then, we turn to the second action, in which the GAR model is optimized using a maximum likelihood loss function, in which the noise values $\{R_{(n)}\}$ constitute the time-series data. This procedure ends when the residual noise is almost white noise, and the GAR model is numerically stationary. At this point, the GLE is fully determined for equilibrium systems and can be used to model mesoscale dynamics under equilibrium conditions. However, when an external driving force $F$ is present, extra training may be necessary. This is done in a third action, which is only executed if needed. In this procedure, the parameters that define $G$ and $F$ are refined with the loss function used in the first action, while keeping the memory kernel and the GAR model fixed. We tested the above procedure and the 1D AIGLE model on a toy system, the infinite harmonic chain, which can be solved analytically within the Mori–Zwanzig formalism. The results of this validation test, reported in SI Appendix, show that AIGLE reproduces with high accuracy the MD data, the analytical Mori–Zwanzig solution, and the 2FDT.

Finally, we generalize AIGLE for general lattice problems. First, we reinterpret Eq. 1 for on-lattice CVs. We let ${\mathit{x}}_i={(x_{i1},\cdots ,x_{i{d}_s})}^T$ represent a $d_s$-dimensional local order parameter associated to site-$i$ of a $d_l$-dimensional Bravais lattice with periodic boundary conditions. $L$ is the number of sites in the simulation supercell. By concatenating $\{{\mathit{x}}_i\}$ we define a $d_{s}L$-dimensional CV $\mathit{x}={({\mathit{x}}_1^T,\cdots ,{\mathit{x}}_L^T)}^T$. We let $A=\{A_{\mathit{ij}}\}$ be the $L\times L$ adjacency matrix of the lattice. We use $i\sim j$ to indicate a neighboring pair ($A_{\mathit{ij}}$=1). For large $L$, it is not practical to model a dense $d_{s}L\times d_{s}L$ memory kernel matrix $K(s)=\{K_{i\alpha }^{j\beta }(s)\}$ ($\alpha ,\beta \in [1,d_s]$) and a long-range correlated $\mathit{R}(t)$ that preserve exactly the velocity correlation matrix or the 2FDT intrinsic to the data. The simplest approximation is to assume locality and set $K_{i\alpha }^{j\beta }(s)$ to be equal to zero whenever the indices $i$ and $j$ do not correspond to the same site, i.e., when $i\ne j$. Even with this drastic approximation, the one-body memory kernel will depend not only on the autocorrelation but also on the cross-correlations of the CVs on different sites. Limiting consideration to close-neighbor correlations, we adopt a variational principle for the optimal memory kernel. We define the orthogonality tensor ${\mathrm{\Omega }}_{i\alpha }^{j\beta }(t)=⟨R_{i\alpha }(t)v_{j\beta }(0)⟩$, and define the corresponding orthogonality loss, a functional of $K$, by

$$\begin{array}{c}\hfill \mathcal{L}[K](t_K)={\int }_0^{t_K}{\displaystyle \sum _{i,j,\alpha ,\beta }}{|{\mathrm{\Omega }}_{i\alpha }^{j\beta }(t)|}^2({\delta }_{\mathit{ij}}+A_{\mathit{ij}})dt.\end{array}$$ [5]

For a given cutoff of the memory time $t_K=m_K\mathrm{\Delta }t$, the optimal one-body memory kernel $K^{\ast }$ minimizes Eq. 5, i.e., $K^{\ast }={\mathrm{argmin}}_K\mathcal{L}[K](t_K)$. We call $K^{\ast }$ a “local kernel approximation” of the exact many-body memory kernel. It enforces a weak form of the 2FDT. The optimality condition for the memory kernel in the case of 1D AIGLE can be regarded as a special case of Eq. 5. We show, in SI Appendix, that the local kernel approximation can be viewed as a special case of a general variational principle for the orthogonality condition. In practice, we still adopt the discretized form of Eq. 1 with the leapfrog scheme. We use a multidimensional version of Eqs. 2 and 3, but keep the notation $f_{(n)}$ for a time-dependent function $f(n\mathrm{\Delta }t)$. We require $K_{(s+\frac{1}{2})}=0$ for $s\ge m_K$. Given the force field $F$, after discretizing Eq. 5, we obtain a least-square solution for the optimal memory kernel $K_{(s+\frac{1}{2})}^{\ast }$ for $0\le s<m_K$. The derivation, given in SI Appendix, is lengthy but straightforward. We also generalize the 1D GAR model to the multidimensional case. Note that, although $K^{\ast }$ is one-body, the noise $R$ is still spatially correlated as required by the 2FDT. Thus, the noise generator can not be defined locally as commonly done with molecular dynamics thermostats. To deal with this complication, we allow the noise ${(R_{(n)})}_i$ at site-$i$ and time step $n$ to depend not only on its own history but also on the history of a finite number of neighboring sites-$j$ defined in terms of the adjacency matrix $A$. The sites-$j$ could include nearest neighbor sites ($A_{\mathit{ij}}=1$), next nearest neighbor sites, and so on. In other words, the multidimensional GAR model is analogous to a graph neural network (76) on the graph of the CVs defined by $A$. The details of the multidimensional GAR model are given in SI Appendix. The training of multidimensional AIGLE follows the same protocol of univariant AIGLE.

2. Applications

A. Domain Wall as a Virtual Particle.

In this section, AIGLE is used to study a prototypical problem of ferroelectric domain switching—the field-driven motion of a twin 180° domain wall in epitaxially grown PbTi${\text{O}}_3$. Schematic drawings of the domain wall and of the crystal structure are shown in Fig. 1. Electric dipole moments (local dipoles) ${\mathit{p}}_{\mathit{j}}$, represented by yellow arrows in Fig. 1, are associated with the Ti-centered elementary cells-$j$ (77). The polarization is defined by $P=\sum _j{\mathit{p}}_{\mathit{j}}/V$, with $V$ the volume of the sample. Polarization changes are experimentally observable. AIGLE is constructed from ab initio electronic structure data within DFT. Microscopic definitions of ${\mathit{p}}_{\mathit{j}}$ and $P$ are given in SI Appendix.

Fig. 1.

Upper panel: Lateral view of the 180° domain wall in PbTi${\text{O}}_3$. The bonds between titanium atoms and the nearest oxygen atoms are shown to visualize the domain separation. The yellow arrows represent the local dipoles, which are weaker near the domain wall. Lower panel (Left): A $20\times 40\times 20$ supercell of PbTi${\text{O}}_3$ with parallel twin domain walls on the $\mathit{xz}$ plane. The $x$ and $y$ dimensions are fixed to match the experimental lattice constant (see text) while the $z$ dimension fluctuates under constant pressure $P_z=28$kbar (see text) at temperature $T=300$ K. Lower panel (Right): The elementary cell of PbTi${\text{O}}_3$.

In epitaxially strained tetragonal PbTi${\text{O}}_3$, polarized along the $[001]$ crystallographic direction at room temperature ($T=300$ K), the ferroelectric domains have narrow 180° domain walls (77). A CV that describes continuously the switch of a domain from $+\widehat{z}$ to $-\widehat{z}$ is $\alpha =\sum _{i}tanh\frac{p_{\mathit{iz}}}{p_{\ast }}/2A_{\mathrm{wall}}$, where the sum extends to the local dipoles. We choose a value of $p_{\ast }$ that is close to the bulk average of $\Vert p_{\mathit{iz}}\Vert$. In the simulations, we set the parameter $A_{\mathrm{wall}}$ to be equal to $400$, the supercell area in the $\mathit{xz}$ plane in units of elementary cells. With this definition, when the $+\widehat{z}$ domain grows by one layer of unit cells in the $y$-direction, the increment of $\alpha$ is approximately equal to $1$.

We model the motion of the domain wall driven by an external electric field $\mathit{E}=E\widehat{z}$. Experimentally, the domain wall velocity, $v_D$, obeys approximately a phenomenological law suggested by Merz (58), according to which $v_D=v_0{e}^{-E_a/E}$, with $v_0$ and $E_a$ empirical parameters. For small $E$, $v_D\to 0$, and the wall dynamics is glassy. This motion, called domain wall creep, is usually initiated by the nucleation and growth of flat nuclei at the separating interface (78). On a coarser time scale, the moving interface can be viewed as a virtual particle that performs a succession of noise-activated, rare hopping events, rather than a steady continuous motion. This behavior cannot be deduced from phenomenological laws, like Merz’s, and is usually ignored in continuum models, but can be probed, in principle, with microscopic simulations (79). However, glassy dynamics can easily exceed all-atom simulation capabilities when the time scale is of the order of the microsecond or longer. To cope with the long-timescale bottleneck, one often turns to kinetic Monte Carlo (78), an approach that typically requires ad hoc iteration rules and assumes Markovianity. AIGLE can simulate non-Markovian dynamics with ab initio accuracy for time scales comparable to those reachable by kinetic Monte Carlo.

We generate training data for AIGLE with MD simulations of PbTi${\text{O}}_3$. We adopt the Deep Potential (DP) model for the interatomic interactions, and an effective Born charge (BC) model for the local dipoles (SI Appendix). The MD supercell is shown in Fig. 1, where the $x$ and $y$ dimensions are fixed to match the experimental lattice constant $a=3.91$ Å, and the $z$ dimension is barostatted at a constant pressure of $P_z=28$ kbar, a value chosen to roughly match the experimental lattice constant $c$ at 300 K and atmospheric pressure (SI Appendix and refs. 80 and 55 for more details). With the above setup, we run MD trajectories at $T=300$ K, with temperature controlled by a stochastic thermostat, in the presence of homogeneous electric fields of varying magnitude $E$, with $0\le E\le 3\mathrm{mV}/$Å, along the $\widehat{z}$ direction. The microscopic data for the CV $\alpha$ are extracted from these trajectories. In these simulations, the atomistic degrees of freedom equilibrate quickly with the environment, and the loss of detailed balance is mostly associated with the CV describing domain motion.

The atomistic simulations suggest that the dynamics of $\alpha$ resemble that of a virtual particle subject to colored noise in a tilted periodic potential (81). When $E$ is small, the particle is trapped in a metastable equilibrium and the velocity ACF of $\alpha$, defined by ${\mathcal{A}}_{\dot{\alpha }\dot{\alpha }}(\tau )=⟨\dot{\alpha }(\tau )\dot{\alpha }(0)⟩$, exhibits several characteristic oscillations (modes), i.e., a behavior dramatically different from the simple exponential decay characteristic of Brownian dynamics driven by white noise. These characteristic modes originate mainly from the optical phonons of PbTi${\text{O}}_3$ and provide the thermal fluctuations that activate nucleation-driven creep events at small driving fields. Taking time scale separation into account, it is convenient, when constructing a CG model, to filter out the frequencies much higher than that of the slowest mode of $\alpha$ ($\approx$30 $\mathrm{c}{m}^{-1}$). Then, a new CV, $x$, is constructed by acting on $\alpha$ with a truncated Gaussian filter in time:

$$\begin{array}{c}\hfill x_{(n)}={\displaystyle {\displaystyle \sum _{q=0}^{3\varsigma }}}\frac{exp(-\frac{q^2}{2{\varsigma }^2})}{{\sum }_{q=0}^{3\varsigma }exp(-\frac{q^2}{2{\varsigma }^2})}\alpha (t=n\mathrm{\Delta }t-q\delta t).\end{array}$$ [6]

Here, $\varsigma$ is the truncation parameter that we set equal to $40$. With this choice, the modes of $\alpha$ in the range $[0,100]\mathrm{c}{m}^{-1}$ are barely affected, while the modes with higher frequency are suppressed (SI Appendix). The AIGLE integration time step $\mathrm{\Delta }t=10$ fs is equal to five times the MD time step $\delta t=2$ fs. This procedure is substantiated by the fact that the residual noise, after training the model, is indeed very close to white noise. The GLE of motion for $x$, deriving from Eq. 2, is

$$\begin{array}{c}\hfill a_{(n)}=-{\partial }_{x}U(x_{(n)})+pE+{\displaystyle {\displaystyle \sum _{s=0}^{n-1}}}{K}_{(s+\frac{1}{2})}{v}_{(n-s-\frac{1}{2})}\mathrm{\Delta }t+\frac{1}{m}{R}_{(n)}.\end{array}$$ [7]

Here, we parameterized the FES of Eq. 2 with the periodic function $G(x)=mU(x)=m{U}_{b}tanh(k(1-cos\omega (x-x_0)))$. $m\approx 1.0\times {10}^3$ amu is a scalar mass, estimated with the equipartition theorem. $U_b$, the barrier height, is predetermined with metadynamics (62) since it is hard to fit it accurately in the near-equilibrium regime without enhanced sampling. The external force in Eq. 2 is represented by $F(x)=mpE$. The values of the parameters $k$, $\omega$, $x_0$, and $p$, are fixed by training. The time cutoffs for the memory kernel $K$ and for the GAR model for $R$ are $m_K\mathrm{\Delta }t=2$ ps and $m_A\mathrm{\Delta }t=0.4$ ps, respectively. Assuming a linear response regime, the model parameters are independent of $E$. The AIGLE model introduced here is trained on several MD trajectories with $E\in [2.0,2.4]$ mV/Å. Details of training and validation can be found in SI Appendix. MD systems are at metastable equilibrium for $E\approx 2$mV/Å and near equilibrium for smaller $E$.

Comparison of $C_{\mathit{vv}}^{\text{MD}}(\tau )$, the normalized autocorrelation function (NACF) of the CV velocity extracted from MD, with its AIGLE counterpart, $C_{\mathit{vv}}^{\text{GLE}}(\tau )$, provides a direct validation of AIGLE. The two NACFs, calculated at metastable equilibrium conditions for $E=2$ mV/Å, are reported in Fig. 2A. They agree well with each other but for minor discrepancies. The figure also indicates that the adopted cutoff $m_K$ is large enough to satisfy the condition $C_{\mathit{vv}}^{\text{MD}}(\tau >m_K\mathrm{\Delta }t)\ll 1$. A major cause of the small differences between MD and AIGLE is apparent in the inset of Fig. 2A, which reports the real parts of Fourier transforms of the velocity NACFs, $\mathfrak{R}({\widehat{C}}_{\mathit{vv}}^{\text{MD}}(\mathrm{\Omega }))$ and $\mathfrak{R}({\widehat{C}}_{\mathit{vv}}^{\text{GLE}}(\mathrm{\Omega }))$. The slowest mode occurs at $\mathrm{\Omega }\approx$ 30 $\mathrm{c}{m}^{-1}$ in both NACFs. In $\mathfrak{R}({\widehat{C}}_{\mathit{vv}}^{\text{MD}}(\mathrm{\Omega }))$ this mode exhibits a fast oscillatory line shape, indicating relaxational origin. The same mode in $\mathfrak{R}({\widehat{C}}_{\mathit{vv}}^{\text{GLE}}(\mathrm{\Omega }))$ has a smooth Lorentzian line shape with a peak frequency that matches the harmonic frequency of the nearly quadratic free energy basin depicted in Fig. 2B. This indicates that the relaxational fluctuation of the domain wall is turned into an effective harmonic oscillation in a potential well. Since the ansatz for $U(x)$ assumes a smooth, rather than fractal, dependence on $x$, the slowest mode of $\mathfrak{R}({\widehat{C}}_{\mathit{vv}}^{\text{GLE}}(\mathrm{\Omega }))$ displays a clean harmonic peak, sharper than the relaxational peak of $\mathfrak{R}({\widehat{C}}_{\mathit{vv}}^{\text{MD}}(\mathrm{\Omega }))$. This subtle difference is, in fact, a desired consequence of coarse-graining the FES. Two other modes (near 80 $\mathrm{c}{m}^{-1}$) are displayed by $\mathfrak{R}({\widehat{C}}_{\mathit{vv}}^{\text{GLE}}(\mathrm{\Omega }))$ and by $\mathfrak{R}({\widehat{C}}_{\mathit{vv}}^{\text{MD}}(\mathrm{\Omega }))$ as well. At higher frequencies, the spectrum of $\mathfrak{R}({\widehat{C}}_{\mathit{vv}}^{\text{GLE}}(\mathrm{\Omega }))$ is quite smooth and agrees well with the MD results.

Fig. 2.

(A) The velocity NACF $C_{\mathit{vv}}^{\text{MD}}(\tau )$ from MD (blue) and the velocity NACF $C_{\mathit{vv}}^{\text{GLE}}(\tau )$ from AIGLE (orange), for $\tau <2$ps. For $\tau >2$ps the correlations are confined to the interval [$-$0.1,0.1] and decay rapidly to zero. The Inset reports $\mathfrak{R}({\widehat{C}}_{\mathit{vv}}^{\text{MD}}(\mathrm{\Omega }))$ (blue) and $\mathfrak{R}({\widehat{C}}_{\mathit{vv}}^{\text{GLE}}(\mathrm{\Omega }))$ (orange) in arbitrary units. The first peak of $\mathfrak{R}({\widehat{C}}_{\mathit{vv}}^{\text{GLE}}(\mathrm{\Omega }))$ is located near 30 $\mathrm{c}{m}^{-1}$. (B) Free energy profiles along the CV $x$ in the presence of driving fields of various strengths. Two periods of the parametrized $U(x)$ (see text) are shown. $U(x)$ is represented by a dashed blue line. Metadynamics results are also reported as blue crosses, for comparison. The Inset shows a magnified plot of the free energy basin including $U(x)$ and metadynamics data. (C) The natural logarithm of the domain wall velocity $v_D$ is plotted against $1/E$. The vertical dashed line indicates $E=E^{\ast }$. Each GLE data point (orange dot) is the average of five AIGLE simulations lasting 0.5 $\mathrm{\mu }$s each. The corresponding error bars are smaller than the size of the dot. Each MD data point (blue diamond) is the average of 100 MD trajectories lasting $0.1$ ns each. The error bars are smaller than the size of the diamond. The dashed blue line is the best fit of Merz’s law with MD data ($E_a=28\mathrm{mV}/$Å). (D) A comparison of AIGLE and LE predictions for $ln{v}_D$ vs. $1/E$. Each LE data point (green triangle) is the average of five LE trajectories lasting 0.5 $\mathrm{\mu }$s each. The error bars are smaller than the size of the triangle.

The optimized free energy profile as a function of $x$ is shown in Fig. 2B. Metastability disappears for $E$ greater than $E^{\ast }\approx 2.8$ mV/Å when the profile becomes monotonic. Hence, $E^{\ast }$ represents the threshold beyond which the near-equilibrium regime appropriate for AIGLE is no longer valid. Under near-equilibrium conditions, the lifetime of a metastable state should be much longer than the relaxation times of the atomic vibrations. To study this phenomenology, we run AIGLE for a dense grid of electric field values in the interval $(1,5)$ mV/Å. To visualize the variation of the domain velocity $v_D$, which spans several orders of magnitude, we display in Fig. 2C the natural logarithm of $v_D$, extracted from AIGLE and MD simulations, respectively, as a function of $1/E$. MD data are only available for $v_D⪆1$m/s due to time limits of fully atomistic simulations. When AIGLE and MD data are both available, the two approaches agree well for $E\le E^{\ast }$, i.e., under near-equilibrium conditions. For $E>E^{\ast }$, the domain velocity of MD is significantly larger than its AIGLE counterpart. From a coarse-graining point of view, this occurs because the 2FDT, valid near equilibrium, has been imposed far from equilibrium. From a microscopic point of view, the electric dipoles, temporarily associated with the moving domain wall, are unable to dissipate energy before separating from the wall. Far from equilibrium memory effects are different from those learned for $E<E^{\ast }$. Thus, the present AIGLE model should only be used when $E\le E^{\ast }$, i.e., within the creep regime of the Markovian theory of elastic interface dynamics (59, 60). When $v_D⪆1$m/s, MD shows linear behavior of $ln{v}_D$ with $1/E$, in agreement with Merz’s law: $ln{v}_D=ln{v}_0-E_a/E$ (58). A best fit of the MD data to this law gives $E_a=28\mathrm{mV}/$Å. AIGLE gives essentially the same result, $E_a=27\mathrm{mV}/$Å, for $E\in [2\mathrm{mV}/$Å, $E^{\ast }]$. Thus, $v_D$ at low electric fields can be estimated from Merz’s law fitted to MD for $v_D⪆1$ m/s, as done, e.g., in refs. 78 and 79. However, when $v_D\ll 1$m/s, direct AIGLE simulations display a gradual deviation from Merz’s law, as illustrated in Fig. 2C. When $1/E>0.6$ Å/mV, AIGLE predicts a $v_D$ higher than Merz’s law by orders of magnitude. This behavior is similar to the stretched exponential inferred from the relation $ln(v_D/v_0)\propto {(E/E_c)}^{-\mu }$ of the Markovian theory of elastic interfaces (59) when the dynamic exponent $\mu$ is less than $1$. This theory assumes a Markovian overdamped regime. Yet, the deviation from Merz’s law, predicted by AIGLE at low fields, is markedly more rapid than the stretched exponential of the Markovian theory. This suggests that memory and inertia play an increasingly important role in the regime of very rare domain motions.

To gauge the implication of non-Markovianity, we approximate AIGLE with AILE. Fig. 2D, shows that LE predicts for $v_D$ a behavior consistent with Merz’s law, which is not surprising because the derivation of Merz’s law requires a Markovian approximation. The same figure shows that LE and AIGLE agree well with each other when $1/E$ is close to $1/E^{\ast }$, a situation in which the external driving force dominates over memory and noise. For larger $1/E$, the LE predicted behavior deviates from a pure exponential in a very minor way, underestimating $v_D$ by orders of magnitude relative to AIGLE at the largest values of $1/E$. Within Markovian dynamics, the friction is always dissipative, hindering thermally activated motion irrespective of the time scale of the creep events. An even simpler dynamics is postulated in the Markovian theory of elastic interfaces (59, 60) that adopts an overdamped Langevin equation, where both memory and inertia effects are absent. By contrast, within AIGLE, memory results from a convolution of oscillating functions and can occasionally lead to a kinetic energy increase over a short time interval. In combination with inertia, this effect enhances the likelihood of barrier crossing. From the perspective of transition state theory, this effect can be understood as effectively enhancing the pre-exponential factor in the formula for the rate. Non-Markovian effects that facilitate barrier-crossing have also been discussed in other contexts, such as, e.g., in the Grote-Hynes theory of chemical reaction rates (82).

Using the domain velocity $v_D(E)$ calculated with AIGLE, we can estimate the hysteresis loop observed experimentally when the polarization is reversed by a driving field. We report in SI Appendix a hysteresis loop calculation using a very simple model of ferroelectric switching that ignores point defects and dependence on the curvature of the domain wall. The results are in semiquantitative agreement with experiments.

B. CG Lattice Dynamics.

Here, we use multidimensional AIGLE to describe the dynamics of lattice CVs, which are either the local dipole moments $\{{\mathit{p}}_{\mathit{j}}\}$ or a CG model of them. The underlying microscopic model is the all-atom DP model of Section A. For each atomic configuration, the local dipoles are provided by a neural network model (SI Appendix).

We run NVT-MD to generate the training data. The lattice parameters are fixed to $a=b=3.93$ Å and $c\approx 4.04$ Å. For $E=0$, we run equilibrium NVT-MD in a $8\text{nm}\times 8\text{nm}\times 5\text{nm}$ supercell comprising a single ferroelectric domain. The system is illustrated in Fig. 3B, where yellow arrows represent the local dipole moments. For $E=0.5$ mV/A and $E=1$ mV/A along $+z$, we run near equilibrium NVT-MD in a $20\text{nm}\times 20\text{nm}\times 5\text{nm}$ supercell that initially contains two opposite ferroelectric domains, i.e., a nearly cylindrical up($+z$)-polarized domain having a radius of about $6\text{nm}$, embedded in an environment of opposite polarization, as illustrated in Fig. 3A. To reduce the energy cost of the cylindrical interface the up-polarized domain shrinks during the simulation, in spite of the applied field favoring up-polarization, with a longer relaxation time when $E$ is larger. In the MD simulations, we calculate the trajectories of all the local dipoles $\{{\mathit{p}}_{\mathit{j}}(t)\}$. Taking the local dipoles in the tetragonal PbTi${\text{O}}_3$ lattice as CVs (Fig. 3C, Middle), the degrees of freedom are one-fifth of the atom coordinates (Fig. 3C, Left). Further coarse-graining is motivated by the following considerations. The time correlations of the Cartesian components of the local dipole velocities, i.e., $⟨{\dot{p}}_{j\alpha }(t_0+\tau ){\dot{p}}_{j\beta }(t_0)⟩$ for $\alpha ,\beta \in \{x,y,z\}$, indicate that the correlations for $\alpha \ne \beta$ are negligible compared to those for $\alpha =\beta$. Thus, we can reduce by one-third the CVs by retaining only the $z$-components, $\{p_{\mathit{jz}}(t)\}$, of the local dipoles, which are related to spontaneous polarization. Nearest neighbor dipoles are strongly correlated, because the oxygen atoms, whose displacements contribute to the polarization the most, are shared between adjacent cells. As a consequence, further coarse-graining is possible by blocking into a single dipole pairs of nearest-neighbor dipoles of the original simple tetragonal lattice $S$. The blocking operation defines two interpenetrating body-centered tetragonal (BCT) lattices $S_1$ and $S_2$ obtained from $S$ by bipartition. We assume that our choice of CG dipoles corresponds to $S_1$, as illustrated in the right panel of Fig. 3C. If $\mathit{a}=a\widehat{\mathit{x}}$, $\mathit{b}=b\widehat{\mathit{y}}$, $\mathit{c}=c\widehat{\mathit{z}}$ are the (conventional) unit cell vectors of $S$, the (conventional) unit cell vectors of $S_1$ are $\mathit{u}=\mathit{a}+\mathit{b}$, $\mathit{v}=-\mathit{a}+\mathit{b}$, and $\mathit{w}=2\mathit{c}$. Let $L$ and $A$ be the size and the adjacency matrix, respectively, of $S_1$. Each site-$i$ of $S_1$ has 12 neighboring sites (in the sense of graph adjacency on $S_1$), displaced by ($\pm \mathit{a}\pm \mathit{b}$), $(\pm \mathit{a}\pm \mathit{c})$, and $(\pm \mathit{b}\pm \mathit{c})$, respectively. The corresponding CVs are denoted by $\tilde{\mathit{p}}={(p_{\mathit{iz}})}_{i\in [1,L]}$. By construction, the degrees of freedom in $\tilde{\mathit{p}}$ are one-thirtieth of the atomic coordinates, but the polarization along $\widehat{\mathit{z}}$ is left unaffected. Then, we apply a truncated Gaussian filter in time to $\tilde{\mathit{p}}(t)$ to remove high-frequency contributions. The resulting CVs are called $\{{\mathit{x}}_{(n)}\}$:

$$\begin{array}{c}\hfill {\mathit{x}}_{(n)}={\displaystyle {\displaystyle \sum _{l=0}^{3\varsigma }}}\frac{exp(-\frac{q^2}{2{\varsigma }^2})}{{\sum }_{l=0}^{3\varsigma }exp(-\frac{q^2}{2{\varsigma }^2})}\tilde{\mathit{p}}(t=n\mathrm{\Delta }t-q\delta t).\end{array}$$ [8]

Fig. 3.

(A) A (nonequilibrium) configuration of PbTi${\text{O}}_3$ showing a cylindrical up-polarized domain in an environment of opposite polarization. Spheres are Ti atoms, Pb and O atoms are not shown. The arrows depict the local dipole moments assigned to Ti-centered elementary cells. (B) A configuration within the up-polarized domain. (C) Coarse-graining procedure. (D and E) Relaxation dynamics of the local dipoles in an atomic layer perpendicular to the direction $\widehat{\mathit{z}}$ of spontaneous polarization under electric fields $E=0.5$ mV/Å (D), and $E=1$ mV/Å (E). The three horizontal sequences of panels in (D) and (E) depict the evolution of the yellow domain from MD, AIGLE, and AILE. The pixels correspond to the $S$ sites. In the MD panels, they give the magnitude and sign of the microscopic dipoles $\{p_{\mathit{jz}}\}$, as per scale on the right. In the AIGLE and AILE panels the pixels associated with the $S_1$ sites give the CG dipoles $\{x_j\}$, while those associated with the $S_2$ sites are the average of the CG dipoles at the neighboring $S_1$ sites. The lifetimes of the cylindrical domain are shown on the two time axes. The solid vertical bars, at 40 ps for $E=0.5$ mV/A, and at 104 ps for $E=1$ mV/A, are extracted from two MD simulations. The gray rectangles are extracted from nine independent AIGLE and AILE simulations for each value of the electric field. (F) Shifted autocorrelation function (ACF) of the local dipole $x_i(t)$, in units of (eÅ)², from MD, AIGLE, and AILE. (G) Equilibrium probability distribution $P(|x_i|)$ of the dipole magnitude from MD and AIGLE. The AILE result coincides with AIGLE and is not reported. The gray dashed line shows the range of the average dipole magnitude $⟨|x_i(t)|⟩$ from different experiments (83–85). (H) ACF of the time derivative, $v_i(t)$, of $x_i(t)$, in units of (eÅ/ps)². (I) Cross-correlation function of $v_i(t)$ and $v_j(t)$ on adjacent sites in the $S_1$ lattice, in units of (eÅ/ps)². (J) ACF of $\dot{\mathcal{P}}(t)$, the time derivative of the polarization $\mathcal{P}(t)$, in units of ${(\mathrm{\mu }C/c{m}^2/ps)}^2$. (K) Gaussian convoluted Fourier transform $S(\mathrm{\Omega })$ of the ACF of $\dot{\mathcal{P}}(t)$ (see text for details). The gray dashed line indicates the peak frequency ${\omega }_{1{\mathrm{A}}_1}^{\mathrm{EXP}}$ of the Raman spectroscopy feature associated to the zone-center $1{\mathrm{A}}_1$ transverse optical phonon (86).

Here $\delta t=2$fs, $\mathrm{\Delta }t=5\delta t$ and $\varsigma =40$, as in Section A. The polarization of the system is $\mathcal{P}(t=n\mathrm{\Delta }t)=2{\displaystyle \sum _i}{x}_{i,(n)}/V$.

The AIGLE model for $\{{\mathit{x}}_{(n)}\}$ is

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill M{\mathit{a}}_{(n)}& =-{\mathrm{\nabla }}_{\mathit{x}}G({\mathit{x}}_{(n)})+pM{\mathit{E}}_{(n)}\hfill \\ \hfill & +{\displaystyle {\displaystyle \sum _{l=0}^{n-1}}}M{K}_{(l+\frac{1}{2})}{\mathit{v}}_{(n-l-\frac{1}{2})}\mathrm{\Delta }t+{\mathit{R}}_{(n)},\hfill \end{array}\end{array}$$ [9]

under local kernel approximation. We use for multidimensional AIGLE the same notation adopted in Eq. 7 for one-dimensional AIGLE. In principle, the external field ${\mathit{E}}_{(n)}$ can vary in space and time, but we consider here only fields that are time-independent and uniform in space. For the free energy $G(\mathit{x})$ we assume a simple polynomial form, $G(\mathit{x})={\displaystyle \sum _i}(b_1{x}_i^2+b_2{x}_i^4)+{\displaystyle \sum _{\mathit{ij}}}{A}_{\mathit{ij}}{b}_{\mathit{ij}}{(x_i-x_j)}^2$, suggested by effective Hamiltonian models (55). By symmetry, $b_{\mathit{ij}}=b_3$ if the sites $i$ and $j$ are separated by ($\pm \mathit{a}\pm \mathit{b}$), and we set $b_{\mathit{ij}}=b_4$, otherwise. The $b$ coefficients are assumed to be independent of $E$, as appropriate in the linear response regime. Hence, we limit simulations to $E\le 1$ mV/A. Our model for $G$ is short-ranged but captures well the dipole–dipole interactions of the DP model within the cutoff radius of the latter. Long-range electrostatic interactions among the dipoles have a negligible effect on the ferroelectric transition in PbTi${\text{O}}_3$ (see, e.g., ref. 80 and references therein).

Training proceeds through several steps. We first predetermine $G$ with equilibrium MD data ($E=0$) by force matching. Then, we calculate the memory kernel with the same data under local kernel approximation and train the multidimensional GAR model using $R_{(n)}$ as time series data. For the noise at site-$j$, the GAR model includes the noise history of site-$j$ and of its neighbors on $S_1$ displaced by ($\pm \mathit{a}\pm \mathit{c}$), ($\pm \mathit{b}\pm \mathit{c}$) or ($\pm 2\mathit{c}$). In the last step, we retrain $G$ and $p$ with nonequilibrium MD data $(E>0)$. The details are in SI Appendix. The corresponding AILE model is defined by the Markovian approximation of AIGLE as in the one-dimensional case.

The relaxation dynamics of the cylindrical domain in Fig. 3A, under weak applied field, is illustrated in Fig. 3D, for $E=0.5$ mV/A, and in Fig. 3E, for $E=1$ mV/A. The noise in AIGLE and AILE trajectories is at the origin of the observed fluctuations in the domain lifetime. Within the uncertainty of the noise, AIGLE and AILE lifetimes coincide, suggesting that non-Markovian effects should be negligible. Indeed, domain shrinking is caused primarily by surface tension, which acts to reduce the area of the interface between domains, a systematic effect originating from the gradient of the free energy. The MD lifetime is deterministic and is extracted from a single trajectory. It agrees with AIGLE/AILE within the uncertainty of the noise for $E=1$ mV/A (Fig. 3E), but is approximately $10$ps shorter than AIGLE/AILE for $E=0.5$ mV/A (Fig. 3D). This discrepancy is likely due to the inaccuracy of the simple polynomial model adopted for the FES. Non-Markovian effects should be more pronounced for larger cylindrical domains, where the surface tension is smaller. Simulation of much larger domains would be feasible with AIGLE and AILE but not with all-atom MD, hampering direct comparison for these settings. The special case of a planar domain wall dynamics under applied field was considered in Section A, where it was found that non-Markovian effects play a role for very weak fields.

Next, we consider a uniformly polarized bulk sample in the absence of an external field ($E=0$). Static and dynamic properties of the dipoles are reported in Fig. 3F–K. Memory and noise effects are more pronounced in the equilibrium dynamics of the bulk than in the relaxation dynamics of a cylindrical interface. Indeed, the MD ACF of an individual CG dipole $x_i$ is reproduced accurately by AIGLE but not by AILE (Fig. 3F). At the same time, nearly identical results are obtained with AIGLE and AILE for static properties like the probability distribution $P(|x_i|)$ of the local dipole, reported in Fig. 3G, as expected from the fact that AIGLE and AILE yield the same equilibrium Boltzmann distribution. On the scale of Fig. 3G, AIGLE and AILE are identical and only AIGLE is reported. The AIGLE distribution overlaps almost perfectly with the MD distribution barring a minor overall shift, much smaller than the range of the average dipole magnitudes extracted from experiments.

The remaining panels in the figure confirm the importance of non-Markovian effects. Fig. 3H shows that the ACF of $v_i(t)$, the time derivative of $x_i(t)$, is reproduced accurately by AIGLE but not by AILE. Also, the cross-correlation function between the time derivatives of neighboring dipoles shown in Fig. 3I is reproduced well, at least up to about $0.5$ ps, by AIGLE but not by AILE. These results suggest that the adopted local kernel approximation, which uses an optimized one-body memory kernel and many-body-correlated noise, can capture the short-range correlations among the dipoles that should dominate the fluctuation and dissipation of observables like the spontaneous polarization $\mathcal{P}$. Indeed, the ACF of $\dot{\mathcal{P}}(t)$, the time derivative of $\mathcal{P}(t)$, displayed in Fig. 3J, shows that AIGLE captures its dominant oscillatory frequency, while AILE misses it completely. However, at larger lagging times $\tau$ in the interval $[0.5,1.5]$ps AIGLE fails to reproduce the weak out-of-phase oscillations observed in MD. This behavior may originate from anharmonic couplings between vibrational modes that are not captured in the CG model. Neglect of long-range correlations in the noise could be another source of errors, as suggested by the observation that AIGLE would overestimate $⟨{\dot{\mathcal{P}}}^2(t)⟩$ by about $30\%$ if the GAR model did not include the history dependence of neighbors separated by $(\pm 2\mathit{c})$. Thus, including longer-range correlations may improve the accuracy of the model. This may be possible by adopting a more elaborate GAR model for the noise while retaining the simple local kernel approximation of AIGLE. It is also instructive to compute $S(\mathrm{\Omega })$, the spectrum of $⟨\dot{\mathcal{P}}(t+\tau )\dot{\mathcal{P}}(t)⟩$, which can be compared with experimental infrared spectroscopy. The spectra from MD, AIGLE, and AILE, given by $S(\mathrm{\Omega })=\mathfrak{R}{\int }_0^{\infty }d\tau exp(-i\mathrm{\Omega }\tau )⟨\dot{\mathcal{P}}(t+\tau )\dot{\mathcal{P}}(t)⟩$, are reported in Fig. 3K, upon Gaussian broadening with full width at half maximum of $12{\mathrm{cm}}^{-1}$. As expected from the real-time data, the AILE peak in Fig. 3K is significantly weaker than the other two, while AIGLE is stronger than MD, reflecting a sharper spectral feature. AIGLE reproduces well the peak frequency of MD, while AILE is red-shifted by approximately $40{\mathrm{cm}}^{-1}$. The spectral feature in Fig. 3K is associated with the zone-center $1{\mathrm{A}}_1$ transverse optical phonon, which is both infrared and Raman active. The corresponding feature from Raman scattering experiments lies at ${\omega }_{1{\mathrm{A}}_1}^{\mathrm{EXP}}=148.5{\mathrm{cm}}^{-1}$ (86), with a full width at half maximum (FWHM) of approximately $30{\mathrm{cm}}^{-1}$, while the MD FWHM is $43{\mathrm{cm}}^{-1}$ and that of AIGLE is $23{\mathrm{cm}}^{-1}$. The red shift of the MD/AIGLE peak at $120{\mathrm{cm}}^{-1}$, relative to the experiment, is mainly due to the adopted DFT approximation.

The above results show that AIGLE with the local kernel approximation can capture to a large extent the dynamic behavior of the CVs predicted by MD for bulk PbTi${\text{O}}_3$. At the same time, AILE, while equivalent to AIGLE for static properties, can not capture dynamical correlations when memory is important.

Finally, a comment on computational efficiency is in order. When modeling the dynamics of a $20\mathrm{nm}\times 20\mathrm{nm}\times 5\mathrm{nm}$ supercell on one Nvidia-A100 GPU, MD runs at $0.5$ ns/d, AIGLE at $0.4\mathrm{\mu }$s/d, and AILE at $1.7\mathrm{\mu }$s/d. Thus, the speedup over MD is of three orders of magnitude for both AIGLE and AILE. Moreover, AIGLE and AILE use significantly less memory than MD, facilitating simulations of significantly larger supercells.

3. Discussion

We have introduced a practical scheme to construct CG GLE models from MD trajectories. Our approach does not rely on the formal projection of MD onto the space of the CVs. As a consequence, the GLE construct is not exact, but should rather be viewed as a physically motivated approximation. While the idea of parameterizing GLE models with data extracted from MD trajectories dates back to at least 50 y (28), we exploit here modern techniques, such as machine learning and deep neural network representations, to generate extensive training datasets with MD and to construct the correlated noise model in the GLE. This enables us to construct AIGLE models, consistent with the microscopic dynamics, for one-dimensional and multidimensional CVs. Multidimensional AIGLE is not a trivial extension of its one-dimensional counterpart, and requires a local variational approximation for the memory kernel and a nearsightedness approximation for the correlated noise. The latter could be formulated only for systems in which local CVs reside on sites with a fixed topology described by an adjacency matrix or a graph, such as crystals and individual polymeric molecules. How to extend the approach to more general disordered systems remains an open issue. Here, we considered mesoscale processes in PbTi${\text{O}}_3$, a ferroelectric crystal, to illustrate the scheme and test its validity.

When used to study one-dimensional interface dynamics, AIGLE can model rare events on glassy landscapes caused by nucleation and growth at the atomistic level, reproducing the interface evolution driven by a weak applied field at a much lower computational cost than MD. In contrast to MD, AIGLE can access very rare events, revealing that, in the “slow” creep regime, when the time scale of the events is much longer than that of the memory, the scaling law for the domain velocity may deviate significantly from that of the “fast” creep regime, due to non-Markovian effects.

When applied to the dynamics of extensive CVs, AIGLE can model the relaxation of an elastic interface of any shape, a special case of extended defects, while still keeping the bulk dynamics of the CVs consistent with MD. These features distinguish AIGLE from other multiscale models with more drastic levels of coarse-graining, such as, e.g., a Landau–Ginzburg field theory of the extensive CVs in the continuum limit. A field theory model can not provide atomistic level resolution of an interface, or correctly describe the vibrational spectrum of a global order parameter like the electric polarization at low but nonzero frequency. In ferroelectric materials polarization dynamics at low frequency is typically dominated by an optical phonon mode that cannot be reduced to white noise, and cannot be modeled by AILE. In this context, AIGLE captures many-body correlations between CV components that are topologically close when the distance is measured in a graph. This feature is the key difference between a truly multidimensional GLE and a set of one-dimensional GLEs with independent frictions and noises.

Our study provides also examples where non-Markovian effects are irrelevant. In the CG lattice dynamics of PbTi${\text{O}}_3$, AIGLE and AILE give similar results for the motion of an interface dominated by a systematic driving force like the surface tension. In that case, memory effects are negligible, but our study shows that they may become important for glassy dynamics. It would be interesting to investigate the effect of driving fields that vary in space and time. Terahertz control of materials is an area of growing importance due to novel experimental developments (87). For controlling fields within the frequency range of atomic/molecular vibrations, non-Markovian memory and noise effects could be in resonance with the external controlling field, coupling the latter to collective behavior associated with domain motion and/or phase transitions.

Modeling CG lattice dynamics with AIGLE or AILE brings us to a scale where phenomena are typically treated with continuum models. These phenomena include general domain dynamics and phase separation in the condensed phase, which occur in ferromagnets (88), ferroelectrics (89), and alloys (9, 10). Other phenomena, important in the fabrication and characterization of nanomaterials, include morphology evolution in epitaxial growth (90) and height fluctuations of two-dimensional membranes (91). In these contexts, the application of phase field models is very popular, whereby a continuum approximation is imposed a priori and partial differential equations are constructed, guided by symmetry and physical intuition. This approach often captures the correct qualitative physics. However, when defects like impurities, grain boundaries, and domain walls are present, ad hoc continuum approximations fitted to few experimental observations, may be insufficient. When the role of defects is important, lattice models for the local dipole moments, local strains, and spins should be more reliable, as defect dynamics could be incorporated in lattice models by coupling homogeneous CVs on a lattice to a finite number of virtual particles representing mobile defects. Along this line, one may be able to model notoriously difficult processes, such as those leading to the fatigue of ferroelectric devices when the dynamics of point defects gradually impacts the dynamics of domain walls over large space and time scales (92).

All the applications discussed in the present work focused on the near-equilibrium regime, where the dynamics is constrained by the 2FDT. However, our methodology could be extended to far-from-equilibrium situations, where a governing principle like the 2FDT does not apply. A regression-based approach like AIGLE can be adapted to deal with these situations, whereas conventional approaches based on ACFs would lose the convenience of direct construction of memory and noise terms. How to extend AIGLE to deal with far-from-equilibrium phenomena is a direction that we intend to explore in future studies.

4. Materials and Methods

Here, we illustrate the learning procedure for the univariant GLE (Eqs. 2–4). We will use ${\mathcal{F}}_{(n)}$ as an abbreviation for $-\mathrm{\nabla }G(x_{(n)})+F_{(n)}$. Also, without loss of generality, we assume $m=1$.

Separation of Noise.

The first step of learning relies on equilibrated MD trajectories $\kappa =\{x_{(n)}|n\in [0,N]\}$ with ergodic fast degrees of freedom. $v_{(n+\frac{1}{2})}$ and $a_{(n)}$ are computed from Eq. 3. $v_{(n)}$ is further determined as the average of $v_{(n+\frac{1}{2})}$ and $v_{(n-\frac{1}{2})}$. We turn the ensemble-averaged orthogonality condition $⟨R_{(n)}{v}_{(0)}⟩=0$ to a time-averaged one. To achieve that, we introduce the shifted GLE with an arbitrary starting point $n_0\ge 0$:

$$\begin{array}{c}\hfill a_{(n)}={\mathcal{F}}_{(n)}+{\displaystyle {\displaystyle \sum _{s=0}^{n-n_0-1}}}{K}_{(s+\frac{1}{2})}{v}_{(n-\frac{1}{2}-s)}\mathrm{\Delta }t+{\tilde{R}}_{(n)}^{(n_0)}.\end{array}$$ [10]

Here, ${\tilde{R}}_{(n)}^{(n_0)}$ is a shifted noise and $n>n_0$ is required. As demonstrated in Ref. (27), $⟨{\tilde{R}}_{(n_0+k)}^{(n_0)}{\tilde{R}}_{(n_0)}^{(n_0)}⟩=⟨R_{(n+k)}{R}_{(n)}⟩$ for a stationary noise series when $n\to \infty$.

For a given CV trajectory, the shifted noise ${\tilde{R}}_{(n)}^{(n_0)}$ is explicitly computed by inverting Eq. 10:

$$\begin{array}{c}\hfill {\tilde{R}}_{(n)}^{(n_0)}=a_{(n)}-{\mathcal{F}}_{(n)}-{\displaystyle {\displaystyle \sum _{s=0}^{n-n_0-1}}}{K}_{(s+\frac{1}{2})}{v}_{(n-s-\frac{1}{2})}\mathrm{\Delta }t.\end{array}$$ [11]

For $n=n_0$, we let ${\tilde{R}}_{(n)}^{(n)}=a_{(n)}-{\mathcal{F}}_{(n)}$. The time-averaged estimator of $⟨R_{(k)}{v}_{(0)}⟩$ can be written as ${\zeta }_k=\frac{1}{N_k}{\displaystyle {\displaystyle \sum _{n_0=0}^{N_k-1}}}{\tilde{R}}_{(n_0+k)}^{(n_0)}{v}_{(n_0)}$, where $N_k=N-k$. Note that ${\zeta }_0$ only depends on the force fields while ${\zeta }_k$ also depends on the memory kernel for $k>0$. It is not recommended, for numerical stability, to train $\mathcal{F}$ by imposing the orthogonality condition ${\zeta }_0=0$ directly. We recommended, instead, to train $\mathcal{F}$ by minimizing the noise within a maximum-likelihood perspective, and further decouple the training of $\mathcal{F}$ and $\mathcal{K}$ for stability and efficiency. To achieve these goals, we first define the constrained optimization problem:

$$\begin{array}{c}\hfill \begin{array}{c}\hfill {\displaystyle \underset{{\mathcal{F}}^{\theta },K^{\theta }}{\text{minimize}}}\&{\displaystyle \underset{\kappa \sim {\pi }_{\kappa }}{\mathbb{E}}}{\displaystyle {\displaystyle \sum _{n=m_K}^{N-1}}}{|{\tilde{R}}_{(n)}^{(0)}|}^2\\ \hfill \text{subject to}\&\underset{\kappa \sim {\pi }_{\kappa }}{\mathbb{E}}{\zeta }_k=0,k\in [1,m_K].\end{array}\end{array}$$ [12]

Here, ${\pi }_{\kappa }$ denotes an ensemble of $\kappa$ trajectories. The ensemble average ${\mathbb{E}}_{\kappa \sim {\pi }_{\kappa }}$ is not necessary when $\kappa$ is ergodic and sufficiently long. But in practice averaging over multiple finite-size trajectories is preferred. $m_K$ is the finite memory cutoff of $K$. ${\mathcal{F}}^{\theta }$ and $K^{\theta }$ are the parameters of $\mathcal{F}$ and $K$, respectively. $\mathcal{F}$ can be any differentiable parameterized function, including neural networks. Eq. 12 should be transformed into an unconstrained problem in practical applications. Notice that the constraint ${\mathbb{E}}_{\kappa \sim {\pi }_{\kappa }}{\zeta }_k=0$ can be written equivalently as

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \underset{\kappa \sim {\pi }_{\kappa }}{\mathbb{E}}{\displaystyle \sum _{n_0=0}^{N_k-1}}& (a_{(n_0+k)}-{\mathcal{F}}_{(n_0+k)})v_{(n_0)}\hfill \\ \hfill & =\underset{\kappa \sim {\pi }_{\kappa }}{\mathbb{E}}{\displaystyle \sum _{s=0}^{k-1}}{K}_{(s+\frac{1}{2})}\mathrm{\Delta }t{\displaystyle \sum _{n_0=0}^{N_k-1}}{v}_{(n_0+k-s-\frac{1}{2})}{v}_{(n_0)}.\hfill \end{array}\end{array}$$ [13]

Considering $k\in [1,m_K]$, Eq. 13 can be written in matrix form as $\mathcal{Y}=\mathcal{C}\mathcal{K}$. $\mathcal{Y}$ and $\mathcal{K}$ are vectors of length $m_K$. $\mathcal{C}$ is a $m_K\times m_K$ lower triangular matrix. The left-hand side of Eq. 13 is the $k$-th entry of $\mathcal{Y}$. The $j$-th entry of $\mathcal{K}$ is ${\mathcal{K}}_j=K_{(j-\frac{1}{2})}\mathrm{\Delta }t$. And ${\mathcal{C}}_{\mathit{kj}}={\mathbb{E}}_{\kappa \sim {\pi }_{\kappa }}{\displaystyle \sum _{n_0=0}^{N_k-1}}{v}_{(n_0+k-j+\frac{1}{2})}{v}_{(n_0)}$ when $m_K\ge k\ge j\ge 1$. Hence, the least-square solution of Eq. 13 can be written as $\mathcal{K}=\text{Inv}({\mathcal{C}}^T\mathcal{C}){\mathcal{C}}^T\mathcal{Y}$. $\text{Inv}$ is the pseudoinverse operator computed from single-value decomposition with a cutoff ratio to avoid numerical instability.

We are then able to approach the solution of Eq. 12 practically by interleaving $n^{\mathrm{GD}}\ge 1$ unconstrained optimization steps toward

$$\begin{array}{c}\hfill \underset{{\mathcal{F}}^{\theta }}{\text{minimize}}\underset{\kappa \sim {\pi }_{\kappa }}{\mathbb{E}}{\displaystyle \sum _{n=m_K}^{N-1}}{|{\tilde{R}}_{(n)}^{(0)}|}^2\end{array}$$ [14]

with one iteration of

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \mathcal{K}& \to (1-ϵ)\mathcal{K}+ϵ\text{Inv}({\mathcal{C}}^T\mathcal{C}){\mathcal{C}}^T\mathcal{Y},\hfill \\ \hfill \mathcal{K}& \to \mathcal{K}-{\mathcal{K}}_{m_K}{\mathbf{1}}_{m_K}.\hfill \end{array}\end{array}$$ [15]

The parameter $ϵ\in (0,1)$ should be small enough for stability. In this work, we use $ϵ=0.01$. The second step in Eq. 15 forces ${\mathcal{K}}_{m_K}=0$ over the course of training.

Training of the GAR Model.

In the previous step, the noise $R_{(n)}={\tilde{R}}_{(n)}^{(0)}$ is extracted from $a_{(n)}$. Then, one can establish a GAR model with $R_{(n)}$ as data. The GAR parameters include the linear coefficients $\varphi =({\varphi }_{(1)},\cdots ,{\varphi }_{(k)})$ and the parameters $\{{\mu }^{\theta },{\sigma }^{\theta }\}$ of the neural network. We define the maximum likelihood loss function

$$\begin{array}{c}\hfill {\mathcal{L}}^{\mathrm{GAR}}=\underset{\kappa \sim {\pi }_{\kappa }}{\mathbb{E}}\sum _{n}ln{\sigma }_{(n)}^2+\frac{{(R_{(n)}-{\sum }_{k=1}^{m_A}{\varphi }_{(k)}{R}_{(n-k)}-{\mu }_{(n)})}^2}{{\sigma }_{(n)}^2}.\end{array}$$ [16]

It is not recommended to minimize ${\mathcal{L}}^{\mathrm{GAR}}$ directly with respect to all the parameters without constraints. Overfitting the data should be avoided for the long-term stationarity of the GAR model. This is crucially important for simulating AIGLE at or above the $\mathrm{\mu }$s scale, much longer than the picosecond/nanosecond duration of the MD trajectories. So, we harness the GAR model by imposing on $\varphi$ the constraint that they should satisfy the Yule–Walker equation. For a given CV trajectory, let ${\lambda }_{(k)}$ be the estimator of the noise ACF, given by ${\lambda }_{(k)}=\frac{1}{N_k-m_A}{\displaystyle \sum _{n=m_A}^{N_k-1}}{R}_{(n+k)}{R}_{(n)}$. Let the vector $\mathrm{\Lambda }$ be $\mathrm{\Lambda }=\underset{\kappa \sim {\pi }_{\kappa }}{\mathbb{E}}({\lambda }_{(1)},\cdots ,{\lambda }_{(m_A)})$. Let the $m_A\times m_A$ matrix $\mathcal{M}$ be ${\mathcal{M}}_{\mathit{jk}}=\underset{\kappa \sim {\pi }_{\kappa }}{\mathbb{E}}{\lambda }_{(|j-k|)}$. The Yule–Walker equation for a standard AR($m_A$) model is $\mathrm{\Lambda }=\mathcal{M}\varphi$, the least square solution of which can be written as ${\varphi }^{\mathrm{YW}}=\text{Inv}({\mathcal{M}}^T\mathcal{M}){\mathcal{M}}^T\mathrm{\Lambda }$. Using the Yule–Walker solution as a constraint, we optimize the GAR model by interleaving $n^{\mathrm{GD}}$ unconstrained optimization steps toward

$$\begin{array}{c}\hfill \underset{{\mu }^{\theta },{\sigma }^{\theta }}{\text{minimize}}{\mathcal{L}}^{\mathrm{GAR}}\end{array}$$ [17]

with one iteration of

$$\begin{array}{c}\hfill \varphi \to (1-ϵ)\varphi +ϵ\text{Inv}({\mathcal{M}}^T\mathcal{M}){\mathcal{M}}^T\mathrm{\Lambda }.\end{array}$$ [18]

Although in the formal presentation, the training of GAR is done after the training of the first step, in practice one can train GAR on the fly to simplify the implementation.

Incorporation of Near-Equilibrium Data.

In this step, we deal with additional datasets that violate detailed balance. We fix the memory kernel and the GAR model obtained for thermal equilibrium, assuming that they are approximately the same in near-equilibrium situations. The optimization task is simply

$$\begin{array}{c}\hfill \underset{{\mathcal{F}}^{\theta }}{\text{minimize}}\underset{\kappa \sim {\pi }_{\kappa }}{\mathbb{E}}{\displaystyle \sum _{n=m_K}^{N-1}}{|{\tilde{R}}_{(n)}^{(0)}|}^2\end{array}$$ [19]

for the extended dataset. Here ${\mathcal{F}}^{\theta }$ may include the parameters of the external driving forces.

Supplementary Material

Appendix 01 (PDF)

Data, Materials, and Software Availability

The DP model and a minimal implementation of AIGLE are publicly available on Github (93).