The seeds of the future are in the present: A blind exploration of metastable states

Significance

Molecular dynamics simulation is an extremely powerful tool of ever-growing relevance in modern science. Here, we push the boundary of this approach by developing a method that allows investigating, in a blind way, the long-time behavior of systems, eliminating kinetic bottlenecks and finding metastable states. The method is machine learning based, and turns into an asset what is usually considered a liability, namely the modest generalization ability of neural networks. Applications to the blind study of complex chemical reactions, ligand binding, and phase transformations, to name just a few, are now made possible by this tool.

Abstract

In this work, we present a type of molecular dynamics simulation that aims at finding, in a blind way, metastable states. Using only data coming from an initial unbiased simulation, and with the help of an appropriately defined loss function, we compute a bias that favors sampling yet unexplored configurational space regions, encouraging the system to leave the initial basin. In our work, we take advantage of what is normally thought to be a defect, namely the difficulty of neural networks to generalize. Contrary to most other enhanced sampling methods, which need previous knowledge of the reactive process, we are able to explore in a blind way metastable states, overcoming otherwise insuperable kinetic bottlenecks. We illustrate the workings of the method with a number of instructive examples.

Molecular dynamics simulations based on an atomistic description of matter play an increasingly relevant role in modern science. Such simulations help understanding and guiding experiments, and finally, gaining insight into physical phenomena. They can also replace experiments and act as a virtual microscope of very high resolution. In most areas of physics and chemistry, it is rare to find experimental papers that are not accompanied by an atomistic simulation (1–3).

One appealing feature of molecular dynamics simulations is that, at least in principle, they proceed like laboratory experiments. One sets up the numerical experiment by choosing a potential that describes as accurately as possible the interatomic interactions and also the thermodynamic conditions of the experiment. Then, finally, and most importantly, once enough data from the simulation have been collected, one tries to understand the science behind the results obtained.

With the help of machine learning, the problem of designing accurate interatomic potentials has made important strides forward (4–7). Ways of controlling the thermodynamic conditions have also been developed (8–10). However, still severe technical problems remain when simulating systems which evolve by visiting different metastable states separated by large barriers. Examples of such scenarios are first-order phase transitions like crystallization, most chemical reactions, or drug–protein binding. The kinetic bottlenecks associated with such phenomena make observing transitions between long-lived metastable states extremely unlikely (11). These transitions are often referred to as rare events. Their study has been the subject of a large number of investigations (12–15) and, although great progress has been made, much remains to be done.

Most enhanced sampling methods assume that the initial ($A$) and final ($B$) metastable state are known beforehand (16–18), either explicitly as in the case of path-based methods (13) or implicitly, as in the case of metadynamics (12) where the choice of collective variables (CVs) encodes also the $B$ state. Despite the many successes, there is something practically and intellectually unsatisfactory in restricting oneself to this scenario. Often, the $B$ state is not known beforehand, and having a method capable of discovering new metastable states would be of the greatest importance. The present state of affairs also implies that molecular dynamics is not yet that perfect method which, like in an experiment, interrogates the system in an unbiased way and comes out with the result.

Here, we propose a method that makes this type of simulations possible, allowing the system overcoming kinetic barriers and moving from metastable state to metastable state without any previous knowledge of the system. Several attempts have been made already to achieve this result. Metadynamics-like approaches (19, 20) have suggested using as CVs the eigenvalues of the adjacency matrix, which encode the molecular structure (21). With this method, successful predictions of reactions relevant to atmospheric (22) and high-pressure chemistry (23) have been reported. However, this approach cannot be easily extended beyond the realm of atmospheric chemistry. More recently, Zhang and Piccini (24) in an elegant approach have suggested using the skewness of the $A$ distribution to drive the system out of the initial metastable state.

Here, we take an approach similar in spirit to that of Zhang and Piccini (24), but one that is more general since it does not rely on quasi-harmonic considerations and which leads to a highly nonlocal bias that can act far away from $A$. Our method is based on a novel principle that might have general relevance in machine learning and allows automatically discovering several metastable states in sequence.

We use here methodologies developed in our recent studies of the committor (25, 26) and of the closely related dynamics operator (27, 28). Following these papers we introduce a loss function that, when minimized using only data in A, has as solution a function $I(x)$ that is constant (i.e., $I(x)\approx 1$) in the sampled region of $A$, but deviates from being a constant outside, where it has to extrapolate. It has been argued that, in this regime, the neural network extrapolates linearly (29), but for what concerns us here, it only matters that the derivative of $I(x)$ changes its value when going outside of the already sampled region. The function ${|\mathrm{\nabla }I(x)|}^2$ is thus approximately $0$ in the convex hull of $A$ and changes to a finite value outside. This is illustrated in Fig. 1, where we generated data following a hyperbolic tangent, which is a simplified model for a committor. We then trained a model using only samples coming from the left of the step. It can be seen that the slope of the model clearly changes when we leave the training set.

Fig. 1.

Example of extrapolation for an objective data following a hyperbolic tangent function. A neural network model was trained only on a part where the target function is constant. The neural network extrapolation behavior is illustrated for $x>0$.

To attract the system toward regions of the configurational space where ${|\mathrm{\nabla }I(x)|}^2$ is nonzero, we can use the following potential by taking inspiration from our works on the committor function (25, 26)

$$\begin{array}{c}\hfill V_{\mathcal{K}}(x)=-\frac{\lambda }{\beta }log(|\mathrm{\nabla }I(x){|}^2+ϵ)+\frac{\lambda }{\beta }logϵ\end{array}$$ [1]

where $\lambda$ modulates the bias strength and $ϵ$ is a regularization parameter. This will tend to push sampling toward the edges of the already sampled $A$ region. We also note that the maximum possible value of $V_{\mathcal{K}}(x)$ is $0$, hence outside the already explored region it has to be negative. Given its link to the functional introduced by Kolmogorov (30), we shall refer to $V_{\mathcal{K}}(x)$ as the Kolmogorov bias. The Kolmogorov bias is related to the derivatives, therefore, it will attract the system where the derivatives are the highest in the extrapolation region, where $I(x)$ is not constant. This will result in an exploration of new regions.

The way the Kolmogorov bias acts is illustrated in Fig. 2: where we show the behavior of $V_{\mathcal{K}}(x)$ in a double-well potential, when $I(x)$ is trained on left basin data only. It can be seen that the Kolmogorov bias removes the barrier at the right of A and even lowers the B state. We also note that the extrapolation reflects the skewness of the potential, and as discussed by Zhang and Piccini (24), this will help moving to $B$. In addition, since it is defined for any $x$ it will make itself felt all over the space, adding a field that encourages the system to visit yet unexplored regions.

Fig. 2.

Kolmogorov bias from a model trained with data only from the left state of a double-well potential. The dashed gray lines indicate the underlying potential, the dashed blue lines the Kolmogorov bias. The orange line is the sum of the Kolmogorov bias and the potential. Finally, the red points are the training data.

1. Results

In this section, we report the results obtained in order of complexity of the studied system. We first start with the sempiternal example of alanine dipeptide in vacuum to show that from an unbiased simulation in one metastable state, we can escape this state and move to another conformation. Then, we show that we can explore chemical reactions, with the example of a Claisen rearrangement (31, 32) in the molecule of 3-ethenoxyprop-1-ene which has already been used as a test for other reaction discovery schemes (20). Finally, we show that even for higher-dimensional systems, our method can find different states in multiple-state systems by tackling the slightly more complex molecule, alanine tetrapeptide.

1.1. Alanine Dipeptide.

Alanine dipeptide is a simple molecule which presents conformational changes associated with the flexibility of its dihedral angles. A very large body of experience has been accumulated on this molecule (17, 33–35). Its conformational space is spanned by the four dihedral angles $\varphi ,\psi ,\theta$ and $\omega$ (Fig. 3A). The angle $\varphi$ is known to be a good CV, and $\theta$ has been found to be part of the reaction coordinate (25, 36).

Fig. 3.

(A) Alanine dipeptide and its four dihedral angles. (B) (blue) An example of an alanine dipeptide trajectory under the combined action of the Kolmogorov bias and of OPES explore metadynamics for a model that uses the dihedral angles as descriptors. The training set is represented with black points The isolines of the underlying free energy surface are drawn, and the background color represents the Kolmogorov bias.

The lowest part of the free energy surface projected on the $\varphi ,\theta$ plane exhibits three well-known conformational states: C5, C7eq, and C7ax. A low barrier separates the C5 and C7eq minima, and at the simulation temperature (T $=$ 300 K), these conformers can easily interconvert. The barrier to go from these two low-lying states to C7ax is much higher and needs some kind of enhanced sampling method to be overcome. The lowest metastable state is thus multimodal, being a combination of the C5 and C7eq conformers. In one such case, the skewness of the distribution does not necessarily identify the escape direction.

In this system, it is natural to use the four dihedral angles introduced above as descriptors. The resulting value $V_{\mathcal{K}}(x)$ projected on the $\varphi ,\theta$ plane has the desired properties of being strongly attractive toward C7ax. Since obtaining $V_{\mathcal{K}}(x)$ is the result of a stochastic process, different models can extrapolate differently. However, while in all circumstances $V_{\mathcal{K}}(x)$ always encourages the system to exit $A$, different models can do so with different efficiency. We will therefore compare the performance of our approach by studying an ensemble of models. Thus, we trained 50 different models with different initial random parameters on the same data. For each model, we ran a 5 ns long simulation. We set the OPES BARRIER parameter to 40 kJ/mol, as the expected barrier to cross to C7ax is of this order of magnitude. Of the 50 simulations, 27 went to the C7ax basin (see Fig. 3B for one example), while 15 visited a higher energy state (see SI Appendix). Finally, to better control the behavior of the system, one can add a cutoff to the Kolmogorov bias to avoid exploring high free energy regions (SI Appendix). In Fig. 3, $V_{\mathcal{K}}(x)$ as a function of the $\varphi$ and the $\theta$ angle. Similarly to the double well potential example (Section ??), $V_{\mathcal{K}}(x)$ is constant in the training basin. In addition, it makes the energy barrier between the two wells vanish and lowers the second basin. The probability of exiting $A$ can be further enhanced if one uses multiple walkers (37) or if appropriate, a multithermal sampling (38).

While the choice of the dihedral angles as descriptors is natural, one might suspect that observing these transitions was made possible by our physical understanding of the system. For this reason, we play ignorant and use as descriptors the 45 interatomic distances between heavy atoms. In such a case, things are slightly more difficult, since this set of descriptors describes the angular arrangement only implicitly. We report in SI Appendix such simulations, and of course, the efficiency is reduced, but still a large number of transitions to the $B$ state could be observed. Interestingly, several transitions to conformational states that are higher in energy were observed.The free energy difference between each state can be computed afterward using standard enhanced sampling methods, as it is demonstrated in the supporting information.

1.2. An Example of Chemical Reaction Discovery.

One example in which this method proved extremely useful was the case of chemical reactions. The example studied here was the reactive behavior of 3-ethenoxyprop-1-ene. Our aim once again was to use as little chemical intuition as possible, and thus, instead of using an intuitive CV, we used again as descriptors the interatomic distances between all heavy atoms of the molecule. One of the products is the result of a Claisen rearrangement, as shown in ref. 20 leading to the pent-4-enal as expected. To explore the possible pathways, we trained ten models with different initial training parameters, and performed simulations with a BARRIER parameter of 200 kJ/mol. To prevent the system from exploring regions with a too high free energy, we put a cutoff to the value of the Kolmogorov bias at 400 kJ/mol. In these simulations, we found two different states: the expected pent-4-enal 5 (Fig. 4, Top panel) and the higher in energy 2-oxabicyclo[2.1.1]hexane state (Fig. 4, Bottom panel), which was also reported in ref. 20. These numerical simulations show that using this method, it is possible to explore efficiently the conformational space and discover possible chemical reactions in a blind and efficient way.

Fig. 4.

Results of our simulations for the Claisen rearrangement of the 3-ethenoxyprop-1-ene molecule. Top panel: example of a successful Claisen rearrangement. Bottom panel: example of a simulation ending up in the 2-oxabicyclo[2.1.1]hexane state.

1.3. Exploring a Multistate System: Alanine Tetrapeptide.

We now apply our exploration method to the conformational changes of a molecule that displays more than two accessible metastable states: alanine tetrapeptide. The natural variables of the problem are the three dihedral angles (${\varphi }_1,{\varphi }_2,{\varphi }_3$) (39, 40) (Fig. 5A). In this representation, the four slowest transitions are related to changes in the sign of the ${\varphi }_2$ and ${\varphi }_3$ angles, for a total of four metastable basins. Once again, in order to be as blind as possible in the exploration phase, instead of using the dihedral angles, which are natural CVs, we used as input of the neural networks all the 190 interatomic distances between heavy atoms, illustrating the ability of our method to deal with high-dimensional descriptor spaces. We show in (Fig. 5 the results of one such simulation that visited all four basins in the allotted 20 ns simulation time. This happens because of the long-range nature of $V_{\mathcal{K}}(x)$, which also acts far away from A and encourages the system to visit other metastable states. This illustrates the ability of our method to explore several unknown basins, as in the case of alanine dipeptide. In the supporting information, we investigate the influence of the descriptor set on the results by using two other different sets of descriptors.

Fig. 5.

Result of one exploration simulation for the alanine tetrapeptide, in the $({\varphi }_2,{\varphi }_3)$ plane. The starting basin is indicated by a blue cross.

2. Discussion and Conclusion

Our method allows a blind exploration of different metastable states based only on data coming from one state and does not require the use of a CV. However, the choice of relevant descriptors might ease the task, as we have seen in the case of alanine dipeptide.

The choice of descriptors can be aided by years of experience in the design of collective variables. Moreover, our method is variational, thus the best descriptors are the ones that lead to the minimal loss. Finally, methods for selecting the most appropriate descriptors can be devised, either by using, when appropriate what is suggested in ref. 24 or more elaborate approaches in which a small set of descriptors is chosen by imposing that the on-sample fluctuations of $I^w(x)$ are minimal. Such an avenue of progress will be explored in the near future. A natural evolution for this method would be to use graph neural networks (41, 42), which are direct functions of the atomic positions, and therefore should naturally encode all the relevant conformational changes.

One of the features which distinguishes our approach from others is the long-range properties of $V_{\mathcal{K}}(x)$, which can also be responsible for the success of the iterative approach for learning the committor in refs. 25 and 26.

Several avenues of further progress are possible. The identification that a transition has taken place can be made automatically using methods like the one in ref. 43 rather than having to resort to manual inspection. Once a new state has been found, committor-based methods can be used to compute accurately the free energy surface and study the transition state ensemble. In a metadynamics-like variant of the method, once the $A\cup B$ has been explored, one could continue to look for other metastable states by iterating the procedure to speed-up further exploration. The use of an equivariant neural network will lift the need to use descriptors and start only with the Cartesian coordinates. This method is also ideally suited to generate on the fly new configurations on which machine learning potential can be trained. One can also envisage using this method as a way of directing the acquisition of new data in other machine learning applications. This would amount to a reinforcement learning interpretation of what we do where $V_{\mathcal{K}}(x)$ acts as a reward.

Finally, it is our hope that this work brings the dream of using molecular dynamics as a totally blind exploration tool a bit closer.

3. Materials and Methods

A standard approach to enhanced sampling is based on filling the metastable states with an external bias that favors transitions to other states. In metadynamics and similar methods, this is done by building on the fly the bias as a sum of repulsive Gaussians that are functions of a set of selected CVs (12, 14, 15). Even though ample literature has been devoted to the choice of CV (44–47), this step is often difficult, and an incorrect choice can affect the outcome. Furthermore, much wasted time is invested in reconstructing the free energy of the initial metastable state before basin $A$ is filled to a level high enough for the system to move out of $A$.

In this paper instead, we construct a bias that encourages the system to sample new and yet unexplored regions, thus eventually inducing the system to leave the initial metastable state.

In order to carry out this program, we first define a functional whose argument $I(x)$ is a function of the atomic coordinates, which is minimized by $I(x)=1$ if the configuration space is fully sampled. A simple example of one such functional is

$$\begin{array}{c}\hfill \mathcal{L}[I]=⟨|{\mathrm{\nabla }}_{u}I(x){|}^2⟩+\alpha {(S-1)}^2,\end{array}$$ [2]

where the angular brackets indicate averages over the Boltzmann ensemble, $S=⟨I^2(x)⟩$ is the second moment of $I(x)$, ${\mathrm{\nabla }}_u$ the mass-weighted gradients as in Kolmogorov functional, and the hyperparameter $\alpha$ controls the normalization of $I(x)$. However, as an alternative to $\mathcal{L}([I])$ we considered using $\mathcal{E}([I],\lambda )$ which depends on the function $I(x)$ and the scalar $\lambda$:

$$\begin{array}{c}\hfill \mathcal{E}([I],\lambda )=\frac{1}{{(\eta +\lambda )}^2}S(\eta S+M)-\frac{2}{\eta +\lambda }S+\alpha {(1-S)}^2,\end{array}$$ [3]

where $S=⟨I^2(x)⟩$ as before, $M=k_{B}T⟨|{\mathrm{\nabla }}_{u}I(x){|}^2⟩$ and $\eta$ a regularization parameter. This functional is minimal for $I(x)=1$ and $\lambda =0$ and is a limiting case of the one introduced in our earlier works (27, 28) (SI Appendix). Since we found that $\mathcal{E}([I],\lambda )$ has better numerical properties than $\mathcal{L}[I]$, $\mathcal{E}([I],\lambda )$ will be used in the following. The extrapolation properties of the neural network are mainly due to the functional used. A deeper understanding of the link between the trivial eigenfunction of the restricted generator and the full one is left for future investigation. However, to justify the use of the functional in (Eq. 3) and its numerical stability, we add in SI Appendix the Kolmogorov bias in the double well potential.

In the practice, we shall transform the functional into a loss function and represent $I(x)$ as a neural network $I^w(x)$ where $w$ are variational parameters. These variational parameters will be optimized by minimizing the loss function through a stochastic gradient descent algorithm (48).

Of course, given an extensive sampling of the Boltzmann distribution, the solution of this variational problem is $I^w(x)=1$ for all $x$. If however, only one basin, say $A$, has been sampled, the solution will be $I^w(x)\approx 1$ for $x\in A$, but $I^w(x)\ne 1$ for $x\notin A$, since in this region, no data are available and the neural network by necessity has to generalize $I^w(x)$. This leads to a change of slope as one crosses the boundary of the explored region.

In practice, $I^w(x)$ is not perfectly constant, and thus, the Kolmogorov bias is not exactly zero in the training basin, which could lead to artificial barriers in the initial well. To overcome this problem, we complement the action of $V_{\mathcal{K}}(x)$ by an OPES Explore term, in a fashion similar to what is done in ref. 26, and use $I^w(x)$ as collective variable. In addition, the variational function that we insert in Eq. 3 has the form

$$I^w(x)=e^{-z^w(x)},$$

where $z^w$ is a neural network. The use of the exponential as activation function leads to a smoother $I^w(x)$ and guides the extrapolation. Different activation functions could be devised, but this first choice has proven efficient enough for our purposes. For all the trainings, we used as activation function for the networks the tanh one. This choice is particularly convenient as tanh is differentiable as well as its derivative, which makes it easy to use for molecular dynamics purposes with $V_K$. However, we believe that our results are general and other function of this kind can be used.

When performing OPES simulations, one has to choose a BARRIER parameter (15), which corresponds to the maximum barrier the system is expected to overcome during the simulation. In order to reduce the number of parameters to be set to a bare minimum, the value of $\lambda /\beta$ for $V_{\mathcal{K}}$ is chosen such that the minimum value of $V_{\mathcal{K}}(x_i)$ among the sampled $x_i$ is the BARRIER parameter. A low OPESBARRIER value will force the system to follow a low free energy path, while a high value will allow exploring excited metastable states.

Numerical details of the calculations will be presented in SI Appendix.

Supplementary Material

Appendix 01 (PDF)

Data, Materials, and Software Availability

All the trainings of the models were performed using the opensource library mlcolvar (49). The models, as well as all the input files to reproduce the results presented in this work are available at the on GitHub (51).