Machine Learning Assisted Free Energy Simulation of Solution–Phase and Enzyme Reactions

Abstract

Despite recent advances in the development of machine learning potentials (MLPs) for biomolecular simulations, there has been limited effort in developing stable and accurate MLPs for enzymatic reactions. Here, we report a protocol for performing machine learning assisted free energy simulation of solution-phase and enzyme reactions at an ab initio quantum mechanical and molecular mechanical (ai-QM/MM) level of accuracy. Within our protocol, the MLP is built to reproduce the ai-QM/MM energy as well as forces on both QM (reactive) and MM (solvent/enzyme) atoms. As an alternative strategy, a delta machine learning potential (ΔMLP) is trained to reproduce the differences between ai-QM/MM and semi-empirical (se) QM/MM energy and forces. To account for the effect of the condensed–phase environment in both MLP and ΔMLP, the DeePMD representation of a molecular system is extended to incorporate external electrostatic potential and field on each QM atom. Using the Menshutkin and chorismate mutase reactions as examples, we show that the developed MLP and ΔMLP reproduce the ai-QM/MM energy and forces with an error on average less than 1.0 kcal/mol and 1.0 kcal/mol/Å for representative configurations along the reaction pathway. For both reactions, MLP/ΔMLP-based simulations yielded free energy profiles that differed by less than 1.0 kcal/mol from the reference ai-QM/MM results, but only at a fractional computational cost.

Graphical Abstract

1. Introduction

To accurately model solution-phase and enzyme reactions, it would be desirable to perform direct ab initio quantum mechanical and molecular mechanical (ai-QM/MM) free energy simulations.^1–11 In a typical ai-QM/MM free energy simulation, the ai-QM/MM potential is evaluated for each configuration of the system of interest, in which a quantum mechanical (QM) reactive region (typically containing up to 150 atoms) is embedded in thousands of or more molecular mechanical (MM) atoms (i.e. the rest of enzyme or solvent atoms).^12,13 In practice, the ai-QM/MM potential needs to be computed for 10⁵ or 10⁶ configurations in umbrella sampling calculations¹⁴ or other enhanced samplings (such as metadynamics), before the mean free energy pathway and corresponding reaction free energy profile can be obtained. As such, direct ai-QM/MM free energy simulations are rather compute-intensive, requiring $\mathcal{O}\left({10}^5\right)$ CPU hours,^15–18 and thus have yet to gain wide use.

To avoid such steep costs of direct ai-QM/MM free energy simulations, Gao,¹⁹ Warshel²⁰ and others^21–25 developed indirect free energy simulations, where sampling is carried out using a reference semi-empirical QM/MM (se-QM/MM) potential and then the free energy result is corrected with a se-QM/MM→ai-QM/MM thermodynamic perturbation or with an interpolation between the two levels of potential, for example along the energy minimized reaction path.^26,27 Alternatively, one can adopt the multiple timestep simulation methodology from Tuckerman,^28–31 Schlick,^32,33 Nam,³⁴ Roux,³⁵ Rothlisberger,³⁶ and others,³⁷ where se-QM/MM trajectory propagation at regular/inner timesteps is coupled with ai-QM/MM trajectory corrections at outer timesteps. Needless to say, the accuracy of both indirect and multiple-timestep simulations is controlled by the quality of the se-QM/MM potential in use. In many cases, it is beneficial to re-optimize the se-QM/MM parameters^23,38 or directly modify the internal forces³⁹ to ensure a proper thermodynamic perturbation or interpolation correction or to maintain a stable multiple-timestep trajectory.

Recently, Yang,^40–42 Gastegger,⁴³ York,⁴⁴ Riniker⁴⁵ and coworkers have proposed machine learning (ML) as a new strategy to address the computational cost of direct ai-QM/MM free energy simulations. Specifically, for configurations of interest, artificial neural network (ANN) models were designed and trained to reproduce either the ai-QM/MM potential, i.e., the machine learning potential (MLP), or the difference between the ai-QM/MM and se-QM/MM potentials, hereafter, referred to as the delta machine learning potential (ΔMLP). Thus developed MLP or ΔMLP was then employed (in lieu of the ai-QM/MM potential) to drive the dynamical sampling of the enzyme system. These MLPs/ΔMLPs led to fairly accurate free energy barriers, with errors of around 1.0 kcal/mol, for several solution-phase reactions.^40–42,44 In these approaches, however, the effects of solvent on the reacting system were rather homogeneous, and their applicability to reactions in a heterogeneous solvent environment, such as in enzyme, has not been fully explored.

When compared to the construction of MLPs for gas-phase and small periodic systems, these are substantial achievements because the training of MLP for reproducing the ai-QM/MM potential is more challenging. Naively adding thousands of MM atoms to commonly-used ML descriptors of a gas-phase molecular system [such as symmetry functions for constructing high-dimensional neural network potentials (HDNNP) descriptors,^46–48 Coulomb matrix,⁴⁹ Faber-Christensen-Huang-Lilienfeld (FCHL) representation,⁵⁰ tensor representation,⁵¹ embedding matrix,^52,53 and PhysNet⁵⁴] would lead to an explosively large number of inputs for the ANN model, thus jeopardizing the convergence (in training) and affordability (for both training and production).

In the development of the above mentioned MLPs/ΔMLPs, two approaches have been proposed to efficiently include MM atoms in the ANN models. The first is an “implicit” approach, where MM atoms are implicitly accounted for through their collective perturbation to the electronic structure of the QM region. For example, Yang and coworkers^40,41 used semi-empirical QM Mulliken charges as perturbed by all the MM charges or the total MM electrostatic potential on each QM atom,⁴² whereas Gastegger and coworkers⁴³ captured the effect of MM atoms through their net dipolar field on each QM atom. The second is an “explicit” approach, in which any MM atom within a “cutoff” distance (such as 6.0Å) from the QM region is included in the computation of ML descriptors. This approach was proposed by York and coworkers⁴⁴ for the QM/MM expansion to the embedding matrix within the DeePMD coding framework,^53,55 and also utilized by Riniker and coworkers⁴⁵ to extend the HDNNP descriptors to QM/MM calculations.

Inspired by these approaches to include MM environment in the MLP development, in this work, we aimed to develop a more robust protocol for training MLPs/ΔMLPs for free energy simulations of enzyme reactions by incorporating the effects of long-range MM electrostatic interactions, such as, under the periodic boundary conditions. [In a separate manuscript from us,⁵⁶ a different ai-QM/MM-based machine learning approach was proposed, where ANN was trained to produce a set of chaperone polarizabilities that augment the insufficient polarizability of the semi-empirical Hamiltonian.] Anticipating some amino acid side chains and/or solvent molecules to move in and out of the cutoff boundary with the progression of the reaction, we opted not to follow the “cutoff” approach, because the number of MM atoms retained in the cutoff might change, for example, between neighboring umbrella sampling windows and thus additional smoothing^57,58 might be needed to ensure a continuous potential energy surface. Furthermore, in enzymatic reactions, the long-range electrostatic interactions play a critical role in many catalytic reactions as well as in the stability of the molecular dynamics (MD) simulations. In addition, within the “implicit” approach, we opted not to utilize semi-empirical Mulliken charges on QM atoms, which would not be available if we aimed to directly produce the ai-QM/MM-quality MLP to propagate the MD trajectories. In the end, our approach would resemble that of Yang and coworkers⁴² and of Gastegger et al,⁴³ but the details differed significantly as described briefly below.

Compared to previous efforts, three key features differentiate our approach from those of other groups. Firstly, long-range electrostatic effects of the MM environment is incorporated rigorously in the training of the MLP models. As will be demonstrated in Section 2, this is achieved by representing the enzyme/solvent environment as MM electrostatic potential and field (ESPF) within our recent QM/MM with augmentary charges (QM/MM-AC) scheme.⁵⁹ Secondly, a more reliable sampling/collection of training configurations is acquired by calibrating the se-QM/MM Hamiltonian. In the case of ΔMLP, it reduces the magnitude of the Hamiltonian differences and thus lessens the need of iterative training of the MLP model. Finally, standalone MLP and ΔMLP are developed side-by-side, thanks to the construction of a single set of ML descriptors, thereby allowing a direct comparison of the two potentials for reactions of interest. With this development, we have simulated the catalytic reaction of an enzyme, chorismate mutase, which can be considered to be the first realistic and successful application of MLP/ΔMLP to model enzyme reactions under the full periodic boundary conditions.

This article is organized as follows. Section 2 introduces our overall methodology (incorporation of electrostatic embedding potential into the MLP, description of long-range QM/MM electrostatics, umbrella sampling), while more computational details for the training and free energy simulations are presented in Section 3. Results for solution-phase Menshutkin reaction and chorismate mutase-catalyzed Claisen rearrangement are presented in Section 4, with concluding remarks made in Section 5.

2. Machine Learning Potential

2.A. Descriptors for the QM atoms

Our current MLP/ΔMLP implementation is based on Deep Potential - Smooth Edition (DeepPot-SE),^52,53 where each QM atom i is represented by its local environment matrix to capture the internal interactions within the QM region (with N QM atoms). Specifically, for the i-th atom in the QM region at position R_i, its has n = N − 1 neighbors. The corresponding environment matrix (n-by-4) is

$${\mathcal{R}}_i=\left(\begin{array}{cccc}s\left(R_{1i}\right)& s\left(R_{1i}\right)\frac{x_{1i}}{R_{1i}}& s\left(R_{1i}\right)\frac{y_{1i}}{R_{1i}}& s\left(R_{1i}\right)\frac{z_{1i}}{R_{1i}}\\ s\left(R_{2i}\right)& s\left(R_{2i}\right)\frac{x_{2i}}{R_{2i}}& s\left(R_{2i}\right)\frac{y_{2i}}{R_{2i}}& s\left(R_{2i}\right)\frac{z_{2i}}{R_{2i}}\\ \dots & \dots & \dots & \dots \\ s\left(R_{ni}\right)& s\left(R_{ni}\right)\frac{x_{ni}}{R_{ni}}& s\left(R_{ni}\right)\frac{y_{ni}}{R_{ni}}& s\left(R_{ni}\right)\frac{z_{ni}}{R_{ni}}\end{array}\right),$$ (1)

where s(R_ji) = 1/|R_j − R_i| contains the Coulomb interaction. No screening was applied to the Coulomb interaction, due to the small size of the QM region. Next, an embedding neural network, ${\mathcal{G}}_i$, maps a single value, s(R_ji), through multiple hidden layers into m₁ outputs

$${\mathcal{G}}_i=\left(\begin{array}{cccc}{\left(G\left[s\left(R_{1i}\right)\right]\right)}_1& {\left(G\left[s\left(R_{1i}\right)\right]\right)}_2& \cdots & {\left(G\left[s\left(R_{1i}\right)\right]\right)}_{m_1}\\ {\left(G\left[s\left(R_{2i}\right)\right]\right)}_1& {\left(G\left[s\left(R_{2i}\right)\right]\right)}_2& \cdots & {\left(G\left[s\left(R_{2i}\right)\right]\right)}_{m_1}\\ \dots & \cdots & \cdots & \cdots \\ {\left(G\left[s\left(R_{ni}\right)\right]\right)}_1& {\left(G\left[s\left(R_{ni}\right)\right]\right)}_2& \cdots & {\left(G\left[s\left(R_{ni}\right)\right]\right)}_{m_1}\end{array}\right).$$ (2)

The encoded feature matrix ${\mathcal{D}}_i$ is m₁-by-m₂

$${\mathcal{D}}_i={\left({\mathcal{G}}_i^1\right)}^T{\mathcal{R}}_i{\mathcal{R}}_i^T{\mathcal{G}}_i^2,$$ (3)

where ${\mathcal{G}}_i^1$ is the same as ${\mathcal{G}}_i$ and ${\mathcal{G}}_i^2$ is a matrix that consists of the first m₂ columns of ${\mathcal{G}}_i$ with m₂ ≤ m₁. Both m₁ and m₂ are hyperparameters of the model.

2.B. Descriptors for the MM environment

For QM/MM interactions, our model closely resembles the regular QM/MM models, where the MM atoms are represented as point charges with no atomic identities for the QM-MM electrostatic interactions, and the QM-MM van der Waals (vdW) interactions are treated at the MM level.^4,7,60–62

In regular QM/MM models, there are three general schemes to treat the QM-MM electrostatic interactions, namely, the continuous, the surrogate, and the hybrid schemes,⁵⁹ depending on whether the continuous electron density or its surrogate charges or a combination interact with the MM atomic charges. Since electrons are not described explicitly in most MLPs, it is natural to adopt a surrogate-like scheme in MLP where QM atoms interact with MM atoms through the electrostatic potential and field generated,

$${\varphi }_i=\sum _{B\in \text{MM}}\frac{q_B}{\left|R_i-R_B\right|},$$ (4)

$${\mathcal{E}}_i=\sum _{B\in \text{MM}}\frac{q_B\left(R_i-R_B\right)}{{\left|R_i-R_B\right|}^3},$$ (5)

or higher-order Taylor expansions at the QM atomic sites. In contrast to normal QM/MM calculations, where one either evaluates the core Hamiltonian contribution associated with ϕ_i and ${\mathcal{E}}_i$ or derives multipole moments on each QM atom to interact with them, we choose to add the electrostatic potential ϕ_i and field ${\mathcal{E}}_i$ on each QM atom directly to the list of the input features of the MLP. In this way, we can control the accuracy of not only the QM-MM interactions but also their variation with respect to the change in the QM or MM atoms, i.e., forces.

Specifically, the contributions of the local electrostatic potential, ϕ_i, and field, ${\mathcal{E}}_i$, are fed into embedding neutral networks similar to Eq. 2, leading to additional feature matrices, ${\mathcal{D}}_i^{\varphi }$ and ${\mathcal{D}}_i^{\mathcal{F}}$, to capture the effect of the MM environment. For example, for the local electrostatic potential, a single value ϕ_i is mapped to $m_1^{\prime }$ outputs through multiple hidden layers, which is used as the feature matrix ${\mathcal{D}}_i^{\varphi }$, i.e.,

$${\mathcal{D}}_i^{\varphi }=\left(\begin{array}{c}{\left(G\left({\varphi }_i\right)\right)}_1\\ {\left(G\left({\varphi }_i\right)\right)}_2\\ \cdots \\ {\left(G\left({\varphi }_i\right)\right)}_{m_1^{\prime }}\end{array}\right).$$ (6)

The electric field on each QM atom is projected onto the vectors pointing toward its neighbors and scaled by the inverse of the distance, i.e.,

$${\mathcal{F}}_i=\left[\begin{array}{c}s\left(r_{1i}\right)\left(\frac{R_{1i}}{R_{1i}}\right)\cdot {\mathcal{E}}_i\\ s\left(r_{2i}\right)\left(\frac{R_{2i}}{R_{2i}}\right)\cdot {\mathcal{E}}_i\\ \cdots \\ s\left(r_{ni}\right)\left(\frac{R_{ni}}{R_{ni}}\right)\cdot \mathcal{E}\end{array}\right].$$ (7)

Then, each element of the projected local field ${\mathcal{F}}_i$ is mapped to $m_1^{\prime \prime }$ outputs through multiple hidden layers, i.e.,

$${\mathcal{G}}_i^{\mathcal{F}}=\left(\begin{array}{cccc}{\left(G\left({\mathcal{F}}_{1i}\right)\right)}_1& {\left(G\left({\mathcal{F}}_{1i}\right)\right)}_2& \cdots & {\left(G\left({\mathcal{F}}_{1i}\right)\right)}_{m_1^{\prime \prime }}\\ {\left(G\left({\mathcal{F}}_{2i}\right)\right)}_1& {\left(G\left({\mathcal{F}}_{2i}\right)\right)}_2& \cdots & {\left(G\left({\mathcal{F}}_{2i}\right)\right)}_{m_1^{\prime \prime }}\\ \cdots & \dots & \cdots & \cdots \\ {\left(G\left({\mathcal{F}}_{ni}\right)\right)}_1& {\left(G\left({\mathcal{F}}_{ni}\right)\right)}_2& \cdots & {\left(G\left({\mathcal{F}}_{ni}\right)\right)}_{m_1^{\prime \prime }}\end{array}\right).$$ (8)

The final feature matrix is a $m_1^{\prime \prime }\text{-by-}{m}_2^{\prime \prime }$ matrix, which can be calculated as

$${\mathcal{D}}_i^{\mathcal{F}}={\left({\mathcal{G}}_i^{\mathcal{F}1}\right)}^T{\mathcal{G}}_i^{\mathcal{F}2},$$ (9)

where ${\mathcal{G}}_i^{\mathcal{F}1}$ is the same as ${\mathcal{G}}_i^{\mathcal{F}}$ and ${\mathcal{G}}_i^{\mathcal{F}2}$ is a matrix that consists of the first $m_2^{\prime \prime }$ columns of ${\mathcal{G}}_i^{\mathcal{F}}$. In Eqs. 6 and 9, $m_1^{\prime }$, $m_1^{\prime \prime }$, and $m_2^{\prime \prime }$ are hyperparameters of the model.

2.C. Fitting Neural Network

After the three sets of feature matrices (i.e., Eqs. 3, 6 and 9) are obtained from the embedding neural networks, they are fed into a fitting neural network (NN) to obtain the total energy,

$$E=\sum _i\text{NN}\left({\mathcal{D}}_i,{\mathcal{D}}_i^{\varphi },{\mathcal{D}}_i^{\mathcal{F}}\right).$$ (10)

With a differentiable neural network model, the analytical energy derivatives in Fig. 1 are also obtained. During the standalone MLP model training, the loss function combines the error in total energy, forces on QM atoms, as well as ESP potential/field values on MM atoms,

$$\mathcal{L}={\lambda }_1{\left|E-E^{\text{ref}}\right|}^2+{\lambda }_2\sum _{i\in \text{QM}}{\left|-\frac{\partial E}{\partial R_i}+\frac{\partial E^{\text{ref}}}{\partial R_i}\right|}^2+{\lambda }_3\sum _{B\in \text{MM}}{\left|\frac{\partial E}{\partial q_B}-{\varphi }_B^{\text{ref}}\right|}^2+{\lambda }_4\sum _{B\in \text{MM}}{\left|-\frac{1}{q_B}\frac{\partial E}{\partial R_B}-{\mathcal{E}}_B^{\text{ref}}\right|}^2,$$ (11)

where the learned forces on QM atoms, $-\frac{\partial E}{\partial R_i}$, and ESP potential/field values, $\frac{\partial E}{\partial q_B}$ and $-\frac{1}{q_B}\frac{\partial E}{\partial R_B}$, are obtained from the differentiable NN model in use. (We note that, with an efficient treatment of the long-range QM/MM interactions as described in the next subsection, the third and fourth terms in Eq. 11 sum over only the charges on the inner MM atoms within 10Å from the QM region. Thus, the number of terms in the summations remains unchanged even for large systems.) The same feature matrices can be applied to the development of ΔMLP model, in which the energy, force, and ESP differences between ai-QM/MM and se-QM/MM models are used as the reference values in Eq. 11. In both MLP models, λ₁, λ₂, λ₃, and λ₄ are tunable prefactors and can be adjusted during the model training.

Figure 1:

Workflow for the training of MLP and its use to generate energy and forces for MD simulations. Q_i is the ESP charge on QM atom i, which is fitted using the electrostatic potential ϕ_B on inner MM atom positions.

2.D. Long-Range QM/MM Interactions

For a condensed-phase reaction, the MM environment contains thousands of (or more) atoms. An explicit account of all these atoms in the MLP training becomes very inefficient, if feasible at all. A common strategy around this is to completely ignore the long-range electrostatics due to MM atoms beyond a cutoff distance.^44,45 In the context of ΔMLP-learning, this strategy means that the long-range electrostatics of ΔMLP is approximated at the low-level method. However, a systematic analysis has yet to be carried out to gauge the impact of solvent/enzyme atoms moving across the cutoff boundary during the simulation on the developed ML potentials, as it can contribute to a discontinuity on the potential energy surface.

In lieu of this approximation, we have decided to adopt our recent QM/MM with Augmented Charges (QM/MM-AC) model⁵⁹ to accurately capture long-range QM/MM electrostatics in this work. Specifically, only inner MM atoms, namely those within a cutoff distance (10Å in this work) from the QM region, are explicitly incorporated into the training of MLP and ΔMLP. This can be easily achieved by replacing the MM charges in Eqs. 4 and 5, with

$$q_B\to q_B^{<,\text{C}}+q_B^{\text{AC}},$$ (12)

where $q_B^{<,\text{C}}$ refer to the “continuous” portion of charges on inner MM atoms and $q_B^{\text{AC}}$ are the augmentary charges projected on the inner MM atoms from all outer MM charges and the “surrogate” portion of charges on the inner MM atoms. More specifically,

$$q_B^{<,C}=S\left(d_B^{\text{min}}\right)q_B^{<},$$ (13)

where $q_B^{<}$ refer to charges on inner MM atoms, $d_B^{\text{min}}$ is a soft minimum distance of MM atom B to the QM atoms. $S\left(d_B^{\text{min}}\right)$ is a switching function that decays from 1 to 0 smoothly when $d_B^{\text{min}}$ increases from 0 to the cutoff distance. For all outer MM charges and the “surrogate” portion of charges on the inner MM atoms, the electrostatic potential ${\varphi }_i^{>}$ that are exerted from them onto the QM atoms are calculated using the particle mesh Ewald (PME) summation method.^63,64 Then the augmentary charges $q_B^{<,\text{AC}}$ are calculated as

$$q_B^{<,\text{AC}}=\sum _i{\varphi }_i^{>}{\left({(W\mathcal{K})}^{-1}\right)}_{i,B}{w}_B,$$ (14)

where $\mathcal{K}$ is Coulomb interaction tensor between QM atoms and inner MM atoms, and w_B is a weighting function that scales the Coulomb interaction tensor to maintain a smooth potential energy surface at the cutoff distance, ${(W\mathcal{K})}_{i,B}=w_B{\mathcal{K}}_{i,B}$.

Then, the QM/MM-AC charges, $q_B^{\text{AC}}$ in Eq. 12, are precomputed for inner MM atoms of each configuration before our training of MLP and ΔMLP. After these models are constructed, the electrostatic potential on inner MM atom positions, ${\varphi }_B=\frac{\partial E}{\partial q_B}$, is used to fit ESP charge Q_i on QM atom i,

$$Q_i=\sum _B{\left({\mathcal{K}}^{-1}\right)}_{i,B}{\varphi }_B.$$ (15)

As shown in Fig. 1, these charges are then used to provide the electrostatic potential to update the forces on outer MM atoms, through the procedure described in Ref. 59.

2.E. Implementation and Training Details

Our method, which combines the Deep Potential – Smooth Edition (DeepPot-SE)⁵² descriptors for QM atoms and QM/MM-AC-based descriptors for the MM environment, was implemented in PyTorch.

In this work, the hyperparameters were not fully optimized, where the recommended values from the DeePMD-kit package were adopted where applicable. For each of the local embedding networks, three hidden layers with 25, 50, and 100 (m₁) neurons, were used. For each of the electrostatic potential embedding networks and the electric field embedding networks, three hidden layers with 5, 10, and 20 ($m_1^{\prime }$ and $m_1^{\prime \prime }$) neurons were used. For the local embedding networks and the electric field embedding networks, the size of the axis filters were chosen to be 4 (m₂ and $m_2^{\prime \prime }$). For the fitting networks, three hidden layers, each of which has 240 neurons, were used.

Due to the pairwise nature of the “interactions” between the QM atoms (no distance cutoff for these interactions in our implementation), the formal computational complexity of the model is quadratic with the size of the QM region. While the embedding network for the QM-QM interactions has a cost that scales quadratically, the calculations of the embedding networks for the QM-MM and fitting network scale linearly.

The (standalone) MLP model was trained using the Adam optimizer. During the training, the batch size was set to 32, and the initial learning rate was set to 10⁻⁴. Following DeePMD-kit,⁵³ the prefactor for the energy error in the loss function in Eq. 11 was much smaller than the other three at the beginning, and they gradually evolved to the same value towards the end of the training. The learning rate decayed exponentially by a factor of 0.95 every epoch. A total of 100 epochs were performed for each system. A separate ΔMLP model was also trained to reproduce the difference between the PM3*/MM and B3LYP/6–31G*/MM models using the same architecture and hyperparameters as the standalone MLP model.

For each of the two reaction systems (i.e., the solution-phase Menshutkin reaction system and the chorismate mutase reaction system), 40,000 frames were collected from the PM3*/MM simulations (see the next Section for details) and then randomly split into sets of 38,400 and 1,600 samples for training and validation, respectively. The testing set consisted of 2,000 additional configurations sampled along the reaction pathway from the ai-QM/MM simulations. For each sample in the training/validation sets, B3LYP/6–31G*/MM single point calculations were performed. The QM/MM-AC method⁵⁹ was used to project the MM charges that were more than 10Å away from any QM atom and all the charges from the periodic images onto the MM atomic sites within the 10Å cutoff. Besides the energies and the gradients of the QM atoms, the electrostatic potentials and fields on the inner MM atomic sites were also collected.

3. Simulation Details

3.A. System Setup and Equilibration

In this work, we started with the Menshutkin reaction, a widely-used model system for solution chemistry, for developing the protocol of training the MLP and ΔMLP. Then the protocol is applied to the chorismate mutase reaction, which is a popular system to test new enzyme simulation methods because of its small QM region (24 QM atoms) and because the QM region is not covalently linked to the MM region.

For the Menshutkin reaction (Figure 2a), the solutes (ammonia and chloromethane) were solvated in a cubic box of 723 TIP3P⁶⁵ waters, and modeled by the general Amber force field (GAFF).⁶⁶ For the chorismate mutase reaction (Figure 2b), the initial structure was prepared based on the X-ray crystal structure (PDB ID: 2CHT⁶⁷) of Bacillus subtilis chorismate mutase complexed with a transition state analog, which was modified to the substrate chorismate manually, and solvated in a cubic box of 13,067 TIP3P⁶⁵ waters. 8 sodium counter ions were added to neutralize the system. The substrate and the enzyme were modeled using GAFF⁶⁶ and the AMBER ff14SB force field,⁶⁸ respectively.

Figure 2:

Schemes for (a) Menshutkin and (b) chorismate mutase reactions.

Both systems were equilibrated at 300 K and 1 atm using Langevin dynamics with a friction coefficient of 5ps⁻¹ and Berendsen barostat with a relaxation time of 1 ps under the periodic boundary conditions. The solutes were restrained to their initial positions by a weak harmonic potential for the Menshutkin reaction, whereas no restraints were applied for the enzymatic system. The particle mesh Ewald (PME) summation method^63,64 was used to treat the electrostatic interactions, while the van der Waals interactions were turned off to zero smoothly at a cutoff of 10 Å. The SHAKE algorithm⁶⁹ was used to constrain all bonds involving hydrogen atoms, and a time step of 2 fs was used for the MD integration using the leapfrog integrator. After equilibration (500 ps and 17 ns for the Menshutkin and chorismate mutase reactions, respectively), the sizes of the simulation boxes were ~28Å × 28Å × 28Å and ~76Å × 76Å × 76Å for the Menshutkin and chorismate mutase reactions, respectively, which were used for the subsequent simulations in the NVT ensemble. The classical MD simulations were performed using the PMEMD program from the Amber20 package.⁷⁰

3.B. QM/MM Calculations

To simulate the bond breaking/forming process, a QM/MM description of the system was needed. To sample at the target level of theory within a moderate amount of computer time (<500,000 CPU hours), the B3LYP functional^71–73 and 6–31G* basis set⁷⁴ (in conjugation with the MM force field) were chosen as the reference ai-QM/MM method for the two test systems in this study. On the other hand, the PM3*/MM model, where the parameters of the standard PM3 method⁷⁵ were recalibrated through force-matching,²³ was used to simulate the reactions under study. We note that despite the same notation, the PM3* model parameters are different between the two reaction systems. For the Menshutkin reaction, the parameters were directly taken from our previous study,²³ whereas for the chorismate mutase reaction, an improved version of the parameter set were used (Table S1).

The setup of the QM/MM MD simulations was overall similar to the classical ones. During the QM/MM MD simulations, the solutes and the substrate were described by the QM method, while the rest of the system (solvent or protein) was described by the force fields used in the classical simulations. The QM/MM-AC method⁵⁹ was used to capture the long-range QM-MM electrostatic interactions. The SHAKE algorithm⁶⁹ was only applied to the MM subsystem, and the integration time step was set to 1 fs. The QM/MM MD simulations were performed using our QM/MM interface QMHub (https://github.com/panxl/qmhub) and a modified version of the SANDER program from the AmberTools20 package.⁷⁰ All DFT/MM calculations were performed using Q-Chem 5.2.⁷⁶

3.C. Umbrella Sampling

To achieve good coverage of all configurations relevant to the reactive process, enhanced sampling MD simulations are needed. Umbrella sampling¹⁴ is a method that can enhance the sampling of the system along one or more predefined reaction coordinates by adding harmonic biasing potentials to restrain the system to the region of interest of the reaction coordinate. Typically, a series of restraint centers ${\xi }_i^0$ are determined to cover the region of interest of the reaction coordinate. Then, for each restraint center ${\xi }_i^0$ (also called window), separate simulations are conducted with the harmonic biasing potential, in the form of,

$$E_{\text{bias},i}(\xi )=k_i{\left(\xi -{\xi }_i^0\right)}^2,$$ (16)

added to the Hamiltonian of the system, where k_i is the force constant of the harmonic potential at window i and ξ refers to the reaction coordinate. In practice, the number and locations of windows and the force constants are determined to ensure sufficient overlapping of the sampled configurations between neighboring windows while minimizing the computational cost. In this study, we also applied the Hamiltonian replica exchange molecular dynamics (HREMD)⁷⁷ technique to accelerate the convergence of the free energy simulation, in which the exchanges of the biasing potentials between the neighboring windows were attempted at a fixed interval of steps set to 100 fs.

To generate the training/validation sets that cover all the relevant configuration space for the reactions, the above-mentioned umbrella sampling technique was used to collect configurations along the predefined reaction coordinates, using the PM3*/MM potential. For both reaction systems, the reaction coordinates were defined as ξ = d_break − d_form, where d_break and d_form were the bond lengths of the breaking and forming bonds, respectively. Specifically, ξ = d_C–Cl −d_C–N for Menshutkin reaction and d_C–O −d_C–C for chorismate mutase reaction (see Figure 2). For each reaction system, a total of 80 windows were evenly distributed with an interval of 0.05Å to cover ξ ranged from −1.975 to 1.975 Å, and the force constants were set to be 150 kcal/mol/Ų for all the windows. For each window, 50 ps simulation was conducted, and the configurations were saved every 0.1 ps, which resulted in 500 configurations. Overall, 40,000 frames were collected for each reaction system, and for each configuration saved, the B3LYP/6–31G*/MM calculation was performed to generate the reference data.

After the ML model was trained, the same umbrella sampling simulations were performed where PM3*/MM was replaced by the MLP/MM or PM3*+ΔMLP/MM model. Multistate Bennett acceptance ratio (MBAR)⁷⁸ method as implemented in the pymbar package (https://github.com/choderalab/pymbar) was used to compute the free energy profile.

4. Results and Discussion

4.A. Energy Conservation During Microcanonical Ensemble Simulations

As a validation of the developed MLP and ΔMLP models, especially, the conservation of the total energy, we first performed 100 ps microcanonical ensemble (NVE) MD simulations for the reactant of the chorismate mutase reaction using the PM3*/MM, MLP/MM, and PM3*+ΔMLP/MM models. The results are presented in Figure 3. Overall, a good conservation of energy was observed, with the energy drifts being 3.96 ± 0.08 × 10⁻³ kcal/mol/ps, −1.81 ± 0.08 × 10⁻³ kcal/mol/ps, and 4.17 ± 0.08 × 10⁻³ kcal/mol/ps for the PM3*/MM, MLP/MM, and PM3*+ΔMLP/MM models, respectively. In addition, we compared the initial 500 fs trajectories with an NVE trajectory using B3LYP/6–31G*/MM under the same initial conditions (Figure S2). It showed that MLP and PM3*+ΔMLP models diverged less from the DFT/MM than the PM3* model.

Figure 3:

Conservation of the total energy in 100ps NVE simulations of the chorismate mutase reaction using A) PM3*, B) MLP, and C) PM3*+ΔMLP models. In each figure, the line shown in orange indicates the drift of energy (see the values mentioned in the main text for each model).

4.B. Overall Free Energy Results

The overall free energy results for the two reactive systems studied are summarized in Table 1. For the aqueous Menshutkin reaction, the B3LYP/MM free energy barrier was calculated to be 15.5 ± 0.1 kcal/mol and the free energy of reaction was −26.9 ± 0.1 kcal/mol, respectively. Despite extensive reparameterization as outlined in Section 3.B, the PM3*/MM model still substantially overestimated the barrier height with a value of 23.3 ± 0.1 kcal/mol. Similarly, the free energy of reaction was overestimated by 18.6 kcal/mol. In contrast, much improved free energy barriers and reaction free energies were obtained with MLP sampling (16.1 ± 0.1 kcal/mol for barrier and −25.9 ± 0.1 kcal/mol for the reaction free energy) and also with PM3*+ΔMLP sampling (15.1 ± 0.1 kcal/mol for barrier and −27.5 ± 0.1 kcal/mol for the reaction free energy), both within 1.0 kcal/mol from the reference B3LYP values.

Table 1:

Free Energy Barriers and Reaction Free Energies for Menshutkin (MEN) and Chorismate Mutase (CM) Reactions^a

	Free Energy Barrier (kcal/mol)				Reaction Free Energy (kcal/mol)
	PM3*	MLPb	PM3* + ΔMLP	B3LYP	PM3*	MLPa	PM3* + ΔMLP	B3LYP
MEN	23.3 ± 0.1	16.1 ± 0.1	15.1 ± 0.1	15.5 ± 0.1	−8.3 ± 0.1	−25.9 ± 0.1	−27.5 ± 0.1	−26.9 ± 0.1
CM	14.9 ± 0.1	13.9 ± 0.1	13.9 ± 0.1	13.6 ± 0.1	−15.5 ± 0.1	−16.3 ± 0.1	−16.6 ± 0.1	−16.8 ± 0.1

^aThe numbers after the plus–minus sign show the uncertainties estimated by the MBAR method.

^bFor the chorismate mutase reaction, the reported results are from the 2nd iteration MLP.

Similar improvements were found for the chorismate mutase reaction. Both MLP and PM3*+ΔMLP simulations reproduced the barrier height of 13.6 ± 0.1 kcal/mol and the free energy of reaction of −16.8 ± 0.1 kcal/mol, both with errors less than 0.5 kcal/mol from the reference B3LYP values. Considering the length of the sampling for each simulation, the three results can be considered to be essentially the same. On the other hand, while remaining excellent, the PM3* model produced a barrier of 14.9 ± 0.1 kcal/mol and a reaction free energy of −15.5 ± 0.1 kcal/mol, which were only 1.3 kcal/mol higher than their corresponding B3LYP values.

We want to note that our standalone MLP and ΔMLP models have not been fully optimized in terms of their computational cost. With the current set of hyperparameters (as specified in Section 2.E), the MLP sampling of the Menshutkin reaction system took about 800 CPU hours, which was 3-fold higher than the PM3* sampling, but it was still 16 times faster than the B3LYP simulation (Table 2). For PM3*+ΔMLP, since both PM3* and ΔMLP energy/force evaluation were performed for each configuration, its cost increased to around 1,300 CPU hours, which was still 10 times faster than the reference B3LYP simulation. It should be noted that, our MLP and ΔMLP models were implemented using the PyTorch framework which supports both CPU and GPU parallelization. However, the timing reported here was only based on CPU calculations.

Table 2:

CPU Time (in 10³h) for the Computation of Free Energy Profiles for Menshutkin (MEN) and Chorismate Mutase (CM) Reactions^a

	PM3*	MLP	PM3* + ΔMLP	B3LYP
MEN	0.3	0.8	1.3	12.4
CM	1.5	11.9b	8.2	379.5

^aUmbrella sampling simulations were performed for 50ps per window for 80 windows. Thus, a total of 4ns umbrella sampling MD simulations were performed for each QM/MM model. The time for recalibrating PM3 is not included, which is ~0.6 × 10³ h and ~0.9 × 10³ h for the Menshutkin and chorismate mutase reactions, respectively.

^bIncluding time from two iterations. Thus, a total of 100ps for each umbrella sampling window.

For the chorismate mutase system, the MLP sampling took 11,900 CPU hours, which was about 8 times slower than the PM3* sampling. The higher MLP/PM3* timing ratio arose largely from two rounds of MLP training performed to obtain a stable MLP model around the reaction transition state region (see Section 4.D for details). In comparison to the 379,500 CPU hours for the B3LYP sampling, the MLP sampling offered a 32-fold speedup. With PM3*+ΔMLP, only one round of ML model training was needed, which led to a net 8,200 CPU hours for the sampling. This was 46 times faster than the reference B3LYP simulation.

Below we go over each reactive system in more detail, and show how such speedups were obtained.

4.C. Menshutkin Reaction

For Menshutkin reaction, the standard PM3 model (marked red), which is shown in the bottom panels in Fig. 4, deviates substantially from the reference B3LYP values. In terms of the root-mean-square error (RMSE), PM3 energy differed by 6.24 kcal/mol, while the corresponding force error on QM atoms was 12.90 kcal/mol/Å (Table S2). Similarly, the mean-unsigned-error (MUE) for energy was 5.00 kcal/mol and for force was 9.62 kcal/mol/Å. The RMS errors in the QM electrostatic potential and field on MM atom position were substantially smaller at 1.33 kcal/mol/e and 0.95 kcal/mol/Å/e, respectively, and their corresponding MUE values were 0.67 kcal/mol/e and 0.24 kcal/mol/Å/e. The maximum errors (MAXs) were largest for PM3 among the test QM/MM models: 21.55 kcal/mol for energy, 54.32 kcal/mol/Å for force, 38.41 kcal/mol/e for electrostatic potential and 49.39 kcal/mol/Å/e for the field, respectively.

Figure 4:

Accuracy of MLP (top), PM3*+ΔMLP (middle), PM3 and PM3* (bottom) energy, forces, electrostatic potential (ϕ) and electric field ($\mathcal{E}$) for the 2,000 testing configurations for aqueous Menshutkin reaction. In each figure, the reference values are obtained from the B3LYP/6–31G*/MM calculations. The root-mean-square error (RMSE) value is also shown for each method.

Through our automated reparameterization (which placed a larger weight on the force errors), the PM3* model (marked blue in bottom panels in Figure 4) substantially reduced the force error to 4.42 kcal/mol/Å for RMSE and 3.21 kcal/mol/Å for MUE, respectively. Meanwhile, the PM3* energy error remained at 6.38 kcal/mol for RMSD and 5.49 kcal/mol for MUE, which was evident from Fig. 5A with a broad distribution of the energy difference between the PM3* and B3LYP models.

Figure 5:

Distribution of high-level (B3LYP/6–31G*) and low-level (PM3*, MLP, and PM3*+ΔMLP energy differences for configuration collected from B3LYP/MM MD trajectories (blue) or low-level MD trajectories (orange) of the aqueous Menshutkin reaction.

When ΔMLP was trained, it was designed to reproduce the difference between the PM3* and B3LYP potentials. As shown in the middle panels in Fig. 4, the energy error of the combined PM3*+ΔMLP model reached 0.78 kcal/mol for RMSE and 0.62 kcal/mol for MUE, which were below 1.0 kcal/mol error and lowest among the models tested. The reduction in the force error was more impressive, reaching a RMSE value of 0.79 kcal/mol/Å and a MUE value of 0.53 kcal/mol/Å. A moderate but more systematic improvement of the ESP and field values on MM atom position could also be noticeable. As a consequence, the distribution of PM3*+ΔMLP and B3LYP energy differences as shown in Fig. 5C became much narrower. The corresponding standard deviation of energy difference was 0.7 kcal/mol, which fell within the 1.7–2.5 kcal/mol threshold for a lower-level theory to accurately reproduce the sampling from a high-level model.^79–81

Alternatively, MLP can be directly trained to reproduce the B3LYP energy/force as well as the QM ESP and field on MM atom positions. As shown in the top panel of Fig. 4, a standalone MLP can reproduce the B3LYP energy with 1.03 kcal/mol RMSE and 0.79 kcal/mol MUE and the forces with 1.17 kcal/mol/Å RMSE and 0.75 kcal/mol/Å MUE, respectively, which were only slightly worse than PM3*+ΔMLP. Therefore, it led to a narrow distribution of energy differences in Fig. 5B, with a slightly larger standard deviation of 1.5 kcal/mol than that of PM3*+ΔMLP.

Due to their capability to reproduce B3LYP energy and forces, both MLP and PM3*+ΔMLP models led to proper sampling of the pathway (shown in Fig. 6A) and thus accurate reaction free energy profile for the Menshutkin reaction. In addition, we expect that the conformational space sampled by both models would overlap very well with that of B3LYP.

Figure 6:

(A) Sampled pathway and (B) potential of mean force for the aqueous Menshutkin reaction based on umbrella sampling using PM3*+ΔMLP and MLP potentials in comparison to PM3* and B3LYP/6–31G* results. The pathways are represented by the average bond lengths for each of the 80 windows in the umbrella sampling simulations (shown in Figure S2). The stars show the locations of the transition states on the pathways.

4.D. Chorismate Mutase Catalysis

Similar to the case of Menshutkin reaction in an aqueous solution, the chorismate mutase system was poorly described by the PM3 model, with an energy error of 12.08 kcal/mol and a force error of 13.59 kcal/mol/Å shown in the bottom panel of Fig. 7. A substantial improvement to the PM3 model was achieved through reparameterization, which reduced the energy error to 2.25 kcal/mol and the force error to 4.88 kcal/mol/Å. The improvement of the PM3* model is also noticeable from the comparison of MUE and MAXE values in Table S3.

Figure 7:

Accuracy of MLP (top), PM3*+ΔMLP (middle), PM3 and PM3* (bottom) energy and forces for the 2,000 testing configurations for the chorismate mutase reaction.

The modeling of the chorismate mutase system was brought within 1.0 kcal/mol with PM3*+ΔMLP, whose RMS energy error was found to be 0.69 kcal/mol and RMS force error was 0.88 kcal/mol/Å. This was also evident in the change of the wide PM3*–B3LYP energy distribution in Fig. 8A to the narrow peaks for ΔMLP-corrected model in Fig. 8D. In addition, we point out the substantial reduction of MAXE for the forces from 33.29 kcal/mol/Å for PM3* to 7.18 kcal/mol/Å for PM3*+ΔMLP, suggesting very robust improvement of the accuracy of the PM3*+ΔMLP model.

Figure 8:

Distribution of high-level (B3LYP/6–31G*) and low-level (PM3*, MLP, and PM3*+ΔMLP energy differences for configuration collected from B3LYP/MM MD trajectories (blue) or low-level MD trajectories (orange) of chorismate mutase reaction.

With standalone MLP, one round of training using the configuration collected from the PM3*/MM MD trajectory did yield a good agreement with the target B3LYP/MM energy (RMSE: 0.99 kcal/mol and MUE: 0.78 kcal/mol). However, the force error was found to be 1.84 kcal/mol/Å for RMSE and 1.36 kcal/mol/Å for MUE. The MAXE of 14.78 kcal/mol/Å was also noticeably large. With such larger errors in the force, we observed some level of instability of the MD simulations using this MLP, especially near the transition state region. To address this issue, we constructed an expanded pool of configurations combining the configurations used in the first MLP training and the configurations selected from this round of MLP sampling, and performed a second round of MLP training using the expanded configuration pool. Previously, Pu et al. has shown that the iterative parameterization of the se-QM model can systematically improve the accuracy of the (reparameterized) QM model³⁸ and the present second round of MLP training can be considered to be a second iteration of the MLP model training. The resultant MLP model, referred here to as MLP(2nd), further reduced the RMS force error to 1.10 kcal/mol/Å (and MUE to 0.81 kcal/mol/Å), which was accompanied by a RMS energy error of 0.73 kcal/mol and MUE of 0.58 kcal/mol, respectively. More impressively, the MAX force error was reduced to 8.32 kcal/mol/Å, which was only 1.1 kcal/mol/Å larger than the MAX force error of PM3*+ΔMLP. The MLP(2nd) model also demonstrated a much narrower MLP(2nd)–B3LYP energy distribution than the one for MLP(1st)–B3LYP one as shown in Fig. 8.

These MLP(2nd) and PM3*+ΔMLP models yielded an accurate sampling of the reaction pathway shown in Fig. 9A. This is in contrast to the PM3* model, whose sampling was shown in Fig. 8d of Ref. 23 to deviate substantially from the B3LYP pathway between the transition and the product. As a result of the accurate energetics and sampling, both MLP(2nd) and PM3*+ΔMLP models produced free energy profiles of the chorismate mutase reaction that matched the B3LYP profile in Fig. 9B.

Figure 9:

(A) Sampled pathway and (B) potential of mean force for the chorismate mutase reaction based on umbrella sampling using PM3*, PM3*+ΔMLP and MLP (the 2nd iteration) potentials in comparison to B3LYP/6–31G* results. The pathways are represented by the average bond lengths for each of the 80 windows in the umbrella sampling simulations (in Figure S3). The stars show the locations of the transition states on the pathways.

5. Conclusions

Inspired by the works from Yang, York, Gastegger, Riniker and coworkers, we explored the construction of system-specific machine-learning potentials (MLPs) to accelerate the generation of ab initio-quality QM/MM free energy profiles of enzymatic reactions. A couple of advances were made in this work:

• The deep-potential of E and coworkers (i.e., DeepPot-SE) was combined with our QM/MM-AC electrostatic model to provide descriptors for a QM reactive region embedded in an enzyme or solution environment (described by MM point charges).

• A reparameterized semi-empirical quantum model, PM3*, was utilized to generate the configurations for training the atomistic machine-learning potentials. The deep-potential and QM/MM-AC descriptors of these configurations were then used to train the embedding and fitting neural networks to provide MLPs or ΔMLPs to propagate MD trajectories of condensed-phase reactive systems.

• The ΔMLP model, when combined with the PM3* model in umbrella simulations, produced accurate reaction free energies and barrier heights of two model reactions (aqueous Menshutkin reaction and chorismate mutase reaction) within 1.0 kcal/mol. A 10–fold or 46–fold reduction in CPU time, respectively, was achieved in comparison to reference B3LYP/6–31G*/MM free energy simulations.

• The standalone MLP, which sought to directly reproduce the B3LYP energy/force on QM atoms and fitted electrostatic potentials and fields on MM atoms, could reproduce the aqueous Menshutkin free energy profile with a 16-fold CPU timing saving. For the chorismate mutase reaction, however, two rounds of MLP training was found to be necessary to ensure stable MD trajectories. With a larger QM region in chorismate mutase reaction, the timing saving was 32–fold.

On the other hand, several limitations need to be addressed in the future:

• The hyperparameters for our MLP and ΔMLP (which were fixed in this work) are needed to be systematically explored to find an optimal set of hyperparameters for each machine learning potential for efficiency and accuracy.

• In our current protocol, the samples from one or more iterations were all labelled using the high-level method. Active learning or concurrent learning^44,55 as well as a more robust data clustering algorithm can be adopted to reduce the cost of both data labelling and training.

• Our protocol for reparameterizing semi-empirical QM models and for training MLPs needs to be tested on typical enzyme reactions, where the computational savings are expected to be even more substantial with a larger QM region.

• Given the generality of our QM/MM electrostatics treatment, a MLP trained with the wild-type enzyme should be applicable to the mutants. The transferability of the trained MLPs has yet to be tested and verified.

• More accurate ab initio QM/MM models beyond B3LYP/6–31G*/MM should be adopted as the labelling method for our MLP and ΔMLP training.

Work in these directions are currently being pursued.