Multireference Generalization of the Weighted Thermodynamic Perturbation Method

Abstract

We describe the generalized weighted thermodynamic perturbation (gwTP) method for estimating the free energy surface of an expensive “high-level” potential energy function from the umbrella sampling performed with multiple inexpensive “low-level” reference potentials. The gwTP method is a generalization of the weighted thermodynamic perturbation (wTP) method developed by Li and co-workers [J. Chem. Theory Comput. 2018, 14, 5583–5596] that uses a single “low-level” reference potential. The gwTP method offers new possibilities in model design whereby the sampling generated from several low-level potentials may be combined (e.g., specific reaction parameter models that might have variable accuracy at different stages of a multistep reaction). The gwTP method is especially well suited for use with machine learning potentials (MLPs) that are trained against computationally expensive ab initio quantum mechanical/molecular mechanical (QM/MM) energies and forces using active learning procedures that naturally produce multiple distinct neural network potentials. Simulations can be performed with greater sampling using the fast MLPs and then corrected to the ab initio level using gwTP. The capabilities of the gwTP method are demonstrated by creating reference potentials based on the MNDO/d and DFTB2/MIO semiempirical models supplemented with the “range-corrected deep potential” (DPRc). The DPRc parameters are trained to ab initio QM/MM data, and the potentials are used to calculate the free energy surface of stepwise mechanisms for nonenzymatic RNA 2′-O-transesterification model reactions. The extended sampling made possible by the reference potentials allows one to identify unequilibrated portions of the simulations that are not always evident from the short time scale commonly used with ab initio QM/MM potentials. We show that the reference potential approach can yield more accurate ab initio free energy predictions than the wTP method or what can be reasonably afforded from explicit ab initio QM/MM sampling.

Graphical Abstract

Introduction

The study of chemical mechanisms of complex systems¹ has broad application to areas of heterogeneous, homogeneous and enzyme catalysis,² synthetic chemistry³ and chemical education.⁴ Chemical mechanism is often explored through calculation of a free energy surface (FES) in a reduced set of collective variables referred to as reaction coordinates.⁵ The FES is used to characterize the location and free energy values of competitive reactive pathways connecting the reactant and product states. Numerous methods have been developed to calculate free energy surfaces from combined quantum mechanical/molecular mechanical (QM/MM) simulation. The approaches have been classified into 3 categories:⁶ methods based on the work of Jarzynski⁷ which analyze nonequilibrium statistics,^8–10 methods that analyze equilibrium statistics generated from biased simulations (umbrella sampling),^11–14 and methods which introduce and integrate auxiliary degrees of freedom, such as λ-dynamics^15–18 and metadynamics.^19,20

Umbrella sampling is a technique that applies an artificial bias to improve sampling efficiency. That is to say, a bias can be chosen to enhance the sampling in the high energy regions of the FES that would otherwise be difficult to sample on a reasonable time scale. If the FES was known a priori, then uniform sampling could be achieved by introducing a bias that exactly canceled the free energy. In practice, one instead performs a series of biased simulations that restrain the sampling to a particular region of reaction coordinates. The umbrella potentials are often chosen to be uncoupled harmonic oscillators, and the biased simulations differ by the choice of biasing force constants and equilibrium positions; however, the methods presented in the present work are general and do not assume a specific functional form for the biasing potential. For relatively simple mechanisms, the umbrella potentials can be chosen to span a predetermined range of reaction coordinates; however, methods have been developed to adaptively focus the sampling^21–30 and locate minimum free energy pathways.^31–38 There are several analysis techniques that can be used to reconstruct the FES from the biased simulations, including: the weighted histogram analysis method (WHAM),^39,40 and unbinned weighted histogram analysis method (UWHAM),^41,42 the multistate Bennett acceptance ratio method (MBAR),⁴³ the variational free energy profile (vFEP) method,^44–46 and umbrella integration.^47–49

A series of recent works have introduced the weighted thermodynamic perturbation (wTP) method.^50–55 This method estimates the FES of an expensive target potential from the umbrella sampling performed with a cost-effective reference potential. One can view the wTP method as being analogous to the reference potential method encountered in alchemical free energy applications,^56–59 and the accuracy of the wTP method similarly relies on close agreement between the target and reference potentials.⁶⁰ By performing simulations with an inexpensive reference potential, much greater sampling can be achieved through longer and/or more numerous independent simulations. Therefore, the wTP method is complimentary to other approaches that extend the sampling, such as multiple timestep integration.⁶¹ Analysis of the extended simulations is useful for identifying unequilibrated sampling, and extended simulations become necessary in situations where multiple structural conformations are thermally accessible.

In the present work, we describe the generalized weighted thermodynamic perturbation (gwTP) method. The gwTP method estimates the FES of an expensive target potential from the aggregate umbrella sampling performed with multiple reference potentials. By combining the sampling, one no longer relies on a single reference potential to obtain good phase space overlap with the target potential throughout the entire FES.^59,62,63 Instead, only one of the reference potentials must have good phase space overlap in each region of the FES. If all of the potentials poorly overlapped with the target, then no reweighting strategy is likely to succeed from a limited number of target potential evaluations. Thus, it is important to have a quantitative measure of agreement between the reference and target potentials to gauge the reliability of the approach. The “reweighting entropy” is an index developed specifically for this purpose,⁶⁴ and we shall discuss it in more detail in subsequent sections. We were motivated to develop the gwTP method by the growing interest in supplementing semiempirical QM/MM Hamiltonians with machine learning potentials (MLP) that are trained to reproduce ab initio QM/MM energies and forces.^51,65–75 If one had a priori confidence in the trained MLP, then one obviously can use it to estimate free energy surfaces without additional correction; however, if one is applying a MLP to a system that was not explicitly included in the training, then a MLP-corrected model naturally serves as an excellent reference potential to estimate the ab initio FES from reweighting. Furthermore, the active learning procedure used to train MLPs produces several neural network parameter sets (several potentials),^76,77 and the gwTP method provides a means to estimate the ab initio FES from the aggregate sampling performed with each potential. As multiple independent simulation runs are typically performed in order to produce robust averages and error estimates, the use of different reference potentials can often be accommodated for no added computational cost. The gwTP framework further offers new possibilities in model design. Because the gwTP method can combine the sampling from several “specific reaction parameter” (SRP) models,^78–80 one can design several potentials that specialize their training to reproduce different chemical events embodied within a complex chemical mechanism, such as the general acid, general base, and phosphoryl transfer steps in RNA cleavage reactions.⁸¹ Alternatively, one could parametrize several potentials that are trained to model the 2′-O-transesterification reaction in different RNA environments, and then perform sampling with each potential when exploring the mechanism in an RNA environment not included in the training. In the above discussion, the trained MLPs are used to perform the sampling; however, MLPs can also be utilized in another strategy whereby an uncorrected reference potential is sampled, and a MLP is trained to a small number of configurations to estimate the target potential energies required for thermodynamic perturbation.^68,82–84

We apply the gwTP method to two non-enzymatic RNA 2′-O-transesterification model reactions shown in Figure 1. The mechanism of closely related non-enzymatic phosphoryl transfer reactions have been explored with linear free energy relationships^85,86 and through the calculation of free energy surfaces.^87,88 These previous works found that the pathway is correlated to the pK_a of the leaving group. Leaving groups with a pK_a < 11 (“enhanced” leaving groups) proceed through a concerted mechanism containing a single, “early” (ξ_PT < 0) transition state, whereas leaving groups with a pK_a > 12 (“poor” leaving groups) proceed through two distinct barriers separated by a minimum. The methoxide and ethoxide leaving groups shown in Figure 1 are “poor” leaving groups, so the FESs are expected to contain an early transition state characterized by partial formation of the O2′-P bond and a second rate-controlling transition state characterized by partial cleavage of the O5′-P bond.

Figure 1:

Non-enzymatic RNA 2′-O-transesterification model reactions explored in this work (atomic numbering of 2′ and 5′ positions in the model systems reflect their analogous positions in RNA). (a) The ethylene phosphate model reaction with a methoxide leaving group. (b) The native model reaction used in ref. 88 with an ethoxide leaving group. The reaction coordinate used in this work is ξ_PT = R_P-O5’ − R_P-O2’.

The ethylene phosphate reaction (Figure 1a) is used to demonstrate the gwTP method’s ability to reconstruct a PBE0/6–31G* target FES from the sampling obtained from 4 disparate reference potentials. For the purpose of providing a stringent test case for demonstration, the 4 reference potentials were specifically designed such that none of them accurately reproduce the target FES throughout the entire range of ξ_PT values. The reference potentials use the MNDO/d semiempirical Hamiltonian supplemented with a range-corrected deep potential^76,88 (DPRc) MLP. We trained 4 ad hoc MNDO/d QM/MM+DPRc potentials using different target data to yield significantly different reference potentials. Each MNDO/d QM/MM+DPRc potential was trained to reproduce the PBE0/6–31G* target energies and forces at different stages along the reaction coordinate (i.e., different subdomains of the ξ_PT reaction coordinate) such that the reference potentials disagree with each other and the target in the regions they were not trained. We will show that the wTP estimates of the target FES (based on a single reference potential) are inaccurate in the untrained regions, whereas the gwTP analysis of the aggregate sampling from all reference potentials accurately reproduces the PBE0/6–31G* surface.

The native model reaction (Figure 1b) is used as a case study to emphasize the benefits offered by the reference potential approach. This reaction was one of 6 nonenzymatic models used to parametrize the DFTB2/MIO QM/MM+DPRc potentials in ref. 88. Free energy surfaces of the PBE0/6–31G* QM/MM, DFTB2/MIO QM/MM, and DFTB2/MIO QM/MM+DPRc potentials were obtained from identical simulation protocols. The DPRc correction was shown to improve the comparison with the ab initio results; however, some discrepancies between the PBE0/6–31G* QM/MM and DFTB2/MIO QM/MM+DPRc native model profiles remained. In the present work, we revisit the calculation of the native model FES. We show that gwTP analysis of the DFTB2/MIO QM/MM+DPRc sampling performed in ref. 88 results in excellent agreement with the PBE0/6–31G* FES. Furthermore, we extend the DFTB2/MIO QM/MM+DPRc sampling from 100 ps/window to 1.2 ns/window of aggregate sampling – an amount of sampling well beyond what is routinely affordable using ab initio QM/MM simulation. The extended sampling made possible by the reference potential approach allows us to demonstrate that the native model reaction ab initio QM/MM free energy profiles presented in ref. 88 are not converged. We show that a better estimate of the ab initio FES can be made using reference potentials with extended sampling than what can reasonably be afforded from ab initio QM/MM simulation.

Theory

Multistate Bennett Acceptance Ratio (MBAR) Method

In this section, we introduce our notation by reviewing the MBAR approach for calculating free energy surfaces. The description also serves to aid the reader’s understanding of the differences between the MBAR, wTP, and gwTP methods. The FES of a system composed of 3N atomic coordinates r is typically expressed in a reduced set of relevant collective variables ξ(r), called reaction coordinates. Umbrella sampling is used to enhance the statistics in the high-energy regions of the FES by performing a series of K_h simulations of biased potential energy (PE) functions U_hk(r) that differ only by varying the bias W_hk(r) applied to the unbiased potential U_h(r).

$$U_{hk}(\text{r})=U_h(\text{r})+W_{hk}(\text{r})$$ (1)

The subscripts h and k denote the unbiased PE function and biasing potential, respectively, and one can interpret (hk) as a combined-index of biased states. The goal is to use MBAR to analyze the biased sampling performed with U_hk(r) to estimate the unbiased FES of U_h(r). One could write the MBAR expressions without introducing the h subscript because only a single unbiased PE function is ever considered; however, the utility of its inclusion will become apparent when describing the wTP and gwTP methods in the ensuing sections. A simulation performed with U_hk(r) at temperature T_hk produces an ensemble of N_hk structures. The 3N array of atomic coordinates of sample n in the ensemble of state hk is denoted r_hkn. Our description of the methods shall write the PE and biasing potential in reduced energy units: u_hk(r_h′_k′_n) = β_h′_k′U_hk(r_h′_k′_n), u_h(r_h′_k′_n) = β_h′_k′U_h(r_h′_k′_n), and w_hk(r_h′_k′_n) = β_h′_k′W_hk(r_h′_k′_n) where β_hk = (k_BT_hk)⁻¹ and k_B is the Boltzmann constant.

The unbiased free energy surface F_h(ξ) is, to within an additive constant, related to the probability of observing a sample at ξ, ρ_h(ξ).

$$F_h(\xi )=-{\beta }^{-1}\text{ln}{\rho }_h(\xi )$$ (2)

In practice, the probability is approximated by discretizing ξ into histogram bins consisting of centers ξ_m and widths Δξ. Let δ(ξ_m − ξ(r_hkn)) denote the indicator function, which is 1 only if the sample is contained within bin m.

$$\delta \left({\xi }_m-\xi \left({\text{r}}_{hkn}\right)\right)=\{\begin{array}{ll}1,\hfill & \text{if}-\Delta \xi /2<{\xi }_m-\xi \left({\text{r}}_{hkn}\right)<\Delta \xi /2\hfill \\ 0,\hfill & \text{otherwise}\hfill \end{array}$$ (3)

The probability of observing a sample in bin m is then given by eq. 4, where ω_h(r_hkn) is the weight of sample r_hkn (see eq. 6).

$${\rho }_h\left({\xi }_m\right)=\sum _{k=1}^{K_h}\sum _{n=1}^{N_{hk}}\delta \left({\xi }_m-\xi \left({\text{r}}_{hkn}\right)\right){\omega }_h\left({\text{r}}_{hkn}\right)$$ (4)

The MBAR expression for the FES of u_h(r) within histogram m is given by inserting eq. 4 into eq. 2.

$$F_h\left({\xi }_m\right)=-{\beta }^{-1}\text{ln}\sum _{k=1}^{K_h}\sum _{n=1}^{N_{hk}}\delta \left({\xi }_m-\xi \left({\text{r}}_{hkn}\right)\right){\omega }_h\left({\text{r}}_{hkn}\right)$$ (5)

If the sampling was performed with an unbiased PE, then the sample weights would be uniform and eqs. 4–5 would merely count the fraction of samples observed within each bin. The situation is more complicated when umbrella sampling is performed because the distributions are skewed by the artificial bias. Reference 43 developed the equations to reweight trajectories using MBAR, and ref. 50 specialized those equations to combine the biased sampling in FES applications. The result of these previous developments is shown in eq. 6.

$$\begin{gathered} {\omega }_h\left({\text{r}}_{hkn}\right)=\frac{\text{exp}\left[f_h-u_h\left({\text{r}}_{hkn}\right)\right]}{{\sum }_{k^{\prime }=1}^{K_h}{N}_{h{k}^{\prime }}\text{exp}\left[f_{h{k}^{\prime }}-u_{h{k}^{\prime }}\left({\text{r}}_{hkn}\right)\right]} \\ =\frac{\text{exp}\left(f_h\right)}{{\sum }_{k^{\prime }=1}^{K_h}{N}_{h{k}^{\prime }}\text{exp}\left[f_{h{k}^{\prime }}-w_{h{k}^{\prime }}\left({\text{r}}_{hkn}\right)\right]} \end{gathered}$$ (6)

The f_hk values are the free energies (in reduced energy units) of the K_h biased states obtained from solution to the MBAR/UWHAM equations.

$$f_{hl}=-\text{ln}\sum _{k=1}^{K_h}\sum _{n=1}^{N_{hk}}\frac{\text{exp}\left[-u_{hl}\left({\text{r}}_{hkn}\right)\right]}{{\sum }_{k^{\prime }=1}^{K_h}{N}_{h{k}^{\prime }}\text{exp}\left[f_{h{k}^{\prime }}-u_{h{k}^{\prime }}\left({\text{r}}_{hkn}\right)\right]},\forall l=1,\cdots ,K_h$$ (7)

The f_h value is the free energy of unbiased PE u_h(r). The numerator of eq. 6 includes the factor $\text{exp}\left(f_h\right)=Q_h^{-1}$; this factor is the inverse of the unbiased state’s partition function, which formally normalizes the weights.^43,50 In practice, the normalization constant shifts the entire FES by a constant.

$$f_h=-\text{ln}\sum _{k=1}^{K_h}\sum _{n=1}^{N_{hk}}\frac{\text{exp}\left[-u_h\left({\text{r}}_{hkn}\right)\right]}{{\sum }_{k^{\prime }=1}^{K_h}{N}_{h{k}^{\prime }}\text{exp}\left[f_{h{k}^{\prime }}-u_{h{k}^{\prime }}\left({\text{r}}_{hkn}\right)\right]}$$ (8)

Weighted Thermodynamic Perturbation (wTP) Method

The wTP method has been developed over a series of recent articles.^50–55 The purpose of the method is to predict the FES of an expensive “target” PE function from the umbrella sampling performed with an inexpensive “reference” PE function. The unbiased target u_t(r_hkn) and reference u_h(r_hkn) PE functions of each sample are evaluated, and the energy differences Δu_th(r_hkn) = u_t(r_hkn) − u_h(r_hkn) are used to predict the target FES. In this sense, the wTP method is analogous to a “reference potential method” often encountered in alchemical free energy applications.

The wTP FES estimate of the unbiased target potential is given by eqs. 9–10.

$$F_t\left({\xi }_m\right)=-{\beta }^{-1}\text{ln}\sum _{k=1}^{K_h}\sum _{n=1}^{N_{hk}}\delta \left({\xi }_m-\xi \left({\text{r}}_{hkn}\right)\right){\omega }_t\left({\text{r}}_{hkn}\right)$$ (9)

$${\omega }_t\left({\text{r}}_{hkn}\right)=\frac{\text{exp}\left[f_t-u_t\left({\text{r}}_{hkn}\right)\right]}{{\sum }_{k^{\prime }=1}^{K_h}{N}_{h{k}^{\prime }}\text{exp}\left[f_{h{k}^{\prime }}-u_{h{k}^{\prime }}\left({\text{r}}_{hkn}\right)\right]}$$ (10)

The f_t quantity is the free energy of the unbiased target potential.

$$f_t=-\text{ln}\sum _{k=1}^{K_h}\sum _{n=1}^{N_{hk}}\frac{\text{exp}\left[-u_t\left({\text{r}}_{hkn}\right)\right]}{{\sum }_{k^{\prime }=1}^{K_h}{N}_{h{k}^{\prime }}\text{exp}\left[f_{h{k}^{\prime }}-u_{h{k}^{\prime }}\left({\text{r}}_{hkn}\right)\right]}$$ (11)

Multiplying the numerator and denominator of eq. 10 by exp[u_h(r_hkn)] allows one to rewrite the expression in terms of the energy difference.

$$\begin{gathered} {\omega }_t\left({\text{r}}_{hkn}\right)=\frac{\text{exp}\left[f_t-\Delta u_{th}\left({\text{r}}_{hkn}\right)\right]}{{\sum }_{k^{\prime }=1}^{K_h}{N}_{h{k}^{\prime }}\text{exp}\left[f_{h{k}^{\prime }}-w_{h{k}^{\prime }}\left({\text{r}}_{hkn}\right)\right]} \\ ={\omega }_h\left({\text{r}}_{hkn}\right)\text{exp}\left[-\Delta u_{th}\left({\text{r}}_{hkn}\right)\right]\text{exp}\left(f_t-f_h\right) \end{gathered}$$ (12)

The second line of eq. 12 suggests that one can interpret the wTP method as effectively performing “exponential averaging” between the reference and target potentials in each histogram bin upon reweighting the biased simulations. The exp(f_t − f_h) factor is a ratio of partition functions that merely shifts the FES by a constant.

Generalized Weighted Thermodynamic Perturbation (gwTP) Method

The gwTP method extends the wTP approach by considering situations where umbrella sampling has been performed with N_PE reference potentials. We place no restrictions on the relationship between the sets of biasing potentials used to enhance the sampling of each reference potential; that is, K_h does not need to be the same as any other K_h′, and w_hk(r) does not need to be the same as any other w_h′_k′(r). It is for this precise reason why the biasing potentials are written with both h and k subscripts. With this freedom, there are a total of $N_{\text{sim }}={\sum }_{h=1}^{N_{\text{PE}}}{K}_h$ biased states, and expressions which sum all biased states must now be written either as a double summation ${\sum }_{h=1}^{N_{\text{PE}}}\sum {}_{k=1}^{K_h}$ or by viewing hk as a combined-index of biased simulations $\sum {}_{(hk)=1}^{N_{\text{sim}}}$. The MBAR/UWHAM equations for the solution of the N_sim biased free energies is shown in eq. 13.

$$f_{il}=-\text{ln}\sum _{h=1}^{N_{\text{PE}}}\sum _{k=1}^{K_h}\sum _{n=1}^{N_{hk}}\frac{\text{exp}\left[-u_{il}\left({\text{r}}_{hkn}\right)\right]}{{\sum }_{h^{\prime }=1}^{N_{\text{PE}}}{\sum }_{k^{\prime }=1}^{K_h}{N}_{h^{\prime }{k}^{\prime }}\text{exp}\left[f_{h^{\prime }{k}^{\prime }}-u_{h^{\prime }{k}^{\prime }}\left({\text{r}}_{hkn}\right)\right]},\forall (il)=1,\cdots ,N_{\text{sim}}$$ (13)

The target potential free energy (eq. 14) and the target FES (eq. 15) are similarly rewritten to account for the additional umbrella sampling.

$$f_t=-\text{ln}\sum _{h=1}^{N_{\text{PE}}}\sum _{k=1}^{K_h}\sum _{n=1}^{N_{hk}}\frac{\text{exp}\left[-u_t\left({\text{r}}_{hkn}\right)\right]}{{\sum }_{h^{\prime }=1}^{N_{\text{PE}}}{\sum }_{k^{\prime }=1}^{K_h}{N}_{h^{\prime }{k}^{\prime }}\text{exp}\left[f_{h^{\prime }{k}^{\prime }}-u_{h^{\prime }{k}^{\prime }}\left({\text{r}}_{hkn}\right)\right]}$$ (14)

$$F_t\left({\xi }_m\right)=-{\beta }^{-1}\text{ln}\sum _{h=1}^{N_{\text{PE}}}\sum _{k=1}^{K_h}\sum _{n=1}^{N_{hk}}\delta \left({\xi }_m-\xi \left({\text{r}}_{hkn}\right)\right){\omega }_t\left({\text{r}}_{hkn}\right)$$ (15)

All that remains is to generalize the wTP expression for the target potential sample weights (eq. 12). As previously stated, the wTP method can be interpreted as a weighted exponential averaging procedure within each histogram bin, so it may be perplexing how this could be adapted to treat multiple reference potentials because exponential averaging has traditionally been applied to the calculation of free energy differences between pairs of potentials. Our approach is to reinterpret the biased potentials as using a single reference (the “selected reference”) perturbed by an effective bias that accounts for the difference in unbiased energies. This is more clearly described by eq. 16 which re-expresses the biased PE u_h′_k′(r) in terms of the selected reference u_h(r) and the effective bias w_k′(r) + Δu_h′_h(r), where Δu_h′_h(r) = u_h′(r) − u_h(r).

$$\begin{gathered} u_{h^{\prime }{k}^{\prime }}\left({\text{r}}_{hkn}\right)=u_{h^{\prime }}\left({\text{r}}_{hkn}\right)+w_{k^{\prime }}\left({\text{r}}_{hkn}\right) \\ =u_h\left({\text{r}}_{hkn}\right)+\left[w_{k^{\prime }}\left({\text{r}}_{hkn}\right)+\Delta u_{h^{\prime }h}\left({\text{r}}_{hkn}\right)\right] \end{gathered}$$ (16)

The decision of which potential to select as the reference is arbitrary; it has no effect on the predicted FES. Our convention is to select the potential which produced the sample. That is to say, the sample r_hkn is a member of the ensemble generated from u_hk(r), whose reference potential is u_h(r), thus we choose u_h(r) to be the selected reference. The gwTP expression for the sample weights (eq. 17) differs from eq. 12 only by considering the additional simulated states and by replacing the biasing potential with the effective bias.

$${\omega }_t\left({\text{r}}_{hkn}\right)=\frac{\text{exp}\left[f_t-\Delta u_{th}\left({\text{r}}_{hkn}\right)\right]}{{\sum }_{h^{\prime }=1}^{N_{\text{PE}}}{\sum }_{k^{\prime }=1}^{N_{h^{\prime }}}{N}_{h^{\prime }{k}^{\prime }}\text{exp}\left[f_{h^{\prime }{k}^{\prime }}-w_{h^{\prime }{k}^{\prime }}\left({\text{r}}_{hkn}\right)-\Delta u_{h^{\prime }h}\left({\text{r}}_{hkn}\right)\right]}$$ (17)

Multiplying the numerator and denominator by exp[−u_h(r_hkn)] yields an expression that illustrates more clearly that the weights are independent of the selected reference.

$${\omega }_t\left({\text{r}}_{hkn}\right)=\frac{\text{exp}\left[f_t-u_t\left({\text{r}}_{hkn}\right)\right]}{{\sum }_{h^{\prime }=1}^{N_{\text{PE}}}{\sum }_{k^{\prime }=1}^{N_{h^{\prime }}}{N}_{h^{\prime }{k}^{\prime }}\text{exp}\left[f_{h^{\prime }{k}^{\prime }}-u_{h^{\prime }{k}^{\prime }}\left({\text{r}}_{hkn}\right)\right]}$$ (18)

The weights shown in eqs. 6, 10, and 18 are very similar because they are all specialized forms of the general expressions for the MBAR weights developed by Shirts and Chodera.⁴³ In their work, an ensemble average of a target thermodynamic state is expressed using the samples obtained from a collection of explicitly-simulated states (which may or may not include the target state). To apply the MBAR approach in a novel way, one must define the target and sampled potentials in the context of the new application and describe how the weights are used. In the context of umbrella sampling, the target state corresponds to an unbiased potential, the sampled states are the biased simulations, and the weights are used to approximate the spatial distribution of the density (eq. 4). The methods described in the present work differ only in their choice of target and sampled potentials.

The reweighting entropy is a useful index to gauge the reliability of a calculated FES.⁶⁴ It can be interpreted as being a measure of “flatness” in the sample weights, such that it is 1 when the distribution of weights in a bin are uniform, and it approaches 0 when the sum of weights is dominated by only a few samples. The gwTP expression for the reweighting entropy is given by eqs. 19–20.

$${𝒮}_t\left({\xi }_m\right)=-\frac{{\sum }_{h=1}^{N_{\text{PE}}}{\sum }_{k=1}^{K_h}{\sum }_{n=1}^{N_{hk}}\delta \left({\xi }_m-\xi \left({\text{r}}_{hkn}\right)\right)\frac{{\omega }_t\left({\text{r}}_{hkn}\right)}{s_{tm}}\text{ln}\frac{{\omega }_t\left({\text{r}}_{hkn}\right)}{s_{tm}}}{\text{ln}{\sum }_{h=1}^{N_{\text{PE}}}{\sum }_{k=1}^{K_h}{\sum }_{n=1}^{N_{hk}}\delta \left({\xi }_m-\xi \left({\text{r}}_{hkn}\right)\right)}$$ (19)

$$s_{tm}=\sum _{h=1}^{N_{\text{PE}}}\sum _{k=1}^{K_h}\sum _{n=1}^{N_{hk}}\delta \left({\xi }_m-\xi \left({\text{r}}_{hkn}\right)\right){\omega }_t\left({\text{r}}_{hkn}\right)$$ (20)

Computational Models and Methods

All simulations described below were performed with a development version of SANDER.⁸⁹ The simulations were propagated with a 1 fs time step. All production sampling was performed in the canonical ensemble using the Langevin thermostat with a 5 ps⁻¹ collision frequency to maintain a temperature of 298 K.⁹⁰ The equilibration of the system densities were performed in the isothermal-isobaric ensemble using the Berendsen barostat to maintain an external pressure of 1 atm.⁹¹ The long-range electrostatics were calculated with a particle mesh Ewald (PME) method using a 1 ų regular grid spacing, 8 Å real-space cutoffs, and tinfoil boundary conditions.⁹² Specifically, the semiempirical QM/MM simulations performed PME with Mulliken charges,^93,94 whereas the ab initio QM/MM calculations used the ambient-potential composite Ewald method.⁸⁷ The ab initio QM/MM simulations were performed with the HFDF software package developed within our group, which we interfaced to SANDER and described in ref. 87. The Lennard-Jones potential was truncated at 8 Å, and a tail correction was applied to model the interactions beyond the cutoff.⁹⁵ The solute structures were initially prepared with the GAFF force field,⁹⁶ and the solvent was modeled with the TIP4P-Ew water model.⁹⁷ The MNDO/d QM/MM+DPRc and DFTB2/MIO^98–100 QM/MM+DPRc potentials include a nonelectrostatic MLP correction to the underlying QM/MM energy and forces. The DPRc potential is an extension of the DeepPot-SE model¹⁰¹ that includes corrections for both the QM-QM and QM-MM interactions, and the neural network parameters are optimized to reproduce the target QM/MM energies and forces. The DPRc model has been previous described,^76,88 and additional details are provided in the Supporting Information.

Simulations of the ethylene phosphate reaction.

The ethylene phosphate (Figure 1a) FES was calculated from umbrella sampling performed at 48 values of ξ_PT ranging from −3.1 to 1.6 Å in steps of 0.1 Å that bias the PE with a spring force constant of 200 kcal mol⁻¹ Å⁻². An initial structure for the unimolecular reactant state was prepared by solvating the system in a truncated octahedron containing 2974 TIP4P-Ew water molecules. A 100 ps MNDO/d QM/MM simulation was performed in the NPT ensemble to equilibrate the system density, and the final real-space lattice vector lengths were 49.03 Å. A series of 3 ps NVT simulations were conducted to slowly progress the structure along the reaction coordinate (that is, the brief simulation of window i + 1 was restarted from simulation i) to generate initial structures for each window. Each window was then equilibrated with MNDO/d QM/MM for 100 ps in the NVT ensemble. This was followed by an additional 20 ps of NVT equilibration with PBE0/6–31G* QM/MM. Production sampling was performed with PBE0/6–31G* QM/MM to obtain a target FES used to validate the wTP and gwTP reference potential methods. The PBE0/6–31G* QM/MM production simulations were performed for 80 ps/(window·trial) and repeated 4 times to analyze the uncertainty in the FES values. The 4 trials differed only by changing the thermostat random number seed value. The aggregate amount of PBE0/6–31G* sampling corresponds to 320 ps/window or 15.36 ns of sampling for the entire FES. The coordinates were saved every 10 fs for analysis.

The parametrization of MLPs often involves the use of an active learning procedure that results in several network parameter sets that produce similar corrections to the energies and forces for the data they were trained against. If the training data adequately represents the ensemble of structures observed in production sampling, then the trial network parameter sets can lead to similar FES values. For illustrative purposes, we’ve chosen to construct 4 ad hoc MNDO/d QM/MM+DPRc potentials that yield disparate FES values. These potentials were trained to reproduce different regions of the ethylene phosphate FES rather than the entire surface. Each potential well-reproduces the ab initio FES in the region where it was parametrized, but the FES values disagree with each other and the ab initio target elsewhere. We will show that the gwTP analysis from the combined sampling of all 4 potentials reproduces the ab initio FES over the full range. The MNDO/d QM/MM+DPRc potentials will be referred to as ML0, ML1, ML2, and ML3. The potentials were trained to reproduce different sets of target PBE0/6–31G* energies and forces. Specifically, the ML0 potential was trained only using the sampling obtained from −0.1 Å ≤ ξ_PT ≤ 1.6 Å, the ML1 potential was trained to −1.3 Å ≤ ξ_PT ≤ 1.0 Å, the ML2 potential was trained to −2.5 Å ≤ ξ_PT ≤ −0.2 Å, and the ML3 potential was trained to −3.1 Å ≤ ξ_PT ≤ −1.4 Å. In the present work, we are making comparison with a FES obtained from explicit ab initio sampling; therefore, we’ve reused the sampling to serve as initial training data for each potential. Specifically, 2.5% of the PBE0/6–31G* trajectory frames in the desired range was chosen at random to parametrize each potential. The optimization of the network parameters was performed with the DeePMD-kit and DP-GEN software.^102,103 The optimization consisted of 200k steps with initial and final learning rates of 10⁻³ and 5·10⁻⁸ respectively. The initial optimization was followed by 9 cycles of active learning to search for additional training data.⁷⁶ Each active learning cycle performs 4 parameter optimizations to yield 4 trial parameter sets. One of the parameter sets is used to generate 20 ps of MNDO/d QM/MM+DPRc sampling for each window in the selected range of ξ_PT values. The atomic forces are then computed with each of the 4 trial parameter sets, and if the maximum standard deviation of atomic forces is within the range 0.08 to 0.25 eV/Å, then the candidate structure is included in the next round of optimization. Upon completion of the active learning procedure, the “DP Compress” compression algorithm was applied to the MLP to improve computational performance during inference.¹⁰⁴ The production sampling of each potential, described below, was performed using only 1 of the 4 parameter sets. In hindsight, the initial parameterization to the ab initio sampling would have been sufficient for our purpose; each round of active learning only labeled 6 candidate structures, on average.

The MNDO/d QM/MM+DPRc production sampling of the ethylene phosphate system was performed for 250 ps/(window·potential·trial), and the coordinates were saved to a trajectory file every 50 fs. Three trials were performed with different random number seed values for error analysis. From these simulations, one can use MBAR or wTP to evaluate 12 surfaces corresponding to the 3 trials of the 4 MNDO/d QM/MM+DPRc potentials. One can further obtain 4 trial-averaged surfaces. Each of the 12 surfaces is analyzed from 12 ns/(potential·trial) of sampling, and each of the 4 trial-averaged surfaces are produced from 36 ns/potential of sampling. We use the notation ΔA(target;reference) to distinguish between the surfaces. This notation signifies that the curve is the FES of the target PE function estimated from the sampling performed with reference. When the target and reference are the same PE function, then the FES is calculated from the MBAR method. If the target FES is calculated from the sampling performed with a single reference other than the target, then the wTP method is used. If the target FES is estimated from multiple reference potentials, then the gwTP method is used. The trial-averaged estimates of the ML0, ML1, ML2, and ML3 potentials calculated from the MBAR method shall be labeled ΔA(MLi;MLi) (where i ∈ [0,3]). The trial-averaged estimates of the PBE0/6–31G* FES obtained from wTP analysis of the individual potentials are denoted ΔA(PBE0;MLi). The gwTP method analyzes the combined sampling from the 4 MNDO/d QM/MM+DPRc potentials. Therefore, one obtains 3 gwTP surfaces corresponding to the 3 trials; each surface consists of 48 ns/trial of sampling, and the trial-averaged gwTP FES includes 144 ns of aggregate sampling. The trial-averaged gwTP FES is referred to as ΔA(PBE0;ML*). One can further make a “best estimate” of the ab initio FES by using gwTP to analyze the 144 ns of MNDO/d QM/MM+DPRc sampling and the 15.36 ns of PBE0/6–31G* sampling. The best estimate is denoted ΔA(PBE0;All).

Simulations of the native model reaction.

The native model reaction (Figure 1b) was previously examined in ref. 88. It was one of 6 nonenzymatic transesterification model reactions used to develop 4 DFTB2/MIO QM/MM+DPRc potentials trained to reproduce PBE0/6–31G* QM/MM energies and forces. The 4 potentials were generated from an active learning procedure described in that work.⁸⁸ In the present work, we reuse the DFTB2/MIO QM/MM+DPRc potentials without modification. The 5 analogous systems included within the training differ from the native model by having replaced one-or-more oxygens with sulfurs at key positions. We do not reconsider the sulfur substituted systems in the present work; instead, our interest in the native model reaction arises from noticeable discrepancies between the ab initio and DFTB2/MIO QM/MM+DPRc FES values. We will show that the native model reaction profiles presented in ref. 88 are not fully converged. The high computational cost of ab initio QM/MM calculations, however, places practical limitations on the amount of sampling that can be reasonably achieved. We will demonstrate that gwTP analysis of extended sampling performed with the reference potentials produces a better estimate of the ab initio FES than what can be afforded from ab initio QM/MM simulation.

An initial structure for the native model unimolecular reactant state was prepared by solvating the system in a box of 1510 TIP4P-Ew waters and performing 200 ps of DFTB2/MIO QM/MM simulation in the NPT ensemble to equilibrate the system density. The final unit cell lattice vectors were approximately 35.8 Å. This was followed by a series of 2 ps DFTB2/MIO QM/MM simulations in the NVT ensemble that slowly incremented the value of the reaction coordinate to obtain initial structures for each window in the range −3.5Å ≤ ξ_PT ≤ 5Å in steps of 0.1 Å. The umbrella potentials biased the simulations with a spring force constant of 100 kcal mol⁻¹ Å⁻². Each of the 86 windows were equilibrated with DFTB2/MIO QM/MM for 100 ps in the NVT ensemble, and this was followed by an additional 25 ps of PBE0/6–31G* QM/MM NVT equilibration. The 25 ps/window of PBE0/6–31G* equilibration is not included in the analysis presented in this work nor ref. 88.

PBE0/6–31G* production sampling of the native model reaction was run for 25 ps/(window·trial), and the coordinates were saved every 25 fs. Each simulation was repeated 4 times using different thermostat random number seed values to yield 4 FES estimates. Each surface is the analysis of 2.15 ns/trial of sampling. The trial-averaged FES, denoted “ΔA(PBE0;PBE0) 100 ps”, includes all 8.6 ns of production sampling (100 ps/window of aggregate sampling).

DFTB2/MIO QM/MM+DPRc production sampling of the native model reaction proceeded similarly. However, the active learning procedure yields 4 neural network parameter sets, so we performed 25 ps/(window·potential) of sampling with each potential. Therefore, the average DFTB2/MIO QM/MM+DPRc FES does not correspond to a single potential; instead, it is the mean of the 4 reference potentials. The average DFTB2/MIO QM/MM+DPRc FES includes 100 ps/window of aggregate sampling, and it is denoted “〈ΔA(ML;ML)〉 100 ps”. The gwTP method can be used to estimate the PBE0/6–31G* from the combined sampling of the DFTB2/MIO QM/MM+DPRc potentials, referred to as “ΔA(PBE0;ML*) 100 ps”.

To examine the behavior of the predicted native model reaction FES on a longer timescale, we extended the DFTB2/MIO QM/MM+DPRc simulations and performed multiple trials with each potential. The extended simulations were sampled for 100 ps/(window·potential·trial), and we performed 3 trials of each simulation with different random number seed values. The 3 trials of the 4 potentials yield 12 surfaces. Each surface is the analysis of 8.6 ns/(potential·trial) of sampling. The average FES is the mean of the 12 estimates, such that it considers 103.2 ns (1.2 ns/window) of aggregate sampling. The average DFTB2/MIO QM/MM+DPRc FES from the extended sampling shall be labeled “〈ΔA(ML;ML)〉 1.2 ns”. The sampling from the 3 trials produce 3 gwTP estimates of the ab initio surface. The PBE0/6–31G* FES from the extended sampling is labeled “ΔA(PBE0;ML*) 1.2 ns”. An analogous set of profiles labeled “〈ΔA(ML;ML)〉 last 600 ps” and “ΔA(PBE0;ML*) last 600 ps” were analyzed from the last 50 ps/(window·potential·trial) of sampling from each simulation. Finally, we shall refer to a profile labeled “ΔA(PBE0;All)”. This FES includes all 103.2 ns of DFTB2/MIO QM/MM+DPRc sampling and the 8.6 ns of PBE0/6–31G* sampling.

Error Analysis.

The uncertainties in the free energy values can be estimated from the MBAR calculation of the sample weight covariance⁴³ between spatial bins, as was done in ref. 52. This approach uses the fluctuations of the statistically independent samples within the observed ensembles to assign error values. Another strategy for estimating errors is the “ensemble average approach” which estimates the standard errors from the distribution of calculated results produced from independent simulations.^105–108 The ensemble average approach can be viewed as a strategy to gauge the uncertainty arising from limited finite sampling. In the present work, we incorporate the ideas that motivate both strategies by analyzing multiple trials with bootstrap analysis. Let $F_t^{(i)}\left({\xi }_m\right)$ be the free energy of target potential t in spatial histogram m from trial i (i ∈ [1, N_trial]). The uncertainty of the FES value $\delta F_t^{(i)}\left({\xi }_m\right)$ is estimated from circular moving block bootstrap error analysis,¹⁰⁹ where the block size of each simulation is chosen from the autocorrelation of the biasing potential time series w_hk(r_hkn). The aggregate sampling from all trials can similarly be analyzed to yield free energy values $F_t^{(a)}\left({\xi }_m\right)$ and uncertainties $\delta F_t^{(a)}\left({\xi }_m\right)$. To compute a trial-average ${\overline{F}}_t\left({\xi }_m\right)$, we choose a set of constants C_i that uniformly shift each surface $F_t^{(i)}\left({\xi }_m\right)+C_i$ to minimize the weighted sum of squared differences with the $F_t^{(a)}\left({\xi }_m\right)$ values.

$$\frac{\partial }{\partial C_i}\sum _m{w}_m^{(i)}{\left\{\left[F_t^{(i)}\left({\xi }_m\right)+C_i\right]-F_t^{(a)}\left({\xi }_m\right)\right\}}^2=0$$ (21)

The weights are chosen to be inversely proportional to the bootstrap uncertainties.

$$w_m^{(i)}=\frac{{\left\{{\left[\delta F_t^{(i)}\left({\xi }_m\right)\right]}^2+{\left[\delta F_t^{(a)}\left({\xi }_m\right)\right]}^2\right\}}^{-1}}{{{\sum }_{m^{\prime }}\left\{{\left[\delta F_t^{(i)}\left({\xi }_{m^{\prime }}\right)\right]}^2+{\left[\delta F_t^{(a)}\left({\xi }_{m^{\prime }}\right)\right]}^2\right\}}^{-1}}$$ (22)

The average value ${\overline{F}}_t\left({\xi }_m\right)$ and its standard error $\delta {\overline{F}}_t\left({\xi }_m\right)$ are then given by eqs. 23 and 24, respectively.

$${\overline{F}}_t\left({\xi }_m\right)=\frac{1}{N_{\text{trial}}}\sum _{i=1}^{N_{\text{trial}}}{F}_t^{(i)}\left({\xi }_m\right)+C_i$$ (23)

$$\delta {\overline{F}}_t\left({\xi }_m\right)={\left(N_{\text{trial}}^{-1}\sum _{i=1}^{N_{\text{trial}}}\frac{{\left[F_t^{(i)}\left({\xi }_m\right)+C_i-{\overline{F}}_t\left({\xi }_m\right)\right]}^2}{N_{\text{trial}}-1}+\frac{{\left[\delta F_t^{(i)}\left({\xi }_m\right)\right]}^2}{N_{\text{trial}}}\right)}^{\frac{1}{2}}$$ (24)

The first and second terms in eq. 24 correspond to the unbiased sample variance between trials and the average bootstrap variance, respectively. The error bars shown in the figures are 95% confidence intervals $\left(1.96\delta {\overline{F}}_t\left({\xi }_m\right)\right)$.

Results and Discussion

Ethylene Phosphate Reaction Profiles

Figure 2 displays free energy profiles of the ethylene phosphate reaction calculated with the MBAR and wTP methods using each of the 4 MNDO/d QM/MM+DPRc reference potentials (ML0, ML1, ML2, and ML3). As previously mentioned, the reference potentials were specifically designed to disagree with each other and the PBE0/6–31G* target in various regions of the FES. The surfaces estimated with the gwTP method are shown in figure 3. These figures demonstrate that the gwTP method can improve upon the wTP estimate by combining the sampling from multiple reference potentials.

Figure 2:

Free energy surfaces (ΔA) of the ethylene phosphate model reaction (Figure 1a) calculated with MBAR and wTP analysis. Reweighting entropies (RE) are shown in the subplots below each FES. Parts (a), (e), (i), and (m) are the 4 MNDO/d QM/MM+DPRc potentials (ML0, ML1, ML2 and ML3) evaluated from MBAR analysis. Parts (b), (f), (j), and (n) are their corresponding wTP estimates of the PBE0/6–31G* FES. Parts (c), (d), (g), (h), (k), (l), (o), and (p) are the reweighting entropy of the corresponding MBAR and wTP analysis. The error bars are 95% confidence intervals.

Figure 3:

Free energy surfaces (ΔA) of the ethylene phosphate model reaction (Figure 1) calculated with gwTP analysis. Reweighting entropies (RE) are shown in the subplots below each FES. (a) The gwTP FES estimated from the combined umbrella sampling produced from the 4 MNDO/d QM/MM+DPRc potentials (ML0, ML1, ML2 and ML3). (b) The PBE0/6–31G* surface estimated from gwTP analysis of all available PBE0/6–31G* and MNDO/d QM/MM+DPRc umbrella sampling. Parts (c) and (d) are the reweighting entropy of the corresponding MBAR and gwTP analysis. The error bars are 95% confidence intervals.

Figure 2 parts a, e, i, and m show that the 4 MNDO/d QM/MM+DPRc potentials agree with PBE0/6–31G* in the regions they were trained to reproduce the target energies and forces. Outside of these regions, the profiles disagree with ab initio and each other. Figure 2 parts b, f, j, and n are the wTP estimates of the target FES from each reference potential. The wTP surfaces generally improve the agreement with the target; however, they suffer from numerical noise and large uncertainties in the untrained regions. This is further emphasized by a corresponding decrease in the reweighting entropies shown in parts d, h, l, and p. The results shown in figure 2 are consistent with previous work which found that reliable free energies are produced by the wTP method when the reweighting entropy is larger than 0.6, but the results become questionable when the reweighting entropy decreases below 0.3.⁵⁰ Every MNDO/d QM/MM+DPRc potential shown in figure 2 exhibits reweighting entropies below 0.3 for some ξ_PT values.

The gwTP estimate of the ab initio FES is shown in figure 3. Figure 3a uses the combined sampling from each MNDO/d QM/MM+DPRc potential, and the resulting FES agrees with the MBAR analysis of the ab initio sampling to within the uncertainties of the calculations throughout the entire range of ξ_PT values. Similarly, the reweighting entropies shown in 3c consistently remain above 0.6. The gwTP FES shown in Figure 3b includes sampling from both the ab initio and MNDO/d QM/MM+DPRc potentials and represents our best estimate of the surface. The inclusion of sampling from the target potential increases the reweighting entropy values, but the FES values do not significantly change.

An approach for smoothing the density-of-states (DoS) in wTP calculations was presented in ref. 52. The motivation for DoS smoothing is to effectively remove statistically unlikely samples from a distribution that are over-represented within the limited amount of finite sampling. We generalized the DoS smoothing procedure in the Supporting Information. Figures S1 and S2 are analogous to figures 2 and 3, but include DoS smoothing. The use of DoS smoothing does not significantly improve the wTP estimates of the ab initio surface, because the failure is produced from the large differences in reference and target potentials, not from the presence of a few, unlikely samples. The gwTP formalism does not introduce any mechanism to prevent the occurrence of over-represented samples. Although the DoS smoothing procedure is unnecessary in the present applications, we continue to regard it as a valuable technique which can help eliminate numerical noise from the predicted free energies. Application of DoS smoothing to the MBAR and gwTP analysis yield profiles that are indistinguishable from those shown in Figures 2 and 3.

Native Model Reaction Profiles

Free energy surfaces of the native model reaction (Figure 1b) are shown in figures 4 and 5. These figures illustrate that the extended sampling made possible by using an affordable reference potential can result in a more accurate FES than what can be estimated from limited sampling performed with an expensive target potential.

Figure 4:

Free energy surfaces (ΔA) of the native model reaction (Figure 1). (a) Comparison of the PBE0/6–31G* FES to the average FES of the 4 DFTB2/MIO QM/MM+DPRc potentials (ML0, ML1, ML2 and ML3). The listed times are the total amount of sampling per umbrella window used in the FES calculation. (b) Comparison of the PBE0/6–31G* FES to those predicted from gwTP analysis. The ΔA(PBE0;ML*) gwTP surfaces are evaluated using the sampling from all 4 DPRc potentials. The ΔA(PBE0;All) surface is calculated from the 1.2 ns/window of DPRc sampling and the 100 ps/window PBE0/6–31G* sampling. The error bars are 95% confidence intervals. The RE values shown in parts (c) and (d) are the reweighting entropies.

Figure 5:

Free energy surfaces (ΔA) of the native model reaction (Figure 1). (a) Comparison of the PBE0/6–31G* FES to the average FES of the 4 DFTB2/MIO QM/MM+DPRc potentials (ML0, ML1, ML2 and ML3). (b) Comparison of the PBE0/6–31G* FES to those predicted from gwTP analysis. The surfaces labeled “last 600 ps” refer to aggregate sampling taken from the last 50 ps of the 12 DFTB2/MIO QM/MM+DPRc simulations. The error bars are 95% confidence intervals. The RE values shown in parts (c) and (d) are the reweighting entropies.

The PBE0/6–31G* and average DFTB2/MIO QM/MM+DPRc surfaces (the black and red surfaces, respectively) shown in figure 4a were originally presented in ref. 88. These surfaces include 100 ps/window of aggregate production sampling. Additional ab initio sampling was deemed too costly to extend, and the DFTB2/MIO QM/MM+DPRc sampling was chosen to mimic the protocol used to perform the ab initio simulations. The ab initio and reference potentials generally agree except in the range ξ_PT ≈ [−1 Å, 0.5 Å], where the reference potential FES is noticeably higher in energy.

We used the gwTP method to estimate the ab initio FES from the 100 ps/window of DFTB2/MIO QM/MM+DPRc sampling, and the result is labeled “ΔA(PBE0;ML*), 100 ps” in figure 4b. This gwTP estimate agrees with the MBAR analysis of the ab initio sampling to within the uncertainties of the calculations, and the reweighting entropy is larger than 0.6. This suggests there is a small discrepancy in the reference potentials’ ability to reproduce the ab initio QM/MM energies and forces which is corrected by reweighting the samples. However, if the reference potential simulations are extended to include 1.2 ns/window of aggregate sampling (the green surfaces shown in figures 4a and 4b), the predicted activation energy increases by 1 kcal/mol even though the reweighting entropies remain largely unchanged. The large reweighting entropies suggest that there is good phase space overlap between the reference and target potentials in both the 100 ps/window and 1.2 ns/window simulations. Similarly, the gwTP ab initio FES values are nearly 1 kcal/mol higher near the transition state when analyzing the combined sampling from both the PBE0/6–31G* and extended DFTB2/MIO QM/MM+DPRc simulations (the blue surface in 4b). These results suggest that another reason for the observed discrepancies between PBE0/6–31G* and the DFTB2/MIO QM/MM+DPRc potentials may due to the limited amount of sampling. In other words, the ab initio simulations may not represent converged equilibrium ensembles. Consequently, the ΔA(PBE0;PBE0) MBAR estimate should not be trusted, and the ΔA(PBE0;All) gwTP result does not represent a “best estimate” of the FES because it is potentially skewed by nonequilibrium sampling.

We performed forward and reverse analysis of the sampling^110,111 to examine the convergence of the extended simulations. The forward analysis generates a hΔA(ML;ML)i FES from the first P% of samples, whereas reverse analysis calculates a FES from the last P% of the simulations. From these surfaces, we monitor the free energy values (and uncertainties) of the two transition states and intermediate relative to the unimolecular reactant state as a function of P%. The convergence behavior of these properties is shown in Figure S3 within the Supporting Information. In summary, the reverse analysis is stable with respect to P%, but the forward analysis display drifts in the free energy values. The forward and reverse analysis do not agree to within their estimated uncertainties until the first 50% of the simulations are discarded. It is difficult to ascertain if the simulations have converged without having extended the sampling (see Figure S4 in the Supporting Information); when the data contains a slight drift, the reverse analysis artificially agrees with the forward analysis simply because it hasn’t been given ample opportunity to approach an equilibrium value. The assertion that the ab initio simulations are not converged is further supported by umbrella integration analysis^47–49 (see Figure S5 in the Supporting Information). The extended simulations yield a smooth free energy gradient profile, whereas the gradient profiles obtained from the limited sampling are contaminated with noise. Figure 5 displays the 〈ΔA(ML;ML)〉 and ΔA(PBE0;ML*) surfaces after discarding the first half of each simulation. These surfaces represent our best estimate of the profiles. It is notable that the best estimate of the ab initio FES does not include any of the available PBE0/6–31G* QM/MM sampling, and the best estimate of the rate-limiting transition state barrier is 1.5 kcal/mol higher in energy than the MBAR analysis of the PBE0/6–31G* QM/MM simulations (see Table 1).

Table 1:

Native model reaction free energies (kcal/mol) of the first transition state ΔA(ξ_TS1), the intermediate ΔA(ξ_int), the rate-limitingtransition state ΔA^‡, and the reaction product ΔA. The uncertainties are standard errors.

Target	Ref.	Sampling	ΔA(ξTS1)	ΔA(ξint)	ΔA‡	ΔA
PBE0	PBE0	100 ps	10.4 ± 0.2	6.4 ± 0.2	19.6 ± 0.2	1.5 ± 0.3
PBE0	ML*	100 ps	10.0 ± 0.2	7.0 ± 0.2	19.9 ± 0.3	1.2 ± 0.3
PBE0	ML*	last 600 ps	11.5 ± 0.2	8.3 ± 0.1	21.1 ± 0.2	3.6 ± 0.1

To reduce the number of target potential evaluations, one should identify correlation within the simulations and only consider the statistically independent samples. The “〈ΔA(ML;ML)〉, last 600 ps” and “ΔA(PBE0;ML*), last 600 ps” surfaces displayed in Figure 5 where analyzed from the correlated samples, and the errors were estimated from block bootstrap analysis. Figure S6 in the Supporting Information compares these surfaces to the analysis performed with uncorrelated samples from the last half of each simulation. The uncorrelated data is the subset of samples extracted with a stride equal to the statistical inefficiency of the biasing energy time series. In summary, analysis of the uncorrelated samples produce surfaces that are visually indistinguishable from the results shown in Figure 5.

The obvious benefit of using gwTP (or wTP) is the reduced computational effort associated with performing the sampling with an affordable reference potential. The total cost of performing MBAR and gwTP estimates of the target FES are given by eqs. 25 and 26, respectively.

$$C(t;t)=\frac{T}{v_t}$$ (25)

$$C(t;h)=\frac{T}{v_h}+\left(1-f_{\text{eq}}\right)f_{\text{file}}{f}_{\text{indep}}{f}_{\text{eval}}\frac{T}{v_t}$$ (26)

T is the aggregate simulation time (ps) of all windows, trials, and potentials (if applicable). ν_t and ν_h are the simulation rates (ps/day) using the target and reference potentials. Equation 25 is, therefore, the net cost of performing the sampling with the target potential, and the first term in eq. 26 is the cost of performing the reference simulations. The second term in eq. 26 is the additional cost of performing target potential evaluations of the saved samples. f_eq and f_file are the fraction of simulation steps excluded as equilibration and the fraction of samples written to file for analysis, respectively. f_indep is the fraction of statistically independent samples, and f_eval is the fraction of the statistically independent samples used to perform target potential evaluations. The ratio C(t;t)/C(t;h) is the computational savings from using the reference potential approach. If one assumes the methods are performed with equal sampling, then the ratio is given by eq. 27.

$$\frac{C(t;t)}{C(t;h)}={\left(\frac{v_t}{v_h}+\left(1-f_{\text{eq}}\right)f_{\text{file}}{f}_{\text{indep}}{f}_{\text{eval}}\right)}^{-1}$$ (27)

In practice, the goal is to use the information provided by the reference simulations to reduce the number of target potential evaluations, thus maximizing the savings. First, one should estimate f_eq by verifying that the reference simulations are converged to avoid unnecessary evaluations of unequilibrated data. In the present work, we chose f_eq by monitoring the energies of relevant stationary points from forward and reverse analysis of the time series.^110,111 Second, one should estimate the correlation in the time series so only the statistically independent samples are evaluated with the target potential. In principle, one could estimate the correlation from the energy differences Δu_t,h, but that approach would defeat the purpose of reducing the number of target potential evaluations. Instead, we’ve chosen to estimate the correlation from the biasing potential w_hk time series. Finally, the value of f_file should generally by chosen such that the statistical inefficiency of the saved frames is close to 1 to avoid storing unnecessarily large files.

To estimate the time savings of the native model reaction, we measured the performance of the PBE0/6–31G* QM/MM (ν_t = 1.565 ps/day) and DFTB2/MIO QM/MM+DPRc (ν_h = 392.7 ps/day) simulations on a single 2.10 GHz Intel Xeon Gold 6230 CPU core. We further note that the first half (f_eq = 1/2) of each simulation was discarded as equilibration, the samples were saved every 50 steps (f_file = 1/50), and the fraction of statistically independent samples was found to be f_indep = 0.763. Equation 27 suggests that the gwTP method is 86 times faster than performing an equivalent amount of ab initio QM/MM sampling for this system even if the full set of uncorrelated samples were used to evaluate the target potential (f_eval = 1).

One may question how many of the uncorrelated samples are necessary to obtain a reasonable estimate of the target FES. The required number of target evaluations will generally depend on the agreement between the target and reference potentials. In the limit that the reference was the target potential, then an accurate target FES would require the minimum number of samples needed to construct the reference FES. If the target potential differed from the reference, then one would expect this limit to be a lower bound on the number of target evaluations. Furthermore, the required number of samples will depend on the acceptable level of uncertainty. If one is comparing different mechanistic pathways, for example, then a high degree of certainty is not necessary to rule out a path exhibiting a large rate-limiting transition state. Table 2 shows how the free energy values, uncertainties, and reweighting entropies behave when gwTP analysis is performed with a varying amount of uncorrelated samples. When f_eval = 1, the analysis is performed with all available uncorrelated samples (after removing the unequilibrated portion of simulations). On average, there are 381.4 samples/(trial·potential·window). In the remaining rows, we perform the analysis using the first N_eval uncorrelated samples from each simulation. For example, N_eval = 15 signifies that the target potential was evaluated using the first 15 uncorrelated samples/(trial·potential·window), on average. In the present application, we have 4 reference potentials and each simulation was performed 3 times, so this corresponds to 15 × 4 × 3 = 180 evaluations/window. As f_eval increases, the value of N_eval ceases to be integer because there are one-or-more simulations that had fewer samples than requested. From a practical perspective, we cannot recommend a precise value of N_eval that the reader should use, for reasons already discussed. Instead, we can offer guidance for a procedure that the reader can follow. First, perform evaluations with a small number of samples and evaluate the FES. If the reweighting entropy is less than some tolerance (e.g., 0.6) or the uncertainty in the free energy values are unacceptably large, then one should increase N_eval accordingly. One could even focus the evaluations in the particular region(s) of the FES where the reweighting entropy was low. Table 2 suggests that N_eval = 15 samples/simulation are required for the minimum observed entropy to exceed 0.6. If the activation energy must be estimated to within an uncertainty of 0.2 kcal/mol, then the number of samples must be increased to N_eval = 50. The surfaces evaluated with N_eval = 15 and N_eval = 50 are compared in Figure S7 within the Supporting Information.

Table 2:

Relative energies of the native model reaction free energy surfaces estimated from gwTP analysis as a function of the number of target potential evaluations.^a

N eval	f eval	min RE	〈RE〉	ΔA(ξTS1)	ΔA(ξint)	ΔA‡	ΔA
5.0	0.013	0.42	0.78	12.0 ± 0.5	8.3 ± 0.4	21.2 ± 0.4	3.1 ± 0.4
10.0	0.026	0.56	0.80	11.7 ± 0.4	8.2 ± 0.4	21.0 ± 0.3	3.2 ± 0.3
15.0	0.039	0.64	0.81	11.7 ± 0.4	8.2 ± 0.4	21.1 ± 0.3	3.5 ± 0.2
25.0	0.066	0.70	0.83	11.8 ± 0.4	8.3 ± 0.3	21.2 ± 0.3	3.6 ± 0.2
49.7	0.130	0.75	0.85	11.7 ± 0.2	8.4 ± 0.2	21.5 ± 0.2	3.6 ± 0.1
97.9	0.257	0.71	0.86	11.5 ± 0.2	8.3 ± 0.2	21.1 ± 0.2	3.5 ± 0.1
381.4	1.000	0.82	0.88	11.5 ± 0.2	8.3 ± 0.2	21.1 ± 0.2	3.6 ± 0.1

^aN_eval is the number of statistically independent samples per simulation used to evaluate the target potential energies. f_eval is the fraction of the statistically independent samples selected for target potential evaluation. The uncertainties are the standard errors. “min RE” and 〈RE〉 are the minimum and average reweighting entropy observed during the gwTP analysis of the aggregate sampling.

The reader may question if MLPs are required for their application or if it would suffice to perform wTP from uncorrected semiempirical QM/MM sampling. After all, the idea of training a MLP and also performing target potential evaluations to analyze the results may seem formidable. The gwTP method allows one to combine the MLP training and analysis so that all of the target potential evaluations used to train the MLP can also be used to estimate the target FES. For example, one could first perform standard semiempirical QM/MM sampling and estimate the target FES with the wTP method from a limited number of target potential evaluations. If the reweighting entropy is high, then no refinement is necessary. If the entropy is low, one may deem it worthwhile to reuse the target evaluations to train a MLP correction. The trained MLP is then sampled, and a limited number of target potential evaluations are made to estimate the FES; however, the gwTP method allows one to analyze the FES using the combined sampling from the uncorrected and MLP-corrected simulations. This process can be iterated, such that one performs “active learning” in a manner which reuses the aggregate sampling from previous iterations to improve the predicted target FES while also generating new training data for the next iteration.

Conclusion

We described a new reference potential approach for calculating free energy surfaces called the generalized weighted thermodynamic perturbation (gwTP) method. The gwTP method extends the formulation of weighted thermodynamic perturbation (wTP) to include sampling from multiple reference potentials. This work was motivated by the growing interest in the use of MLPs being trained to correct semiempirical QM/MM energies and forces to match target ab initio values. The active learning procedure often employed to train these corrections results in several neural network parameter sets. Each parameter set corresponds to a different reference potential, and the gwTP method can be used to analyze the aggregate sampling from each potential to estimate the target free energies.

We envision other strategies where gwTP analysis may be useful. One could train separate MLPs for different systems, such as RNA catalysis reactions in different enzyme environments, or create a single MLP that tries to model all systems. One can perform sampling with all of these potentials and use gwTP to make a best estimate of another system not included in the training. Alternatively, one could construct potentials that specialize their parametrizations to have high accuracy at different stages of a multistep enzyme reaction. In this way the gwTP method can be used to combine the sampling from several “specific reaction parametrizations”^78–80 and thus provide new opportunities in model design.

To this end, we created 4 ad hoc MNDO/d QM/MM+DPRc neural network parameter sets to calculate the free energy surface of an ethylene phosphate model reaction. Each parametrization limited the training data such that the potentials well-reproduced PBE0/6–31G* energies and forces in the regions they were trained, but the potentials disagree with each other and the ab initio target in all other regions. We showed that the wTP method fails to accurately predict the target free energies in the untrained regions. We further demonstrated that the gwTP method accurately reconstructs the ab initio surface from the aggregate sampling.

We used the gwTP method to examine the free energy surface of a non-enzymatic RNA 2′-O-transesterification model with an ethoxide leaving group. This system was used as a case study to emphasize the benefits offered by the reference potential approach. Because the reference potentials are much more affordable, the sampling could be extended. The extended simulations allowed us to determine that the ab initio QM/MM sampling was not converged. Excluding the first half of each simulation from the analysis produced a rate limiting transition state 1.5 kcal/mol higher than the value predicted from ab initio QM/MM sampling; however, the reweighting entropy remained high. In other words, we made a better estimate of the ab initio free energy surface using gwTP analysis than what direct ab initio QM/MM simulation can reasonably afford. For this system, the computational cost of estimating the ab initio surface from gwTP analysis of the DFTB2/MIO QM/MM+DPRc sampling was found to be 86 times faster than an equivalent amount of PBE0/6–31G* QM/MM simulation. The cost savings is expected to increase for systems with larger QM regions due to the inherent differences in the formal computational scaling behaviors of ab initio and semiempirical Hamiltonians. The computational cost is minimized by choosing an efficient reference potential and reducing the number of target potential evaluations. The number of target potential evaluations is reduced by analyzing the reference potential simulations to identify and eliminate the unequilibrated portions of the simulations and by extracting the statistically independent samples.