A variational method for network-wide analysis of relative ligand binding free energies with loop closure and experimental constraints

Abstract

We describe an efficient method for the simultaneous solution of all free energies within a relative binding free energy (RBFE) network with cycle closure and experimental/reference constraint conditions using Bennett Acceptance Ratio (BAR) and Multistate BAR (MBAR) analysis. Rather than solving the BAR or MBAR equations for each transformation independently, the simultaneous solution of all transformations are obtained by performing a constrained minimization of a global objective function. The nonlinear optimization of the objective function is subjected to affine linear constraints that couple the free energies between the network edges. The constraints are used to enforce the closure of thermodynamic cycles within the RBFE network, and to enforce an additional set of linear constraint conditions demonstrated here to be subsets of (1 or 2) experimental values. We describe details of the practical implementation of the network BAR/MBAR procedure, including use of generalized coordinates in the minimization of the free energy objective function, propagation of bootstrap errors from those coordinates, and performance and memory optimization. In some cases it is found that use of restraints in the optimization is more practical than use of generalized coordinates for enforcing constraint conditions. The fast BARnet and MBARnet methods are used to analyze the RBFEs of 6 prototypical protein-ligand systems, and it is shown that enforcement of cycle closure conditions reduces the error in the predictions only modestly, and further reduction in errors can be achieved when one or two experimental RBFEs are included in the optimization procedure. These methods have been implemented into FE-ToolKit, a new free energy analysis toolkit. The BARnet/MBARnet framework presented here opens the door to new, more efficient and robust free energy analysis with enhanced predictive capability for drug discovery applications.

Graphical Abstract

1. Introduction

Alchemical free energy methods play a key role in lead optimization by enabling the prediction and ranking of the relative binding affinities of ligands to their protein targets in order to prioritize them for further synthesis and testing.^1,1–11 Often these calculations take the form of computing the relative binding free energy (RBFE) between ligands by alchemically mutating one ligand into another, both in solution and bound to the protein.^12–17 The ease at which such transformations can be computed robustly to high precision depends in part on the similarity of the ligands.^18–20 To take advantage of this in practice, a topological thermodynamic network can be constructed to connect ligands in such a way that their RBFEs can be optimally computed.^21–25 This network can be thought of as a “directed graph” where each edge corresponds to an alchemical transformation between ligands. When solving for the RBFE values between ligands, one can independently analyze the corresponding edge using an established free energy method such as Bennett’s Acceptance Ratio (BAR) method,²⁶ the multistate-BAR (MBAR) method,²⁷ unbinned weighted histogram analysis method (UWHAM),²⁸ or thermodynamic integration (TI) method²⁹ However, these original approaches will not guarantee that certain theoretical cycle closure constraints are obeyed, nor do they allow integration of experimental values as constraints into the analysis.

Herein we present a robust, efficient method for the network-wide BAR and MBAR analysis of RBFEs of entire sets of ligands with arbitrary linear constraints, including both theoretical cycle closure conditions and experimental (or generally derived) reference value constraints or restraints. The former leads to more precise computed values that obey the cycle closure conditions in the limit of infinite precision (complete sampling), whereas the latter leads to improved prediction for unknown ligands by constraining RBFE values of ligands with known binding affinities. These methods have been implemented into the graphmbar program distributed within FE-ToolKit, a new free energy analysis toolkit that is available from the authors.³⁰

A few methods have been introduced that enable the enforcement of theoretical cycle closure constraints.^31,32 Here, we build upon the MBAR/UWHAM equations^{27,28,33–35} which can be solved efficiently by non-linear optimization of a convex function.²⁸ Rather than solving these equations for each transformation independently, the simultaneous solution of all edges of the thermodynamic network are obtained by performing a constrained minimization of a global objective function. The nonlinear optimization of the global objective function is subjected to affine linear constraints that couple the free energies between the network edges. This is a general approach that is not restricted to simple cycle closure constraints, but could include select experimental or high-precision reference values, or any linear combination thereof. We formulate global objective functions that correspond to both BAR and MBAR solutions for the thermodynamic network, referred to as BARnet and MBARnet, respectively. Practical considerations in terms of computational efficiency and memory requirements for the network data are discussed. The methods are demonstrated in the calculation of RBFEs of 6 prototypical protein-ligand systems, and it is shown that enforcement of cycle closure conditions can lead to a modest reduction of the error in the predictions, and further error reduction can be achieved when one or two experimental RBFEs are included in the network analysis.

In order to establish context and motivation for the present methods, we outline a typical use case for alchemical free energy calculations in the lead optimization stage of drug discovery.^5,11 At the lead optimization phase, initial lead compounds have been identified through high throughput screening and lead generation. The goal of lead optimization is to develop and synthesize new compounds with improved potency, selectivity and pharmacokinetics. This optimization is achieved by creating trial modifications of initial lead compounds that are informed by structure-activity relationships, and in many cases structural data of the target-lead (protein-ligand) complex. Computational free energy simulations are used at this point to make predictions about the relative binding affinities (and in some cases selectivity) in order to prioritize the most promising compounds for synthesis and further characterization. The goal is often to rank a series of trial compounds that involve chemical modifications of a common molecular scaffold in terms of their binding affinity to the target protein. To achieve this, a thermodynamic network^21–24 is constructed such that the RBFEs of the series can be optimally computed, as discussed above. This network will contain the unknown compounds for which predictions are desired, but also contains some known reference compounds for which crystallographic and binding affinity data has been measured. As the free energy is a state function, the number of linearly independent RBFEs is N_lig−1, where N_lig is the number of ligands. However, the number of alchemical transformation edges in the thermodynamic network, N_edges, is typically considerably larger than the theoretical degrees of freedom (N_edges > N_lig − 1). This overdetermined set of computational variables (the free energy values for each edge transformation) give rise to a number of theoretical “cycle closure” conditions that need to be satisfied (this set of conditions is not unique, but has fixed rank). Further, as some of the compounds have been measured, the RBFEs between these compounds are also known. Nonetheless, by including these known compounds in the calculations along with the unknown compounds, the data can, in principle, be leveraged to improve the predictions for the RBFEs of the unknown compounds. More generally, if for some reason, it is known that values for certain sets of edges (or linear combinations thereof) are more reliable, then it might be advantageous to either constrain or restrain these values in the global optimization. It should be acknowledged that in some cases where data may be systematically biased, imposition of constraints or restraints could lead to worse predictions. In the present work, we create a tool to explore the use of experimental or other reference constraints (or restraints), in addition to the theoretical cycle closure constraints, in the global optimization of the network-wide free energy function with the goal to improve predictive capability.

2. Methods

Fast Solution for Large Scale MBAR/UWHAM Equations.

We begin by reviewing the MBAR/UWHAM equations,^{27,28,33–35} first derived in Ref. 28, using a notation based on the description found within Ref. 34. In the context of their work, they considered an alchemical transformation, e, that mutates state A to state B using M_e intermediate alchemical states (λ-states). In the present work, we use the subscript “e” to identify this transformation as particular edge within the RBFE graph. The goal is to calculate M_e free energy values, $G_{ie}^{*}$, where i indexes the value of λ-state within transformation e. Simulations are performed for the M_e states, each generating N_ie frames of coordinates $r_{ie}^k$, where k indexes the frame within the trajectory. Furthermore, the reduced potential energy u_ie (scaled by (k_BT)⁻¹, where k_B and T are the Boltmann constant and absolute temperature, respectively) of each state must be evaluated for each of the $N_e={\sum }_{i=1}^{M_e}{N}_{ie}$ frames. The MBAR/UWHAM objective function is shown in Eq. 1.

$$\begin{gathered} f\left(G_{1e}^{*},\cdots ,G_{M_{e}e}^{*}\right)=f\left(G_e^{*}\right) \\ =\frac{1}{N_e}\sum _{j=1}^{M_e}\sum _{k=1}^{N_{je}}\text{ln}\left(\sum _{l=1}^{M_e}\text{exp}\left(-\left[u_{le}\left(r_{je}^k\right)+b_{le}\right]\right)\right)+\sum _{i=1}^{M_e}\frac{N_{ie}}{N_e}{b}_{ie} \end{gathered}$$ (1)

The b_ie values (Eq. 2) are used for notational compactness.

$$b_{ie}=-\text{ln}\frac{N_{ie}}{N_e}-G_{ie}^{*}$$ (2)

The expression does not contain β = (k_BT)⁻¹ terms within it because it is presumed throughout this manuscript that the potential and free energies are in reduced energy units; that is, they have been pre-multiplied by β. The gradient of the objective function (Eq. 1), which may or may not be necessary depending on the chosen nonlinear optimization algorithm, is given in Eq. 3.

$$\frac{\partial f}{\partial G_{ie}^{*}}=\frac{1}{N_e}\sum _{j=1}^{M_e}\sum _{k=1}^{N_{je}}\frac{\text{exp}\left(-\left[u_{ie}\left(r_{je}^k\right)+b_{ie}\right]\right)}{{\sum }_{l=1}^{M_e}\text{exp}\left(-\left[u_{le}\left(r_{je}^k\right)+b_{le}\right]\right)}-\frac{N_{ie}}{N_e}$$ (3)

Network Optimization using the Multistate Bennet’s Acceptance Ratio Method.

We extend the MBAR/UWHAM equations, first derived in Ref. 28, by minimizing a global objective function that weights and sums Eq. 1 for each edge, and subjects the minimization to linear constraints that couple their simultaneous solution.

$$\begin{gathered} \text{min}F\left(G^{*}\right)=\text{min}\left\{\sum _e^{N_{\text{edges }}}{w}_{e}f\left(G_e^{*}\right)\right\} \\ \text{subject to }{h}_c\left(G_e^{*}\right)=0\text{ for }c=1,\cdots N_{\text{con.}} \end{gathered}$$ (4)

$$h_c\left(G_e^{*}\right)=\sum _e^{N_{\text{edges }}}\sum _i^{M_e}{C}_{\text{con.,}(c,ie)}{G}_{ie}^{*}-\Delta G_{\text{con.},c}^{*}$$ (5)

The w_e values weight each edge in the sum. In the current work, we set all weights to unity. N_con. is the number of constraints and $\Delta G_{\text{con.},c}^{*}$ is the target value of constraint c. The constraint is a linear combination of free energy values, where C_con.,(c,ie) is the contribution from λ-state i within edge e to contraint c. The grouping of the subscripts in C_con.,(c,ie) is meant to view this quantity as a matrix with N_con. rows and $M={\sum }_e^{N_{\text{edges}}}{M}_e$ columns.

The constraint coefficients are typically nonzero only for the λ = 0 and λ = 1 states of an alchemical transformation because the free energy of the process is the difference between those two states, ΔG = G (λ = 1) − G (λ = 0). As an example, if a ΔG value is constrained, then the elements of C_con.,(c,ie) corresponding to the G (λ = 1) and G (λ = 0) states would be 1 and −1, respectively. As a more complicated example, consider a case where N_trial independent simulations of an alchemical transformation are included in the analysis. Each trial will produce a slightly different ΔG value. A constraint on ΔG could be applied to each of the N_trial trials (one constraint for each independent trial); however, we prefer to apply a single constraint to the average 〈ΔG〉 value. In this case, the nonzero C_con.,(c,ie) values are $-N_{\text{trial}}^{-1}$ and $+N_{\text{trial}}^{-1}$ for each trial’s λ = 1 and λ = 0 states, respectively. The application of constraints to trial averages easily extends to more elaborate constraints. For example, an “edge free energy” – the free energy difference between two physical states – could be divided into a series of stages, such as “decharge”, “softcore Lennard-Jones”, and “recharge” stages. A constraint on the edge free energy average involves all λ = 0 and λ = 1 states from each trial of each stage. Furthermore, a constraint on a cycle closure average involves all λ = 0 and λ = 1 states from each trial of each stage for each edge tracing the closed path.

There are many numerical methods for performing the constrained optimization in Eq. 4. The constrained problem has only equality constraints, and the method of Lagrange multipliers could be used to convert it into an unconstrained problem involving N_con. + M parameters by constructing a Lagrange function $\mathcal{L}$ (Eq. 6) and searching for its saddle point(s): $\underset{\lambda }{\text{max}}\underset{G^{*}}{\text{ min}}\mathcal{L}$.

$$\mathcal{L}\left(G^{*},\lambda \right)=F\left(G^{*}\right)+\sum _c{\lambda }_c{h}_c\left(G^{*}\right)$$ (6)

This approach is not ideal only because many of the widely available numerical optimization software libraries are designed to find local minima (or maxima) rather than saddle points. Fortunately, many unconstrained optimization algorithms can be adapted to constrained problems via “the penalty method” or the closely related “augmented Lagrangian method”.^36,37 In the special case that the equality constraints are linear, one can use a “substitution method”,³⁸ whereby the explicit presence of constraint conditions are removed by replacing the full set of parameters by a smaller set of generalized parameters that only (but fully) span the space of feasible solutions.

The penalty method is a well-known approach for finding approximate solutions to constrained problems.^36,37 The method augments the primary objective function with penalty functions that deter the optimization algorithm from exploring the unfeasible solutions. The objective function using a quadratic penalty function to enforce equality constraints is: O(G*) = F(G*)+Σ_ck_ch_c(G*)². The procedure is to set k_c = 0 and optimize O to obtain a guess at the parameters. To enforce the constraints, k_c is increased by 10 (or some chosen amount) and O is reoptimized starting from the previous solution. The process of increasing k_c and reoptimizing the objective is repeated until the constraints are satisfied to within a desired tolerance. The constraints are strictly enforced as the k_c values approach infinity; however, if strict enforcement of the constraints are required, use of the augmented Lagrangian method should be prefered. If it is satisfactory to enforce the constraints to only several digits of accuracy, then the penalty method is quite efficient and widely applicable. One might consider the k_c values to be additional parameters introduced by the penalty method; however, it is better to view the penalty method as introducing a tolerance on the acceptable enforcement of the constraints – in much the same way that one places tolerances on the numerical algorithm to terminate the search for an optimal set of parameters.

The substition method can enforce linear equality constraints by performing the optimization in a reduced set of generalized parameters whose freedom is limited to the feasible regions of the constrained optimization.³⁸ The goal is to find a set of M_free = M − N_con. generalized parameters q that will always satisfy the constraints. A relationship must be found to express the full set of parameters as a function of the generalized parameters $G_{ie}^{*}(q)$ to rewrite the constrained optimization problem as an unconstrained optimization of the generalized parameters.

$$\underset{G^{*}}{\text{min}}F\left(G^{*}\right)\text{ subject to: }{h}_c=0,c\in \left[1,N_{\text{con.}}\right]\to \underset{q}{\text{min}}F\left(G^{*}(q)\right)$$ (7)

To begin, consider a solution for the parameters from the linear equality constraints (Eq. 8).

$$C_{\text{con.}}\cdot G^{*}=\Delta G_{\text{con.}}^{*}$$ (8)

If there are fewer linearly independent constraints than parameters, then a strictly-enforced, nonunique solution can be found from a generalized inverse; this solution will be denoted by $G_{ie}^{*,°}$.

$$\begin{gathered} G^{*,°}=C_{\text{con.}}^{+}\cdot \Delta G_{\text{con.}}^{*} \\ =V\cdot {\Sigma }^{+}\cdot U^T\cdot \Delta G_{\text{con.}}^{*} \end{gathered}$$ (9)

As we shall see below, Eq. 9 is only one of many possible solutions that satisfy the constraints, and we seek to find a formula that generalizes Eq. 9 to include all feasible solutions satisfying the constraints. The generalized inverse of C_con. (denoted $C_{\text{con.}}^{+}$) can be computed from a singular value decomposition (SVD, see Eq. 10), where U is a N_con. × N_con. matrix of left-singular vectors, V is a M × M matrix of right-singular vectors, and Σ is a N_con. × M matrix of singular values along its diagonal.

$$C_{\text{con.}}=U\cdot \Sigma \cdot V^T$$ (10)

Given N_con. linearly independent constraints, the diagonal of Σ will contain N_con. nonzero elements. The remaining M_free columns of Σ are the null space of C_con., and the corresponding M_free rows of V^T span the space of feasible solutions satisfying the constraints. Let us define a M_free × M transformation matrix, T, whose M_free rows are the row vectors of V^T spanning the null space C_con.. Perturbing Eq. 9 by q^T · T for any vector q will continue to satisfy the constraints; therefore, the general expression for the parameters that satisfy the constraints is given by Eq. 11.

$$G_{ie}^{*}=G_{ie}^{*,°}+\sum _{j=1}^{M_{\text{free }}}{q}_j{T}_{j,(ie)}$$ (11)

By rearranging Eq. 11, one can derive the reverse transformation for the generalized parameters.

$$q_i=\sum _{e=1}^{N_{\text{edges }}}\sum _{j=1}^{M_e}{T}_{i,(je)}{G}_{je}^{*}$$ (12)

There are three types of constraints that we will consider: (1) We constrain the free energy of each λ = 0 alchemical state to be zero. This constraint does not have a practical effect on the results other than improving the readability of the output. (2) If a partial list of reference RBFEs are known, then we constrain the calculated RBFEs to match the reference values. The motivation behind this is for situations when a few reference RBFEs (either experimental RBFE values or highly-converged simulation results) are known and one attempts to use that partial knowledge to aid the prediction of RBFEs that remain experimentally unknown. (3) We enforce thermodynamic cycle closure conditions. The free energy is a thermodynamic state function; therefore, the sum of free energies along a closed path should be zero. The reader is free to implement the constraints using whatever method is readily available to them. In the present work, we choose to use the substitution method described above for linear equality conditions for enforcing the first two types of constraints described above. One advantage of our use of singular value decompositions to come up with generalized coordinates that obey the constraint conditions is that the constraint coefficient matrix itself can be overdetermined. From a user perspective, this is convenient in that one can impose multiple redundant cycle closure and other constraint conditions without concern of linear dependencies, and the SVD provides a robust method for determining the generalized coordinates that spanned by the free parameters under the constraint conditions.

We also use the penalty method for enforcing the cycle closure constraints, because of the potential of encountering linear dependencies between the cycle closure conditions. In principle, the presence of linear dependencies is not an insurmountable obstacle; however, in practice the singular vectors defining the free parameters can become highly oscillatory as a linear dependency is approached, thereby limiting the effective precision due to round off error. Our experience is that cycle closure constraints are often satisfied to within 0.001 kcal/mol using restraint force constants of 10(k_BT)², which is much smaller than the uncertainty in the free energy values. This degree of constraint enforcement is sufficient for our purposes, so the penalty method can be terminated after the first use of nonzero k_c values. The penalty method, in our application, can thus be viewed as a “restraint” applied to the primary objective function. In this view, the cycle closure constraint penalties can be explicitly written into the objective function and refered to as restraints, denoted by the subscript “res.”.

$$F\left(G^{*}\right)=\sum _e^{N_{\text{edges }}}{w}_{e}f\left(G_e^{*}\right)+\sum _r^{N_{\text{res.}}}{k}_r{\left(\sum _e^{N_{\text{edges}}}\sum _i^{M_e}{C}_{\text{res.,}(r,ie)}{G}_{ie}^{*}-\Delta G_{\text{res.},r}^{*}\right)}^2$$ (13)

The algorithm for finding and choosing the closed paths, given a list of edges (a list of molecule pairs), follows:

• Assign each molecule a unique (but otherwise arbitrary) index.

• For each molecule, generate a list of directly connected neighbors from the set of edges.

• For each connected neighbor, use a Depth First Traversal algorithm to find the shortest path(s) that connects the molecule to the neighbor, excluding the direct connection. The resulting path is a list of molecules starting with the entry molecule and ending with the connected neighbor.

• To avoid redundant duplication of the same path with others that may differ only from the starting molecule or traversal direction, shift the entries in the path list such that the first element has the lowest molecule index and the second element has a lower molecule index than the last element. This second condition controls the “clockwise-ness” of the cycle.

• Append the end of the path list with the first element to close the cycle.

• If the path has not yet been restrained, then include it as an additional restraint.

The algorithm does not include all possible closures; instead, it includes all smallest closed paths such that the selected paths do not encircle two or more smaller closed paths. Although this was our chosen algorithm, other choices for selecting the cycle closure conditions are certainly possible.

As a technical note, the constrained objective function F is convex if w_e ≥ 0 for all edges, because each f is convex and the sum of convex functions is also convex. Furthermore, one can show that the Hessian in generalized parameters, ∂²F/∂q_i∂qj, is positive semidefinite because the Hessian in the full set of paramters, ${\partial }^{2}F/\partial G_{ie}^{*}\partial G_{jf}^{*}$, is positive semidefinite²⁸ and there is a linear relationship between the generalized set and full set of parameters (Eq. 11).

In summary, the MBARnet procedure consists of the following steps:

• Read the potential energies from file and convert to reduced energy units.

• Make an initial guess for the reduced free energies $G_{je}^{*}$.

• Generate the generalized coordinate transformation matrix T_i,(je) and vector $G_{je}^{*,°}$ from singular value decomposition of the constraint matrix (Eqs. 9–10 and).

• Use Eq. 12 to obtain an intial guess for the free parameters.

• Initiate the nonlinear optimizer, providing it the objective function and M_free generalized coordinates.

• For each objective function evaluation, convert the generalized coordinates to $G_{je}^{*}$ values and evaluate Eq. 11.

• If the optimization method requires parameter gradients, then evaluate Eqs. 14 and 15.

• When a minimum is found, convert the generalized coordinates to reduced free energies and divide them by the appropriate value β to express them with the desired energy units.

$$\frac{\partial F}{\partial G_{ie}^{*}}=w_e\frac{\partial f\left(G_e^{*}\right)}{\partial G_{ie}^{*}}+2\sum _{r=1}^{N_{\text{res.}}}{k}_r\left(C_{\text{res}.,(r,ie)}-\Delta G_{\text{res}.,r}^{*}\right)$$ (14)

$$\frac{\partial F}{\partial q_i}=\sum _{e=1}^{N_{\text{edges}}}\sum _{j=1}^{M_e}\frac{\partial F}{\partial G_{je}^{*}}{T}_{i,(je)}$$ (15)

Network Optimization using the Bennett Acceptance Ratio Method.

We extend the MBARnet method by defining an objective function for BAR analysis such that the free energy network can be similarly constrained and restrained during the optimization. The BARnet optimization is similar to Eq. 13.

$$\begin{gathered} \text{min}{F}_{\text{BAR}}\left(G^{*}\right)=\text{min}\left\{\sum _e^{N_{\text{edges}}}{w}_e{f}_{\text{BAR}}\left(G_e^{*}\right)+\sum _r^{N_{\text{res.}}}{k}_r{\left(\sum _e^{N_{\text{edges}}}\sum _i^{M_e}{C}_{\text{res}.,(r,ie)}{G}_{ie}^{*}-\Delta G_{\text{res}.,r}^{*}\right)}^2\right\} \\ \text{subject to }\sum _e^{N_{\text{edges}}}\sum _i^{M_e}{C}_{\text{con.,}(c,ie)}{G}_{ie}^{*}=\Delta G_{\text{con.,}c}^{*}\text{ for }c=1,\cdots ,N_{\text{con.}} \end{gathered}$$ (16)

The f_BAR objective function is a sum of BAR objective functions corresponding to each adjacent pair of alchemical states:

$$f_{\text{BAR}}\left(G_e^{*}\right)=\sum _{u=1}^{M_e-1}f\left(G_{ue}^{*},G_{u+1,e}^{*}\right)$$ (17)

For example, the expression for $f\left(G_{ue}^{*},G_{u+1,e}^{*}\right)$ is shown in Eq. 18.

$$\begin{gathered} f\left(G_{ue}^{*},G_{u+1,e}^{*}\right)=\frac{1}{N_{ue}+N_{u+1,e}}\sum _{j=u}^{u+1}\sum _{k=1}^{N_{je}}\text{ln}\left(\sum _{l=u}^{u+1}\text{exp}\left(-\left[u_{le}\left(r_{je}^k\right)+b_{le}\right]\right)\right) \\ +\sum _{i=u}^{u+1}\frac{N_{ie}}{N_{ue}+N_{u+1,e}}{b}_{ie} \end{gathered}$$ (18)

Bootstrap Error Analysis.

To estimate the BARnet and MBARnet errors in the calculated values of $G_{ie}^{*}$, we perform many optimizations of F to obtain many optimal sets of generalized coordinate parameters. The optimizations differ by having constructed new ensembles for each state by random sampling with replacement. To account for correlation within the data, we calculate the statistical inefficiency of each trajectory’s reduced potential energy timeseries and group the trajectory into blocks, whose size is chosen to be twice the statistical inefficiency. The bootstrap is performed blockwise by sampling from the available blocks. The resulting distributions for each generalized coordinate has a mean value ${\overline{q}}_i$ and unbiased sample variance $S_{q_i}^2$. Given sufficient resampling effort, the transformation of the average values will match the optimized parameters from the initial ensembles; that is,

$$G_{ie}^{*}=G_{ie}^{*,°}+\sum _j^{M_{\text{free}}}{\overline{q}}_j{T}_{j,(ie)}$$ (19)

The standard errors of $G_{ie}^{*}$ are propagated from the generalized coordinate variances:

$${\sigma }_{G_{ie}^{*}}=\sqrt{\sum _j^{M_{\text{free }}}{s}_{q_j}^2{T}_{j,(ie)}^2}$$ (20)

In the present work, we estimate the errors from 300 bootstrap calculations.

Network Anaylsis using Multiple, Independent Simulations.

The bootstrap BARnet and MBARnet error analysis provides a measure of the uncertainty caused by fluctuations within the observed ensembles. Because the simulations are performed for a finite length of time, the observed ensembles are only an approximation of the theoretically converged ensembles generated from infinite sampling. One can estimate the error caused by finite time length simulations by performing multiple, independent trial simulations that differ only by their initial conditions. This is sometimes called the “Ensemble Average Approach”.^39,40 Each simulation’s trial is included in the global optimization and they are each optimized with their own free energy parameters. The constraints and restraints involving the multiple trials are chosen such that the average value across all trials satisfy the condition, rather than having each trial satisfy the condition.⁴¹ When performing the error analysis, we combine the standard deviation among the trials with the bootstrap errors from each trial.⁴² For notational purposes, let $G_{i_{t}e}^{*}$ be the free energy of trial t of state i within edge e, and ${\sigma }_{G_{i_{t}e}^{*}}$ is the corresponding standard error from bootstrap analysis. We compute the average-across-trials for state i in edge e, $G_{\overline{i}e}^{*}$, and combined standard error, ${\sigma }_{G_{\overline{i}e}^{*}}$, from Eqs. 21 and 22, respectively.

$$G_{\overline{i}e}^{*}=\frac{1}{N_{\text{trial}}}\sum _{t=1}^{N_{\text{trial}}}{G}_{i_{t}e}^{*}$$ (21)

$${\sigma }_{G_{\overline{i}e}^{*}}=\sqrt{N_{\text{trial}}^{-1}(\sum _{t=1}^{N_{\text{trial}}}{\sigma }_{G_{i_{t}e}^{*}}^2+\sum _{t=1}^{N_{\text{trial}}}\frac{{\left[G_{i_{t}e}^{*}-G_{\overline{i}e}^{*}\right]}^2}{N_{\text{trial}}-1})}$$ (22)

Adjustment of edge weights.

The w_e weights appearing in Eqs. 4 and 16 are unity in the present work. If constraints are not applied to the objective functions, then the unequal weighting would not effect the result because each edge is decoupled from all other edges. When a constrained optimization is performed, the parameters (free energies) within each edge become coupled throughout the thermodynamic network and the optimization solution then depends on the relative weight applied to each edge. Undoubtedly, approaches can be adopted such that these weights can be adjusted to improve robustness and predictive capability. This is an area of ongoing active research, but not one that we are able to address in the present work. We note that one of the major challenges to developing any such approach is the lack of very high-precision benchmark quality simulation results for non-trivial protein-ligand systems that can serve as target reference data. Nonetheless, an important direction for future research is to test different approaches for adjustment of the weights in order to reduce statistical errors and improve predictions.

Computational details.

The tables and figures summarize the RBFE analysis of CDK2 (PDBID: 1H1Q),⁴³ MCL1 (PDBID: 4HW3),⁴⁴ p38 (PDBID: 3FLY), Tyk2 (PDBID: 4GIH),⁴⁵ PTP1B (PDBID: 2QBS),⁴⁶ and Thrombin (PDBID: 2ZFF) protein targets previously studied in Ref. 8 The CDK2 system has 16 ligands whose RBFEs are connected by 25 edges. The MCL1, p38, Tyk2, PTP1B, and Thrombin systems have 42, 33, 16, 23, 10 ligands, respectively, connected by 71, 54, 24, 48, 10 edges, respectively. Each ligand transformation is performed in three stages (decharge, softcore, and recharge) and two environments (protein-bound and in solution), and the RBFE is the free energy difference between the protein-bound and aqueous-phase transformation free energies. The decharge stage removes the charges of the atoms that are being deleted; the recharge stage adds the charges of the atoms that being inserted. The softcore stage linearly mutates the remaining potential energy terms between the initial and final states except for the Lennard-Jones (LJ) interactions, which are modeled using the (nonlinear) softcore LJ potential described in Ref. 47. For completeness, the general form of the alchemical potential energy function is given by Eqs. 23–27.

$$\begin{gathered} U(\lambda )=(1-\lambda )U_{\text{elec.}}^{(0)}+\lambda U_{\text{elec.}}^{(1)} \\ +(1-\lambda )U_{\text{bonded}}^{(0)}+\lambda U_{\text{bonded}}^{(1)} \\ +U_{\text{SCLJ}}(\lambda ) \end{gathered}$$ (23)

$$\begin{gathered} U_{\text{bonded}}^{(0)}=\sum _{b=1}^{N_{\text{bonds}}}{k}_b^{(0)}{\left(r-r_0^{(0)}\right)}^2 \\ +\sum _{a=1}^{N_{\text{angles}}}{k}_{\theta ,a}^{(0)}{\left(\theta -{\theta }_0^{(0)}\right)}^2 \\ +\sum _{d=1}^{N_{\text{dihed.}}}\sum _n\frac{V_{in}^{(0)}}{2}\left[1+\text{cos}\left(n{\omega }_i-{\gamma }_i^{(0)}\right)\right] \end{gathered}$$ (24)

$$U_{\text{elec.}}^{(0)}=\frac{q_i^{(0)}{q}_j^{(0)}}{r_{ij}}$$ (25)

$$U_{\text{SCLJ}}(\lambda )=(1-\lambda )4{ϵ}_{ij}\left(u^{-2}-u^{-1}\right)$$ (26)

$$u=\alpha \lambda +{\frac{r_{ij}}{{\sigma }_{ij}}}^6$$ (27)

The decharge and recharge stages linearly scale the electrostatic interactions U_elec. The superscript (0) within Eq. 25, for example, indicate that the parameter values describe the λ = 0 state. Similar equations can be written for the λ = 1 state. The bonded energy (Eq. 24) contains terms that model bond, angle, and dihedral components. The k_b and k_θ values are spring force constants. The r₀ and θ₀ values are spring equilibrium positions. The V_in values control the mangitude of the periodic torsion potential, and γ_i is a phase offset. The q_i values are electric charges, and ϵ_ij and σ_ij are the LJ well-depth and the point where the LJ crosses zero, respectively. The α = 0.5 is a control parameter of the softcore LJ potential. The softcore stage linearly transforms the bonded energy and nonlinearly modifies the nonbonded Lennard-Jones interactions using Eqs. 26–27. The decharge and recharge stages were performed using 5 evenly-spaced λ states. The softcore stages were performed using 12 states: λ = 0.0, 0.0479, 0.1151, 0.2063, 0.3161, 0.4374, 0.5626, 0.6839, 0.7937, 0.885, 0.9521, and 1.0. Each simulation was performed with Amber’s graphics processing unit (GPU) accelerated version of PMEMD for 2 ns using a 4 fs timestep and hydrogen mass repartitioning.^42,48,49 The ligand was modeled using the GAFF2 force field,⁵⁰ and the condensed phase environment was explicitly modeled with TIP3P⁵¹ waters. MBAR potential energies were output every 100 steps (0.4 ps). Each simulation was performed 10 times with differing initial random number seeds.

The simulations were run in the isothermal-isobaric ensemble (NPT). Pressure was regulated with Berendsen barostat to maintain a pressure of 1 atm using a 5 ps collision frequency.⁵² The Langevin thermostat was used to maintain a temperature of 298.15 K.⁵³ The Lennard-Jones potential was truncated at 8 Å, and a a long-range tail correction is used to model the interactions beyond the cutoff. The long-range electrostatics were evaluated with the particle mesh Ewald method using a 1 ų grid spacing.^54,55 The simulation data was taken from Ref. 56 and further details can be found therein.

To perform the nonlinear optimizations of the BARnet and MBARnet global objective functions, we used the Low-storage Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) algorithm⁵⁷ implemented in the NLopt software library.⁵⁸ When constraints and restraints are unused, the initial guess for each free energy is zero. When constraints and/or restraints are used, we first perform an optimization without constraints nor restraints and then reoptimize the free energies with the constraints and/or restraints activated.

The simulations were performed on NVidia GeForce GTX 1080 Ti GPUs. The protein-bound and solution-phase simulations require approximately 0.3 and 0.06 GPU hours to complete, respectively. Each edge requires approximately 80 GPU hours to complete 10 trials of each simulation in both environments. The 6 protein systems consist of a total of 232 edges, corresponding to an aggregate of 2 GPU years of simulation.

After the simulations are performed, the MBAR energies are extracted from the output files. We store the energies in energy timeseries files. A timeseries file is a text file containing two columns of numbers. The first column is the simulation time and the second column is a potential energy. MBAR requires each λ state trajectory be evaluated with all λ state potentials within the stage. For example, a single trial of a decharge stage produces 25 timeseries files because the stage is performed with 5 λ states. All 3 stages of a single trial produce 194 files. Considering that the transformations need to be performed in 2 environments and repeated 10 times, each edge produces 3880 files. Each timeseries file contains 5000 rows and uses 140 KB of disk storage. The storage of each edge’s timeseries files thus requires 530.5 MB of disk space.

The calculated RBFEs will be compared to experimental values. The experimental ligand binding free energies for the protein systems examined in this work were compiled in Ref. 8. The CDK2⁵⁹ and P38⁶⁰ binding free energies were computed from the reported IC50 values using Eq. 28.

$$\Delta G_{\text{expt.}}=RT\text{ ln}IC50$$ (28)

The Tyk2,^61,62 MCL1,⁶³ and PTP1B⁶⁴ binding free energies were computed from the reported K_i dissociation constants using Eq. 29

$$\Delta G_{\text{expt.}}=RT\text{ ln}{K}_i$$ (29)

The Thrombin^8,65 binding free energies were obtained from isothermal titration calorimetry.

3. Results and Discussion

Figure 1 illustrates the correspondence of the BARnet/MBARnet results with and without cycle closure constraints for the set of 71 MCL1 RBFEs. The RBFEs computed with BARnet and MBARnet are in good agreement and yield similar error estimates. When the MBARnet analysis is treated as the target values, the MCL1 MUE of BARnet is only 0.024 to 0.040 kcal/mol, depending on the use of cycle closure restraints, which is an order of magnitude smaller than the uncertainties in both the BARnet and MBARnet values. Analogous comparisons for the other protein RBFEs yield similar results. The BARnet method requires far fewer reduced potential energies to be stored, which is its primary advantage when performing network-wide analysis. For the specific case of the MCL1 RBFEs, MBARnet analysis requires 37 gigabytes (GB) of potential energy timeseries files, whereas BARnet analysis requires storage of only 12 GB of raw data. Only MBARnet results will be presented and discussed henceforth because of the similarity between the BARnet and MBARnet results.

Figure 1:

Comparison between BARnet and MBARnet RBFEs for MCL1 both (a) without and (b) with cycle closure restraints. The red line is a linear fit to the data. The error bars are the standard error of the calculated values.

Table 1 summarizes the number of restrained thermodynamic cycles in the RBFE network and the associated error in the free energy closure conditions. The cycles included in the summary only include those thermodynamic paths that cannot be decomposed into two or more smaller cycles. The number of closed paths included in the summary is shown in the column labeled N_res.. The free energy is a state function, so the sum of free energies along a closed path should theoretically be zero. In practice, limited sampling often causes the free energy sum to erroneously be nonzero. We compute the free energy sum for each of the N_res. cycles and report the average and standard deviation of the N_res. error values in the ΣΔG* and σ columns, respectively. The cycle closure errors are less than 1 kcal/mol on average when restraints are not applied. Application of restraints within the optimization procedure lower the cycle closure errors to 0.001 kcal/mol.

Table 1:

Cycle closure information for each system. The N_lig. and N_edge columns list the number of ligands and connected edges in the transformation graph, respectively. The N_res. values are the number of thermodynamic cycles included in the summary. The columns labeled “CCR” and “no CCR” indicate whether cycle closure restraints are applied to the optimization. The ΣΔG* and σ columns report the average and standard deviation of the cycle closure errors, respectively.

System	Nlig.	Nedge	Nres.	no CCR		CCR
				Σ ΔG*	σ	Σ ΔG*	σ
CDK2	16	25	22	0.88	1.65	0.00	0.00
P38	33	54	42	0.83	1.42	0.00	0.00
MCL1	42	71	70	0.91	1.20	0.00	0.00
Tyk2	16	24	18	0.24	0.35	0.00	0.00
PTP1B	23	48	50	0.47	0.68	0.00	0.00
Thrombin	10	10	2	0.13	0.17	0.00	0.00

Table 2 summarizes the edge RBFE mean unsigned errors (MUEs) with and without cycle closure restraints for each protein target, and it further examines how the edge RBFE MUEs are affected when 0, 1, or 2 edges in the graph are constrained to match the experimental RBFE. The MUEs measure the agreement between the calculated RBFEs and corresponding differences between experimental binding free energy values. For a protein target graph consisting of N_edge edges, there are N_edge possible ways of constraining 1 edge. The RBFEs are reoptimized using each of the possible constraint conditions, and the values reported in the table are the mean and standard deviation from the distribution of MUEs. There is only one MUE to consider when there are no experimental constraints, so no standard deviation is reported in this case. When two edges are constrained, there are formally N_edge(N_edge − 1)/2 possible constraint conditions; however, to generate the statistics, we randomly selected 100 constraint conditions to generate the MUE distribution. The use of cycle restraints appears to lower the MUEs in most cases, but the average change (less than 0.1 kcal/mol) is less than the uncertainty of the calculations. The Tyk2 RBFE MUE relative to experiment increased by 0.01 kcal/mol upon enforcement of the cycle closure conditions. This observation emphasizes that there is no guarantee that closure conditions enforcement alone will cause better agreement with experiment. The magnitude of this change is much smaller than the calculation’s uncertainty and a rigorous exploration of systematic bias in the comparison requires a set of highly converged simulation results, which may necessitate further development of enhanced sampling techniques to sufficiently explore the ensemble of bound ligand conformations. As expected, including experimental RBFE constraints decreases the MUEs. When cycle closure restraints are also included, the reduction is amplified because the constraint(s) then effect the solution for the other RBFEs via their coupling through the restraints. The improvements continue to be modest, however, with MUE reductions on the order of 0.1 kcal/mol when two experimental constraints are applied.

Table 2:

MBARnet-calculated RBFE average mean unsigned errors relative to experiment when 0, 1, or 2 graph edges are constrained to match experiment.

System	Number of Reference (Expt.) Constraints
	0		1		2
	no CCR	CCR	no CCR	CCR	no CCR	CCR
CDK2	0.95	0.93	0.91 ± 0.03	0.88 ± 0.05	0.87 ± 0.04	0.82 ± 0.07
P38	0.70	0.66	0.69 ± 0.01	0.64 ± 0.02	0.68 ± 0.01	0.63 ± 0.02
MCL1	1.31	1.28	1.29 ± 0.02	1.25 ± 0.04	1.27 ± 0.02	1.21 ± 0.08
Tyk2	0.96	0.97	0.92 ± 0.03	0.91 ± 0.05	0.89 ± 0.04	0.86 ± 0.06
PTP1B	0.90	0.89	0.88 ± 0.02	0.85 ± 0.06	0.86 ± 0.02	0.81 ± 0.07
Thrombin	0.41	0.41	0.37 ± 0.04	0.37 ± 0.03	0.33 ± 0.05	0.32 ± 0.05

Figure 2 compares the convergence of the MBARnet-analyzed RBFEs with and without cycle restraints. In the context of this figure, our interest is comparing how much data each method requires to approach the force field’s expected result; therefore, the reference RBFEs are computed from MBARnet using all available production data and optimized with cycle restraints. The abscissa of the plots shown in Figure 2 are percentages of the production data used in the analysis. For example, the values shown at 10% are the RBFE MUEs when only the first 1/10^th of the production data is analyzed. In addition to illustrating the convergence of the RBFEs with and without cycle restraints, we also make comparison to the maximum likelihood estimator (MLE) method described in Ref. 32 for enforcing cycle closure conditions. Unlike the optimization method described in this work, the MLE method does not enforce cycle closures from the analysis of the raw data. Instead, the MLE method maximizes the following objective function to obtain cycle-corrected estimates of the RBFEs, $\Delta G_{a\to b}^{\text{MLE}}$, provided one’s best estimate of the RBFEs ΔG_a→b and their standard errors ${\sigma }_{\Delta G_{a\to b}}$.

$$\begin{gathered} \text{max}\sum _{c=1}^{N_{\text{cyc}}}\prod _{e\in c}^{N_{\text{edge}}}\frac{\text{exp}\left(-\frac{{|\Delta G_e^{\text{MLE}}-\Delta G_e|}^2}{2{\sigma }_{\Delta G_e}^2}\right)}{\sqrt{2\pi {\sigma }_{\Delta G_e}^2}} \\ \text{subject to}\sum _{e\in c}^{N_{\text{edge}}}\Delta G_e^{\text{MLE}}=0\text{ for each cycle},c \end{gathered}$$ (30)

The product operator appearing in Eq. 30 multiplies the normal distributions of each edge, e, in the cycle c.

Figure 2:

Convergence of the MBARnet mean unsigned errors relative to the MBARnet analysis of all production data with cycle restraints. Filled black circles: MBARnet optimization with cycle restraints. Open red circles: MBARnet optimization with cycle restraints and post-optimization MLE correction. Filled green squares: MBARnet optimization without cycle restraints. Open blue squares: MBARnet optimization without cycle restraints and post-optimization MLE correction.

Figure 2 shows that inclusion of cycle closure restraints in the optimization of partial sets of data produces results that match the analysis of the complete set of data more closely than other approaches. The MBARnet optimizations without cycle closure restraints yield the largest MUEs. Application of the MLE method to our cycle-restrained optimized RBFEs has no effect because, in this case, our RBFEs already enforce the cycle closure conditions. For this reason, the red and black circles appearing in Figure 2 always coincide. When the MLE method is applied to the unrestrained MBARnet results, the RBFE MUEs are reduced, and they appear to approach our cycle-restrained MBARnet results. The extent to which the MLE method succeeds in approaching our cycle-restrained results varies. The MLE method does well for Tyk2 and Thrombin likely because the unrestrained MBAR results on which they are based are already similar to the cycle-restrained values.

4. Conclusions

We develop BARnet and MBARnet methods for use in network-wide free energy analysis with restraints and affine linear constraints. The BARnet and MBARnet results are nearly identical (within 0.04 kcal/mol), however the BARnet objective function requires only a fraction of the amount of disk storage relative to the MBARnet approach. Restraints were used in the non-linear optimization to enforce the closure of thermodynamic cycles within the free energy network, and the constraints were chosen to enforce the reproduction of known RBFEs. The utility of the constraints is demonstrated in situations where a partial list of experimental free energies are known, in which case the solution for the other RBFEs are affected by coupling their solutions through the cycle closure restraints. We analyzed the RBFEs of 6 protein targets and showed that the use of cycle closure restraints yields a modest improvement relative to the experimental RBFEs, and the estimates were improved more substantially when one or two constraints to experimental values were included. The BARnet/MBARnet framework enables efficient and robust free energy analysis with enhanced predictive capability for drug discovery applications.