Finite-sample bias in free energy bridge estimators

Abstract

In practical free energy estimation, the bias is often neglected once it has been shown to vanish in the large-sample limit. Yet finite-sample bias always exists and ought to be considered in any rigorous study. This work develops a metric for bias in a broad class of free energy “bridge estimators” (e.g., Bennett’s method). The framework complements existing variance estimation methods and provides a means for comparing systematic and statistical errors. Examples show that, contrary to what is often assumed, the bias can be quite substantial when the sample size is modest.

I. INTRODUCTION

Accurate and precise free energy calculations are a central aim of statistical physics and molecular simulations.^1–3 Indeed, numerous methods have been developed for this purpose,^4–16 although all of them roughly fall into the two categories of thermodynamic integration (TI)⁴ and free energy perturbation (FEP)⁵ (including analogs based on nonequilibrium processes^17,18). Much effort has been exerted in evaluating the strengths and weaknesses of these approaches,^19–25 and the main concern is the trade-off between errors due to systematic bias and statistical variance.

It was noted by Shirts and Chodera¹⁴ that nearly all methods in the FEP category can be viewed within the “bridge estimator” framework from the statistics literature. Consequently, most FEP-like estimators are now rigorously known to be unbiased in the large-sample limit.²⁶ The framework also provides a general path for developing analytic estimates of the variance.^14,16 By contrast, there is rarely, if ever, an a priori guarantee that TI is asymptotically unbiased. Accordingly, the general practice for free energy calculations of both kinds is simply to estimate sample variances and then assume that statistical noise exceeds any and all bias. However, just because an estimator is asymptotically unbiased does not mean that a finite sample does not contain bias nor does it strictly guarantee that statistical error dominates.²⁷ Therefore, it would be of significant use to have a practical quantitative metric for bias.

This work develops a simple protocol for estimating bias in free energy calculations based on bridge estimators. This includes the widely used Bennett acceptance ratio (BAR) method⁸ as well as its close cousin simple overlap sampling (SOS).¹³ Ultimately, the approach generalizes an ansatz introduced by Lu et al. to analyze conventional FEP calculations.^13,28–30 In addition to practical computations, the framework here permits a stronger understanding of why methods like BAR are so effective and offers new avenues to think about the relationships between different sources of error.

II. THEORY AND BACKGROUND

Following the standard development of free energy calculations,^1–3 the relative free energy Δf (in reduced units, absorbing factors of k_BT) between two ensembles (or “states”) is being sought. As is customary in FEP, Δf is defined in terms of a transition from some “forward” ensemble to a “backward” ensemble—the subscripts “f” and “b” will be used as appropriate. Data collection in each ensemble proceeds by gathering the reduced energy difference Δu between states (defined as the reduced energy in the backward ensemble less that in the forward ensemble). Sampling from the two ensembles thus consists in drawing values of Δu from the probability distribution functions (PDFs) p_f and p_b. Ensemble averages (indicated by angle brackets ⟨⋅⟩) can be computed from these PDFs in the usual way.

In a seminal work, Bennett⁸ derived exact and finite-sample relationships for Δf computed from two ensembles (see also the work of Shirts et al.¹⁸). Most notably, he showed that Δf can be written in terms of an arbitrary function α which is now identified as a bridge estimator as follows:^14,26

$$e^{\epsilon -\mathrm{\Delta }f}=\frac{{⟨\alpha \left(\mathrm{\Delta }u-\epsilon \right)⟩}_f}{{⟨e^{\mathrm{\Delta }u-\epsilon }\alpha \left(\mathrm{\Delta }u-\epsilon \right)⟩}_b}.$$ (1)

Here, ε is an arbitrary energy shift parameter. A related perspective was articulated in the overlap sampling framework explored by Lu, Singh, and Kofke.¹³

Bennett derived Eq. (1) by invoking detailed balance and interpreting α as the acceptance ratio for a special kind of post hoc Monte Carlo move (hence, the name BAR). A similar view is also at the core of Crooks’ theorems pertaining to nonequilibrium work,^31,32 and it is worth noting that the results here hold equally well in the case that the Δu are work values. Bennett’s particular choice, α(x) = exp(−x/2)sech(x/2), belongs to a class of functions with the property exp(x)α(x) = α(−x) such that the denominator in Eq. (1) becomes ⟨α(ε − Δu)⟩_b. This equality holds for essentially all estimators in common use and will be used hereafter for simplicity but is otherwise unnecessary.

In the context of a finite-sample estimate of Eq. (1) with (effective) sample sizes N_f and N_b, Bennett also showed that there is a unique, self-consistent value $\epsilon =\mathrm{\Delta }\widehat{f}-\ln \left(N_f/N_b\right)$ that, along with his choice for α, minimizes the asymptotic variance of the free energy estimate $\mathrm{\Delta }\widehat{f}$. Using a similar line of reasoning, Lu, Singh, and Kofke¹³ suggested that the even simpler choice α(x) = exp(−x/2) with ε = 0 (which they called SOS) is nearly as optimal as BAR, at least when Δf is small. Yet another choice briefly considered by Bennett is the standard Metropolis criterion α(x) = min[1, exp(−x)]. All of these choices are everywhere positive and monotonically decrease to zero. These characteristics are hereafter assumed.

A. Bias in FEP calculations

Long after Bennett’s work, Lu and Kofke^28,29 presented a systematic error analysis for conventional FEP calculations. Standard FEP eschews the two-ensemble approach by making two separate choices: α_f(x) = exp(−x) (“forward FEP”) and α_b(x) = 1 (“backward FEP”)—each of these eliminates one of the ensemble averages. In a variation on the previous identity, these choices satisfy the relation exp(x)α_f(x) = α_b(−x). Unfortunately, both types of FEP have rather disappointing properties.^27–29 In particular, the forward estimate $\mathrm{\Delta }{\widehat{f}}_f$ tends to overestimate Δf (i.e., the bias is positive) while the backward estimate $\mathrm{\Delta }{\widehat{f}}_b$ tends to underestimate it (i.e., the bias is negative).

Lu and Kofke²⁸ demonstrated the bias in FEP by first noting that even very large samples tend to leave the ensembles undersampled beyond some threshold value. That is, samples from the forward ensemble neglect values below some cutoff Δu_f while backward samples neglect values above another cutoff Δu_b > Δu_f. Using this threshold ansatz, it can be shown that the biases are²⁸

$$\mathrm{\Delta }{\widehat{f}}_f-\mathrm{\Delta }f=-\ln \left[1-P_b\left(\mathrm{\Delta }{u}_f\right)\right]\approx +P_b\left(\mathrm{\Delta }{u}_f\right),$$ (2a)

$$\mathrm{\Delta }{\widehat{f}}_b-\mathrm{\Delta }f=+\ln \left[1-\overline{P_f}\left(\mathrm{\Delta }{u}_b\right)\right]\approx -\overline{P_f}\left(\mathrm{\Delta }{u}_b\right),$$ (2b)

where P_b is the cumulative distribution function (CDF) associated with p_b and $\overline{P_f}\left(x\right)\equiv 1-P_f\left(x\right)$ is the complementary CDF associated with p_f,

$$P_b\left(\mathrm{\Delta }{u}_f\right)={\int }_{-\infty }^{\mathrm{\Delta }{u}_f}d\mathrm{\Delta }u{p}_b\left(\mathrm{\Delta }u\right),$$ (3a)

$$\overline{P_f}\left(\mathrm{\Delta }{u}_b\right)={\int }_{\mathrm{\Delta }{u}_b}^{\infty }d\mathrm{\Delta }u{p}_f\left(\mathrm{\Delta }u\right).$$ (3b)

Examples of these PDFs and sample threshold values are shown in the top half of Fig. 1. Taylor expansions for small values of the bias (or CDFs) make the signs more explicit.

FIG. 1.

Viewed graphically, the threshold ansatz implies that the bias is proportional to the area under the distributions beyond the sample threshold. In this example, the threshold values Δu_f and Δu_b are observed from short simulations while the “exact” distributions are observed from long simulations. The FEP biases are related to integrals of p_f and p_b, whereas the bridge estimate biases depend on integrals of the modified PDFs defined by Eq. (5).

B. Bias in bridge sampling calculations

The same threshold ansatz used for FEP can also be applied to Eq. (1). A value of Δf computed with such thresholds will not be the exact value but rather some particular (biased) estimate $\mathrm{\Delta }\widehat{f}$,³³

$$e^{\epsilon -\mathrm{\Delta }\widehat{f}}=\frac{{\int }_{\mathrm{\Delta }{u}_f}^{\infty }d\mathrm{\Delta }u\alpha \left(\mathrm{\Delta }u-\epsilon \right)p_f\left(\mathrm{\Delta }u\right)}{{\int }_{-\infty }^{\mathrm{\Delta }{u}_b}d\mathrm{\Delta }u\alpha \left(\epsilon -\mathrm{\Delta }u\right)p_b\left(\mathrm{\Delta }u\right)}.$$ (4)

These new bounds of integration are subsumed by those in the exact expression and can be separated out leaving integrals over the tail regions. A series of equalities (see Appendix A) can be used to transform the integrals into CDFs analogous to those from the FEP result [Eq. (3)] but instead with two new PDFs

$$p_b^{\alpha }\left(\mathrm{\Delta }u;\epsilon \right)\equiv \frac{\alpha \left(\epsilon -\mathrm{\Delta }u\right)p_b\left(\mathrm{\Delta }u\right)}{{\int }_{-\infty }^{\mathrm{\Delta }{u}_b}d\mathrm{\Delta }{u}^{\prime }\alpha \left(\epsilon -\mathrm{\Delta }{u}^{\prime }\right)p_b\left(\mathrm{\Delta }{u}^{\prime }\right)},$$ (5a)

$$p_f^{\alpha }\left(\mathrm{\Delta }u;\epsilon \right)\equiv \frac{\alpha \left(\mathrm{\Delta }u-\epsilon \right)p_f\left(\mathrm{\Delta }u\right)}{{\int }_{\mathrm{\Delta }{u}_f}^{\infty }d\mathrm{\Delta }{u}^{\prime }\alpha \left(\mathrm{\Delta }{u}^{\prime }-\epsilon \right)p_f\left(\mathrm{\Delta }{u}^{\prime }\right)}.$$ (5b)

The shapes of these modified PDFs for BAR and SOS are shown in the bottom frames of Fig. 1. Interestingly, in the case of BAR, ε is adjusted so that the finite sample estimates for both of the denominators are equal and identical to the overlap integral noted by Bennett⁸ [Eq. (11) therein].

Solving Eq. (4) for the sample bias yields

$$\mathrm{\Delta }\widehat{f}-\mathrm{\Delta }f=\ln \left[\frac{1-\overline{P_f^{\alpha }}\left(\mathrm{\Delta }{u}_b;\epsilon \right)}{1-P_b^{\alpha }\left(\mathrm{\Delta }{u}_f;\epsilon \right)}\right]\approx P_b^{\alpha }\left(\mathrm{\Delta }{u}_f;\epsilon \right)-\overline{P_f^{\alpha }}\left(\mathrm{\Delta }{u}_b;\epsilon \right),$$ (6)

which is the main result of this work. It is worth reiterating that this is valid for any choice of α and ε, including those choices that correspond with the well-known BAR and SOS estimators.³⁴ Taking the expectation of both sides shows that using an empirical estimate for $P_b^{\alpha }\left(\mathrm{\Delta }{u}_f;\epsilon \right)$ and $\overline{P_f^{\alpha }}\left(\mathrm{\Delta }{u}_b;\epsilon \right)$ yields an estimate of the bias that goes to zero no faster than ∼1/N (see Appendix B). Details on how to construct and evaluate estimators for finite data sets are given in Sec. III B.

Equation (6) has some instructive properties. For example, the numerator and denominator are effectively the biases due to sampling from only one ensemble. In certain limits, the same bracketing behavior derived for FEP is encountered

$$\underset{\mathrm{\Delta }{u}_b\to +\infty }{lim}\mathrm{\Delta }\widehat{f}-\mathrm{\Delta }f\approx +P_b^{\alpha }\left(\mathrm{\Delta }{u}_f;\epsilon \right),$$ (7a)

$$\underset{\mathrm{\Delta }{u}_f\to -\infty }{lim}\mathrm{\Delta }\widehat{f}-\mathrm{\Delta }f\approx -\overline{P_f^{\alpha }}\left(\mathrm{\Delta }{u}_b;\epsilon \right).$$ (7b)

It is also readily apparent that the right-hand side of Eq. (6) vanishes not only when both $\overline{P_f^{\alpha }}\left(\mathrm{\Delta }{u}_b;\epsilon \right)$ and $P_b^{\alpha }\left(\mathrm{\Delta }{u}_f;\epsilon \right)$ are zero but also whenever $\overline{P_f^{\alpha }}\left(\mathrm{\Delta }{u}_b;\epsilon \right)=P_b^{\alpha }\left(\mathrm{\Delta }{u}_f;\epsilon \right)$—the two bias contributions expressly cancel. Separately estimating the two quantities can thus be used to establish conservative bounds on the net bias.

Interestingly, the extremely common occurrence Δu_f ≤ ⟨Δu⟩_b leads to $P_b^{\alpha }\left(\mathrm{\Delta }{u}_f;\epsilon \right)\le P_b\left(\mathrm{\Delta }{u}_f\right)$ (see Appendix C). This implies that any choice of α (that meets the criteria so far) leads to a positive contribution to the bias that is very often less than the analogous value from forward FEP. Similarly, Δu_b ≥⟨Δu⟩_f leads to $\overline{P_f^{\alpha }}\left(\mathrm{\Delta }{u}_b;\epsilon \right)\le \overline{P_f}\left(\mathrm{\Delta }{u}_b\right)$. Nonetheless, because of bias cancellation, these special occurrences are unnecessary for the overall bias to be less than that from FEP.

Finally, Eq. (6) is quite different from other bias expressions in the literature. It rigorously utilizes data from both ensembles and makes no assumptions as to the underlying form of the PDFs.²⁰ Although there is some resemblance, it does not have the same bias cancellation properties as a simple average of the FEP estimates [i.e., half the sum of Eqs. (2a) and (2b)]. Indeed, such an estimate is not effective because the two components often do not decrease with sample size at the same rate¹³ (more clever weighting likely will not help^21,22). With regard to BAR, Shirts and Pande²¹ derived an expression for the large-sample bias that is proportional to one-half the variance times a constant containing multiple derivatives of the log-likelihood function evaluated at Δf. This is informative but lacks a practical form for estimation. Finally, Lu, Singh, and Kofke¹³ actually did propose an approximate expression for the bias that can be made to look very similar to Eq. (6). However, that expression utilized an equality involving the product p_f(Δu)p_b(Δu) and thus requires some additional manipulations before it can be employed as an estimator.

III. METHODS

A. Example calculations

Some bias calculations are presented here for simple, but realistic, multistage free energy calculations using BAR and FEP. The data come from alchemical simulations of 15 neutral amino acid side-chain analogs designed to compute their absolute solvation free energies. BAR and FEP were applied to successive pairs of alchemical simulations, and the net free energy, variance, and bias across all stages of the transformation were appropriately summed. Three independent trials were performed. The “observed” bias (a proxy for the exact expected bias) was calculated as the difference in free energies computed from a subsample and the average from the three complete data sets (a best guess for Δf). The “estimated” biases were evaluated with either Eq. (2) or (6) (see below for details). For each sample size, the results represent the mean of equal sized, equally spaced, and nonoverlapping blocks (e.g., 100 samples would yield the average of 6 blocks of size 15 with 10 samples discarded between blocks). Additional data were generated for a single-step ethane-to-ethane null transformation so that Δf is known to be exactly zero. This was repeated 500 times when extremely detailed statistics were needed.

All simulations were performed with NAMD 2.13,³⁵ and analysis was done with in-house Python tools that replicate and extend the functionality of ParseFEP.³⁶ The exact details of the simulations will be published elsewhere, and the data shown here are only intended to represent a diverse set of PDFs as might be found, for example, in ligand binding applications.

B. Practical estimation of the bias

When evaluating FEP bias, it is common practice to approximate the CDFs by first making a histogram estimate of the PDF and then summing over a subset of the bins. However, this approach can be sensitive to the choice of bin width. Worse yet, because the forward and backward PDFs tend to have very different shapes, it is difficult (or impossible) to choose a bin width that is simultaneously optimal for both. A simple alternative is to use the empirical CDFs by counting the number of data points below a particular value. This can be thought of as the average of a Heaviside function H [H(0) = 1], and thus, all of the necessary terms in Eqs. (2) and (6) can be evaluated directly from sample means. An estimate of the forward and backward bias components in Eq. (6) can thus be computed as

$$\widehat{\overline{P_f^{\alpha }}}\left(\mathrm{\Delta }{u}_b;\epsilon \right)=\frac{{\sum }_{n=1}^{N_f}\alpha \left(\mathrm{\Delta }{u}_n-\epsilon \right)H\left(\mathrm{\Delta }{u}_n-\mathrm{\Delta }{u}_b\right)}{{\sum }_{n=1}^{N_f}\alpha \left(\mathrm{\Delta }{u}_n-\epsilon \right)},$$ (8a)

$${\widehat{P}}_b^{\alpha }\left(\mathrm{\Delta }{u}_f;\epsilon \right)=\frac{{\sum }_{n=1}^{N_b}\alpha \left(\epsilon -\mathrm{\Delta }{u}_n\right)H\left(\mathrm{\Delta }{u}_f-\mathrm{\Delta }{u}_n\right)}{{\sum }_{n=1}^{N_b}\alpha \left(\epsilon -\mathrm{\Delta }{u}_n\right)},$$ (8b)

where the sums run over the forward and backward samples, respectively. For BAR, one first estimates $\mathrm{\Delta }\widehat{f}$ and then takes $\epsilon =\mathrm{\Delta }\widehat{f}-\ln \frac{N_f}{N_b}$, while for SOS ε = 0 and the form of α also changes.

Finally, it is worth considering the compromises to be made in selecting values for Δu_f and Δu_b. In principle, these can simply be chosen as the minimum and maximum of their respective samples. However, the integrity of the threshold ansatz also hinges on the accuracy of the observed distributions and these may not be reliable for data near the tail regions. This trade-off can be reduced to a single parameter by setting Δu_f equal to the qth percentile value of the forward data set and Δu_b to the (100 − q)th percentile value of the backward data set [q ∈ (0, 50)]. Provided that the value is reasonably small (1–10, say), there does not appear to be a significant relationship between q and the quality of the bias estimate (Appendix D). A value of q = 1 is used for all calculations.

IV. RESULTS AND DISCUSSION

A. Bias cancellation and sample size invariance

Example bias calculations for multiple systems and sample sizes confirm the predicted trends. Most notably, for all systems studied here, the separate components of the BAR bias are always smaller than their FEP counterparts with the same sign (Fig. 2, top, dashed lines). This follows from the fact that the observed threshold values are smaller than the mean of the opposite distribution (see, for example, Fig. 1). Only the individual components obviously decay with sample size, and the net bias is instead flat and near zero. Bias cancellation thus appears to be rather efficient and has a fortuitous side effect of the variance minimization procedure in BAR. Both log terms in Eq. (6) vanish slightly slower than ∼1/N and with opposite signs (Appendix B). The signed bias thus oscillates near zero while the squared bias has the more expected ∼1/N² behavior (Fig. 2, fit lines in the bottom panel). This is to be contrasted with the FEP biases which decay even more slowly (∼1/N^0.6 or worse).

FIG. 2.

Bias estimates (top) for Δf computed over a broad set of molecules illustrate the expected trend, with BAR (circles) yielding considerably lower values than FEP (triangles and squares). The variance is generally the dominant contributor to the squared error (bottom). The reported sample size is the number of (presumed independent) data points from each simulation. Trend lines are nonlinear least squares fits to the power law 1/N^α, α ≤ 1.

With some caveats (discussed later), the estimated squared bias [computed from Eq. (6)] correlates relatively well with the observed squared bias (the squared difference between $\mathrm{\Delta }\widehat{f}$ and a best guess of Δf). This can be seen in the reasonable overlap of the square and circle data points in Fig. 2. The notable underestimation of some points appears to be an outlier due to the reference value being poorly determined for one particular molecule (i.e., the observed bias calculation itself possessed a high variance). The observed and estimated biases are also seen to vanish with essentially the same sample size dependence (prefactors of 172 ± 9 vs 162 ± 7, respectively). Similar, but slightly worse, agreement is seen for estimates using SOS (data not shown).

B. Bias/variance trade-off

By construction, the expected variance reaches a minimum (which is dependent on the sample size) when ε is optimally chosen according to Eq. (1). However, this is not the same as minimizing the sample variance [see Eq. (10a) from the work of Bennett⁸] or the squared sample bias estimated from Eq. (6); the sample quantities are only minimized on average. Indeed, the variances and squared biases from a given sample may have completely different minima with respect to ε, and neither of these minima need correspond to $\mathrm{\Delta }\widehat{f}-\ln \left(N_f/N_b\right)$. Figure 3 illustrates this with representative data compared against the average of 500 repeated free energy calculations. This may initially appear paradoxical and somewhat counter to (arguably imprecise) statements from the literature. However, it is obviously true from the data here and does indeed make sense in light of the fact that multiple variance expressions for BAR have been suggested.^14,16 All such expressions are correct in the large-sample limit but likely have different sample size dependences and/or prefactors. Unlike Bennett’s variance expression, others may also not be valid for arbitrary ε.

FIG. 3.

The squared error components vary as a function of the shift constant ε and, on average, attain minima near ε = Δf (top). The sample quantities are not, in general, minimized for a given data set (top, illustrative data shown as dots). The mean of multiple simulations shows how this pattern emerges (bottom).

A consequence of the energy shift dependence is that the correlation between systematic and statistical error is effectively random within a given sample. That is, since BAR will only jointly minimize both errors on average, there is no guarantee that either is minimized in practice. Furthermore, the BAR optimum might actually lie in between the sample minima such that improvement in one exacerbates the other (Fig. 3, top panel). However, it is not at all clear that minimizing the sample variance ought to be preferred in practice due to (1) the inherent error in the sample variance itself and (2) the accompanying bias that might be introduced. Nonetheless, investigating these trade-offs further could lead to weakly biased estimators that may be preferable to BAR in practice.

C. Bias correction vs bias detection

It is worth distinguishing between two distinct purposes in estimating bias. The obvious inclination is to compute the bias for an estimator so that it can be corrected. This ultimately implies a new estimator, such as the classic Bessel correction for sample variance calculations. However, a corrected estimator can also have different (i.e., worse) variance properties, and so the value of employing the correction should be weighed against the size of the bias that it is meant to correct. An alternative approach is simply to detect the effective presence or absence bias but otherwise leave the original estimator unchanged. The most prudent use of this work is likely along this latter route. To be sure, predicting the exact bias would be quite spectacular, as it would essentially suggest a new method of free energy estimation that outperforms the best methods to date. The accuracy of Eq. (6) does not appear to rise to this level.

The scatter of estimated vs observed biases indicates rather poor predictive power (Fig. 4). Indeed, the correct sign is only predicted 55%–65% of the time for the three subsample sizes shown. For the ethane-to-ethane transformation data, the distributions of both biases are normally distributed (albeit with different variances), but the distribution of their ratio suggests zero correlation (data not shown). Nonetheless, there is some improvement when considering the root mean square bias (i.e., the magnitude), although this still shows some degree of underestimation (by more than 0.2k_BT roughly 20%–40% of the time). Doubling the estimated bias, which is akin to taking a larger confidence interval with the variance, reduces this failure rate to less than 10% and often eliminates it entirely. Another conservative alternative not explored extensively here might be to take the larger magnitude from the two bounding estimates, although this would discount the effects of bias cancellation.

FIG. 4.

Plots of the estimated vs observed bias show a relatively low correlation when considering absolute bias (top). This is rather improved for the root mean square bias (bottom, taken over repeated subsamples of the given size). All energies are in k_BT units, and dotted lines indicate an interval of ±0.2.

V. CONCLUSION

Any serious free energy calculation should also include estimates of its statistical and systematic errors. However, nearly all published calculations only supply variance estimates. This approach is frequently justified on the grounds that the bias vanishes in the large-sample limit, and thus, systematic errors are generally not examined any further. This work provides a clear and practical method for making bias estimates due to undersampling and permits a low-cost examination of this assertion. The method works with the widely popular BAR estimator as well as any of a broad class of free energy bridge estimators. Realistic examples show that variance does indeed tend to dominate for large sample sizes and the framework used here also gives insights into why this is so. Overall, the accuracy observed in test data suggests that the method is best when used to verify that the true bias is small. Future work might explore the usage of this framework to potentially introduce bias into new estimators in an effort to reduce variance. While such weakly biased estimators have been long supposed, the bias defined here provides new ways to think about them.

SUPPLEMENTARY MATERIAL

See supplementary material for the raw data used in this work.

APPENDIX A: DETAILED DERIVATION OF BRIDGE SAMPLING BIAS

The full bounds of integration implied in Eq. (1) can be divided into those from Eq. (4) plus an integral over each of the tail regions

$$e^{\epsilon -\mathrm{\Delta }f}=\frac{{\int }_{-\infty }^{\mathrm{\Delta }{u}_f}d\mathrm{\Delta }u\alpha \left(\mathrm{\Delta }u-\epsilon \right)p_f\left(\mathrm{\Delta }u\right)+{\int }_{\mathrm{\Delta }{u}_f}^{\infty }d\mathrm{\Delta }u\alpha \left(\mathrm{\Delta }u-\epsilon \right)p_f\left(\mathrm{\Delta }u\right)}{{\int }_{-\infty }^{\mathrm{\Delta }{u}_b}d\mathrm{\Delta }u\alpha \left(\epsilon -\mathrm{\Delta }u\right)p_b\left(\mathrm{\Delta }u\right)+{\int }_{\mathrm{\Delta }{u}_b}^{\infty }d\mathrm{\Delta }u\alpha \left(\epsilon -\mathrm{\Delta }u\right)p_b\left(\mathrm{\Delta }u\right)}$$ (A1)

[note the use of the identity exp(x)α(x) = α(−x)]. Making further use of Eq. (4), the quantity in the denominator can be factored out of the numerator and vice versa. This immediately reduces to factors involving $\epsilon -\mathrm{\Delta }\widehat{f}$,

$$e^{\epsilon -\mathrm{\Delta }f}=e^{\mathrm{\Delta }\widehat{f}-\epsilon }\frac{\frac{{\int }_{-\infty }^{\mathrm{\Delta }{u}_f}d\mathrm{\Delta }u\alpha \left(\mathrm{\Delta }u-\epsilon \right)p_f\left(\mathrm{\Delta }u\right)}{{\int }_{-\infty }^{\mathrm{\Delta }{u}_b}d\mathrm{\Delta }u\alpha \left(\epsilon -\mathrm{\Delta }u\right)p_b\left(\mathrm{\Delta }u\right)}+e^{\epsilon -\mathrm{\Delta }\widehat{f}}}{e^{\mathrm{\Delta }\widehat{f}-\epsilon }+\frac{{\int }_{\mathrm{\Delta }{u}_b}^{\infty }d\mathrm{\Delta }u\alpha \left(\epsilon -\mathrm{\Delta }u\right)p_b\left(\mathrm{\Delta }u\right)}{{\int }_{\mathrm{\Delta }{u}_f}^{\infty }d\mathrm{\Delta }u\alpha \left(\mathrm{\Delta }u-\epsilon \right)p_f\left(\mathrm{\Delta }u\right)}}.$$ (A2)

The numerators of the remaining integrals can be transformed by first using the equality p_f(Δu) = exp(Δu − Δf)p_b(Δu) and then the same identity imposed on α (this is only used for brevity and can be undone later). The integrands then become

$$\begin{array}{ccc}\hfill \alpha \left(\mathrm{\Delta }u-\epsilon \right)p_f\left(\mathrm{\Delta }u\right)& =\hfill & \alpha \left(\mathrm{\Delta }u-\epsilon \right)e^{\mathrm{\Delta }u-\epsilon }{p}_b\left(\mathrm{\Delta }u\right)e^{\epsilon -\mathrm{\Delta }f}\hfill \\ \hfill & =\hfill & \alpha \left(\epsilon -\mathrm{\Delta }u\right)p_b\left(\mathrm{\Delta }u\right)e^{\epsilon -\mathrm{\Delta }f}\hfill \end{array}$$

and

$$\begin{array}{ccc}\hfill \alpha \left(\epsilon -\mathrm{\Delta }u\right)p_b\left(\mathrm{\Delta }u\right)& =\hfill & \alpha \left(\epsilon -\mathrm{\Delta }u\right)e^{\epsilon -\mathrm{\Delta }u}{p}_f\left(\mathrm{\Delta }u\right)e^{\mathrm{\Delta }f-\epsilon }\hfill \\ \hfill & =\hfill & \alpha \left(\mathrm{\Delta }u-\epsilon \right)p_f\left(\mathrm{\Delta }u\right)e^{\mathrm{\Delta }f-\epsilon },\hfill \end{array}$$

which (up to a constant factor) are the same as the integrands in their denominator. Using the assumptions about normality and noting that Δu_f < Δu_b permits the integral ratios to be written in terms of CDFs with the PDF definitions from the main text. Substituting these definitions and multiplying through by common factors leads to

$$e^{\mathrm{\Delta }\widehat{f}-\mathrm{\Delta }f}=\frac{P_b^{\alpha }\left(\mathrm{\Delta }{u}_f;\epsilon \right)e^{\mathrm{\Delta }\widehat{f}-\mathrm{\Delta }f}+1}{1+\overline{P_f^{\alpha }}\left(\mathrm{\Delta }{u}_b;\epsilon \right)e^{\mathrm{\Delta }f-\mathrm{\Delta }\widehat{f}}},$$ (A3)

which can be solved in terms of $\mathrm{\Delta }\widehat{f}-\mathrm{\Delta }f$ to give the result in the main text.

APPENDIX B: SAMPLE SIZE DEPENDENCE

The approximate sample size dependence of the bias can be determined by estimating Δu_f and Δu_b as sample minima and maxima. For independently drawn samples, these have well-defined distributions in terms of the PDF and CDF as follows:

$$p_b^{\text{min}}\left(\mathrm{\Delta }{u}_f;N_b\right)=N_b{p}_b\left(\mathrm{\Delta }{u}_f\right)\overline{P_b}{\left(\mathrm{\Delta }{u}_f\right)}^{N_b-1},$$ (B1a)

$$p_f^{\text{max}}\left(\mathrm{\Delta }{u}_b;N_f\right)=N_f{p}_f\left(\mathrm{\Delta }{u}_b\right)P_f{\left(\mathrm{\Delta }{u}_b\right)}^{N_f-1}.$$ (B1b)

Rearranging and taking the logarithm of each side yields

$$\ln \left[1-P_b\left(\mathrm{\Delta }{u}_f\right)\right]=\frac{\ln \left[\frac{p_b^{\text{min}}\left(\mathrm{\Delta }{u}_f;N_b\right)}{N_b{p}_b\left(\mathrm{\Delta }{u}_f\right)}\right]}{N_b-1},$$ (B2a)

$$\ln \left[1-\overline{P_f}\left(\mathrm{\Delta }{u}_b\right)\right]=\frac{\ln \left[\frac{p_f^{\text{max}}\left(\mathrm{\Delta }{u}_b;N_f\right)}{N_f{p}_f\left(\mathrm{\Delta }{u}_b\right)}\right]}{N_f-1},$$ (B2b)

which, on the left-hand side, are essentially identical to the FEP bias expressions.

It is reasonable to approximate $p_b^{\text{min}}\left(\mathrm{\Delta }{u}_f\right)$ and $p_f^{\text{max}}\left(\mathrm{\Delta }{u}_b\right)$ as (small) constants with respect to sample size since the threshold ansatz assumes that Δu_f and Δu_b do not lie too far into the tails of the opposing distribution. In any event, it is improbable that these terms grow with sample size. With this approximation, the log sample size terms in the numerators mean that both FEP biases fall off no faster than ∼1/N. However, substituting instead the new bridge estimator PDFs and combining to get Eq. (6) leads to the cancellation of the log terms. Assuming for simplicity that N_f = N_b = N/2 gives

$$\mathrm{\Delta }\widehat{f}-\mathrm{\Delta }f=\frac{2\ln \left[\frac{p_f^{\alpha ,\text{max}}\left(\mathrm{\Delta }{u}_b\right)p_b^{\alpha }\left(\mathrm{\Delta }{u}_f\right)}{p_f^{\alpha }\left(\mathrm{\Delta }{u}_b\right)p_b^{\alpha ,\text{min}}\left(\mathrm{\Delta }{u}_f\right)}\right]}{N-2},$$ (B3)

where the N and ε dependence of the PDFs has been suppressed. This is ∼1/N as is usually expected of the bias. The PDF ratios in the logarithm lead to oscillations in the sign since, in most instances, $p_f^{\alpha ,\text{max}}\left(\mathrm{\Delta }{u}_b\right)>p_f^{\alpha }\left(\mathrm{\Delta }{u}_b\right)$ and $p_b^{\alpha ,\text{min}}\left(\mathrm{\Delta }{u}_f\right)>p_b^{\alpha }\left(\mathrm{\Delta }{u}_f\right)$. The sign of the bias is dependent on which of these inequalities is more dramatic.

APPENDIX C: BRIDGE ESTIMATE BIAS RELATIVE TO FEP

The bridge sampling bias estimate is composed of two contributions of opposite signs which cancel and improve the net bias. Nonetheless, the two individual components are themselves almost always less than the corresponding FEP bias. This can be demonstrated by comparing the magnitudes of $P_b^{\alpha }$ and $\overline{P_f^{\alpha }}$ relative to P_b and $\overline{P_f}$. We shall focus here on the backward CDFs and simply outline the similar arguments for the forward CDFs.

In order to show that $P_b^{\alpha }\left(\mathrm{\Delta }{u}_f;\epsilon \right)\le P_b\left(\mathrm{\Delta }{u}_f\right)$, it suffices to show that $p_b^{\alpha }\left(\mathrm{\Delta }u;\epsilon \right)<p_b\left(\mathrm{\Delta }u\right)$ for Δu ≤ Δu_f. Rearranging the definition of $p_b^{\alpha }\left(\mathrm{\Delta }u;\epsilon \right)$ from Eq. (5), one has

$$\frac{p_b^{\alpha }\left(\mathrm{\Delta }u;\epsilon \right)}{p_b\left(\mathrm{\Delta }u\right)}=\frac{\alpha \left(\epsilon -\mathrm{\Delta }u\right)}{{\int }_{-\infty }^{\mathrm{\Delta }{u}_b}d\mathrm{\Delta }{u}^{\prime }\alpha \left(\epsilon -\mathrm{\Delta }{u}^{\prime }\right)p_b\left(\mathrm{\Delta }{u}^{\prime }\right)}.$$ (C1)

Applying Jensen’s inequality to the denominator (this again requires the assertion that the PDF is correctly normalized on the integration interval), consequently,

$$\frac{p_b^{\alpha }\left(\mathrm{\Delta }u;\epsilon \right)}{p_b\left(\mathrm{\Delta }u\right)}\le \frac{\alpha \left(\epsilon -\mathrm{\Delta }u\right)}{\alpha \left(\epsilon -{⟨\mathrm{\Delta }u⟩}_b\right)}.$$ (C2)

Since α(−x) is monotonically increasing, the right-hand side is always less than or equal to unity provided that Δu ≤ ⟨Δu⟩_b. This is not a guaranteed outcome, but it can be easily verified for a given sample. A similar argument follows for the forward distributions when Δu_b ≥⟨Δu⟩_f since α(x) is monotonically decreasing. The results can be summarized as

$$P_b^{\alpha }\left(\mathrm{\Delta }{u}_f;\epsilon \right)\le P_b\left(\mathrm{\Delta }{u}_f\right)\text{\hspace{1em}given that\hspace{1em}}\mathrm{\Delta }{u}_f\le {⟨\mathrm{\Delta }u⟩}_b,$$ (C3a)

$$\overline{P_f^{\alpha }}\left(\mathrm{\Delta }{u}_b;\epsilon \right)\le \overline{P_f}\left(\mathrm{\Delta }{u}_b\right)\text{\hspace{1em}given that\hspace{1em}}\mathrm{\Delta }{u}_b\ge {⟨\mathrm{\Delta }u⟩}_f,$$ (C3b)

which give clear conditions on when the bias components are smaller than those from FEP.

APPENDIX D: DETERMINATION OF THRESHOLD VALUES

As described in the main text, the threshold values can be chosen quite generally as the qth and (100 − q)th percentile values of each data set. However, for small data sets, extracting a specific value from the observed data can lead to repeat results (e.g., the 1st–10th percentile of a 10 element set would all yield the first sorted element). For more flexibility, we consider a set of (sorted) data points {Δu_n}, n = 1, 2, …, N and compute the qth percentile Δu_q as a weighted combination of two elements. That is, elements i = ⌊q(N − 1)/100 + 1⌋and j = min(i + 1, N) from the set yield the estimate Δu_q = (1 − w)Δu_i + wΔu_j, where w ≡ [q(N − 1)/100 + 1 − i]. This is the default algorithm implemented, for example, in NumPy.

Clearly, q must be chosen to be relatively small, as the goal is to compromise between a reasonable estimate of the sample minima/maxima and an accurate representation of the density. However, within this constraint, the data shown here do not seem to suggest any clear correlation between the value of q and the quality of the bias estimate (Fig. 5). That is, increasing q leads to neither a clear increase nor decrease in the accuracy of the bias estimate.

FIG. 5.

Plots of the estimated vs observed bias do not seem to show a discernible pattern as a function of the percentile cutoff, q. This seems to hold for a range of sample sizes. All energies are in k_BT units, and dotted lines indicate an interval of ±0.2.

One minor concern for small data sets is that a small value of q may lead to the entirety of the opposing data set being in the tail region. In this case, $\overline{P_f^{\alpha }}$ and/or $P_b^{\alpha }$ would evaluate to one and the bias estimate diverges. This can of course also happen if the underlying distributions only weakly overlap and thus may indicate a deeper pathology in the simulation setup. Larger values of q may recover a finite estimate of the bias in these cases, but it seems unlikely that this would be considerably more informative.