Optimal estimators and asymptotic variances for nonequilibrium path-ensemble averages

Abstract

Existing optimal estimators of nonequilibrium path-ensemble averages are shown to fall within the framework of extended bridge sampling. Using this framework, we derive a general minimal-variance estimator that can combine nonequilibrium trajectory data sampled from multiple path-ensembles to estimate arbitrary functions of nonequilibrium expectations. The framework is also applied to obtain asymptotic variance estimates, which are a useful measure of statistical uncertainty. In particular, we develop asymptotic variance estimates pertaining to Jarzynski’s equality for free energies and the Hummer–Szabo expressions for the potential of mean force, calculated from uni- or bidirectional path samples. These estimators are demonstrated on a model single-molecule pulling experiment. In these simulations, the asymptotic variance expression is found to accurately characterize the confidence intervals around estimators when the bias is small. Hence, the confidence intervals are inaccurately described for unidirectional estimates with large bias, but for this model it largely reflects the true error in a bidirectional estimator derived by Minh and Adib.

INTRODUCTION

Path-ensemble averages play a central role in nonequilibrium statistical mechanics, akin to the role of configurational ensemble averages in equilibrium statistical mechanics. Expectations of various functionals over processes where a system is driven out of equilibrium by a time-dependent external potential have been shown to be related to equilibrium properties, including free energy differences^{1, 2} and thermodynamic expectations.^{3, 4} The latter relationship, between equilibrium and nonequilibrium expectations, has been applied to several specific cases, such as the potential of mean force (PMF) along the pulling coordinate^{5, 6, 7} (or other observed coordinates⁸) in single-molecule pulling experiments, RNA folding free energies as a function of a control parameter,⁹ the root mean square deviation from a reference structure,¹⁰ the potential energy distribution¹⁰ and average,¹¹ and the thermodynamic length.¹²

Compared to equilibrium sampling, nonequilibrium processes may be advantageous for traversing energetic barriers and accessing larger regions of phase space per unit time. This is useful, for example, in reducing the effects of experimental apparatus drift or increasing the sampling of barrier-crossing events. Thus, there has been interest in calculating equilibrium properties from nonequilibrium trajectories collected in simulations or laboratory experiments. Indeed, single-molecule pulling data have been used to experimentally verify relationships between equilibrium and nonequilibrium quantities.^{13, 14}

While many estimators for free energy differences^{3, 15, 16, 17} and equilibrium ensemble averages can be constructed from nonequilibrium relationships, they will differ in the efficiency with which they utilize finite data sets, leading to varying amounts of statistical bias and uncertainty. Characterization of this bias and uncertainty is helpful in comparing the quality of different estimators¹⁸ and assessing the accuracy of a particular estimate. The statistical uncertainty of an estimator is usually quantified by its variance in the asymptotic, or large sample, limit, where estimates from independent repetitions of the experiment often approach a normal distribution about the true value due to the central limit theorem. It is an important goal to find an optimal estimator which minimizes this asymptotic variance.

Although numerical estimates of the asymptotic variance may be provided by bootstrapping (e.g., Ref. 19), closed-form expressions can provide computational advantages in the calculation of confidence intervals, allow comparison of asymptotic efficiency,^{18, 20} and facilitate the design of adaptive sampling strategies to target data collection in a manner that most rapidly reduces statistical error.^{21, 22, 23} In the asymptotic limit, the statistical error in functions of the estimated parameters can be estimated by propagating this variance estimate via a first-order Taylor series expansion. While this procedure is relatively straightforward for simple estimators, it can be difficult for estimators that involve arbitrary functions (e.g., nonlinear or implicit equations) of nonequilibrium path-ensemble averages.

Fortunately, the extended bridge sampling (EBS) estimators,^{20, 24, 25, 26} a class of equations for estimating the ratios of normalizing constants, are known to have both minimal-variance forms and associated asymptotic variance expressions. Recently, Shirts and Chodera²⁷ applied the EBS formalism to generalize the Bennett acceptance ratio,¹⁵ producing an optimal estimator combining data from multiple equilibrium states to compute free energy differences, thermodynamic expectations, and their associated uncertainties. Here, we apply the EBS formalism to estimators utilizing nonequilibrium trajectories. We first construct a general minimal-variance path-average estimator that can use samples collected from multiple nonequilibrium path ensembles. We then show that some existing path-average estimators using uni- and bidirectional data are special cases of this general estimator, proving their optimality. This also allows us to develop asymptotic variance expressions for estimators based on Jarzynski’s equality^{1, 2} and the Hummer–Szabo expressions for the PMF.^{5, 6, 28} We then demonstrate them on simulation data from a simple one-dimensional system and comment on their applicability.

EXTENDED BRIDGE SAMPLING

Suppose that we sample N_i paths (trajectories) from each of K path ensembles indexed by i=1,2,…,K. The path-ensemble average of an arbitrary functional F[X] in path ensemble i is defined by

$${⟨\mathcal{F}⟩}_i\equiv \int dX\mathcal{F}\left[X\right]{\rho }_i\left[X\right],$$ (1)

where ρ_i[X] is a probability density over trajectories,

$${\rho }_i\left[X\right]=c_i^{-1}{q}_i\left[X\right];c_i=\int dX{q}_i\left[X\right],$$ (2)

with unnormalized density q_i[X]>0 and the normalization constant c_i (a path partition function). The above integrals, in which dX is an infinitesimal path element, are taken over all possible paths, X. EBS estimators provide a way of estimating ratios of normalization constants c_i∕c_j, which will prove useful in estimating free energies and thermodynamic expectations.

To construct these estimators, we first note the importance sampling identity,

$$c_i{⟨{\alpha }_{ij}{q}_j⟩}_i=\left[\int dX{q}_i\left[X\right]\right]\frac{\int dX{\alpha }_{ij}\left[X\right]q_i\left[X\right]q_j\left[X\right]}{\int dX{q}_i\left[X\right]}=\left[\int dX{q}_j\left[X\right]\right]\frac{\int dX{\alpha }_{ij}\left[X\right]q_i\left[X\right]q_j\left[X\right]}{\int dX{q}_j\left[X\right]}=c_j{⟨{\alpha }_{ij}{q}_i⟩}_j,$$ (3)

where j is another path-ensemble index, α_ij[X] is an arbitrary functional of X, and all normalization constants are nonzero.

Summing over the index j in Eq. 3 and using the sample mean, $N_i^{-1}{\sum }_{n=1}^{N_i}\mathcal{F}\left[X_{in}\right]$, as an estimator for ⟨F⟩_i, we obtain a set of K estimating equations, indexed byi=1,…,K,

$$\sum _{j=1}^K\frac{{\widehat{c}}_i}{N_i}\sum _{n=1}^{N_i}{\alpha }_{ij}\left[X_{in}\right]q_j\left[X_{in}\right]=\sum _{j=1}^K\frac{{\widehat{c}}_j}{N_i}\sum _{n=1}^{N_j}{\alpha }_{ij}\left[X_{jn}\right]q_i\left[X_{jn}\right],$$ (4)

whose solutions yield estimates ${\widehat{c}}_i$ for the normalization constants c_i, up to an irrelevant scalar multiple. Each path, X_in, is indexed by the ensemble i from which it is sampled, and the sample number n=1,2,…,N_i. This coupled set of nonlinear equations defines a family of estimators parametrized by the choice of α_ij[X], all of which are asymptotically consistent, but whose statistical efficiencies will vary.²⁰

With the choice,

$${\alpha }_{ij}\left[X\right]=\frac{N_j{\widehat{c}}_j^{-1}}{{\sum }_{k=1}^K{N}_k{\widehat{c}}_k^{-1}{q}_k\left[X\right]},$$ (5)

Eq. 4 simplifies to the optimal EBS estimator,

$${\widehat{c}}_i=\sum _{j=1}^K\sum _{n=1}^{N_j}{\left[\sum _{k=1}^K\frac{N_k}{{\widehat{c}}_k}\frac{q_k\left[X_{jn}\right]}{q_i\left[X_{jn}\right]}\right]}^{-1}.$$ (6)

This choice for α_ij[X] is optimal in that the asymptotic variance of the ratios ${\widehat{c}}_i∕{\widehat{c}}_j$ is minimal.^{20, 27} These equations may be solved by any appropriate algorithm, including a number of efficient and stable methods suggested by Shirts and Chodera.²⁷

The asymptotic covariance of Eq. 6 is estimated by

$$\hat{\mathbf{\Theta }}={\mathit{M}}^T{\left({\mathit{I}}_N-\mathit{MN}{\mathit{M}}^T\right)}^{+}\mathit{M},$$ (7)

where the elements of Θ are the covariances of the logarithms of the estimated normalization constants, ${\Theta }_{ij}\equiv \text{cov}\left({\widehat{\gamma }}_i,{\widehat{\gamma }}_j\right)$, and ${\widehat{\gamma }}_i\equiv \text{ln}{\widehat{c}}_i$.²⁶ The superscript (⋯)⁺ denotes an appropriate generalized inverse, such as the Moore–Penrose pseudoinverse, I_N is the N×N identity matrix (where $N={\sum }_{i=1}^K{N}_i$ is the total number of samples), N≡diag(N₁,N₂,…,N_K) is the diagonal matrix of sample sizes, and M is the N×K weight matrix with elements,

$$M_{ni}={\widehat{c}}_i^{-1}\frac{q_i\left[X_n\right]}{{\sum }_{k=1}^K{N}_k{\widehat{c}}_k^{-1}{q}_k\left[X_n\right]}.$$ (8)

In this matrix, the distribution from which samples are drawn from is irrelevant and X is only indexed by n=1,…,N. We note that the sum over each column, ${\sum }_{n=1}^N{M}_{ni}$, is unity. (Efficient methods for computing $\hat{\mathbf{\Theta }}$ are discussed in Appendix D of Ref. 27.)

For arbitrary functions of the logarithms of the normalization constants, $\varphi \left({\widehat{\gamma }}_1,\dots ,{\widehat{\gamma }}_K\right)$ and $\psi \left({\widehat{\gamma }}_1,\dots ,{\widehat{\gamma }}_K\right)$, the asymptotic covariance $\text{cov}\left(\widehat{\varphi },\widehat{\psi }\right)$ can be estimated from $\hat{\mathbf{\Theta }}$ according to

$$\text{cov}\left(\widehat{\varphi },\widehat{\psi }\right)\approx \sum _{i,j=1}^K\frac{\partial \varphi }{\partial {\widehat{\gamma }}_i}{\widehat{\Theta }}_{ij}\frac{\partial \psi }{\partial {\widehat{\gamma }}_j},$$ (9)

through first-order Taylor series expansion of ϕ and ψ.

GENERAL PATH-ENSEMBLE AVERAGES

Following previous work,^{27, 29} we estimate nonequilibrium expectations by defining additional path ensembles with “unnormalized densities,”

$$q_{{\mathcal{F}}_i}\left[X\right]=\mathcal{F}\left[X\right]q_i\left[X\right];c_{{\mathcal{F}}_i}=\int dX{q}_{{\mathcal{F}}_i}\left[X\right].$$ (10)

Using Eqs. 1, 2, 10, we can express nonequilibrium expectations as a ratio of the appropriate normalization constants, ⟨F⟩_i=c_Fi∕c_i. Notably, this can be estimated without actually sampling path ensembles biased by some function of F[X] (although it is sometimes possible to do so in computer simulations^{30, 31} via transition path sampling^{32, 33}). If no paths are drawn from the path ensemble corresponding to q_Fi[X], then N_Fi=0 and it is no longer required that q_Fi[X]>0.^{20, 27}

For each defined path ensemble, the weight matrix M is augmented by one column with elements,

$$M_{n{\mathcal{F}}_i}={\widehat{c}}_{{\mathcal{F}}_i}^{-1}\frac{\mathcal{F}\left[X_n\right]q_i\left[X_n\right]}{{\sum }_{k=1}^K{N}_k{\widehat{c}}_k^{-1}{q}_k\left[X_n\right]}.$$ (11)

The estimator for the path-ensemble average, ${\overline{\mathcal{F}}}_i\approx {⟨\mathcal{F}⟩}_i$, can be expressed in terms of weight matrix elements,

$${\overline{\mathcal{F}}}_i=\sum _{n=1}^N{M}_{ni}\mathcal{F}\left[X_n\right],$$ (12)

and its uncertainty estimated by

$${\sigma }^2\left({\overline{\mathcal{F}}}_i\right)\approx {\overline{\mathcal{F}}}_i^2\left({\widehat{\Theta }}_{{\mathcal{F}}_i{\mathcal{F}}_i}-2{\widehat{\Theta }}_{{\mathcal{F}}_{i}i}+{\widehat{\Theta }}_{ii}\right).$$ (13)

EXPERIMENTALLY RELEVANT PATH ENSEMBLES

The above formalism is fully general, and may be applied to any situation where the ratio q_i[X]∕q_j[X] can be computed. For arbitrary path ensembles, unfortunately, calculating this ratio is only possible in computer simulations unless certain assumptions are made about the dynamics.³⁴ In a few special path ensembles, however, we can use the Crooks fluctuation theorem^{35, 36} to estimate this ratio, allowing us to apply the EBS estimator to laboratory experiments. We examine these here.

First, consider a forward process, in which a system, initially in equilibrium, is propagated under some time-dependent dynamics for a time τ, which may cause it to be driven out of equilibrium. The time dependence of the evolution law (e.g., Hamiltonian dynamics in a time-dependent potential) is the same for all paths sampled from this ensemble.

For a sample of paths only drawn from this ensemble, the optimal EBS estimator of ⟨F⟩_f reduces to the sample mean estimator, which we call the unidirectional path-ensemble average estimator,

$${\overline{\mathcal{F}}}_f=\frac{1}{N_f}\sum _{n=1}^{N_f}\mathcal{F}\left[X_{fn}\right],$$ (14)

and the associated asymptotic variance from Eq. 9 reduces to the variance of the sample mean (see Appendix A)

$${\sigma }^2\left({\overline{\mathcal{F}}}_f\right)\approx \frac{1}{N_f}\left[\frac{1}{N_f}\sum _{n=1}^{N_f}{\left(\mathcal{F}\left[X_{fn}\right]-{\overline{\mathcal{F}}}_f\right)}^2\right].$$ (15)

The forward process has a unique counterpart known as the reverse process. Here, the system moves via the opposite protocol in thermodynamic state space; after initial configurations are drawn from the final thermodynamic state of the forward path ensemble, they are driven toward the initial state. If the dynamical law satisfies detailed balance when the control parameters are held constant at each fixed time t, the path probabilities in the conjugate forward and reverse path ensembles are related according to the Crooks fluctuation theorem,^{35, 36}

$$\frac{{\rho }_f\left[X\right]}{{\rho }_r\left[\tilde{X}\right]}=\frac{q_f\left[X\right]}{q_r\left[\tilde{X}\right]}\frac{c_r}{c_f}=e^{w_{\tau }\left[X\right]-\Delta f_{\tau }}\equiv e^{\Omega \left[X\right]},$$ (16)

in which $\tilde{X}$ is the time reversal, or conjugate twin,³⁷ of X, Δf_t=−ln(c_t∕c₀) is the dimensionless free energy difference between thermodynamic states at times 0 and t (with τ being the fixed total trajectory length), and w_t[X] is the appropriate dimensionless work. With Hamiltonian dynamics, for example, this work is $w_t\left[X\right]=\beta {\int }_0^{t}d{t}^{\prime }\left(\partial H∕\partial t^{\prime }\right)$. For convenience, we define the total dissipative work as Ω[X]≡w_τ[X]−Δf_τ.

We will refer to data sets which only include realizations from the forward path ensemble as “unidirectional,” and those with paths from both path ensembles as “bidirectional.” Notably, sampling paths from these conjugate ensembles and calculating the associated work w_t[X] are possible in single-molecule pulling experiments as well as computer simulations (cf. Refs. ^{6, 14}). To combine bidirectional data to estimate ⟨F⟩_f, we apply the Crooks fluctuation theorem^{35, 36} to Eq. 6 and divide by ${\widehat{c}}_f$, leading to

$${\overline{\mathcal{F}}}_f=\sum _{n=1}^{N_f}\frac{\mathcal{F}\left[X_{fn}\right]}{N_f+N_r{e}^{-\widehat{\Omega }\left[X_{fn}\right]}}+\sum _{n=1}^{N_r}\frac{\mathcal{F}\left[X_{rn}\right]}{N_f+N_r{e}^{-\widehat{\Omega }\left[X_{rn}\right]}},$$ (17)

which is bidirectional path-average estimator of Minh and Adib,²⁸ derived here by a different route which demonstrates its optimality. (The asymptotic variance estimator for this equation is written in a closed form in Appendix B.) In these bidirectional expressions, samples drawn from the reverse path ensemble are time reversed to obtain the paths X_rn. The dissipated work estimate, $\widehat{\Omega }\left[X\right]\equiv w_{\tau }\left[X\right]-\Delta {\widehat{f}}_{\tau }$, requires an estimate of Δf_τ . A method for obtaining this estimate will be described next.

FREE ENERGY

Jarzynski’s equality,^{1, 2}

$$e^{-\Delta f_t}={⟨e^{-w_t}⟩}_f,$$ (18)

relates nonequilibrium work and free energy differences. To facilitate the use of EBS in Jarzynski’s equality, we define a path ensemble by choosing F[X]=e^−w_t[X] in Eq. 10, leading to

$$q_{w_t}\left[X\right]=e^{-w_t\left[X\right]}{q}_f\left[X\right];c_{w_t}=\int dX{e}^{-w_t\left[X\right]}{q}_f\left[X\right].$$ (19)

When only unidirectional data is available, the optimal EBS estimator for Jarzynski’s equality is

$$e^{-\Delta {\widehat{f}}_t}=\frac{1}{N_f}\sum _{n=1}^{N_f}{e}^{-w_t\left[X_{fn}\right]}$$ (20)

and its asymptotic variance is straightforwardly given by error propagation.³⁸Estimators^{30, 31, 39} and asymptotic variances^{39, 40} have also been developed for unidirectional importance sampling forms of the equality.

When bidirectional data is available, the same choice of F[X] in Eq. 17 gives the estimator

$$e^{-\Delta {\widehat{f}}_t}=\sum _{n=1}^{N_f}\frac{e^{-w_t\left[X_{fn}\right]}}{N_f+N_r{e}^{-\widehat{\Omega }\left[X_{fn}\right]}}+\sum _{n=1}^{N_r}\frac{e^{-w_t\left[X_{rn}\right]}}{N_f+N_r{e}^{-\widehat{\Omega }\left[X_{rn}\right]}}.$$ (21)

In this equation, choosing t=0 or t=τ leads to an implicit function mathematically equivalent to the Bennett acceptance ratio method,^{3, 15} as previously explained.^{28, 41} The asymptotic variance of $\Delta {\widehat{f}}_t$ is calculated by augmenting the matrices M and $\hat{\mathbf{\Theta }}$ and using ϕ=ψ=Δf_t=−ln(c_wt∕c_f) in Eq. 9, such that

$${\sigma }^2\left(\Delta {\widehat{f}}_t\right)={\widehat{\Theta }}_{w_t{w}_t}-2{\widehat{\Theta }}_{w_{t}f}+{\widehat{\Theta }}_{ff}.$$ (22)

POTENTIAL OF MEAN FORCE

Building on Jarzynski’s equality, Hummer and Szabo developed expressions for the PMF,^{5, 6} the free energy as a function of a order parameter rather than a thermodynamic state, which may be used to interpret single-molecule pulling experiments. In these experiments, a molecule is mechanically stretched by a force-transducing apparatus, such as a laser optical trap or atomic force microscope tip (cf. Ref. 6). The Hamiltonian governing the time evolution in these experiments, H(x;t)=H₀(x)+V(z(x);t), is assumed to contain both a term corresponding to the unperturbed system, H₀(x), and a time-dependent (typically harmonic) external bias potential imposed by the apparatus, V(z;t), which acts along a pulling coordinate, z(x). As the coordinate z_t≡z(x(t)) is observed at fixed intervals Δt over the course of the experiment, we will henceforth use t=0,1,…,T as an integer time index. We calculate the work with a discrete sum as $w_t={\sum }_{n=1}^t\left[V_n\left(z_n\right)-V_{n-1}\left(z_n\right)\right]$, where V_n(z)≡V(z;nΔt).

While the expressions in Sec. 5 provide an estimate of relative free energies of the equilibrium thermodynamic states defined by H(x;t), they are not immediately useful as an estimate for the PMF along z.^{5, 6, 42} By applying the nonequilibrium estimator for thermodynamic expectations,^{3, 4} it was shown that the PMF in the absence of an external potential is given by^{5, 6}

$$e^{-g_0\left(z\right)}={⟨\delta \left(z-z_t\right)e^{-w_t}⟩}_f{e}^{V\left(z_t;t\right)},$$ (23)

where the dimensionless PMF, g₀(z), is defined in relation to the normalized density as g₀(z)=−ln p₀(z)−δg. In this equation, δg is a time-independent constant, e^−δg=∫dxe^−H(x;0)∕∫dxe^−H₀(x).⁶

This theorem [Eq. 23] can be used to develop estimators for the PMF by replacing the delta function using a kernel function of finite width, such as

$$h\left(z-z_t\right)=\{\begin{array}{ll}\begin{array}{c}\frac{1}{\Delta z}\end{array},\hfill & \text{if}\mid z-z_t\mid <\frac{\Delta z}{2},\hfill \\ 0,\hfill & \text{else}.\hfill \end{array}\phantom{\}}$$ (24)

The width Δz must be small so that e^V(z;t) does not vary substantially across it.

As this theorem is valid at all times, it is possible to obtain an asymptotically unbiased density estimate ${\widehat{p}}_t$ from each time slice. It is far more efficient, however, to estimate the PMF using all recorded time slices. While any linear combination of time slices will lead to a valid estimate, certain choices will be more statistically efficient (leading to lower variance) than others. One way to combine time slices is to use the asymptotic covariance matrix in the method of control variates,²⁰ leading to a generalized least-squares optimal estimate of the PMF. Unfortunately, we empirically found this approach to be numerically unstable. A more numerically stable approach, which was proposed by Hummer and Szabo,^{5, 6} is based on the weighted histogram analysis method,^{43, 44} which was generalized to,^{45, 46, 47}

$${\widehat{p}}_0\left(z\right)=\frac{{\sum }_t{\mu }_t\left(z\right){\widehat{p}}_t\left(z\right)}{{\sum }_t{\mu }_t\left(z\right)};{\mu }_t\left(z\right)\equiv e^{-V\left(z;t\right)+\Delta {\widehat{f}}_t}.$$ (25)

While this weighting scheme is optimal, in a minimal-variance sense, for independent samples from multiple equilibrium distributions, these assumptions do not hold for time slices from nonequilibrium trajectories. However, Oberhofer and Dellago did not observe substantial improvement in PMF estimates when using other time-slice weighting schemes.⁴⁸

By defining the path ensemble,

$$q_{z_t}\left[X\right]=\delta \left(z-z_t\right)e^{-w_t\left[X\right]}{q}_f\left[X\right];c_{z_t}=\int dX{q}_{z_t}\left[X\right],$$ (26)

and making use of Jarzynski’s equality [Eq. 18] for $e^{-\Delta {\widehat{f}}_t}$, we can write Hummer and Szabo’s PMF estimator as

$$e^{-{\widehat{g}}_0\left(z\right)}=\frac{{\sum }_t\left({\widehat{c}}_{z_t}∕{\widehat{c}}_{w_t}\right)}{{\sum }_t{e}^{-V\left(z;t\right)}\left({\widehat{c}}_f∕{\widehat{c}}_{w_t}\right)},$$ (27)

which can be readily analyzed in terms of EBS. While Hummer and Szabo proposed using the unidirectional path average estimator [Eq. 14] to estimate the expectations in Eq. 27, Minh and Adib later applied a bidirectional estimator [Eq. 17], leading to significantly improved statistical properties.²⁸

The asymptotic variance of these estimators can be determined by choosing ϕ=ψ=p₀(z) in Eq. 9. For the bidirectional estimator, the matrices M and $\hat{\mathbf{\Theta }}$ will contain one column each for the f and r path ensembles, and T+1 columns each for the path ensembles associated with ${\left\{w_t\right\}}_{t=0}^{\text{T}}$ and ${\left\{z_t\right\}}_{t=0}^{\text{T}}$. The relevant partial derivatives are

$$\frac{\partial p_0\left(z\right)}{\partial {\gamma }_f}=-p_0\left(z\right),$$ (28)

$$\frac{\partial p_0\left(z\right)}{\partial {\gamma }_{w_t}}=-\frac{1}{\mathcal{D}}\frac{c_{z_t}}{c_{w_t}}+\frac{\mathcal{N}}{{\mathcal{D}}^2}\left(e^{-V\left(z;t\right)}\frac{c_f}{c_{w_t}}\right),$$ (29)

$$\frac{\partial p_0\left(z\right)}{\partial {\gamma }_{z_t}}=\frac{1}{\mathcal{D}}\frac{c_{z_t}}{c_{w_t}},$$ (30)

where γ_i=ln c_i, N=∑_t(c_zt∕c_wt) is the numerator of Eq. 27, and D=∑_te^−V(z;t)(c_f∕c_wt) is its denominator. These lead to an estimate for ${\sigma }^2\left({\widehat{p}}_0\left(z\right)\right)$. Finally, the asymptotic variance in the PMF is given by the error propagation formula, ${\sigma }^2\left({\widehat{g}}_0\left(z\right)\right)\approx {\sigma }^2\left({\widehat{p}}_0\left(z\right)\right)∕{\widehat{p}}_0{\left(z\right)}^2$.

ILLUSTRATIVE EXAMPLE

We demonstrate these results with Brownian dynamics simulations on a one-dimensional potential with U₀(z)=(5z³−10z+3)z, which were run as previously described.²⁸ A time-dependent external perturbation, $V\left(z;t\right)=k_s{\left(z-\overline{z}\left(t\right)\right)}^2∕2$, with k_s=15 is applied, such that the total potential is U(z;t)=U₀(z)+V(z;t). After 100 steps of equilibration at the initial $\overline{z}\left(t\right)$, $\overline{z}\left(t\right)$ is linearly moved over 750 steps from −1.5 to 1.5 in forward processes and 1.5 to −1.5 in the reverse. The position at each time step is calculated using the equation z_t=z_t−1−(dU(x_t−1)∕dx)DΔt+(2DΔt)^1∕2R_t, where the diffusion coefficient is D=1, the time step is Δt=0.001, and R_t∼N(0,1) is a random number from the standard normal distribution.

As previously noted,^{28, 49, 50, 51} unidirectional sampling leads to significant apparent bias in estimates of Δf_t (Fig. 1). In addition to the increased bias as the system is driven further from equilibrium, we further observe that the estimated variance also increases. Bidirectional sampling, on the other hand, leads to a significant reduction in bias and variance,²⁸ such that free energy estimate is within error bars of the actual free energy. Because $\Delta {\widehat{f}}_t$ represents the estimated free energy difference with respect to t, the estimated ${\sigma }^2\left(\Delta {\widehat{F}}_t\right)$ increases with t, becoming equal to the well-known Bennett acceptance ratio asymptotic variance estimate^{15, 41} when t=τ.

Figure 1.

Comparison of estimators for Δf_t. This figure is similar to Fig. 1 of Ref. 28, except that error bars are now included and the sample size is halved. The unidirectional estimator [Eq. 20] is applied to 250 forward (rightward triangles) or reverse (leftward triangles, time reversed) sampled paths, and the bidirectional estimator [Eq. 21] to 125 paths in each direction (upward triangles). The exact Δf_t is shown as a solid line. Error bars (sometimes smaller than the markers) denote one standard deviation of $\Delta {\widehat{f}}_t$, estimated using the expressions presented here. The vertical dashed lines are at the times considered in Fig. 4.

Similar trends are observed with the Hummer–Szabo PMF estimates (Fig. 2). For unidirectional sampling, the finite-sampling bias and estimated variance increases when the PMF is far from the region sampled by the initial state. With bidirectional sampling, the bias is significantly reduced; the PMF estimate is largely within error bars of the actual PMF.

Figure 2.

Comparison of PMF estimators. This figure is similar to Fig. 2 of Ref. 28, except that error bars are now included and the sample size is halved. In the left panel, the unidirectional Hummer and Szabo estimator is applied to (a) 250 forward (rightward triangles) or 250 reverse (leftward triangles) sampled paths. In the right panel, the bidirectional estimator is applied to 125 sampled paths in each direction (upward triangles). The exact PMF is shown as a solid line in both panels. Error bars (sometimes smaller than the markers) denote one standard deviation of $\Delta {\widehat{g}}_0\left(z\right)$, estimated using the expressions presented here. The vertical dashed lines are at the positions considered in Fig. 4.

To analyze these trends more quantitatively, we repeated the experiment 1000 times. For both Δf_t and g₀(z), we calculated the bias as $\overline{B}\left({\overline{\mathcal{F}}}_f\right)=\left(1∕S\right){\sum }_{s=1}^S\left({\overline{\mathcal{F}}}_{f,s}-⟨\mathcal{F}⟩\right)$ and the standard deviation as $\overline{\sigma }\left({\overline{\mathcal{F}}}_f\right)=\sqrt{\left(1∕S\right){\sum }_{s=1}^S{\left({\overline{\mathcal{F}}}_{f,s}-⟨\mathcal{F}⟩\right)}^2}$, where S=1000 is the number of replicates. The results from these more extensive simulations support our described trends (Fig. 3). For unidirectional sampling, the bias in both Δf_t and g₀(x) appear to significantly increase around the barrier crossing. In the bidirectional free energy estimate, however, the bias is small relative to the variance at all times. Notably, in the bidirectional PMF estimate, there is a small spike in the bias near the barrier, potentially due to reduced sampling in the region.

Figure 3.

Ratio of estimator bias to standard deviation. This ratio is calculated for the (a) free energy and (b) PMF, using 1000 independent estimates. Each estimate is obtained and the type of path sample is indicated as in Figs. 12. The vertical dashed lines are at the times/positions considered in Fig. 4.

While in the large sample limit, the bias in the unidirectional estimate is expected to be small compared to the variance,⁵⁰ our distribution of unidirectional e^−Δf_t estimates is significantly skewed and does not resemble a Gaussian distribution expected by the central limit theorem (data not shown). Hence, the asymptotic limit has not been reached and the large relative bias is caused by insufficient sampling of rare events with low work values that dominate the exponential average.³⁷ Larger sample sizes would be necessary for the distribution of estimates to be normally distributed and for the error to be dominated by the variance (which we estimate here) rather than the bias.

The accuracy of variance estimates may be assessed by comparing predicted and observed confidence intervals. If the estimates are indeed normally distributed about the true value, about 68% of estimates from many independent replicates of the experiment should be within one standard deviation of the true value, 95% within two, and so forth. Figure 4 compares confidence intervals predicted using the described asymptotic variance estimators and the actual fraction of estimates within the interval.

Figure 4.

Validation of asymptotic variance estimators. Predicted vs observed fraction of 1000 independent estimates that are within an interval of the true value for [(a)–(c)] Δf_t and [(d)–(f)] g₀(z) at the indicated times or positions. Each estimate is obtained and the type of path sample is indicated as in Figs. 12. Error bars on these fractions are 95% confidence intervals calculated using a Bayesian scheme described in Appendix B of Chodera et al. (Ref. ⁶²) except that, for numerical reasons, the confidence interval was estimated from the variance of the beta distribution assuming approximate normality, rather than from the inverse beta cumulative distribution function.

We observe that the accuracy of our asymptotic variance estimate in characterizing the confidence interval largely depends on the presence of bias. In the bidirectional $\Delta {\widehat{f}}_t$ estimate, where there is little bias, the asymptotic variance estimate works very well. For the unidirectional $\Delta {\widehat{f}}_t$ estimates, it works well near the initial state but underestimates the error as the system is driven further away from equilibrium, concurring with the bias trend. In the bidirectional PMF estimate, the asymptotic variance estimate accurately describes the confidence interval except near the barrier, where it slightly underestimates the uncertainty, probably due to the small spike in bias.

In the regime where the bias is much smaller than the variance, $\overline{B}⪡\overline{\sigma }$, the asymptotic variance estimate provides a good estimate of the actual statistical error in the estimate. This also permits us to model the posterior distribution of quantity being estimated as a multivariate normal distribution with mean $\overline{\mathcal{F}}$ and covariance $\hat{\mathbf{\Theta }}$. Doing so provides a route to combining estimates from independent data sets collected from different path ensembles—such as different pulling speeds or from equilibrium and nonequilibrium path ensembles—without knowledge of path probability ratios. This is achieved by maximizing the product of these posterior distributions in a manner similar to the Bayesian approach for estimating Δf_τ described in Ref. 17.

DISCUSSION

In addition to the described experimentally relevant applications, we anticipate the use of our estimator in conjunction with a number of computational techniques. Recent work suggested^{30, 31, 40, 52, 53} that the convergence of Jarzynski’s equality may be improved by sampling from an alternate path ensemble, biased according to the work. Unfortunately, the optimal bias for this path ensemble includes the free energy itself,⁵³ limiting its practicality. However, an iterative procedure in which progressively improving free energy estimates are used in successive biased path ensembles is feasible, and our estimator provides a way to combine all of the data. Free energy estimates may also be improved by optimizing the switching protocol in thermodynamic state space to minimize the dissipated work.^{54, 55, 56, 57, 58, 59, 60, 61} When the optimal protocol is known,^{59, 60, 61} the bidirectional form of our estimator is likely to improve free energy estimates over a unidirectional procedure. In the general case, protocol optimization may require an iterative procedure for which our estimator can also be used to combine the data.

Lastly, while we emphasized the use of EBS in nonequilibrium path-ensemble averages, the formalism described in Secs. 2, 3 is equally applicable to other path ensembles. Given the explosion in popularity of transition path sampling,^{32, 33} in which novel path ensembles can be designed and sampled from, our estimator should find use in some of its applications.

APPENDIX A: CLOSED-FORM EXPRESSION FOR THE ASYMPTOTIC VARIANCE, GIVEN UNIDIRECTIONAL DATA

In this appendix, we show that given unidirectional data, the optimal EBS estimate is the sample mean and its variance simplifies to the variance of a sample mean. First, consider the application of the optimal EBS estimator, Eq. 6, to estimating a nonequilibrium path-ensemble average from a unidirectional data set,

$${\widehat{c}}_{{\mathcal{F}}_f}=\sum _{n=1}^{N_f}{\left[\frac{N_f}{{\widehat{c}}_f}\frac{q_f\left[X_{fn}\right]}{q_{{\mathcal{F}}_f}\left[X_{fn}\right]}\right]}^{-1},$$ (A1)

$$=\sum _{n=1}^{N_f}\frac{\mathcal{F}\left[X_{fn}\right]{\widehat{c}}_f}{N_f}.$$ (A2)

Dividing both sides by ${\widehat{c}}_f$, we obtain the sample mean estimator,

$${\overline{\mathcal{F}}}_f=\frac{1}{N_f}\sum _{n=1}^{N_f}\mathcal{F}\left[X_{fn}\right].$$ (A3)

We shall now simplify the asymptotic variance estimate by closely following the procedure of Shirts and Chodera.²⁷ When M has full column rank, $\hat{\mathbf{\Theta }}$ can be written as (Eq. D7 of Ref. 27)

$$\hat{\mathbf{\Theta }}={\left[{\left({\mathit{M}}^{\text{T}}\mathit{M}\right)}^{-1}-\mathit{N}+b{\mathbf{1}}_K{\mathbf{1}}_K^{\text{T}}\right]}^{-1},$$ (A4)

where b is an arbitrary multiplicative factor and 1_K is a 1×K matrix of ones.

The weight matrix M consists of two columns,

$$M_{nf}=\frac{{\widehat{c}}_f^{-1}{q}_f\left[X_{fn}\right]}{N_f{\widehat{c}}_f^{-1}{q}_f\left[X_{fn}\right]}=\frac{1}{N_f},$$ (A5)

$$M_{n{\mathcal{F}}_f}=\frac{{\widehat{c}}_{{\mathcal{F}}_f}^{-1}{q}_{{\mathcal{F}}_f}\left[X_{fn}\right]}{N_f{\widehat{c}}_f^{-1}{q}_f\left[X_{fn}\right]}=\frac{\mathcal{F}\left[X_{fn}\right]}{N_f{\overline{\mathcal{F}}}_f},$$ (A6)

obtained by applying Eqs. 8, 11. This leads to,

$${\mathit{M}}^{\text{T}}\mathit{M}=\left[\begin{array}{cc}{N}_f^{-1}& N_f^{-1}\\ N_f^{-1}& {\sum }_{n=1}^{N_f}{M}_{n{\mathcal{F}}_f}^2\end{array}\right]\equiv \left[\begin{array}{cc}{a}_{11}& a_{12}\\ a_{21}& a_{22}\end{array}\right].$$ (A7)

The matrix M^TM has the determinant

$$D=\frac{1}{N_f}\sum _{n=1}^{N_f}{M}_{n{\mathcal{F}}_f}^2-\frac{1}{N_f^2},$$ (A8)

which allows us to write the inverse covariance matrix as

$${\hat{\mathbf{\Theta }}}^{-1}=\left[\begin{array}{cc}\frac{a_{22}}{D}-N_f+b& -\frac{a_{21}}{D}+b\\ -\frac{a_{12}}{D}+b& \frac{a_{11}}{D}+b\end{array}\right].$$ (A9)

By applying the same steps as Appendix E of Shirts and Chodera,²⁷ we then obtain the determinant

$$\mid {\hat{\mathbf{\Theta }}}^{-1}\mid =\frac{4ab}{D},$$ (A10)

where a=a₁₂=a₂₁. We then obtain the asymptotic covariance estimate

$$\hat{\mathbf{\Theta }}=\frac{D}{4ab}\left[\begin{array}{cc}\frac{a_{11}}{D}+b& \frac{a}{D}-b\\ \frac{a}{D}-b& \frac{a_{22}}{D}-N_f+b\end{array}\right].$$ (A11)

To estimate the variance, we apply Eq. 13, leading to

$${\sigma }^2\left({\overline{\mathcal{F}}}_f^2\right)\approx {\overline{\mathcal{F}}}_f^2\left({\Theta }_{{\mathcal{F}}_f{\mathcal{F}}_f}-2{\Theta }_{{\mathcal{F}}_{f}f}+{\Theta }_{ff}\right),$$ (A12)

$$={\overline{\mathcal{F}}}_f^2\left(\sum _{n=1}^{N_f}{M}_{n{\mathcal{F}}_f}^2-\frac{1}{N_f}\right),$$ (A13)

$$=\sum _{n=1}^{N_f}\frac{\mathcal{F}{\left[X_{fn}\right]}^2}{N_f^2}-\frac{{\overline{\mathcal{F}}}_f^2}{N_f},$$ (A14)

$$=\frac{1}{N_f}\left[\frac{1}{N_f}\sum _{n=1}^{N_f}\mathcal{F}{\left[X_{fn}\right]}^2-{\left(\frac{1}{N_f}\sum _{n=1}^{N_f}\mathcal{F}\left[X_{fn}\right]\right)}^2\right],$$ (A15)

$$=\frac{1}{N_f}\left[\frac{1}{N_f}\sum _{n=1}^{N_f}{\left(\mathcal{F}\left[X_{fn}\right]-{\overline{\mathcal{F}}}_f\right)}^2\right],$$ (A16)

which is the variance of a sample mean estimate.

APPENDIX B: CLOSED-FORM EXPRESSION FOR THE ASYMPTOTIC VARIANCE, GIVEN BIDIRECTIONAL DATA

In this appendix, we obtain a closed-form expression for the asymptotic variance of the optimal EBS estimate for ${\overline{\mathcal{F}}}_f$, given bidirectional data. We will follow a similar procedure as in Appendix A. For the bidirectional case, the weight matrix M consists of three columns, M=[m_fm_rm_Ff], where m_i is a column matrix of weights from Eqs. 8, 11 corresponding to path ensemble i. The elements of M are

$$M_{nf}=\frac{{\widehat{c}}_f^{-1}{q}_f\left[X_n\right]}{N_f{\widehat{c}}_f^{-1}{q}_f\left[X_n\right]+N_r{\widehat{c}}_r^{-1}{q}_r\left[{\tilde{X}}_n\right]}=\frac{1}{N_f+N_r{e}^{-\widehat{\Omega }\left[X_n\right]}}=N_f^{-1}ϵ\left(L_n\right),$$ (B1)

$$M_{nr}=\frac{{\widehat{c}}_r^{-1}{q}_r\left[{\tilde{X}}_{fn}\right]}{N_f{\widehat{c}}_f^{-1}{q}_f\left[X_n\right]+N_r{\widehat{c}}_r^{-1}{q}_r\left[{\tilde{X}}_n\right]}=\frac{1}{N_f{e}^{\widehat{\Omega }\left[X_n\right]}+N_r}=N_r^{-1}ϵ\left(-L_n\right),$$ (B2)

$$M_{n{\mathcal{F}}_f}=\left(\frac{\mathcal{F}\left[X\right]}{{\overline{\mathcal{F}}}_f}\right)\frac{1}{N_f+N_r{e}^{-\widehat{\Omega }\left[X_n\right]}}=\left(\frac{\mathcal{F}\left[X\right]}{{\overline{\mathcal{F}}}_f}\right)N_f^{-1}ϵ\left(L_n\right),$$ (B3)

where ϵ is defined as the Fermi function, ϵ(L_n)=1∕1+e^−L_n, and we define $L_n=W\left[X_n\right]-\Delta {\widehat{f}}_t+\text{ln}\left(N_f∕N_r\right)$. This allows us to write M^TM as

$${\mathit{M}}^{\text{T}}\mathit{M}=\sum _{n=1}^N\left[\begin{array}{ccc}\frac{1}{N_f^2}ϵ{\left(L_n\right)}^2& \frac{1}{N_f{N}_r}ϵ\left(L_n\right)ϵ\left(-L_n\right)& \frac{1}{N_f^2}\left(\frac{\mathcal{F}\left[X\right]}{{\overline{\mathcal{F}}}_f}\right)ϵ{\left(L_n\right)}^2\\ \frac{1}{N_f{N}_r}ϵ\left(L_n\right)ϵ\left(-L_n\right)& \frac{1}{N_r^2}ϵ{\left(-L_n\right)}^2& \frac{1}{N_f{N}_r}\left(\frac{\mathcal{F}\left[X\right]}{{\overline{\mathcal{F}}}_f}\right)ϵ\left(L_n\right)ϵ\left(-L_n\right)\\ \frac{1}{N_f^2}\left(\frac{\mathcal{F}\left[X\right]}{{\overline{\mathcal{F}}}_f}\right)ϵ{\left(L_n\right)}^2& \frac{1}{N_f{N}_r}\left(\frac{\mathcal{F}\left[X\right]}{{\overline{\mathcal{F}}}_f}\right)ϵ\left(L_n\right)ϵ\left(-L_n\right)& \frac{1}{N_f^2}{\left(\frac{\mathcal{F}\left[X\right]}{{\overline{\mathcal{F}}}_f}\right)}^2ϵ{\left(L_n\right)}^2\end{array}\right]\equiv \left[\begin{array}{ccc}{a}_{ff}& a_{fr}& a_{f{\mathcal{F}}_f}\\ a_{fr}& a_{rr}& a_{r{\mathcal{F}}_f}\\ a_{{\mathcal{F}}_f\mathcal{F}}& a_{r{\mathcal{F}}_f}& a_{{\mathcal{F}}_f{\mathcal{F}}_f}\end{array}\right].$$ (B4)

Using the determinant,

$$D=-a_{{\mathcal{F}}_f{\mathcal{F}}_f}{a}_{fr}^2+2a_{f{\mathcal{F}}_f}{a}_{fr}{a}_{r{\mathcal{F}}_f}-a_{ff}{a}_{r{\mathcal{F}}_f}^2-a_{f{\mathcal{F}}_f}^2{a}_{rr}+a_{{\mathcal{F}}_f{\mathcal{F}}_f}{a}_{ff}{a}_{rr},$$ (B5)

we write the inverse covariance matrix estimator as

$${\hat{\mathbf{\Theta }}}^{-1}=\left[\begin{array}{ccc}\frac{-a_{r{\mathcal{F}}_f}^2+a_{{\mathcal{F}}_f{\mathcal{F}}_f}{a}_{rr}}{D}-N_f+b& \frac{-a_{{\mathcal{F}}_f{\mathcal{F}}_f}{a}_{fr}+a_{f{\mathcal{F}}_f}{a}_{r{\mathcal{F}}_f}}{D}+b& \frac{a_{fr}{a}_{r{\mathcal{F}}_f}-a_{f{\mathcal{F}}_f}{a}_{rr}}{D}+b\\ \frac{-a_{{\mathcal{F}}_f{\mathcal{F}}_f}{a}_{fr}+a_{f{\mathcal{F}}_f}{a}_{r{\mathcal{F}}_f}}{D}+b& \frac{-a_{f{\mathcal{F}}_f}^2+a_{{\mathcal{F}}_f{\mathcal{F}}_f}{a}_{ff}}{D}-N_r+b& \frac{a_{f{\mathcal{F}}_f}{a}_{fr}-a_{ff}{a}_{r{\mathcal{F}}_f}}{D}+b\\ \frac{a_{fr}{a}_{r{\mathcal{F}}_f}-a_{f{\mathcal{F}}_f}{a}_{rr}}{D}+b& \frac{a_{f{\mathcal{F}}_f}{a}_{fr}-a_{ff}{a}_{r{\mathcal{F}}_f}}{D}+b& \frac{-a_{fr}^2+a_{ff}{a}_{rr}}{D}+b\end{array}\right].$$ (B6)

By applying the same steps as Appendix E of Shirts and Chodera,²⁷ we obtain the determinant

$$\mid {\hat{\mathbf{\Theta }}}^{-1}\mid =\frac{9b\left(a_{fr}^2{N}_f+a_{fr}{a}_{rr}{N}_f\right)}{D}.$$ (B7)

Applying Eq. 13 to $\hat{\mathbf{\Theta }}$ and simplifying, it can be shown that the variance estimate is

$${\sigma }^2\left({\overline{\mathcal{F}}}_f^2\right)\approx {\overline{\mathcal{F}}}_f^2\left({\Theta }_{{\mathcal{F}}_f{\mathcal{F}}_f}-2{\Theta }_{{\mathcal{F}}_{f}f}+{\Theta }_{ff}\right),$$ (B8)

$$=\frac{a_{{\mathcal{F}}_f{\mathcal{F}}_f}{a}_{fr}-a_{f{\mathcal{F}}_f}{a}_{fr}-a_{f{\mathcal{F}}_f}{a}_{r{\mathcal{F}}_f}+a_{ff}{a}_{r{\mathcal{F}}_f}}{a_{fr}\left(a_{fr}{N}_f+a_{rr}{N}_r\right)}.$$ (B9)