Error and efficiency of simulated tempering simulations

Abstract

We derive simple analytical expressions for the error and computational efficiency of simulated tempering (ST) simulations. The theory applies to the important case of systems whose dynamics at long times is dominated by the slow interconversion between two metastable states. An extension to the multistate case is described. We show that the relative gain in efficiency of ST simulations over regular molecular dynamics (MD) or Monte Carlo (MC) simulations is given by the ratio of their reactive fluxes, i.e., the number of transitions between the two states summed over all ST temperatures divided by the number of transitions at the single temperature of the MD or MC simulation. This relation for the efficiency is derived for the limit in which changes in the ST temperature are fast compared to the two-state transitions. In this limit, ST is most efficient. Our expression for the maximum efficiency gain of ST simulations is essentially identical to the corresponding expression derived by us for replica exchange MD and MC simulations [E. Rosta and G. Hummer, J. Chem. Phys. 131, 165102 (2009)] on a different route. We find quantitative agreement between predicted and observed efficiency gains in a test against ST and replica exchange MC simulations of a two-dimensional Ising model. Based on the efficiency formula, we provide recommendations for the optimal choice of ST simulation parameters, in particular, the range and number of temperatures, and the frequency of attempted temperature changes.

INTRODUCTION

Molecular dynamics (MD) and Monte Carlo (MC) simulations are powerful tools to explore the configuration space of molecular systems. However, for many systems the sampling efficiency is limited by the slow rate of interconversion between different conformational basins on the energy surface. To accelerate the conformational sampling, a variety of extended ensemble schemes have been developed. Among the most widely used ones are simulated tempering (ST),^{1, 2} replica exchange Monte Carlo (REMC),^{3, 4} and replica exchange molecular dynamics (REMD).^{5, 6} Replica exchange is also known as parallel tempering. These extended ensemble methods aim to establish an equilibrium between canonical systems at different temperatures (or, more generally, systems with different Hamiltonians⁷), an idea that can be traced back to the seminal papers by Bennett,⁸ and Swendsen and Wang.⁹ By coupling simulations at a low temperature of interest with simulations at high temperatures where the relaxation is fast, one hopes to transfer the improved sampling efficiency at the higher temperature down to the lower temperature.

In ST simulations,^{1, 2} the reference thermostat temperature is itself a dynamic variable. At fixed intervals along an otherwise regular MD or MC simulation, attempts are made to change the current reference temperature T_i to a trial temperature T_j picked from a discrete set (e.g., T_i±1 in a series of temperatures T₁,T₂,…,T_N). These attempted temperature changes are accepted with a criterion designed to maintain detailed balance for canonical distributions. In a long run (assuming mixing in the resulting Markov chain), canonical distributions will be obtained at each of the temperatures T_i. In REMD (Refs. ^{5, 6}) and REMC,^{3, 4} MD and MC simulations of N identical molecular systems are performed in parallel. Each replica is thermostatted at a different temperature T_i to establish corresponding canonical distributions.¹⁰ With a given frequency, one attempts to exchange the configurations of replica pairs, i↔j (typically i↔i±1). These replica exchanges are accepted with a probability that conserves the respective canonical distributions. ST and REMD have been proven particularly useful to enhance the sampling of biomolecular systems.^{5, 6, 11, 12, 13, 14, 15}

Key questions in ST, REMD, REMC, or other extended ensemble simulations are as follows: (1) How large is the possible gain in computational efficiency, (2) for which simulation parameters can this maximum gain be realized, and (3) how do the different methods compare with respect to their efficiency gains? We have previously developed a simple analytical formula for the efficiency gain in REMD and REMC simulations¹⁶ for the important case that the slow dynamics of the system of interest can be described by two-state transitions. Here, we derive a formula for the error and computational efficiency of ST simulations. The formula applies to systems with two-state dynamics at long-time scales (with an extension to the multistate case being discussed). For the sake of concreteness, we will frequently refer to two-state protein folding, but the theory is general. We measure the error in the estimator of an equilibrium property by its variance over repeated simulations. The computational efficiency is then defined as the rate with which the variance in the estimator decreases with length of the simulations.

The main result of this paper is that the relative efficiency of a ST simulation compared to a regular MD (or MC) simulation of the same duration t_sim, run at the temperature T_m of interest alone, is

$${\eta }_m=\frac{{\sigma }_{\text{MD}}^2}{{\sigma }_{\text{ST}}^2}=\sum _{i=1}^N{Q}_i\frac{{\tau }_m^{+}+{\tau }_m^{-}}{{\tau }_i^{+}+{\tau }_i^{-}}$$ (1)

independent of the property of interest. In the ST simulation, the system spends a fraction Q_i at each of the N temperatures T₁,T₂,…,T_N, with ${\sum }_{i=1}^N{Q}_i=1$. Equation 1 is valid for systems whose slow dynamics can be described by two-state transitions, and in the asymptotic limit of long simulation times t_sim. The lifetimes of the system in its two long-lived (unfolded and folded) states at temperature T_i are ${\tau }_i^{+}\equiv 1∕k_i^{+}$ and ${\tau }_i^{-}\equiv 1∕k_i^{-}$, where $k_i^{+}$ and $k_i^{-}$ are the corresponding (folding and unfolding) rates. A ST simulation is more efficient than a MD simulation of the same duration at temperature T_m if η_m>1.

We can interpret Eq. 1 for the relative efficiency of ST simulations as the ratio of the net reactive fluxes given by the number of folding and unfolding transitions per unit time in the ST simulation, and in the MD simulation: $2t_{\text{sim}}∕\left({\tau }_m^{+}+{\tau }_m^{-}\right)$ is the expected number of two-state transitions in MD and ${\sum }_{i}2Q_i{t}_{\text{sim}}∕\left({\tau }_i^{+}+{\tau }_i^{-}\right)$ in the ST simulation. Remarkably, Eq. 1 is essentially identical to the efficiency formula obtained by us for REMD (Ref. ¹⁶) on a different, more involved route. We will show that this is no coincidence: The derivation followed here for ST can be adapted for replica exchange. Practically, the equivalence of the efficiency formulas means that at least asymptotically for two-state systems and ignoring any issues of computational overhead, ST and REMD have the same efficiency.

To illustrate and test the efficiency formula, we perform MC simulations of a two-dimensional (2D) Ising system at zero magnetic field. Below the critical temperature of the infinite system, a finite spin system will only slowly interconvert between the “up” and “down” states of the net magnetization, if only single spin flips are attempted in the MC sampling. We show that the slow sampling of the net magnetization of a finite system is accurately captured by our theory. We also show that Eq. 1 quantitatively reproduces the observed gains in efficiency from using ST, and that the large predicted gains in efficiency can indeed be realized for the 2D Ising system with its thermally activated transition in magnetization. In conclusion, we discuss procedures to accelerate the sampling of temperature space and strategies to optimize the choice of parameters.

THEORY

Rate model of simulated tempering

Building on our recent work on REMD,¹⁰ we will analyze the statistical error and efficiency of ST simulations for the important case of systems whose dynamics at long times is governed by a single slow exponential relaxation process. Even though complex molecular systems normally have multiple states, and thus a broad spectrum of relaxation processes, at sufficiently long times the relaxation to equilibrium is often dominated by a single slow exponential process. The folding of small “two-state” proteins is a specific example, with measured relaxation times that range from microseconds to seconds.

For such systems, a reduced two-state description captures the long-time dynamics. For the sake of concreteness, here we will refer to protein folding and unfolding as the slow processes, but the results are general. In the following, we assume that the interconversion between the same two states U and F dominate the relaxation at low and high temperatures (but we note that this may not always be the case). In the absence of ST temperature changes, we assume that the interconversion between the folded and unfolded states F_i and U_i, respectively, at each of the temperatures T_i follows first-order kinetics,

$$U_i\underset{k_i^{-}}{\overset{k_i^{+}}{\rightleftharpoons }}{F}_i\left(i=1,2,\dots ,N\right).$$ (2)

At temperature T_i, the equilibrium folded and unfolded populations are $p_i=k_i^{+}∕\left(k_i^{+}+k_i^{-}\right)$ and q_i=1−p_i, respectively.

In ST simulations,^{1, 2} one periodically attempts to change the simulation temperature from T_i to T_j. These changes are accepted with a probability that maintains detailed balance with respect to a canonical ensemble at each temperature. If E=E(x) is the energy of a configuration x, a change from temperature T_i to T_j is accepted with probability

$$p_{\text{acc}}\left(T_i\to T_j\right)=\text{min}\left(1,e^{E∕k_B{T}_i-E∕k_B{T}_j+f_j-f_i}\right),$$ (3)

where k_B is Boltzmann’s constant. Here we assume that the generation probabilities for changes T_i→T_j and T_j→T_i are identical. If x is a purely configurational coordinate of the system, as in typical MC sampling, E is the potential energy; in contrast, if x is a position in phase space, as in MD simulations, E is the total energy. Alternatively, in ST combined with MD simulations, one can also use the potential energy E in the acceptance criterion, combined with either rescaling momenta or redrawing them from a Maxwell–Boltzmann distribution at the new temperature T_j upon acceptance. The f_i are constants that determine the fraction Q_i of time spent at temperature T_i in long runs,

$$Q_i=\frac{e^{f_i}{Z}_i}{{\sum }_{j=1}^N{e}^{f_j}{Z}_j},$$ (4)

where Z_i=∫dxe^{−E(x)∕k_BT_i} is the partition function at T_i and 0<Q_i≤1 with ${\sum }_{i=1}^N{Q}_i=1$. Note that otherwise the Q_i are arbitrary, determined by the specific choice of ST simulation parameters. In practice, the coefficients f_i can be gradually optimized to achieve a targeted distribution, e.g., Q_i=1∕N. These optimization strategies are typically based on the average energies at the different temperatures.^{2, 17} Here, we will instead solve the N−1 coupled linear equations

$$Q_i\sum _{j=1}^N{Q}_j^{\text{obs}}{g}_j=Q_i^{\text{obs}}{g}_i$$ (5)

for $g_i=\text{exp}\left(f_i^{\text{new}}-f_i^{\text{old}}\right)$ with g₁=1, $Q_i^{\text{obs}}$ is the observed fraction at T_i in a preliminary run with weight factors $f_i^{\text{old}}$, and $f_i^{\text{new}}$ are the new weight factors for subsequent runs.

To model ST with a kinetic model with 2N states, we allow changes in the temperature along the simulation trajectory. Here, we focus on the case of very frequent temperature change attempts (which will result in the highest possible efficiency gains). In this limit of fast ST, we can use a rate model also for changes in the temperature in ST, not just for the two-state transitions of the system,

$$F_i\underset{r_{i+1}^{-}}{\overset{r_i^{+}}{\rightleftharpoons }}{F}_{i+1}\text{and}{U}_i\underset{s_{i+1}^{-}}{\overset{s_i^{+}}{\rightleftharpoons }}{U}_{i+1}\left(i=1,2,\dots ,N-1\right).$$ (6)

In the kinetic model, the rates $r_i^{+}$ and $r_{i+1}^{-}$ ($s_i^{+}$ and $s_{i+1}^{-}$) for temperature changes in a folded (unfolded) structure should satisfy detailed balance to maintain the proper equilibria,

$$r_i^{+}=r{\left(Q_{i+1}{p}_{i+1}∕Q_i{p}_i\right)}^{1∕2},$$ (7)

$$r_{i+1}^{-}=r{\left(Q_i{p}_i∕Q_{i+1}{p}_{i+1}\right)}^{1∕2},$$

$$s_i^{+}=r{\left(Q_{i+1}{q}_{i+1}∕Q_i{q}_i\right)}^{1∕2},$$

$$s_{i+1}^{-}=r{\left(Q_i{q}_i∕Q_{i+1}{q}_{i+1}\right)}^{1∕2},$$

where r is a constant rate that will be chosen large enough to ensure frequent temperature changes. To model an actual simulation, the rate coefficients can be estimated from the acceptance probabilities p_acc, given the folding state of the system, and the time intervals Δt between attempted changes of a certain type, e.g., $s_i^{+}=p_{\text{acc}}\left(i\to i+1|U\right)∕\Delta t\left(i\to i+1\right)$. In our ST implementations, we will also consider non-nearest-neighbor transitions, for which temperature change rates can be estimated by analogy. Similar master-equation descriptions resulting in simple first-order kinetic models have previously been used to model replica exchange simulations.^{16, 18, 19, 20, 21}

Error and efficiency of MD and ST simulations

Calculating equilibrium properties is the main goal in both MD, MC, and ST simulations. We accordingly define the computational efficiency of the simulations as the rate with which the statistical error in the estimate of the property of interest decreases with the simulation time. Following our earlier work on the efficiency of REMD simulations,¹⁶ we note that for simulation times t_sim long compared to the overall longest relaxation time of the system, the central limit theorem implies that the error in the estimate $\overline{A}$ of the exact mean ⟨A⟩ of any property A decreases as $\text{var}\left(\overline{A}\right)=c∕t_{\text{sim}}$, where $\text{var}\left(\overline{A}\right)$ indicates the variance about the true mean in multiple simulations of the same duration t_sim. The constant c depends on the simulation method (MD versus ST). Note that we do not consider the approach of $\overline{A}$ to equilibrium for a given (“nonequilibrium”) initial condition. This approach to the true mean ⟨A⟩ occurs asymptotically as 1∕t_sim, faster than the decrease in the statistical error (t_sim^−1∕2).

The ratio of the variances of the estimators from MD and ST simulation methods allows us to compare their respective efficiencies. Even though we explicitly consider only the case of two-state protein folding, the results apply generally to systems with two metastable states that interconvert slowly compared to the relaxation processes within each state. Importantly,¹⁶ in the two-state case the relative computational efficiencies will not depend on the particular property A. In such systems, A will quickly relax to the average values _⟨A⟩F and _⟨A⟩U in folded and unfolded states F and U, respectively. If those averages differ, then $\text{var}\left(\overline{A}\right)\approx {\left({⟨A⟩}_F-{⟨A⟩}_U\right)}^2\text{var}\left(\overline{s}\right)$, where s indicates the folding state with s=1 if the system is folded, and s=0 if it is unfolded, such that _⟨s⟩i=p_i at temperature T_i. It thus suffices to compare the convergence of estimates of the fraction folded, _⟨s⟩1, at the temperature T₁ of interest.

For regular MD (and, by analogy, MC) simulations, one finds either directly¹⁶ or from the theory of statistical errors in single-molecule experiments^{22, 23} that the estimator $\overline{s}$, defined as the fraction of time spent in the folded state in a simulation of duration t_sim has a variance of

$${\sigma }_{\text{MD}}^2\left(t_{\text{sim}}\right)\equiv \text{var}\left(\overline{s}\right)=\frac{2}{t_{\text{sim}}}\frac{p_1\left(1-p_1\right)}{k_1^{+}+k_1^{-}}=\frac{2}{t_{\text{sim}}}\frac{k_1^{+}{k}_1^{-}}{{\left(k_1^{+}+k_1^{-}\right)}^3}$$ (8)

at the temperature T₁ of interest. Equation 8 applies in the asymptotic limit of $t_{\text{sim}}⪢{\left(k_1^{+}+k_1^{-}\right)}^{-1}$.

For ST simulations, we consider the limit of very frequent temperature changes (r large). In this fast ST limit, the different temperatures will be visited frequently before a change in the folding state occurs at any one of the temperatures T_i. We can thus describe the kinetics of the folding state with effective folding and unfolding rates,

$$k_{\text{eff}}^{+}=\sum _{i=1}^N{k}_i^{+}p\left(i|U\right),$$ (9)

$$k_{\text{eff}}^{-}=\sum _{i=1}^N{k}_i^{+}p\left(i|F\right),$$

where p(i|U) and p(i|F) are the probabilities of being at temperature T_i, given that the system is unfolded and folded, respectively. From Bayes’ theorem, p(i|U)=p(U|i)p(i)∕p(U) with p(U|i)=q_i, p(i)=Q_i, and $p\left(U\right)={\sum }_{i=1}^N{Q}_i{q}_i$ [and analogous relations for p(i|F)] it follows that

$$k_{\text{eff}}^{+}=\kappa ∕\left(1-p\right),$$ (10)

$$k_{\text{eff}}^{-}=\kappa ∕p,$$

where

$$\kappa =\sum _{i=1}^N\frac{Q_i}{1∕k_i^{+}+1∕k_i^{-}}=\sum _{i=1}^N\frac{Q_i}{{\tau }_i^{+}+{\tau }_i^{-}}$$ (11)

and $p=p\left(F\right)=k_{\text{eff}}^{+}∕\left(k_{\text{eff}}^{+}+k_{\text{eff}}^{-}\right)$ is the net fraction folded irrespective of temperature. In the last expression, we substituted the lifetimes ${\tau }_i^{\pm }=1∕k_i^{\pm }$ for the rates of interconversion.

To estimate the sampling error, we define the estimator x₁ of p₁≡_⟨s⟩1 (i.e., the fraction folded at the temperature T₁ of interest) as x₁=t_F1∕(t_F1+t_U1), where t_F1 and t_U1 are the times spent folded and unfolded, respectively, at temperature T₁ during a simulation of duration t_sim. In the following, we will calculate the variance ${\sigma }_{\text{ST}}^2\equiv \text{var}\left(x_1\right)\equiv \text{var}\left(\overline{s}\right)$ of x₁ over repeated ST simulations of duration t_sim.

In the limit of fast ST, we have effective two-state behavior with folding and unfolding rates $k_{\text{eff}}^{+}$ and $k_{\text{eff}}^{-}$, respectively. We define t_F and t_U=t_sim−t_F as the total time folded and unfolded in the simulation, irrespective of the temperature. Then, in the limit of fast ST, we have t_F1=p(1|F)t_F and t_U1=p(1|U)t_U, i.e., perfect sampling of the fractions of time spent at temperature T₁ for given times t_F and t_U spent folded and unfolded, respectively,

$$x_1\equiv \frac{t_{F1}}{t_{F1}+t_{U1}}=\frac{t_{F}p\left(1|F\right)}{t_{F}p\left(1|F\right)+\left(t_{\mathrm{sim}}-t_F\right)p\left(1|U\right)}.$$ (12)

We further assume that the simulation time t_sim is sufficiently long compared to the characteristic time 1∕k_eff, where$k_{\text{eff}}=k_{\text{eff}}^{+}+k_{\text{eff}}^{-}$ is the effective folding relaxation rate irrespective of temperature. We can then use the asymptotic formula^{22, 23} for the variance of t_F and linear error propagation,

$$\text{var}\left(x_1\right)=\text{var}\left(t_F\right){\left({\phantom{|}\frac{d{x}_1}{d{t}_F}|}_{t_F=⟨t_F⟩}\right)}^2,$$ (13)

where^{22, 23}

$$\text{var}\left(t_F\right)=\frac{2t_{\text{sim}}p\left(1-p\right)}{k_{\text{eff}}}=\frac{2t_{\text{sim}}{p}^2{\left(1-p\right)}^2}{\kappa }.$$ (14)

After differentiating Eq. 12 with respect to t_F, and substituting p(1|F)p+p(1|U)(1−p)=p(1)=Q₁ and ⟨t_F⟩=pt_sim, we find that

$${\phantom{|}\frac{d{x}_1}{d{t}_F}|}_{t_F=⟨t_F⟩}=\frac{k_1^{+}{k}_1^{-}}{t_{\text{sim}}p\left(1-p\right){\left(k_1^{+}+k_1^{-}\right)}^2}.$$ (15)

Substituting Eqs. 14, 15 into Eq. 13 results in

$$\text{var}\left(x_1\right)=\frac{2}{t_{\text{sim}}\kappa }\frac{{\left(k_1^{+}{k}_1^{-}\right)}^2}{{\left(k_1^{+}+k_1^{-}\right)}^4}.$$ (16)

Comparing this expression to the corresponding variance of the estimator in a regular MD simulation at temperature T₁, Eq. 8, we find that

$$\text{var}\left(x_1\right)\equiv {\sigma }_{\text{ST}}^2=\frac{{\sigma }_{\text{MD}}^2{k}_1^{+}{k}_1^{-}}{\left(k_1^{+}+k_1^{-}\right)\kappa }=\frac{{\sigma }_{\text{MD}}^2}{\sum _{i=1}^N{Q}_i\frac{{\tau }_1^{+}+{\tau }_1^{-}}{{\tau }_i^{+}+{\tau }_i^{-}}}.$$ (17)

In the last expression, we used Eq. 11 for κ and$k_1^{\pm }=1∕{\tau }_1^{\pm }$ for the rate coefficients. By rewriting Eq. 17 in terms of the ratio of variances ${\sigma }_{\text{MD}}^2∕{\sigma }_{\text{ST}}^2$ for a more general temperature of interest, T_m instead of T₁, we obtain our main result, Eq. 1, for the relative efficiency of ST simulations compared to MD simulations using the same computational resources.

We note that one obtains the same result by using error propagation, including covariances. Specifically, if one defines covariances of the times spent folded and unfolded irrespective of temperature as C_αβ=⟨t_αt_β⟩−⟨t_α⟩⟨t_β⟩ with α,β∊{F,U}, then

$$\text{var}\left(x_1\right)=\sum _{\alpha ,\beta }\frac{\partial x_1}{\partial t_{\alpha }}{C}_{\alpha \beta }\frac{\partial x_1}{\partial t_{\beta }}.$$ (18)

With this procedure it is possible to extend the efficiency analysis worked out above for two-state systems to systems whose slow dynamics involves M>2 states. In the limit of fast ST, one can again determine effective transition rates $K_{\alpha \beta }^{\text{eff}}={\sum }_i{K}_{\alpha \beta }p\left(i|\beta \right)$ from state β to α, averaged over the different temperatures T_i, analogous to Eq. 9. From the resulting effective rate matrix K^eff, the matrix of correlation coefficients,

$$C_{\alpha \beta }=⟨t_{\alpha }{t}_{\beta }⟩-⟨t_{\alpha }⟩⟨t_{\beta }⟩,$$ (19)

of the times t_α and t_β spent in states α,β and β irrespective of temperature (α=1,2,…,M) can be determined in the asymptotic limit of large t_sim as

$$\mathbf{C}=2t_{\text{sim}}\left[a^{-1}\mathbf{P}-{\left({\mathbf{K}}^{\text{eff}}+a\mathbf{P}\right)}^{-1}\right]{\mathbf{P}}^{\prime },$$ (20)

where P=(p₁,p₂,…,p_N)^T(1,1,…,1) is a projector onto the equilibrium distribution, P^′ is a diagonal matrix,$P_{\alpha \beta }^{\prime }={\delta }_{\alpha \beta }{p}_{\beta }$, and a is a an arbitrary nonzero number with dimensions of reciprocal time. Equation 20 corresponds to Eq. (12) in Ref. 24, noting that τ_αβ in that paper is related to C_αβ through C_αβ=−2t_simp_ατ_αβp_β for α≠β andC_αα=2t_simp_ατ_αα(1−p_α). If we define $x_{\alpha 1}=t_{\alpha 1}∕{\sum }_{\beta =1}^M{t}_{\beta 1}$ as the estimator of the relative population p(α|T₁) in state α at temperature T₁, then

$$\frac{\partial x_{\alpha 1}}{\partial t_{\beta }}=\frac{1}{t_{\text{sim}}}\left[{\delta }_{\alpha \beta }\frac{p\left(\alpha |T_1\right)}{p\left(\alpha \right)}-\frac{p\left(\alpha |T_1\right)p\left(\beta |T_1\right)}{p\left(\beta \right)}\right],$$ (21)

where p(α) is the fraction in state α averaged over all temperatures. Substitution of Eqs. 20, 21 into Eq. 18 allows us to evaluate the variance of the estimator x_α1 for a system with more than two states.

Relation to REMD and REMC

Equation 1 is identical to our previous result for the relative efficiency of REMD simulations¹⁶ if we identifyQ_i=1∕N. In REMD simulations, the different temperatures indeed have equal weights by construction. As a consequence, REMD and ST have the same computational efficiency for the systems studied here in the asymptotic limit of long simulations, and if the sampling of temperature space is uniform in ST. However, the methods are only equivalent in the limits of fast temperature changes in ST, and fast replica exchanges in REMC and REMD simulations (and for a large number of replicas).

The efficiency result for REMD simulations was derived by us on a very different route. Specifically, in Ref. 16 we studied the global relaxation dynamics of the total number n(t) of folded replicas at time t, requiring us to estimate the first nonzero eigenvalue of a large rate matrix. In the following, we sketch a rederivation of the efficiency formula for REMD (and, by analogy, REMC) simulations following the route in the present paper.

In REMD and REMC simulations, every replica is statistically equivalent.²⁵ So if we follow each replica, as it travels through temperature space, we have in effect the same dynamics as we have here for a single ST run (except that in REMC and REMD, Q_i=1∕N is fixed). In particular, if temperature changes are fast compared to folding and unfolding, the folding dynamics can be described with the effective rates in Eq. 10. As a consequence, in a REMD simulation of duration t_sim∕N (which is N times shorter than a MD or ST simulation using the same resources), the variance in the estimator x from an individual replica will be N times larger than that in Eq. 17. However, in the limit of a large number of replicas, the estimators from different replicas are statistically independent. As a consequence of the central limit theorem, the variance in the estimator x obtained by pooling all N replicas is thus reduced by a factor N, and we arrive at Eq. 17 also for REMD, with Q_i=1∕N. We note, however, that in the case of replica exchange the efficiency formula is only approximate (because of the assumption of statistical independence of the estimators), even in the limit of fast replica exchange, whereas it is exact for ST in the fast limit. We note further that in REMC and REMD, one gets weights Q_i different from 1∕N simply by running more replicas at a certain temperature.

As an interesting note, we showed in Ref. 16 that the estimators of all typical observables in a REMD simulation (and, in particular, the total number of folded replicas) relax at long times with a characteristic rate ${\sum }_{i=1}^N\left(k_i^{+}+k_i^{-}\right)q_i{p}_i∕{\sum }_{i=1}^N{q}_i{p}_i$. This rate is different from the relaxation rate predicted for ST, $k_{\text{eff}}=k_{\text{eff}}^{+}+k_{\text{eff}}^{-}$. However, k_eff should give the relaxation of the estimator evaluated for an individual replica in REMD (or REMC), as it performs its random walk through temperature space.

RESULTS

Error and efficiency in simulated tempering

To test Eq. 1 for the efficiency, we have performed MC simulations of a 2D Ising model with Hamiltonian,

$$H=\sum _{\left(i,j\right)}{\sigma }_i{\sigma }_j.$$ (22)

The sum extends over all distinct pairs (i,j) of neighboring spins σ_i=±1 on a periodic square lattice of size K×K. In a canonical ensemble with Boltzmann factor exp(−H∕T) and partition function ∑_{σi}exp(−H∕T) (with k_B=1 in this section), in the absence of an external magnetic field, and below the critical temperature T_c=2∕ln(1+2^1∕2)≈2.3 for the infinite system, finite 2D Ising models exhibit bistability with respect to their reduced magnetization m=∑_iσ_i∕K². At sufficiently low temperatures, m fluctuates between m≈−1 and 1.

The dynamics of this interconversion between up and down states of m will depend on the chosen MC move set. If in the MC sampling the move set is restricted to flips of single spins, the relaxation time of m will be long, measured in units of MC steps (one attempted move per spin) or MC passes (one attempted move for each of the K² Ising spins). The reason is that intermediate states with m≈0 are energetically unfavorable.

The inset in Fig. 1a shows the magnetization m(t) for a trajectory segment at T=2 of a 2D periodic Ising systems with 12×12 spins obtained from a regular MC simulation with single spin flips. From the decay of the autocorrelation function of m(t) at temperatures between 2 and 2.45 we have determined the relaxation time τ(T). The Arrhenius plot of τ(T) in Fig. 1a shows that the relaxation time grows exponentially with 1∕T at low temperatures. As a consequence, sampling of ⟨m⟩ with naive MC is challenging at low temperatures. The slow interconversion rate of the up and down states of m at low temperatures is also reflected in a high barrier at m=0 in the free energy profile F(m)∕T=−ln p(m), where p(m) is the probability density of m [Fig. 1b].

Figure 1.

Magnetization of the 12×12 2D Ising model. (a) Arrhenius plot of relaxation time τ(T) of magnetization m in units of MC passes vs 1∕T. Symbols: MC data; line: Arrhenius fit. Inset: MC trajectory segment of the reduced magnetization m(t) at T=2. (b) Free energy profileF(m)∕T=−ln p(m) of the reduced magnetization m at temperaturesT=2,2.1,…,2.4 (top to bottom at m=0).

In addition to the regular MC simulations, we have also performed ST simulations at temperatures T_i=2+(i−1)∕20, i=1,2,…. We have run simulations for N=2 (T₁=2, T₂=2.05), N=3 (T₁=2, T₂=2.05, T₃=2.1), etc., up to N=10 (with T₁=2 and T₁₀=2.45). The weight factors f_i determining the relative populations Q_i of the different temperatures in the ST runs are adjusted by solving Eq. 5 such thatQ_i=1∕N. After every MC pass, we randomly pick a trial temperature T_j (j=1,2,…,i−1,i+1,…,N) different from the current temperature T_i, and accept the change from T_i to T_j according to Eq. 3.

From each of the simulations, we estimated the average magnetization $\overline{m}$ at the lowest temperature, T=2, as the simple mean of the instantaneous magnetization along the trajectory. In the limit of perfect sampling, $\overline{m}$ converges to the exact limit ⟨m⟩=0. From multiple ST simulations of the same duration (number of MC passes between t_sim=10⁵ and 5×10⁶) we have determined the variance of the estimator $\overline{m}$ and confirmed that it scales as ${\sigma }^2\left(\overline{m}\right)=c_N∕t_{\text{sim}}$ for both regular MC (N=1) and ST simulations (N>1). The coefficients c_N are determined from a fit, with c₁ being the coefficient that determines the rate of convergence for regular MC, c₂ for ST with two temperatures, T₁=2 and T₂=2.05, etc. As shown in Fig. 2, the fitted coefficients c_N are in excellent agreement with the theoretical predictions from Eq. 8 for regular MC, and Eq. 17 for the ST simulations.

Figure 2.

Variance in the estimator of the mean magnetization as a function of the simulation length t_sim measured in units of MC passes. The symbols show the results of a regular MC simulation (top, inverted triangles) and ST simulations with N=3, 5, 7, and 9 temperatures (top to bottom) and corresponding upper temperatures T_N=2.1, 2.2, 2.3, and 2.4. The solid lines are the theoretical predictions, c_N∕t_sim, with c_N=(_⟨m⟩+−_⟨m⟩−)²var(x₁)=⟨2|m^|⟩2var(x₁) and $\text{var}\left(x_1\right)\equiv \text{var}\left(\overline{s}\right)$ defined in Eq. 8 for the MC data (N=1) and in Eq. 17 for the ST data (N>1). The dashed lines are fits of c_N∕t_sim. Note that for N=9, the actual error is slightly larger than predicted.

Figure 3 plots the relative computational efficiencies η₁=c₁∕c_N of the ST simulations compared to a regular MC simulation at temperature T=2, with the c_i obtained from fits. Also plotted is the relative efficiency gain η₁ predicted from Eq. 1 using the life times ${\tau }_i^{+}+{\tau }_i^{-}=4\tau \left(T_i\right)$, where τ(T_i) is the relaxation time of the magnetization m at temperature T_i plotted in Fig. 1a. We find that the relative efficiencies of sampling in the actual ST and MC simulations is accurately predicted by Eq. 1 up to an upper temperature of T_N=2.3 (with N=7 temperatures, resulting in an efficiency gain of a factor of ≈30). For upper temperatures T_N>2.3, Eq. 1 slightly overestimates the efficiency gain. At these high temperatures, the magnetization is barely bimodal [Fig. 1b, with a low barrier of only ≈1k_BT in the free energy profile along m], and both the assumption of two-state dynamics and fast temperature change begin to break down.

Figure 3.

Efficiency gain η₁ in ST (squares) and REMC simulations (triangles) over regular MC sampling of the magnetization m of the finite 2D Ising model at T₁=2. The theoretical prediction, Eq. 1, is shown as a line. The bottom axis indicates the upper temperature T_N in the ST and REMC simulations. The top axis indicates the corresponding number of evenly spaced temperatures in the ST simulations, and replicas in the REMC simulations.

A comparison of the normalized autocorrelation function of the magnetization C(t) from simulations and theory further demonstrates the quantitative accuracy of the kinetic model of ST (Fig. 4). We find that C(t)=⟨Δm(t)Δm(0)⟩∕⟨Δm²⟩ with Δm(t)=m(t)−⟨m⟩, as obtained from the ST simulations, matches the theoretical prediction of $C\left(t\right)\approx \text{exp}\left[-\left(k_{\text{eff}}^{+}+k_{\text{eff}}^{-}\right)t\right]$ with $k_{\text{eff}}^{\pm }$ defined in Eq. 9. The correlation functions are calculated from the time series of m(t) irrespective of the instantaneous temperatures in the ST simulations. Figure 4 shows that the magnetization relaxes exponentially with a rate of k_eff, with only small deviations even at a high upper temperature of T₉=2.4.

Figure 4.

Normalized autocorrelation function C(t) of the magnetizationm(t) irrespective of temperature in ST simulations (solid lines) with N=3, 5, 7, and 9 evenly spaced temperatures and upper temperatures of T_N=2.1, 2.2, 2.3, and 2.4. Dashed lines show the theoretical predictions C(t)=exp(−k_efft). For reference, the top curve shows C(t) for a regular MC run at temperature T₁=2.

Comparison to replica exchange Monte Carlo

For comparison, we have also run REMC simulations with replicas at the same temperatures as in the ST simulations. Figure 3 shows that the resulting efficiency gains for sampling the magnetization at T₁=2 are nearly identical to those of the ST simulations, and again in full agreement with Eq. 1 up to an upper temperature of T_N=2.3.

Effect of tempering protocol

To test the influence of slowing down the sampling of the different temperatures T_i (i.e., the diffusion in temperature space), we have also performed ST simulations in which temperature changes are only allowed with neighboring temperatures (j=i±1) instead of picking the trial temperature T_j randomly from the whole temperature range. For upper temperatures T_N<2.3, the efficiency gains are identical to those in ST simulations with random choice of the trial temperature T_j (not shown); for higher temperatures T_N, the efficiency gains fall below the theoretical limit of Eq. 1 and those in ST simulations with faster sampling of temperature space. We found that for T_N=2.45, the relaxation time of the temperature in the ST simulations with neighbor switching is comparable to the relaxation time of the magnetization at T_N. As a consequence, the assumption of fast temperature change is no longer valid, explaining the reduced gain in efficiency. In essence, with temperature relaxation being slow, transitions between positive and negative magnetizations at a high temperature have a high probability of reverting before the ST system “cools down” to the temperature of interest.

Effect of the number of temperatures

A somewhat surprising prediction from the efficiency formula equation 1 is that for a fixed temperature range T₁ to T_N, and with a given distribution of temperatures (e.g., uniform), the efficiency gain η_m will approach a constant in the limit of large N.¹⁶ This means that even though the fraction Q_m of time spent at the target temperature decreases as 1∕N, the sampling efficiency η_m is predicted to be unaffected. However, Eq. 1 applies only in the limit of fast temperature change, which may not be the case for large N, depending on the tempering protocol used. If temperature changes are only performed between nearest neighbors in temperature, then for large N the mean first passage time (in units of attempted temperature changes) to diffuse from a high temperature T_N to a low temperature T₁ will scale as N². Specifically, for uniform weights Q_i=1∕N, we can estimate the relaxation time of the temperature from the corresponding time for diffusion on a finite interval,

$${\tau }_T\approx \frac{N^2\Delta t}{10\overline{p_{\text{acc}}}},$$ (23)

where Δt is the time between attempted temperature changes in the same direction (e.g., up), and $\overline{p_{\text{acc}}}$ is the average probability of acceptance. Here we assume $\overline{p_{\text{acc}}}$ to be independent of temperature and the state of the system and p(T_i|F) to be roughly constant. We found Eq. 23 to be an excellent approximation for the relaxation time of the temperature (Fig. 5).

Figure 5.

Normalized autocorrelation function of the temperature in ST simulations with nearest-neighbor temperature exchanges (solid lines), N=3, 5, 7, and 9 evenly spaced temperatures and upper temperatures of T_N=2.1, 2.2, 2.3, and 2.4. Dashed lines show the theoretical predictions exp(−t∕τ_T) with τ_T from Eq. 23. Also shown is the curve for N=9 with global temperatures changes (i→1,2,…,i−1,i+1,…,N) permitting distant jumps in the temperature.

For large N with only nearest-neighbor temperature exchanges the assumption of fast temperature change will be violated. We tested this prediction by performing simulations with a fixed temperature range, but covered with different numbers N of uniformly spaced temperatures. For T₁=2 and T_N=2.4 with nearest-neighbor exchange, we found a significant gain in efficiency when using only N=3 or 5 temperatures (η₁≈53) instead of 9 (η₁≈30), but still somewhat lower than what is obtained by picking the trial temperature T_j at random from N=9 uniformly spaced temperatures (η₁≈59). In contrast, global temperature changes,i→1,2,…,i−1,i+1,…,N, result in much faster temperature relaxation (Fig. 5). In practice, one should monitor andoptimize the rate of temperature relaxation [e.g., from the autocorrelation function of T(t)] and compare it to the rate of relaxation of the two-state process. Fast temperature change is essential to achieve the maximum possible efficiency.

Effect of temperature weights

According to Eq. 1, the computational efficiency can be increased by giving higher weights Q_i to temperatures T_i, where ${\tau }_i^{+}+{\tau }_i^{-}$ is small and the system crosses frequently between the states. Typically, these will be the higher temperatures. We have tested Eq. 1 for nonuniform Q_i by running ST simulations of the Ising system with linearly increasing weights, $Q_i=i∕{\sum }_{j=1}^{N}j$. We again find excellent agreement between the predicted and observed gains in efficiency up to about N=7 with an upper temperature of T₇=2.3 (data not shown). For higher upper temperatures, the gains are somewhat below those predicted, as with uniform weights. Overall, using a linear bias toward higher temperatures results in gains ≈35% higher than those for uniform weights. Gains from increasing the weight of the higher temperatures were previously observed in simulations by Zhang and Ma.²⁶

Accelerated tempering protocol

We have also tested a scheme to speed up the temperature relaxation. Instead of first picking the temperatures T_j from 1,2,…,i−1,i+1,…,N (or from j=±1) and then accepting a change from T_i to T_j according to Eq. 1, we choose the temperatures T_j at random according to their probability for a given conformation with energy E,

$$p\left(T_j|E\right)=\frac{\text{exp}\left(-E∕k_B{T}_j+f_j\right)}{{\sum }_{k=1}^N\text{exp}\left(-E∕k_B{T}_k+f_k\right)}.$$ (24)

To sample T_j from this distribution, we calculate the cumulative distribution $P_j={\sum }_{k=1}^{k}p\left(T_k|N\right)$, with P₀=0 and P_N=1. We then draw a random number r with uniform distribution between 0 and 1. The new temperature T_j is determined suchthat P_j−1<r≤P_j. Care has to be taken to avoid over-∕underflows in the numerical calculation, e.g., by adding a suitable constant to all energies E. As an alternative, one can also repeatedly apply the standard ST temperature change protocol for a fixed configuration of energy E, and with detailed balance properly enforced.

In simulations of the Ising model with N=10 temperatures, and T₁=2 and T₁₀=2.45, the efficiency gains from this protocol over picking trial temperatures at random are negligible [η₁≈58 in both cases, compared to η₁=88 from Eq. 1]. We conclude from this result (and similar results from simulations run with attempted temperature changes after every attempted spin flip) that the gradual breakdown of the two-state assumption is primarily responsible for the actual efficiency being slightly lower than predicted for upper temperatures T_N>2.3.

CONCLUDING REMARKS

We have derived an analytical expression for the relative computational efficiency of ST simulations over regular MC or MD simulations. The expression, Eq. 1, applies to the important case that the slow dynamics of the system can be described by the relaxation between two metastable states, and is derived under the assumption that the temperature change in ST is fast compared to the transitions between the two states. We also sketch an extension to systems requiring more than two states. In our efficiency calculation, we have not considered the possibility of reweighting the data collected at the different temperatures. Combining data from all temperatures, for instance, by using the weighted histogram method,^{27, 28} can further enhance the sampling efficiency at the temperature of interest, in particular if ST is not in the fast limit; but gains should be minimal if ST is fast with respect to the property of interest.

We have tested the expression for the efficiency gain against MC calculations of the magnetization in a 2D Ising model. We have found that Eq. 1 predicts the efficiency gains quantitatively. Only at the highest upper temperatures T_N considered, the observed sampling efficiency gains are somewhat below those predicted. The reasons for this drop are, on the one hand, that the assumption of fast temperature change is increasingly violated at high T_N (where the relaxation between up and down magnetizations becomes faster than the relaxation from high to low temperatures); and, on the other hand, that the two-state assumption for the magnetization breaks down as the temperature T_N exceeds T_c.

Interestingly, the formula equation 1 for the efficiency gain in ST simulations is essentially identical to that derived previously on a very different route for REMD and REMC simulations. However, in the limit of fast exchange and a large number N of replicas, we have shown that the equivalence of the two approaches follows from simple statistical arguments. As an interesting consequence, in the asymptotic limit running multiple ST simulations in parallel without communication is as efficient as running REMD or REMC simulations with communication for replica exchange. This makes ST particularly useful for distributed computing.¹⁵

Our analytical result for the efficiency gain in ST simulations has practical implications in the setup of ST simulations as follows.

Temperature range T₁ to T_N

As in REMC and REMD simulations,¹⁶ one should cover a range of temperatures T_i, where the sum of relaxation times ${\tau }_i^{+}+{\tau }_i^{-}$ is smaller than ${\tau }_m^{+}+{\tau }_m^{-}$ at the temperature of interest, T_m. In this way, the more frequent transitions (corresponding to a higher reactive flux) at the higher temperatures result in improved sampling at the lower temperatures of interest. To further maximize η_m with respect to T₁ and T_N, the strategies discussed in Sec. II.E of our previous work¹⁶ for REMD and REMC can be adapted.

Importantly, because the sum of lifetimes ${\tau }_i^{+}+{\tau }_i^{-}$ enters the efficiency formula equation 1, not the relaxation rate $k_i^{+}+k_i^{-}$, ST, REMC, and REMD are not as powerful to enhance the sampling of entropy dominated processes such as protein folding as they are for enthalpy dominated transitions. Over a fairly broad range of temperatures, the folding rate of proteins is often found to be nearly constant or even slowing down with increasing temperature. Despite the acceleration of the enthalpy dominated unfolding process at high temperatures, the efficiency gain is limited by the unaccelerated folding process, resulting in long lifetimes of the unfolded state.

Temperature weights

Traditionally, one attempts to find weight factors f_i to ensure equal sampling Q_i≈1∕N for all temperatures. From the perspective of sampling efficiency, equal weighting is not optimal. Based on Eq. 1, it is advantageous to give higher weights to temperatures T_i at which ${\tau }_i^{+}+{\tau }_i^{-}$ is smaller than ${\tau }_m^{+}+{\tau }_m^{-}$ at the temperature of interest, T_m. In typical problems, one would thus want to give a disproportionally high weight to the higher temperatures. However, any optimization of the target weights Q_i has to consider that the relaxation in temperature space should be fast, as discussed next.

Temperature relaxation

In addition to choosing the optimal temperature range, it is important to ensure that temperature relaxation is fast compared to the relaxation of the system state at any given temperature [as has been found before for REMD (Refs. ^{16, 19, 20, 21})]. Temperature relaxation can be accelerated by spacing the temperatures T_i more narrowly (if the acceptance of temperature changes is low), by making more frequent temperature change attempts, by using nonlocal temperature change moves (that must, however, satisfy detailed balance, e.g., by picking the trial temperatures T_j at random), or by picking temperatures T_j according to Eq. 24. Alternative implementations of ST,^{14, 29, 30} often related to approaches in REMD and REMC simulations,³¹ may also help enhance the sampling of temperature space, and in turn configuration space.

There has been some discussion in the literature of the optimal frequency of attempted temperature exchanges, in particular, in the context of replica exchange simulations.^{21, 32, 33} On formal grounds, and consistent with other studies,^{21, 26, 33} we conclude that it is optimal to attempt temperature changes in ST, and temperature exchanges in REMC and REMD, as frequent as possible (as long as the computational cost of the exchange itself can be ignored). However, in practical implementations it is important that exchanges maintain a proper Boltzmann distribution. This is a particular concern in MD simulations, where one should use proper thermostats¹⁰ and integrators that evaluate positions and momenta at the same time step, such as velocity Verlet,³⁴ for proper rescaling (or redrawing) of the momenta.

Number N of temperatures

Interestingly, for a large number of temperatures N spaced according to a given distribution in a fixed range T₁ to T_N, the efficiency gain becomes independent of N, as long as the temperature change is fast. As a consequence, there is no loss in sampling efficiency from increasing the number N of temperatures, as long as the temperature relaxation rate does not suffer. However, circumventing the N² scaling of the temperature relaxation time for large N will require nonlocal temperature changes [such as the use of Eq. 24].

In summary, we have shown that ST can substantially enhance the sampling efficiency over regular MC and MD simulations, as measured by a decrease in the simulation time required to achieve a certain statistical accuracy. In the limit of fast temperature change, this gain is largest, and equivalent to that in REMD and REMC simulations covering the same temperatures. Achieving high computational efficiency in ST simulations requires a careful selection of the simulation parameters, a process that can be guided by the analytical expression Eq. 1 for the maximum gain achievable.