Single-Molecule Test for Markovianity of the Dynamics along a Reaction Coordinate

Abstract

In an effort to answer the much-debated question of whether the time evolution of common experimental observables can be described as one-dimensional diffusion in the potential of mean force, we propose a simple criterion that allows one to test whether the Markov assumption is applicable to a single-molecule trajectory x(t). This test does not involve fitting of the data to any presupposed model and can be applied to experimental data with relatively low temporal resolution.

Graphical Abstract

Kinetic measurements of protein folding and other complex biomolecular process are often interpreted using the model that assumes simple diffusion along some reaction coordinate x^1,2 with a (generally coordinate-dependent) diffusion coefficient D(x). This is a Markov process that lacks memory. Although this model and its low-dimensional extensions successfully account for numerous experimental^3–6 and simulational^7–9 observations, it remains a phenomenological description. On theoretical grounds, however, memory effects are expected to be ubiquitous^10,11 and to have important consequences on the observed dynamics and rates.^12–21 Moreover, they are commonly observed in computer simulations, and, less commonly, in experimental studies of biomolecules.^22–32

Single-molecule measurements that directly probe the time evolution x(t) of an experimentally measurable quantity (such as a protein’s extension in single-molecule pulling studies) offer an opportunity to estimate the magnitude of memory effects,^18,32 but no consensus yet exists regarding their importance. On one hand, several studies report that the dynamics of protein and DNA extension, as measured by single-molecule force spectroscopy, is consistent with the one-dimensional (1D) diffusion model with no memory.^5,33 On the other hand, this model appears inadequate as a global fit of both rates and transition path times, while describing each of these data sets separately;³⁴ this finding was interpreted as evidence that a one-dimensional model without memory is insufficient.^24,35

A key difficulty in estimating memory effects from single-molecule trajectories is their relatively low time resolution. For this reason, previous comparisons of the 1D Markov diffusion model with experimental data have been indirect and employed fitting certain experimental observables (e.g., transition rates, distributions of transition path times, etc.). This is in contrast to trajectories obtained from molecular simulations, for which, for example, the memory kernel in the generalized Langevin equation can be directly computed,^36–38 or the assumption of diffusive motion can be tested by examining short-time evolution of mean-square displacements.²³ In what follows we propose a simple and direct test of the Markov assumption that can be applied to any sufficiently long single-molecule trajectory measured experimentally or simulated on a computer. The test does not make any assumptions about the specific equations of motion governing the system and, importantly, does not require high temporal resolution of the observed trajectory x(t). We expect that this test would be straightforward to apply to existing single-molecule data and, in anticipation of such applications, illustrate how it works by analyzing (i) a trajectory obtained by numerically integrating a generalized Langevin equation with memory and (ii) a trajectory of a protein obtained from molecular simulations.³⁹

We consider a long trajectory x(t) that is bounded in space, as is usually the case for common reaction coordinates considered in protein folding and other intramolecular processes. We would like to establish whether x(t) is a Markov process. To this end we pick an interval (a, b) (where a < b) that is sufficiently well-sampled by the trajectory. Suppose at some time a point x lying within this interval is visited by the trajectory. This event could be part of four possible scenarios (see Figure 1): The trajectory could arrive through the boundary a (b), visit x, and then exit the interval through the same boundary. Or the trajectory could also arrive through the boundary a (b), pass the point x, and exit through the other boundary b (a). In the latter case, we say that x belongs to a transition path from a to b (b to a). Thus, all trajectories passing through x are either transition paths (from a to b or b to a) or loops that escape the interval through the same boundary through which they entered.

Figure 1.

Examples of trajectories crossing some intermediate point x that are transition paths from a to b (the combined trajectory composed of the green and red lines) and loops (green and red lines). A transition path from a to b can be thought of as a combination of a forward trajectory originating from some intermediate point x and proceeding to b and a time-reversed backward trajectory originating at x and going to a.

For each of the above events, we can define corresponding conditional probabilities P(a → alx), P(b → blx), P(a → blx), and P(b → alx). For example, P(a → blx) is the probability that a chosen point x belongs to a transition path from a to b, P(b → blx) is the probability that the trajectory passing through x is a loop entering and exiting the interval (a,b) through the boundary b, etc. In what follows, we will show that when the dynamics along x is Markovian, we have

$${\text{max}}_{a<x<b}P(a\to b|x)={\text{max}}_{a<x<b}P(b\to a|x)=1/4$$ (1)

$${\text{min}}_{a<x<b}[P(a\to a|x)+P(b\to b|x)]=1/2$$ (2)

In contrast, for non-Markov dynamics in the overdamped regime we have

$${\text{max}}_{a<x<b}P(a\to b|x)={\text{max}}_{a<x<b}P(b\to a|x)<1/4$$ (3)

$${\text{min}}_{a<x<b}[P(a\to a|x)+P(b\to b|x)]>1/2$$ (4)

Violation of the equalities of eqs 1 and 2 is, therefore, a signature of non-Markovian dynamics allowing one to assess the validity of the one-dimensional Markov diffusion model without attempting to use this model to fit the data.

Note that eq 2 follows from eq 1 and eq 4 follows from eq 3 because the sum of all four probabilities is unity, P(a → alx) + P(b → blx) + P(a → blx) + P(b → alx) = 1. Also note that the probabilities P(a → blx) and P(b → alx) are identical because of the time reversal symmetry assumed here; indeed, upon time reversal, any transition path from a to b becomes a transition path from b to a and vice versa. Because the direct and the time-reversed trajectories are statistically indistinguishable,⁴⁰ the above two probabilities must be the same. For these reasons, throughout the rest of this Letter, we focus on the behavior of the probability P(a → blx).

To explain the physical basis of the inequalities of eqs 3 and 4, consider first the simplest example of Markovian dynamics described by the overdamped Langevin equation:

$$0=-U^{\prime }(x)-{\gamma }_x\dot{x}+{\zeta }_x(t)$$ (5)

where U(x) is the potential of mean force, γ_χ a friction coefficient, and ζ_χ a Gaussian random force with zero mean, which satisfies the fluctuation–dissipation theorem (FDT), 〈ζ_x(t) ζ_x(t′)〉 = 2γ_xk_BTδ(t – t′), with the angular brackets denoting averaging over realizations of the random force. In this case, the probability P(a → blx) is

$$P(a\to b|x)={\Phi }_a(x){\Phi }_b(x)={\Phi }_b(x)\left[1-{\Phi }_b(x)\right]$$ (6)

Here Φ_a(b)(x) is the splitting probability⁴¹ (also known as committor^42–46 or PFOLD⁴⁷) that the system starting in x will reach the boundary a (b) before b (a). Equation 6 takes advantage of the fact that, in a Markov case, P(a → blx) is the product of two probabilities: (i) the probability Φ_b(x) that the trajectory exits the interval (a, b) through the right boundary and (ii) the probability Φ_a(x) that the trajectory enters through the left boundary, where time reversal symmetry of the Langevin dynamics has been taken into account. In Figure 1, the exiting and entering segments of the transition path are colored red and green.

The splitting probability Φ_b(x) is a monotonically increasing function with Φ_b(a) = 0 and Φ_b(x) = 1; therefore, P(a → blx), eq 6, is a nonmonotonic function of x that vanishes at the boundaries (P(a → bla) = P(a → blb) = 0) and has a maximum at x = x* where Φ_a(x*) = Φ_b(x*) = 1/2:

$${\text{max}}_{a<x<b}P(a\to b|x)=P\left(a\to b|x^{*}\right)=1/4$$ (7)

As the simplest example of non-Markovian dynamics, consider now the overdamped generalized Langevin equation (GLE) of the form

$$0=-U^{\prime }(x)-{\gamma }_x\dot{x}-k{\int }_0^t{\text{e}}^{-\left(t-t^{\prime }\right)/\tau }\dot{x}\left(t^{\prime }\right)\text{d}{t}^{\prime }+\zeta (t)$$ (8)

where ζ(t) is a Gaussian random force with zero mean, and where the FDT now takes the form 〈ζ(t)ζ(t′)〉 = 2γ_xk_BTδ(t − t′) + k_BTke^{−|t−t′|/τ}. A key observation is that the dynamics described by the GLE with a single-exponential memory kernel is equivalent to two-dimensional memoryless Langevin dynamics with one additional degree of freedom y:^48,49

$$\begin{gathered} 0=-\partial U_{2\text{D}}(x,y)/\partial x-{\gamma }_x\dot{x}+{\zeta }_x(t)\text{} \\ 0=-\partial U_{2\text{D}}(x,y)/\partial y-{\gamma }_y\dot{y}+{\zeta }_y(t) \end{gathered}$$ (9)

where U_2D(x, y) = U(x) + k(y − x)²/2. The two components of the random force, ζ_x(t) and ζ_y(t), are uncorrelated Gaussian forces with zero means; ζ_x(t) satisfies the FDT given below eq 5, while ζ_y(t) satisfies 〈ζ_y(t)ζ_y(t′)〉 = 2γ_yk_BTδ(t – t′), where γ_y = kτ is the friction coefficient along y.

Because the dynamics described by eq 9 is Markovian, an equivalent of eq 6 holds in the extended space, where the splitting probability is viewed as a function of x and y:

$$\begin{gathered} \tilde{P}(a\to b|x,y)={\tilde{\Phi }}_a(x,y){\tilde{\Phi }}_b(x,y) \\ ={\tilde{\Phi }}_b(x,y)\left[1-{\tilde{\Phi }}_b(x,y)\right] \end{gathered}$$ (10)

Here, the tilde above a quantity indicates that it is a function of the coordinates in the extended space. Whether a fragment of a trajectory in the xy-space is a transition path is determined exclusively by the dynamics of its x-component, which is the experimental observable. We can then obtain the probability P(a → blx) from $\tilde{P}(a\to b|x,y)$ by averaging over y. More precisely, consider the joint probability density $\tilde{\rho }(x,y,a\to b)$ that the system is found at (x, y) while on a transition path (from a to b) and similarly the joint probability

$$\rho (x,a\to b)=\int \text{d}y\tilde{\rho }(x,y,a\to b)$$ (11)

of finding the system at x while on a transition path. By definition of the conditional probability, we have

$$\tilde{\rho }(x,y,a\to b)=\tilde{P}(a\to b|x,y){\tilde{p}}_{\text{eq}}(x,y)$$ (12)

where ${\tilde{p}}_{\text{eq}}(x,y)$ is the equilibrium distribution of x and y.

Similarly, we have

$$\rho (x,a\to b)=P(a\to b|x)p_{\text{eq}}(x)$$ (13)

for the corresponding quantities defined in x-space only. Combining eqs 11–13, we obtain

$$\begin{gathered} P(a\to b|x)=\int \text{d}y\tilde{P}(a\to b|x,y)p_{\text{eq}}(y|x)\text{} \\ ={\langle \tilde{P}(a\to b|x,y)\rangle }_x \end{gathered}$$ (14)

Here, angular brackets indicate averaging over y at fixed x

$${\langle \dots \rangle }_x=\int \text{d}y(\dots )p_{\text{eq}}(y|x)$$ (15)

weighted with the local in x equilibrium density of y

$$p_{\text{eq}}(y|x)=\frac{{\tilde{p}}_{\text{eq}}(x,y)}{p_{\text{eq}}(x)}=\frac{{\tilde{p}}_{\text{eq}}(x,y)}{\int \text{d}y{\tilde{p}}_{\text{eq}}(x,y)}$$ (16)

Using the same definition of the average, we also find

$${\Phi }_b(x)={\langle {\tilde{\Phi }}_b(x,y)\rangle }_x$$ (17)

Finally, using eqs 10 and 14 and the fact that ${\langle {\tilde{\Phi }}_b^2(x,y)\rangle }_x\ge {\langle {\tilde{\Phi }}_b(x,y)\rangle }_x^2$ we obtain

$$\begin{gathered} P(a\to b|x)={\langle {\tilde{\Phi }}_b(x,y)\rangle }_x-{\langle {\tilde{\Phi }}_b^2(x,y)\rangle }_x \\ \le {\langle {\tilde{\Phi }}_b(x,y)\rangle }_x-{\langle {\tilde{\Phi }}_b(x,y)\rangle }_x^2={\Phi }_b(x)-{\Phi }_b^2(x) \end{gathered}$$ (18)

The maximum value of ${\Phi }_b(x)-{\Phi }_b^2(x)$ achieved on the interval (a, b) is 1/4 (cf. eq 7). The left-hand side (lhs) of eq 18, therefore, cannot exceed 1/4, but it can be less. In fact, the only way the lhs of eq 18 can attain its maximum value of 1/4 at some x = x* is if ${\tilde{\mathbf{\Phi }}}_b\left(x^{*},y\right)=1/2$ for any y. This implies that the dynamics along x and y are decoupled and, therefore, the dynamics along x is Markovian. The inequality stated by eq 3 is thus a fingerprint of non-Markovian dynamics.

Figure 2 illustrates the inequality of eq 3 for the system obeying eq 8. The probability P(a → blx) was computed from a long simulated trajectory x(t) using the identity⁴⁵

$$P(a\to b|x)=\frac{p(x|a\to b)P(a\to b)}{p_{\text{eq}}(x)}$$ (19)

where the equilibrium distribution p_eq(x) is the distribution of all the points belonging to the trajectory x(t), p(xla → b) the distribution of all the points that belong to transition paths from a to b only, and P(a → b) the fraction of time the system is found on a transition path from a to b. For short memory times τ, we expect the dynamics described by eq 8 to be close to the Markov limit. Indeed, for τ = 0.05, the function P(a → blx) reaches a maximum value that is close to 1/4. As the memory time increases, however, the non-Markovian character of the dynamics is readily detected by observing that the maximum of P(a → blx) becomes less than 1/4.

Figure 2.

Non-Markovian character of GLE dynamics, eq 8, readily observed in the behavior of P(a → blx) (right). Here, the potential (measured in units of thermal energy) is given by U(x) = x⁸ − x + 0.5 sin(7x) and is depicted on the left, with the boundaries a and b shown as vertical dashed lines. Other parameters are γ_x = 1; k = 10; and τ = γ_y/k = 0.05 (red), 0.5 (green), and 5.0 (blue).

Although eq 3 was derived for the GLE with a single-exponential memory kernel, eq 8, it holds for an overdamped GLE with a memory kernel of arbitrary form. Indeed, introducing a set of auxiliary degrees of freedom y_n coupled to x via a potential k_n(y_n − x)²/2 results in a GLE of the form similar to eq 8, but with memory kernel equal to the sum of exponentials

$$0=-U^{\prime }(x)-{\gamma }_x\dot{x}-{\int }_0^t\sum _n{k}_n{\text{e}}^{-\left(t-t^{\prime }\right)/{\tau }_n}\dot{x}\left(t^{\prime }\right)\text{d}{t}^{\prime }+\zeta (t)$$ (20)

The derivation of eq 3 can be generalized to such a multidimensional system. By allowing a continuum of auxiliary degrees of freedom, we can now extend the proof to a GLE with arbitrary memory kernel. Moreover, as seen from its proof, eq 3 does not require that the time evolution of x obeys a GLE as long as it can be viewed as a projection of some higher-dimensional Markov process in an extended configuration space. We further emphasize that our results are valid regardless of how the boundaries a and b are chosen. In particular, it is not assumed that the transition region a < x < b separates two basins of attraction and contains a barrier; indeed, no such basins of attraction exist in the potential chosen in the example of Figure 2.

As a more realistic case, we now apply our test of the Markov assumption to the folding of a designed double norleucine mutant of 35-residue villin headpiece C-terminal fragment (HP-35) protein, for which a long trajectory displaying many folding–unfolding transitions has been simulated by Piana, Lindorff-Larsen, and Shaw.³⁹ Following their work, the root-mean-square deviation from the native structure is used here as a dynamic variable x(t). Based on the decay of its autocorrelation function and on the distributions of transition-path times, an earlier study²⁴ concluded that the dynamics of this coordinate is strongly non-Markovian. Consistent with these conclusions, P(a → blx) peaks at a value of ~0.11, much lower than the 0.25 expected for a Markov process (Figure 3). Without even attempting to model the data as a one-dimensional diffusion process, with position-dependent or position-independent diffusion coefficient, one establishes that this would not be an adequate model for the folding dynamics along x.

Figure 3.

Probability P(a → blx), where x is the root-mean-square deviation from the native structure for a designed HP-35 protein, peaking at value that is much less than the 1/4 expected in the Markov case. This immediately rules out one-dimensional diffusion, with a position-dependent or position-independent diffusion coefficient, as an adequate model for the dynamics along x. In this analysis, the interval (a, b) was chosen exactly as the transition region in previous studies.^24,39

In the preceding discussion, we considered dynamics in configuration space. In particular, eqs 3 and 4 were derived by assuming that the dynamics of x is a projection of Markovian dynamics occurring in some extended configuration space. This is a common assumption for many biophysical phenomena that take place in solution, where overdamped dynamics is a good approximation for most relevant degrees of freedom. When inertial effects are not negligible, however, dynamics in phase space must be considered. Here we briefly discuss how inertial effects are reflected in P(a → blx), focusing on the case where the dynamics along x alone is not Markovian, but the dynamics in the phase space (x,v) is Markovian, as exemplified by the (nonoverdamped) Langevin equation

$$m\ddot{x}=m\dot{v}=-U^{\prime }(x)-{\gamma }_x\dot{x}+{\zeta }_x(t)$$ (21)

where m is a mass. The equivalent of eq 10 written in phase space is then

$$\begin{gathered} \tilde{P}(a\to b|x,v)={\tilde{\Phi }}_a(x,-v){\tilde{\Phi }}_b(x,v) \\ ={\tilde{\Phi }}_b(x,v)\left[1-{\tilde{\Phi }}_b(x,-v)\right] \end{gathered}$$ (22)

There is an essential difference between eqs 10 and 22: to form a transition path from two trajectory segments originating from x, the velocity of one of them must be reversed (cf. green and red trajectory segments in Figure 1). This renders our previous analysis, which led to the conclusion that the maximum value of P(a → blx) cannot exceed 1/4, inapplicable. For a symmetric potential, U(x) = U(−x), and for a symmetric transition region, a = −b, one can, in fact, show that P(a → b|0) always exceeds 1/4! Indeed, because of symmetry, we have ${\tilde{\Phi }}_a(0,-v)=1-{\tilde{\Phi }}_b(0,-v)={\tilde{\Phi }}_b(0,v)$, and therefore

$$\tilde{P}(a\to b|0,v)={\tilde{\Phi }}_b^2(0,v)$$ (23)

By averaging $\tilde{P}(a\to b|0,v)$ over the velocity (analogous to averaging over y in eq 14), we obtain

$$\begin{gathered} P(a\to b|0)={\langle \tilde{P}\left(a\to b|0,v\right)\rangle }_v={\langle {\tilde{\Phi }}_b^2(0,v)\rangle }_v \\ \ge {\langle {\tilde{\Phi }}_b(0,v)\rangle }_v^2={\Phi }_b^2(0)=\frac{1}{4} \end{gathered}$$ (24)

because symmetry dictates that Φ_b(0) = 1/2. Finding values of P(a → blx) that exceed 1/4, therefore, would indicate importance of inertial effects (Figure 4).

Figure 4.

Peak value of P(a → blx) (right panel) exceeding 1/4 for Langevin dynamics (eq 21) in a symmetric potential βU(x) = 2(x² − 1)² with a mass m = 1. The potential is depicted in the left panel, with the boundaries a and b shown as vertical dashed lines. For low friction (γ_x = 0.01, red), the maximum of P(a → blx) approaches 1/2 as expected for ballistic dynamics (at x coinciding with the barrier top, P(a → blx) equals the probability to have a positive velocity, which is 1/2). As the friction coefficient is increased (γ_x = 1 green, γ_x = 10, blue), the overdamped Markov limit is approached, where max_a<x<bP(a → blx) = 1/4.

In conclusion, we showed that the probability P(a → blx) = P(b → alx) that a given point x belongs to a transition path between arbitrarily chosen boundaries a and b and the probability P(a → alx) + P(b → blx) that x is belongs to a loop (i.e., not a transition path) obey inequalities (eqs 3 and 4), which turn into equalities (eqs 1 and 2) only in the Markov case. Testing whether these equalities are satisfied for an experimentally observable reaction coordinate x(t) directly shows whether its underlying dynamics is Markovian. Because the above probabilities do not depend on time explicitly, this Markovianity test can be applied to data that have limited time resolution, as is often the case with experimental signals.