Generalizing HMMs to Continuous Time for Fast Kinetics: Hidden Markov Jump Processes

Abstract

The hidden Markov model (HMM) is a framework for time series analysis widely applied to single-molecule experiments. Although initially developed for applications outside the natural sciences, the HMM has traditionally been used to interpret signals generated by physical systems, such as single molecules, evolving in a discrete state space observed at discrete time levels dictated by the data acquisition rate. Within the HMM framework, transitions between states are modeled as occurring at the end of each data acquisition period and are described using transition probabilities. Yet, whereas measurements are often performed at discrete time levels in the natural sciences, physical systems evolve in continuous time according to transition rates. It then follows that the modeling assumptions underlying the HMM are justified if the transition rates of a physical process from state to state are small as compared to the data acquisition rate. In other words, HMMs apply to slow kinetics. The problem is, because the transition rates are unknown in principle, it is unclear, a priori, whether the HMM applies to a particular system. For this reason, we must generalize HMMs for physical systems, such as single molecules, because these switch between discrete states in “continuous time”. We do so by exploiting recent mathematical tools developed in the context of inferring Markov jump processes and propose the hidden Markov jump process. We explicitly show in what limit the hidden Markov jump process reduces to the HMM. Resolving the discrete time discrepancy of the HMM has clear implications: we no longer need to assume that processes, such as molecular events, must occur on timescales slower than data acquisition and can learn transition rates even if these are on the same timescale or otherwise exceed data acquisition rates.

Significance

Hidden Markov models (HMMs) have been a workhorse of single-molecule data analysis for the past 50 years. Yet, HMMs are inappropriate for molecular systems as they must assume, by construction, that single-molecule events occur much more slowly than the timescale of data acquisition. To move beyond fundamental HMM limitations, we must treat single-molecule events in continuous time as they occur in nature. Here, we exploit and generalize inverse methods for Markov jump processes to treat single-molecule events in continuous time. The implications of our work are profound and we can learn 1) the kinetics of single-molecule events without assuming these to be slower than the measurement timescale and 2) the rates on timescales faster than data acquisition.

Introduction

Hidden Markov models (HMMs) have been important tools of time series analysis for over 50 years (1,2). Under some modeling assumptions, detailed shortly, HMMs have been used to self-consistently determine dynamics of physical systems under noise and the properties of the noise obscuring the system’s dynamics itself.

Originally developed for applications in speech recognition (3,4), the relevance of HMMs to single-molecule time series analysis was quickly recognized (5, 6, 7, 8, 9, 10, 11, 12, 13, 14). Since then, HMMs and variants have successfully been used in the interpretation of ion channel patch-clamp data (15, 16, 17, 18, 19), fluorescence resonant energy transfer (FRET) (10,20, 21, 22, 23, 24, 25, 26, 27, 28, 29), force spectroscopy (30, 31, 32), among many other physical applications (3,33,34).

For HMMs to apply to single molecules and other physical systems, the assumptions underlying the HMMs must hold for such systems. There are several such assumptions worthy of consideration.

• 1.HMMs assume that the system under study evolves in a discrete state space (whether physical of conformational). This is a reasonable approximation for biomolecules visiting different conformations (34, 35, 36) or fluorophores visiting different photostates (34,35,37). Of parallel interest to this point is the notion that the number of discrete states is known (though the transition probabilities between states is unknown). The assumption of a known number of states has been lifted thanks to extensions of the HMM (35,38, 39, 40, 41, 42, 43, 44) afforded by nonparametrics that we discuss elsewhere (34,35,39, 40, 41,44).

• 2. HMMs assume that measurements are obtained at discrete time levels. That is, successive measurements are reported at regular time levels separated by some fixed period $\Delta t$. For clarity, we call $\Delta t$ the “data acquisition period”. This assumption is consistent with a number of experimental biophysical settings (10,11,39, 40, 41,45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57).

• 3. HMMs assume that physical systems transition between states in discrete time steps. Put differently, HMMs apply under the implicit assumption that the underlying system switches between states “rarely” as compared to the data acquisition period, $\Delta t$. This can only really be assured if the transition rates (as required in a continuous-time description) are slow. This assumption is implicit to the very definition of an HMM, which requires that the system’s switching occurs “precisely” at the end of the data acquisition periods (3,39,40,58, 59, 60).

This last assumption is problematic and presents the following conundrum: on the one hand, the transition rates are unknown and their analog for discrete time processes (transition probabilities) are typically to be determined using HMMs. On the other hand, we must assume that these unknowns are slow as compared to the data acquisition rate. Even if, optimistically, transition rates are slow, molecular events themselves are stochastic and, as with all physical processes, occur in continuous time. As such, any one event has a probability of occurring on timescales faster than the data acquisition period.

As an example, Fig. 1 illustrates the types of dynamical measurements that we can and cannot analyze within the HMM paradigm. The top panel shows an example of single-molecule measurements characterized by slow kinetics that can be analyzed within an HMM paradigm. In contradistinction to the above is the bottom panel illustrating an example of fast kinetics, as compared to the data acquisition rate, that cannot currently be analyzed within the HMM paradigm. The reason for this is simple: the fast kinetics give rise to a large number of apparent states that go beyond the two real states. This is because the measurements reported at each time point averages the molecular signal from all states visited in each acquisition period. Yet it is clear that the information on the transition rates between the rapidly switching molecular states is encoded in the time trace however uninterpretable.

Figure 1.

A conceptual illustration of single-molecule continuous-time-switching kinetics between discrete states probed in discrete time. For illustrative purposes only, the trajectory of a single molecule between two states $\left({\sigma }_1,{\sigma }_2\right)$ is shown in cyan in (a1) and (b1). For concreteness, we can think of these molecular states as conformational states. The state levels, i.e., signal level in the absence of noise, for these states is ${\mu }_{{\sigma }_1}$ and ${\mu }_{{\sigma }_2}$, respectively, and shown by the horizontal gray dotted line in (a1) and (b1). This synthetic experiment starts at time $t_0=0.05$ s, and ends at time $t_N=20$ s, and the data acquisition period, $\Delta t$, is 0.1 s. Next, again only for illustration, we assume that the measurements are acquired by a detector that has a fixed integration period $\tau =90$ ms (light gray) for each fixed data acquisition period shown in (a1) and (a2). As a molecule may switch between states during an integration period, the measurements represent the signal levels capturing the average of the amount of time spent at each state levels $\left({\mu }_{{\sigma }_1},{\mu }_{{\sigma }_2}\right)$ visited (in addition to added noise). (a1) and (a2) are associated with single-molecule kinetics that are slower than the data acquisition rate. Instead, in (b1) and (b2), we show a single-molecule trajectory when a molecule’s kinetics are faster than the data acquisition rate. In (b1), slow kinetics result in well separated state occupancy histograms around the average state levels. In (b2), we do not have well separated histograms centered around the average state levels because of the fast kinetics of the molecule. To see this figure in color, go online.

Indeed, to address the last assumption, a recent method (46), termed ${\text{H}}^2\text{MM}$, was proposed. ${\text{H}}^2\text{MM}$ has been applied to single-molecule FRET photon arrival time series analyses (47,48). This method handles fast switching kinetics within an HMM framework by embedding a finer discrete timescale into the HMM: in this case, one fine enough to avoid the arrival of two photons within the same time bin. The ${\text{H}}^2\text{MM}$ applies to a scenario different from that provided in Fig. 1 for which the detector model produces measurement that coincide with noise on top of average molecular signal obtained throughout the detector’s exposure time.

Statistical analysis methods exploiting finer time grids, to approximate faster “continuous” time processes, had previously been considered albeit, for applications outside the natural sciences (61,62). Such statistical methods (61,62) have been criticized (63) for two main reasons: 1) they sometimes, though not always, introduce additional computational load because of the finer time grid and, almost certainly, 2) introduce bias by discretely approximating a continuous-time process. In the mathematical literature, these two challenges are what motivated the development of strategies to infer kinetic rates for genuinely continuous-time processes albeit measured in discrete time (63).

It is, therefore, natural to propose an analysis method that treats physical processes as they occur in “continuous” time to extract rates directly from traces with fast kinetics without relying on the artificial assumption that the physical processes involved occur on timescales much slower than the data acquisition period. To do so, we must fundamentally upgrade both key ingredients of the HMM model: 1) the system dynamics must be in continuous time; and 2) the measurement output must realistically reflect an average over the dynamics of the system over the data acquisition period. The output then encodes fast dynamics that can be retrieved (21,57,64, 65, 66, 67).

It is indeed to address processes evolving in continuous time that continuous-time Markov models, so-called Markov jump processes (MJPs), were developed (45,68, 69, 70). MJPs describe continuous-time events using rates (rather than transition probabilities) and recent advances in computational statistics (63,71, 72, 73, 74, 75) have made it possible to learn these rates given data. However, an important challenge remains: how to infer MJP rates under the assumption that the measurement process averages the probed signal over each measurement period? The nature of the measurement process and the inherent continuous dynamics, therefore, suggest a hidden MJPs (HMJPs) framework that we put forward herewith.

In the Methods section below, we start with the formulation of our HMJP model and also, briefly, summarize the HMM. Next, in the Results section, we move on to the head-to-head comparison of HMMs and HMJPs (showing in what limit the HMM exactly reduces to the HMJP). We focus on their respective performance in learning molecular trajectories and transition probabilities. We show how HMJPs successfully outperform HMMs, especially for kinetics occurring on timescales on the order of or exceeding the data acquisition period. Finally, in the Discussion section, we discuss the broader potential of HMJPs to biophysics. Fine details on the implementation of these two methods can be found in the Supporting Materials and Methods Section A.

Methods

In this section, we describe a physical system that evolves in continuous time alongside a measurement model. We also discuss how to generate realistic synthetic data from such a model and subsequently analyze time traces reflecting both fast and slow dynamics. We analyze the traces using two different methods: HMMs, as they are broadly used across the literature, and our proposed HMJPs, which we describe in detail. We compare the analyses in the Results section.

Model description

Using the experimental data, and the model of the experiment that we will describe, our goal is to learn 1) the switching rates between the states of the system (i.e., transition rates), 2) the state of the system at any given time (which we call the trajectory of the system), 3) initial conditions of the system, and 4) parameters describing the measurement process (i.e., parameters of the emission distribution).

Dynamics

We start by defining the trajectory $\mathcal{T}(\cdot )$ that tracks the state of the system over time. Here, $\mathcal{T}\left(t\right)$ is the state of the system at time t and, as such, it is a “function” over the time interval $[t_0,t_N]$. We adopt functional notation and distinguish between $\mathcal{T}(\cdot )$ and $\mathcal{T}\left(t\right)$ to avoid confusion with the entire trajectory and the value attained at particular time levels, critical to the ensuing presentation.

We label the states to which the system has access with ${\sigma }_k$ and use the subscript $k=1,\dots ,K$ to distinguish them. For example, ${\sigma }_1/{\sigma }_2$ may represent a protein in folded/unfolded conformation or an ion channel in an on/off state. With this convention (borrowed from (63)), if the system is at ${\sigma }_k$ at time t, then we write $\mathcal{T}\left(t\right)={\sigma }_k$.

As with most molecular systems (36,39,76,77), the switching dynamics are faithfully modeled as memoryless. That is, the waiting time of the system in a state is exponentially distributed. Such memoryless processes are termed “MJPs” and below, we present their mathematical formulation. Memoryless dynamics often result from kinetic schemes relying on master equations, and in subsequent section, we explore the connections between our model and kinetic schemes in detail.

At the experiment’s onset, we assume the state of the system $\mathcal{T}\left(t_0\right)$ is chosen stochastically among ${\sigma }_k$. We use ${\rho }_{{\sigma }_k}$ to denote the probability of the system starting at ${\sigma }_k$ and collect all initial probabilities in $\overline{\rho }=\left({\rho }_{{\sigma }_1},{\rho }_{{\sigma }_2},\dots ,{\rho }_{{\sigma }_K}\right)$, which is a probability vector (78, 79, 80, 81).

Memoryless switching kinetics are described by “switching rates” between all possible state pairs. These switching rates are labeled with ${\lambda }_{{\sigma }_k\to {\sigma }_{k^{\prime }}}$ and, in biophysics, are most commonly termed transition rate coefficients. By convention, all self-switching rates are zero ${\lambda }_{{\sigma }_k\to {\sigma }_k}=0$, which, in general, allows for at most $K\left(K-1\right)$ nonzero rates (36). Although, the switching rates ${\lambda }_{{\sigma }_k\to {\sigma }_{k^{\prime }}}$ fully describe the system’s kinetics, as we will see shortly, it is mathematically more convenient to work with an alternative parametrization. In this alternative parametrization, we keep track of the “escape rates”

$${\lambda }_{{\sigma }_k}=\sum _{k^{\prime }=1}^K{\lambda }_{{\sigma }_k\to {\sigma }_{k^{\prime }}},$$ (1)

which, for simplicity, we gather in $\overline{\lambda }=\left({\lambda }_{{\sigma }_1},{\lambda }_{{\sigma }_2},\dots ,{\lambda }_{{\sigma }_K}\right)$. In biophysics, each ${\lambda }_{{\sigma }_k}$ is understood to correspond to the reciprocal of a mean dwell time. Furthermore, instead of keeping track of each rate ${\lambda }_{{\sigma }_k\to {\sigma }_{k^{\prime }}}$, we keep track of the rates normalized by the escape rates, namely

$${\pi }_{{\sigma }_k\to {\sigma }_{k^{\prime }}}=\frac{{\lambda }_{{\sigma }_k\to {\sigma }_{k^{\prime }}}}{{\lambda }_{{\sigma }_k}}$$ (2)

Gathering all normalized rates out of the same state in ${\overline{\pi }}_{{\sigma }_k}=\left({\pi }_{{\sigma }_k\to {\sigma }_1},{\pi }_{{\sigma }_k\to {\sigma }_2},\cdots ,{\pi }_{{\sigma }_k\to {\sigma }_M}\right)$, we see that each ${\overline{\pi }}_{{\sigma }_k}$ forms a probability vector (80). The entries of these probability vectors can also be termed “splitting probabilities.”

In summary, instead of $K\left(K-1\right)$ switching rates ${\lambda }_{{\sigma }_k\to {\sigma }_{k^{\prime }}}$, we describe the system’s kinetics with K escape rates ${\lambda }_{{\sigma }_k}$ and K switching probability vectors ${\overline{\pi }}_{{\sigma }_k}$. The latter have, by convention, ${\pi }_{{\sigma }_k\to {\sigma }_k}=0$, and so, the total number of scalar parameters is the same in both parametrizations. Below, for simplicity, we gather all transition probability vectors into a matrix

$$\overline{\overline{\pi }}=\left(\begin{array}{l}{\overline{\overline{\pi }}}_{{\sigma }_1}\hfill \\ {\overline{\overline{\pi }}}_{{\sigma }_2}\hfill \\ ⋮\hfill \\ {\overline{\overline{\pi }}}_{{\sigma }_K}\hfill \end{array}\right)$$ (3)

Measurements

The overall input to our method consists of the measurements $\mathbf{x}=\left(x_1,x_2,\dots ,x_N\right)$ acquired in an experiment. Here, $x_n$ indicates the n^th measurement and, for clarity, we assume measurements are time ordered, so $n=1$ labels the earliest acquired measurement and $n=N$ the latest. These measurements may be image values, photon counts, FRET efficiencies, derived intermolecular extensions, or any other quantity determinable in an experiment.

Each $x_n$ is reported at a time $t_n=t_{n-1}+\Delta t$, which is $\Delta t$ later than the time $t_{n-1}$ at which the previous measurement $x_{n-1}$ was reported. For completeness, together with the time levels $t_1,t_2,\dots ,t_N$ at which a measurement is reported, we also consider an additional time level $t_0$, that marks the onset of the experiment, which is not associated with any measurement, Fig. 1.

The most common assumption made almost universally by HMMs is that the instantaneous state of the system at $t_n$ determines $x_n$. Yet, for realistic detectors, the reported value $x_n$ is influenced by the entire trajectory of our system during the n^th integration period, which we represent by the time window $\left[t_n-\tau ,t_n\right]$. Here, τ is the duration of each integration time (such as an exposure period for optical experiments).

We account for detector features in the generation of the measurements via characteristic state levels that we label with ${\mu }_{{\sigma }_k}$ and, for simplicity, gather these in $\overline{\mu }=\left({\mu }_{{\sigma }_1},{\mu }_{{\sigma }_2},\dots ,{\mu }_{{\sigma }_K}\right)$. Informally, we think of each ${\mu }_{{\sigma }_k}$ as corresponding to a distinct signal level. In this formulation, each ${\sigma }_k$ is associated with its own characteristic level ${\mu }_{{\sigma }_k}$. If the system remains at a single state ${\sigma }_k$ throughout an entire integration period $\left[t_n-\tau ,t_n\right]$, then the detector is triggered by ${\mu }_{{\sigma }_k}$ and so, provided that the measurement noise is negligible, the reported measurement $x_n$ is similar to ${\mu }_{{\sigma }_k}\tau$. However, if the system switches multiple states “during” an integration period, the detector is influenced by the levels of every state attained and the time spent in each state.

More specifically, the n^th signal level triggering the detector during the n^th integration period, $\left[t_n-\tau ,t_n\right]$, is obtained from the time average of ${\mu }_{\mathcal{T}(\cdot )}$ over this integration period. Mathematically, this time average equals $\frac{1}{\tau }{\int }_{t_n-\tau }^{t_n}dt{\mu }_{\mathcal{T}\left(t\right)}$ and, provided measurement noise is negligible, the reported measurement $x_n$ is similar to the value of this average.

In the presence of measurement noise, such as shot-noise (82, 83, 84, 85, 86), quantification noise (87, 88, 89), or other degrading effects common to detectors currently available, each measurement $x_n$ depends “stochastically” upon the signal that triggers the detector (34,90,91). Of course, the precise relationship depends on the detector employed in the experiment and differs between the various types of cameras, single-photon detectors, or other devices used. To continue with our formulation, we assume that measurement noise is additive, which results in

$$x_n|\mathcal{T}\left(\cdot \right)\sim \mathbf{N}\mathbf{o}\mathbf{r}\mathbf{m}\mathbf{a}\mathbf{l}\left(\frac{1}{\tau }{\int }_{t_n-\tau }^{t_n}dt{\mu }_{\mathcal{T}\left(t\right)},v\right).$$ (4)

The latter expression is a statistical shorthand for the following items: the measurement $x_n$ is a random variable that is sampled from a normal distribution whose mean is $\frac{1}{\tau }{\int }_{t_n-\tau }^{t_n}dt{\mu }_{\mathcal{T}\left(t\right)}$ and whose variance is v. For the normal distribution, the variance is related to the detector’s full-width-at-half-maximum (FWHM) by $v=\left({\text{FWHM}}^2/8\text{log}2\right)$.

Our Eq. 4 is general enough to capture the effect of the history of the system during the detector’s integration time or, put differently, to capture the effect of a low pass filter. Of course, our choice of normal distribution itself is incidental and can be modified depending on the type of detector used to obtain the measurements $x_n$. For example, in an accompanying article (92), we adapt Eq. 4 to FRET measurements in separate donor and acceptor channels with shot-noise and background as follows

$$x_n^D|\mathcal{T}\left(\cdot \right)\sim \mathbf{P}\mathbf{o}\mathbf{i}\mathbf{s}\mathbf{s}\mathbf{o}\mathbf{n}\left(\underset{t_n-\tau }{\overset{t_n}{\int }}dt{\mu }_{\mathcal{T}\left(t\right)}^D\right),$$

$$x_n^A|\mathcal{T}\left(\cdot \right)\sim \mathbf{P}\mathbf{o}\mathbf{i}\mathbf{s}\mathbf{s}\mathbf{o}\mathbf{n}\left(\underset{t_n-\tau }{\overset{t_n}{\int }}dt{\mu }_{\mathcal{T}\left(t\right)}^A\right),$$

where $x_n=\left(x_n^D,x_n^A\right)$ denotes the measurements acquired in the donor’s and the acceptor’s channels, respectively.

Simulation

Given $\overline{\rho }$ and $\overline{\lambda },\overline{\overline{\pi }}$, a trajectory $\mathcal{T}(\cdot )$ that mimics real systems may be simulated using the Gillespie algorithm (76), which we describe briefly here only in an effort to introduce necessary notation. Simulations by means of the Gillespie algorithm are also known as “kinetic Monte Carlo simulations.” Often, the simulated dynamics, which are characteristic of MJPs, are obtained by formulations involving “master equations.”

To begin, an initial state $s_0$ is chosen among ${\sigma }_1,{\sigma }_2,\cdots ,{\sigma }_K$ with probability ${\rho }_{{\sigma }_k}$. Then, the period $d_1$ that the system spends in $s_0$ is chosen from the exponential distribution with mean $1/{\lambda }_{s_0}$. Subsequently, the next state $s_1$ is chosen among ${\sigma }_1,{\sigma }_2,\cdots ,{\sigma }_K$ with probability ${\pi }_{s_0\to {\sigma }_k}$. Because ${\pi }_{s_0\to s_0}=0$, any chosen $s_1$ is different from $s_0$; therefore, the transition $s_0\to s_1$ is a jump in the system’s time course that occurs at time $t_0+d_1$. Next, a new period $d_2$ is sampled from an exponential distribution with mean $1/{\lambda }_{s_1}$ and a new state $s_2$ is chosen among ${\sigma }_k$ with probability ${\pi }_{s_1\to {\sigma }_k}$, and so on. These steps are repeated until the end of the experiment, which, in our setup, is the same as the time $t_N$ of the last measurement.

More formally, we summarize the sampling of a Gillespie trajectory as follows

$$s_0\sim \mathbf{C}\mathbf{a}\mathbf{t}\mathbf{e}\mathbf{g}\mathbf{o}\mathbf{r}\mathbf{i}\mathbf{c}\mathbf{a}\mathbf{l}\left(\bar{\rho }\right),$$ (5)

$$d_m|s_m\sim \mathbf{E}\mathbf{x}\mathbf{p}\mathbf{o}\mathbf{n}\mathbf{e}\mathbf{n}\mathbf{t}\mathbf{i}\mathbf{a}\mathbf{l}\left({\lambda }_{s_m}\right),$$ (6)

$$s_{m+1}|s_m\sim \mathbf{C}\mathbf{a}\mathbf{t}\mathbf{e}\mathbf{g}\mathbf{o}\mathbf{r}\mathbf{i}\mathbf{c}\mathbf{a}\mathbf{l}\left({\bar{\pi }}_{s_m}\right),$$ (7)

for $m=\mathrm{0,1,2},\dots ,M-1$, where $M-1$ is the lowest value such that

$$t_0+\sum _{m=0}^{M-1}{d}_m\ge t_N$$ (8)

The categorical distribution we use here is the generalization of the Bernoulli distribution for which more than two outcomes are possible (58).

The successive states of the system $s_0,s_1,\cdots ,s_{M-1}$ and the associated durations $d_0,d_1,d_2,\cdots ,d_{M-1}$, which we term “holding states” and “holding times,” respectively, encode $\mathcal{T}(\cdot )$ throughout the experiment’s time course $[t_0,t_N]$. Namely,

$$\mathcal{T}\left(t\right)=\{\begin{array}{ll}{s}_0\hfill & \text{if}{t}_0\le t<t_0+d_0\hfill \\ s_1\hfill & \text{if}{t}_0+d_1\le t<t_0+d_0+d_1\hfill \\ ⋮\hfill & ⋮\hfill \\ s_{M-1}\hfill & \text{if}{t}_0+d_0+\cdots +d_{M-2}\le t<t_0+d_0+d_1+\cdots +d_{M-1}\hfill \end{array}$$ (9)

For convenience, we summarize the representation of $\mathcal{T}(\cdot )$ in a triplet $\left(\overrightarrow{S},\overrightarrow{D},M\right)$, where $\overrightarrow{S}$ = {s₀, s₁, …, s_{M – 1}}, $\overrightarrow{D}$ = {d₀, d₁, …, d_{M – 1}}, and M is the size of $\overrightarrow{S}$ and $\overrightarrow{D}$.

Once a trajectory is obtained through the Gillespie algorithm just described, the signal levels ${\int }_{t_n-\tau }^{t_n}dt{\mu }_{\mathcal{T}\left(t\right)}$ for each integration period can be computed. For instance, as the trajectory is piecewise constant, the integrals reduce to sums that can be easily calculated. Therefore, given an appropriate detector model, such as Eq. 4, and a trajectory’s triplet $\left(\overrightarrow{S},\overrightarrow{D},M\right)$, we can obtain simulated measurements by adding noise according to the detector’s distribution. The graphical model, shown in Fig. 2, illustrates all of the dependencies of the parameters discussed above.

Figure 2.

Graphical representation for HMJP framework. Here, we provide a graphical representation for our HMJP framework. The notation followed in this figure is consistent with that presented in the main text. To see this figure in color, go online.

Model inference

Using experimentally obtained measurements x and the model of the experiment that we have just described, our goal is now to learn initial probabilities ${\rho }_{{\sigma }_k}$, switching rates ${\lambda }_{{\sigma }_k\to {\sigma }_{k^{\prime }}}$, and state levels ${\mu }_{{\sigma }_k}$ for all states as well as the trajectory of the system $\mathcal{T}(\cdot )$ throughout the experiment’s time course $[t_0,t_N]$. Below, we attempt to learn these model parameters by using time series analysis with an HMM and then introduce a novel, to our knowledge, time series analysis relying on HMJPs.

Model inference via HMMs

An HMM requires that each measurement $x_n$ depends exclusively on the “instantaneous” state of the system, namely $\mathcal{T}\left(t_n\right)$. In view of Eq. 4, this is achieved by the trajectory of the system $\mathcal{T}(\cdot )$ remaining constant during the integration period $\left[t_n-\tau ,t_n\right]$. To a sufficiently good approximation, this is satisfied provided

$$\tau {\lambda }_{{\sigma }_k}\ll 1,$$ (10)

for all ${\sigma }_k$. Further details for the bound in Eq. 10 are provided in Appendix A.1.9 in the Supporting Materials and Methods. Thereby, the system rarely exhibits switching during periods that last shorter than τ. This approximation allows for $\frac{1}{\tau }{\int }_{t_n-\tau }^{t_n}dt{\mu }_{\mathcal{T}\left(t\right)}\approx {\mu }_{\mathcal{T}\left(t_n\right)}$ to be used. Accordingly, in an HMM, Eq. 4 is replaced with

$$x_n|\mathcal{T}\left(\cdot \right)\sim \mathbf{N}\mathbf{o}\mathbf{r}\mathbf{m}\mathbf{a}\mathbf{l}\left({\mu }_{\mathcal{T}\left(t_n\right)},v\right).$$ (11)

Again, as with Eq. 4, the exact choice of probability distribution (whether normal or otherwise) is incidental. HMMs can treat any emission distribution “provided” $x_n$ only depends on $\mathcal{T}\left(t_n\right)$ as opposed to the full history of the trajectory over the integration time.

With the measurements described by Eq. 11, we can use an HMM to learn the probabilities of the transitions $\mathcal{T}\left(t_{n-1}\right)\to \mathcal{T}\left(t_n\right)\to \mathcal{T}\left(t_{n+1}\right)$. For clarity, from now on, we will use $c_n=\mathcal{T}\left(t_n\right)$ and denote these transitions with $c_{n-1}\to c_n\to c_{n+1}$. That is, $c_n$ is the state of the system “precisely” at the time $t_n$.

For an HMM, transition probabilities are denoted with $P_{c_{n-1}\to c_n}$. Because the system can attain K different states ${\sigma }_k$, in general, an HMM possesses $K\times K$ transition probabilities $P_{{\sigma }_k\to {\sigma }_{k^{\prime }}}$. Now, we gather the transition probabilities out of the same ${\sigma }_k$ in a vector ${\overline{P}}_k=\left(P_{{\sigma }_k\to {\sigma }_1},P_{{\sigma }_k\to {\sigma }_2},\cdots ,P_{{\sigma }_k\to {\sigma }_K}\right)$ and, for clarity, gather all of the vectors in a matrix

$$\overline{\overline{P}}=\left(\begin{array}{l}{\overline{\overline{P}}}_{{\sigma }_1}\hfill \\ {\overline{\overline{P}}}_{{\sigma }_2}\hfill \\ ⋮\hfill \\ {\overline{\overline{P}}}_{{\sigma }_K}\hfill \end{array}\right)$$ (12)

The matrix $\overline{\overline{P}}$ is related to the system’s switching rates ${\lambda }_{{\sigma }_k\to {\sigma }_{k^{\prime }}}$ and escape rates ${\lambda }_{{\sigma }_k}$. Specifically, if we gather them in

$$\overline{\overline{G}}=\left[\begin{array}{llll}-{\lambda }_{{\sigma }_1}\hfill & {\lambda }_{{\sigma }_1\to {\sigma }_2}\hfill & \dots \hfill & {\lambda }_{{\sigma }_1\to {\sigma }_K}\hfill \\ {\lambda }_{{\sigma }_2\to {\sigma }_1}\hfill & -{\lambda }_{{\sigma }_2}\hfill & \dots \hfill & {\lambda }_{{\sigma }_2\to {\sigma }_K}\hfill \\ ⋮\hfill & ⋮\hfill & \ddots \hfill & ⋮\hfill \\ {\lambda }_{{\sigma }_K\to {\sigma }_1}\hfill & {\lambda }_{{\sigma }_K\to {\sigma }_2}\hfill & \dots \hfill & -{\lambda }_{{\sigma }_K}\hfill \end{array}\right],$$ (13)

termed the “generator or rate matrix” (80,93), then the time evolution of the probability of being in conformational state ${\sigma }_k$ at time t, $p_{{\sigma }_k}\left(t\right)$, is governed by the “master equation,”

$$\frac{d{p}_{{\sigma }_k}\left(t\right)}{dt}=\sum _{k^{\prime }\ne k}{p}_{{\sigma }_{k^{\prime }}}\left(t\right){\lambda }_{{\sigma }_{k^{\prime }}\to {\sigma }_k}-p_{{\sigma }_k}\left(t\right)\sum _{k^{\prime }\ne k}{\lambda }_{{\sigma }_k\to {\sigma }_{k^{\prime }}},$$ (14)

for all $k=\mathrm{1,2},\dots ,K$. For finite conformational states, the analytical solution to the master equation” is

$$\overline{p}\left(t\right)=\overline{p}\left(t_0\right)\text{exp}\left(\overline{\overline{G}}\left(t-t_0\right)\right),$$ (15)

where

$$\overline{p}\left(t\right)=\left(\begin{array}{l}{p}_{{\sigma }_1}\left(t\right),p_{{\sigma }_2}\left(t\right),\dots ,p_{{\sigma }_K}\left(t\right)\hfill \end{array}\right)$$ (16)

Now, following Eq. 15, we have set $\overline{p}\left(t_0\right)=\overline{\rho }$, where

$$\overline{\rho }=\left({\rho }_{{\sigma }_1},{\rho }_{{\sigma }_2},\dots ,{\rho }_{{\sigma }_K}\right).$$ (17)

For the evolution over $\Delta t$, we have $\overline{p}\left(t_{n+1}\right)=\overline{p}\left(t_n\right)\overline{\overline{P}}$, where we define the transition probability matrix, $\overline{\overline{P}}$, as

$$\overline{\overline{P}}=\text{exp}\left(\overline{\overline{G}}\text{Δ}t\right),$$ (18)

where $\text{exp}(\cdot )$ denotes the matrix exponential. We point out that $\overline{\overline{\pi }}$ and $\overline{\overline{P}}$ are both probability matrices; however, they assume quite “different” properties. For instance, ${\pi }_{{\sigma }_k\to {\sigma }_k}=0$, whereas $P_{{\sigma }_k\to {\sigma }_k}>0$.

Although knowing $\overline{\overline{G}}$ is sufficient to specify $\overline{\overline{P}}$, the inverse is “not” true: knowing $\overline{\overline{P}}$ does not necessarily lead us to a unique $\overline{\overline{G}}$ and so the switching rates “cannot” simply be inferred from $\overline{\overline{P}}$. This is a consequence of the multivalued nature of the logarithm. As such, one transition probability matrix may corresponds to multiple rate matrices (93). Instead, provided ${\lambda }_{{\sigma }_k}\Delta t\ll 1$ for all ${\sigma }_k$, we may approximate Eq. 18 by

$$\overline{\overline{P}}\approx \overline{\overline{I}}+\overline{\overline{G}}\Delta t,$$ (19)

where $\overline{\overline{I}}$ is the identity matrix of size $K\times K$. Under this approximation, we can estimate transition rates by $\overline{\overline{G}}\approx \left(\overline{\overline{P}}-\overline{\overline{I}}\right)/\Delta t$. Otherwise, when ${\lambda }_{{\sigma }_k}\Delta t\ll 1$ for some ${\sigma }_k$ does not hold, then the transition rates estimated with Eq. 19 are inaccurate.

Below, we highlight the steps necessary to estimate the quantities of interest in an HMM. Specifically, an HMM relies on the statistical model

$$c_0\sim \mathbf{C}\mathbf{a}\mathbf{t}\mathbf{e}\mathbf{g}\mathbf{o}\mathbf{r}\mathbf{i}\mathbf{c}\mathbf{a}\mathbf{l}\left(\bar{\rho }\right),$$ (20)

$$c_{n+1}|c_n\sim \mathbf{C}\mathbf{a}\mathbf{t}\mathbf{e}\mathbf{g}\mathbf{o}\mathbf{r}\mathbf{i}\mathbf{c}\mathbf{a}\mathbf{l}\left({\bar{P}}_{c_n}\right),$$ (21)

$$x_n|c_n\sim \mathbf{N}\mathbf{o}\mathbf{r}\mathbf{m}\mathbf{a}\mathbf{l}\left({\mu }_{c_n},v\right).$$ (22)

To model the full distribution over the quantities of interest (e.g., initial probabilities $\overline{\rho }$, transition probabilities ${\overline{P}}_{{\sigma }_k}$, state levels $\overline{\mu }$, and the trajectory of the system $\mathcal{T}(\cdot )$, which is encoded by $\overrightarrow{c}=\left(c_0,c_1,\dots ,c_N\right)$), we follow the “Bayesian paradigm” (78,94). Within this paradigm, we place prior distributions over the parameters, and we discuss the appropriate choices next.

On the transition probabilities ${\overline{P}}_{{\sigma }_k}$, we place a Dirichlet prior with concentration parameter A

$${\bar{P}}_{{\sigma }_1}\sim \mathbf{D}\mathbf{i}\mathbf{r}\mathbf{i}\mathbf{c}\mathbf{h}\mathbf{l}\mathbf{e}\mathbf{t}\left(\frac{A}{K},\frac{A}{K},\dots ,\frac{A}{K}\right),$$ (23)

$${\bar{P}}_{{\sigma }_2}\sim \mathbf{D}\mathbf{i}\mathbf{r}\mathbf{i}\mathbf{c}\mathbf{h}\mathbf{l}\mathbf{e}\mathbf{t}\left(\frac{A}{K},\frac{A}{K},\dots ,\frac{A}{K}\right),$$ (24)

$${\bar{P}}_{{\sigma }_K}\sim \mathbf{D}\mathbf{i}\mathbf{r}\mathbf{i}\mathbf{c}\mathbf{h}\mathbf{l}\mathbf{e}\mathbf{t}\left(\frac{A}{K},\frac{A}{K},\dots ,\frac{A}{K}\right),$$ (25)

which is conjugate to the categorical distribution (39,40,42,81). We consider a similar prior distribution, with concentration parameter α, also for the initial transition probability $\overline{\rho }$, namely

$$\bar{\rho }\sim \mathbf{D}\mathbf{i}\mathbf{r}\mathbf{i}\mathbf{c}\mathbf{h}\mathbf{l}\mathbf{e}\mathbf{t}\left(\frac{\alpha }{K},\frac{\alpha }{K},\dots ,\frac{\alpha }{K}\right).$$ (26)

Subsequently, we place priors on the state levels $\overline{\mu }=\left({\mu }_{{\sigma }_1},{\mu }_{{\sigma }_2},\dots ,{\mu }_{{\sigma }_K}\right)$. The prior that we choose is the conjugate normal prior

$${\mu }_{{\sigma }_k}\sim \mathbf{N}\mathbf{o}\mathbf{r}\mathbf{m}\mathbf{a}\mathbf{l}\left(H,V\right),$$ (27)

with hyperparameters H, V.

Once the choices for the priors are made, we then form the posterior distribution (35,39, 40, 41, 42, 43, 44)

$$\mathbb{P}(\overline{\rho },\overline{\overline{P}},\overline{\mu },\mathcal{T}(\cdot )|\mathbf{x})=\mathbb{P}(\overline{\rho },\overline{\overline{P}},\overline{\mu },\overrightarrow{c}|\mathbf{x}),$$ (28)

containing all unknown variables that we wish to learn. Given that the posterior above can be constructed from the likelihood that we have defined in Eq. 11 and the priors that we have defined in Eqs. 20, 21, 22, 23, and 24, Eqs. 26 and 27 can be more explicitly written as follows:

$$\mathbb{P}(\overline{\rho },\overline{\overline{P}},\overline{\mu },\overrightarrow{c}|\mathbf{x})\propto \mathbb{P}(\mathbf{x}|\overline{\rho },\overline{\overline{P}},\overline{\mu },\overrightarrow{c})\mathbb{P}\left(\overline{\rho },\overline{\overline{P}},\overline{\mu },\overrightarrow{c}\right),$$ (29)

$$\propto \mathbb{P}\left(\mathbf{x}|c_{0:N},\overline{\mu }\right)\prod _{n=0}^{N-1}\mathbb{P}\left(c_{n+1}|c_n,\overline{\overline{P}}\right)\mathbb{P}\left(c_0|\overline{\rho }\right)\mathbb{P}\left(\overline{\overline{P}}\right)\mathbb{P}\left(\overline{\rho }\right)\mathbb{P}\left(\overline{\mu }\right),$$ (30)

$$\propto \prod _{n=1}^N\mathbf{N}\mathbf{o}\mathbf{r}\mathbf{m}\mathbf{a}\mathbf{l}\left(x_n;{\mu }_{c_n},v\right)\mathbb{P}\left(c_{n+1}|c_n,\overline{\overline{P}})\mathbb{P}(c_0|\overline{\rho }\right)\mathbb{P}\left(\overline{\overline{P}}\right)\mathbb{P}\left(\overline{\rho }\right)\mathbb{P}\left(\overline{\mu }\right),$$ (31)

However, the posterior distribution in Eq. 29 does not attain an analytical form. Therefore, we develop a specialized computational scheme exploiting Markov Chain Monte Carlo (MCMC) to generate pseudorandom samples from this posterior. We explain the details of this scheme in Computational Inference.

Model inference via HMJPs

HMJP apply directly on the formulation of eq:measure and, unlike with HMM (see Eq. 10), no approximations are required on the system-switching kinetics. Therefore, to proceed with inference, we need only provide appropriate prior distributions on the parameters, namely $\overline{\rho },\overline{\overline{\pi }},\overline{\lambda },\overline{\mu }$.

We start with the prior distribution for the escape rates $\overline{\lambda }=\left({\lambda }_{{\sigma }_1},{\lambda }_{{\sigma }_2},\dots ,{\lambda }_{{\sigma }_K}\right)$. We put priors on each of the ${\lambda }_{{\sigma }_k}$ for all $k=\mathrm{1,2},\dots ,K$. The prior we select is

$${\lambda }_{{\sigma }_k}\sim \mathbf{G}\mathbf{a}\mathbf{m}\mathbf{m}\mathbf{a}\left(\eta ,\frac{b}{\eta }\right),$$ (32)

for all $k=\mathrm{1,2},\dots ,K$ with hyperparameters $\eta ,b$. We note that this prior is conjugate to the exponential distribution given in Eq. 6. Next, we place a prior on ${\overline{\pi }}_{{\sigma }_k}$ for all $k=\mathrm{1,2},\dots ,K$. For this, we place independent conjugate Dirichlet priors with concentration parameter A such that ${\pi }_{{\sigma }_k\to {\sigma }_k}=0$ holds for all $k=\mathrm{1,2},\dots ,K$

$${\bar{\pi }}_{{\sigma }_1}\sim \mathbf{D}\mathbf{i}\mathbf{r}\mathbf{i}\mathbf{c}\mathbf{h}\mathbf{l}\mathbf{e}\mathbf{t}\left(0,\frac{A}{K-1},\dots ,\frac{A}{K-1}\right),$$ (33)

$${\bar{\pi }}_{{\sigma }_2}\sim \mathbf{D}\mathbf{i}\mathbf{r}\mathbf{i}\mathbf{c}\mathbf{h}\mathbf{l}\mathbf{e}\mathbf{t}\left(\frac{A}{K-1},0,\dots ,\frac{A}{K-1}\right),$$ (34)

$${\bar{\pi }}_{{\sigma }_K}\sim \mathbf{D}\mathbf{i}\mathbf{r}\mathbf{i}\mathbf{c}\mathbf{h}\mathbf{l}\mathbf{e}\mathbf{t}\left(\frac{A}{K-1},\frac{A}{K-1},\dots ,0\right).$$ (35)

Finally, on $\overline{\rho }$ and $\overline{\mu }$, we place the same prior distributions as in Eqs. 27 and 28, respectively.

Once the choices for the priors are made, we then form the posterior distribution

$$\mathbb{P}(\overline{\rho },\overline{\overline{\pi }},\overline{\lambda },\overline{\mu },\mathcal{T}(\cdot )|\mathbf{x})=\mathbb{P}(\overline{\rho },\overline{\overline{\pi }},\overline{\lambda },\overline{\mu },\left(\overrightarrow{S},\overrightarrow{D},M\right)|\mathbf{x}),$$ (36)

containing all unknown variables that we wish to learn. This posterior also can be expanded in a way that is proportional to the product of the likelihood introduced in Eq. 4 and the priors introduced in Eqs. 5, 6, 7, 28, 27, 34, and 36. More explicitly, the form for this posterior distribution is as follows:

$$\mathbb{P}(\overline{\rho },\overline{\overline{\pi }},\overline{\lambda },\overline{\mu },\left(\overrightarrow{S},\overrightarrow{D},M\right)|\mathbf{x})\propto \mathbb{P}(\mathbf{x}|\overline{\rho },\overline{\overline{\pi }},\overline{\lambda },\overline{\mu },\left(\overrightarrow{S},\overrightarrow{D},M\right))\mathbb{P}\left(\overline{\rho },\overline{\overline{\pi }},\overline{\lambda },\overline{\mu },\left(\overrightarrow{S},\overrightarrow{D},M\right)\right),$$ (37)

$$\propto \mathbb{P}\left(\mathbf{x}|s_{0:M-1},d_{0:M-1},\overline{\mu })\stackrel{M-1}{\prod _{m=0}}\mathbb{P}(d_m|s_m,\overline{\lambda }\right)\prod _{m=0}^{M-1}\mathbb{P}\left(s_{m+1}|s_m,\overline{\overline{\pi }})\mathbb{P}(s_0|\overline{\rho }\right)\mathbb{P}\left(\overline{\overline{\pi }}\right)\mathbb{P}\left(\overline{\rho }\right)\mathbb{P}\left(\overline{\lambda }\right)\mathbb{P}\left(\overline{\mu }\right),$$ (38)

$$\propto \prod _{n=1}^N\mathbf{N}\mathbf{o}\mathbf{r}\mathbf{m}\mathbf{a}\mathbf{l}\left(x_n;\frac{1}{\tau }\sum _{m=0}^{M-1}{d}_m{\mu }_{s_m},v\right)\prod _{m=0}^{M-1}\mathbb{P}\left(d_m|s_m,\overline{\lambda }\right)\prod _{m=0}^{M-1}\mathbb{P}\left(s_{m+1}|s_m,\overline{\overline{\pi }}\right)\mathbb{P}\left(s_0|\overline{\rho }\right)\mathbb{P}\left(\overline{\overline{\pi }}\right)\mathbb{P}\left(\overline{\rho }\right)\mathbb{P}\left(\overline{\lambda }\right)\mathbb{P}\left(\overline{\mu }\right).$$

However, once more, the posterior distribution does not attain an analytical form. Therefore, we develop a specialized computational scheme exploiting MCMC.

Computational inference

We carry out the analyses, shown in the Results section, evaluating the associated posteriors with an MCMC scheme (95) relying on Gibbs sampling (35,39, 40, 41, 42, 43, 44,63). The overall sampling strategy, for either the HMM or the HMJP, is as follows:

• 1)update the trajectory $\mathcal{T}(\cdot )$, that is, $\overrightarrow{c}$ for HMM or $\left(\overrightarrow{S},\overrightarrow{D},M\right)$ for HMJP;
• 2)update the kinetics, that is,${\overline{P}}_{{\sigma }_k}$ for HMM or ${\overline{\pi }}_{{\sigma }_k}$ and ${\lambda }_{{\sigma }_k}$ for HMJP;
• 3)update the initial probabilities $\overline{\rho }$; and
• 4)update state levels $\overline{\mu }$.

We repeat these updates to obtain a large number of samples. The end result is a sampling of the posterior $\mathbb{P}\left(\overline{\rho },\overline{\overline{P}},\overline{\mu },\overrightarrow{c}|\mathbf{x}\right)$ for the HMM and $\mathbb{P}\left(\overline{\rho },\overline{\overline{\pi }},\overline{\lambda },\overline{\mu },\left(\overrightarrow{S},\overrightarrow{D},M\right)|\mathbf{x}\right)$ for the HMJP. Each of the conditional probabilities used in the Gibbs sampling scheme that we have defined above is as follows: for step one, we use $\mathbb{P}\left(\overrightarrow{c}|\mathbf{x},\overline{\rho },\overline{\overline{P}},\overline{\mu }\right)$ and $\mathbb{P}\left(\left(\overrightarrow{S},\overrightarrow{D},M\right)|\mathbf{x},\overline{\rho },\overline{\overline{\pi }},\overline{\lambda },\overline{\mu }\right)$ for the trajectory; for step two, we use $\mathbb{P}\left({\overline{P}}_k|\mathbf{x},\overrightarrow{c},\overline{\mu }\right)$ and $\mathbb{P}\left({\overline{\pi }}_k|\mathbf{x},\left(\overrightarrow{S},\overrightarrow{D},M\right),\overline{\lambda },\overline{\mu }\right)$ for the transition probabilities; for step three, we use $\mathbb{P}\left({\overline{\lambda }}_{{\sigma }_k}|\overline{\overline{\pi }},\mathbf{x},\left(\overrightarrow{S},\overrightarrow{D},M\right),\overline{\mu }\right)$ for the switching rates for all $k=\mathrm{1,2},\dots ,K$; for step four, we use $\mathbb{P}\left({\mu }_{{\sigma }_k}|{\mu }_{{\sigma }_{k^{\prime }}},\overrightarrow{c},\mathbf{x},\overline{\overline{\pi }},\overline{\rho }\right)$ and $\mathbb{P}\left({\mu }_{{\sigma }_k}|{\mu }_{{\sigma }_{k^{\prime }}},\left(\overrightarrow{S},\overrightarrow{D},M\right),\mathbf{x},\overline{\overline{\pi }},\overline{\lambda },\overline{\rho }\right)$ for the state levels for all $k,k^{\prime }=\mathrm{1,2},\dots ,K$ and $k\ne k^{\prime }$ in HMM and HMJP frameworks. The formulae for each conditional probability distribution are expanded in Appendix A.2.3 in the Supporting Materials and Methods. Both samplings can be used to estimate switching rates ${\lambda }_{{\sigma }_k\to {\sigma }_{k^{\prime }}}$ (for example, HMJP by Eq. 2 and HMM by Eq. 19).

Finally, as can be seen, Gibbs sampling for both HMM and HMJPs requires sampling of the corresponding trajectories. This is achieved by means of a forward-filtering backward-sampling algorithm in the HMM and by means of uniformization in HMJPs. The former is well known (3,58, 59, 60,96) and widely applied in biophysical applications (10,11,39, 40, 41,45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57); however, the latter has been only recently achieved (63) and, to this day, it has never been used to solve in biophysics. Because of their importance to this study, we provide a detailed description of both procedures in Supporting Materials and Methods Section A. Here, we emphasize that uniformization is only a computational tool used in Gibbs sampling scheme developed for HMJPs that has nothing to do with the HMM. In the Supporting Materials and Methods Section A.2, we also provide a thorough description of all steps and a working code through the authors’ website.

Results

To demonstrate how HMJPs work and highlight their advantages over HMMs, in this section, we use synthetic data that mimic a single-molecule experiment. Synthetic data are ideal for this purpose because they allow us to benchmark the results against the exact, readily available, “ground truth.” We obtain such data from the Gillespie algorithm described in Simulation and we explain our simulation choices below.

We focus on two data sets: one where the system exhibits slow kinetics and another where the system exhibits fast kinetics as compared with data acquisition, see Fig. 1. We provide the values for the hyperparameters in all analyses, as well as any other choices made, in Supporting Materials and Methods Section A.3. To be clear, we only assume to have access to the data, i.e., the gray dashes of Fig. 1, a1 and b1. The cyan (ground truth) trajectories are assumed unknown and to be determined.

In our results, we first benchmark the HMJP on the easy (i.e., slow kinetics) case shown in Fig. 1 a1; see Fig. 3 and 4. This is the regime where the HMM also works well and the expected (good) results for the HMM are relegated to the appendix; see Supporting Materials and Methods Section A.1. Next, we turn to the more complex case of fast kinetics. A sample time trace is shown in Fig. 1 b1. The results for both the HMJP and HMM are shown in Figs. 5 and 6. Afterwards, in Figs. 7 and 8, we demonstrate the effect of proportion of integration period to the total data acquisition period, called the “duty cycle,” on the HMJP’s performance for estimating state levels (see Fig. 7) and switching kinetics (see Fig. 8) in the case of fast kinetics provided in Fig. 1 b1. Further results demonstrating the effect of the duty cycle size on the HMJP for faster kinetics are provided in Appendix A.1.1 in the Supporting Materials and Methods. in addition, we present the performance of the HMJP for various detector FWHMs in estimating state levels and switching rates for fast kinetics in Appendix A.1.2 in the Supporting Materials and Methods.

Figure 3.

HMJP trajectory estimates for slow state switching. Here, we provide trajectory estimates obtained with the HMJP when the switching rate is slower than the data acquisition rate, $1/\Delta t=10$ (1/s). In this figure’s (a1), the measurements are shown as gray rectangles (the width of the rectangle coincides with the integration period as shown in Fig. 1) generated based on the description provided in Model Description. We superposed the true trajectory (cyan) with the measurements in (a). Next, in (a2), we provide the histogram of all measurements to visualize the system kinetics. For illustrative purposes, we only show the MAP estimates of the HMJP on a zoomed-in region of (a1). Next, we provide that region of the (a1) in (b). In (b), we show the the MAP trajectory estimates of HMJP (magenta) that are superposed with the measurements and the true trajectory (cyan). For visual purposes only, we offset the HMJP MAP trajectory estimate by slightly shifting it downward. We observe that the HMJP MAP trajectory is able to capture switching occurring roughly in the middle of the integration time. This is not something that the HMM can capture. Here, simulated measurements are generated with ${\lambda }_{{\sigma }_1\to {\sigma }_2},{\lambda }_{{\sigma }_2\to {\sigma }_1}$Eq. 40 where the data acquisition happens at every $\Delta t=0.1$ s with ${\tau }_f=0.8$ s and $\tau =0.09$ s starting at $t_0=0.05$ s until $t_N=20$ s. To see this figure in color, go online.

Figure 4.

HMJP state level and rate estimates for slow state switching. Here, we provide posterior state level and rate estimates obtained with HMJP whose time trace we discussed in Fig. 3. We expect HMJPs to perform well in estimating the true state levels and rates when these rates are slower than the data acquisition rate. In all figure panels, we superposed the posterior distributions over state levels and rates for HMJP (blue) along with their 95% confidence intervals, the true state levels (dashed green lines) and the corresponding prior distributions (magenta lines). Here, we emphasize that the posterior distributions over each parameter are obtained based on the Gibbs sampling scheme by drawing samples from the full posterior distribution. In all panels of this figure, we histogram the sampled values for each parameter irrespective of all other parameters. As such, we can call these posterior distributions, over each parameters, marginals. We start with the information in (a1) and (a2). We observe in these panels that the HMJP posterior distributions over state levels contain the true state levels within their 95% confidence intervals. Next, we move to the (b1) and (b2), which show the posterior distributions over the rates labeled ${\lambda }_{{\sigma }_k\to {\sigma }_{k^{\prime }}}$ for all $k,k^{\prime }=\mathrm{1,2}$. Again, the HMJP does quite well in estimating these rates as measured by the fact that the ground truth lies within the 95% confidence intervals of the posteriors. In this figure, the analyzed simulated measurements are generated with the same parameters as those provided in Fig. 3. To see this figure in color, go online.

Figure 5.

HMJP with HMM trajectory estimates for fast state switching. Here, we provide trajectory estimates obtained with HMJP and HMM when the switching rate is faster than the data acquisition rate, $1/\Delta t=10$ (1/s). We expect HMMs to perform poorly in estimating the true trajectory when switching is fast. In this figure’s (a1), the measurements are shown with gray rectangles (the width of the rectangle coincides with the integration period as shown in Fig. 1) that are generated based on the description provided in Model Description. We follow the same color scheme and layout as in Fig. 3 except for (a4), where we provide the MAP trajectory estimate provided by the HMJP as well as the HMM. In (a4), the magenta dashed line shows the HMJP MAP trajectory estimate and the blue line shows the HMM MAP trajectory estimate. For visual purposes, we offset the HMJP MAP trajectory estimate and HMM MAP trajectory estimate by shifting these downward. Here, simulated measurements are generated with ${\lambda }_{{\sigma }_1\to {\sigma }_2},{\lambda }_{{\sigma }_2\to {\sigma }_1}$ (see Eq. 40), where the data acquisition happens at every $\Delta t=0.1$ s with ${\tau }_f=1/15$ s and $\tau =0.09$ s starting at $t_0=0.05$ s until $t_N=20$ s. To see this figure in color, go online.

Figure 6.

HMJP with HMM state level and transition probability estimates for fast state switching. Here, we provide posterior state level and transition probability estimates obtained with HMJP and HMM when the switching rate is faster than the data acquisition rate, $1/\Delta t=10$ (1/s). We expect HMMs to perform poorly in estimating the true state levels and transition probabilities when the system switching is fast. In all of this figure’s panels, we superposed the posterior distributions over state levels for both HMJP (blue) and HMM (orange) along with their 95% confidence intervals, the true state levels and true transition probabilities (dashed green lines). Next, we move to transition probability estimates provided in (b1)–(b4). In these panels, we wish to test the performance of HMJPs and HMMs in estimating the transition probabilities. In this figure, each panel is corresponding to the posterior distribution of a transition probability labeled as $P_{{\sigma }_k\to {\sigma }_{k^{\prime }}}$ for all $k,k^{\prime }=\mathrm{1,2}$. Here, simulated measurements are generated with the same parameters as those provided in Fig. 5. To see this figure in color, go online.

Figure 7.

HMJP state level estimates for fast state switching with different duty cycles. Here, we present how the duty cycle affects the HMJP state level estimates for fast kinetics. The parameters for fast dynamics are the same as in Figs. 5 and 6. Here, we simulated four data sets with kinetics presented in Figs. 5 and 6 for four different specified duty cycles. These are 90, 50, 5, and 1%. For clarity, a 90% duty cycle represents $90\text{\%}$ integration period of the total data acquisition period. In the data set with 90% duty cycle, what we mean is having 0.09-s-long integration period with 0.01-s dead time period of the detector where the total data acquisition period is $\Delta t=0.1$ s. In each of this figure’s panels, we have state level histograms (blue) estimated from HMJPs superposed with their 95% confidence intervals (light blue), true state levels (dashed green lines) and prior distributions (magenta line). We emphasize that we use normal prior distributions and the mean of the distribution is set by the data. The FWHM is 0.75 au. In the top panels (a1)–(d1), we have state level estimates for the first physical state with the HMJP. The bottom panels, (a2)–(d2), illustrate the state level estimates of the HMJP for the second physical state. Here, we observe that shorter integration periods lead to sharper state level estimates. To see this figure in color, go online.

Figure 8.

HMJP switching rate estimates for fast state switching with different duty cycles. Here, we show how the duty cycle affects the HMJP switching rate estimates for fast kinetics. The parameters for fast dynamics are the same as in Figs. 5 and 6. In this figure, we have the same color pattern in each panel for the estimated quantities as presented in Fig. 7. Also, we consider the same duty cycles as in Fig. 7. In the top panels, (a1)–(d1), we have HMJP switching rate estimates for the first physical state that is ${\lambda }_{{\sigma }_1}$. In the bottom panels, (a2)–(d2), we show the HMJP ${\lambda }_{{\sigma }_2}$ estimates. In each panel, we have HMJP switching rate estimates superposed with their 95% confidence intervals and their true values. Here, we demonstrate that longer integration periods give rise to sharper switching rate estimates, namely the uncertainty decreases as the integration period approaches the total data acquisition period. To see this figure in color, go online.

Data simulation

To simulate the synthetic data, we assumed $K=2$ distinct states, such as on/off or folded/unfolded states for illustrative purposes only. We assumed well-separated state levels, which we set at ${\mu }_{{\sigma }_1}=1$ au and ${\mu }_{{\sigma }_2}=7$ au where “au” denote arbitrary units. The prescribed detector FWHM was set at 0.75 au.

Additionally, for sake of concreteness only, we assumed an acquisition period of $\Delta t=0.1$ s and consider long integration periods by setting τ equal to $90\text{\%}$ of $\Delta t$. In terms familiar to microscopists, our setting corresponds to a frame rate of 10 Hz with exposure time of 90 ms and a dead time of 10 ms (41). The onset and concluding time of the experiment are the same for all simulated measurements and set at $t_0=0.05$ s and $t_N=20$ s, respectively.

To specify kinetics, we use the following structure for the switching rates ${\lambda }_{{\sigma }_1\to {\sigma }_2},{\lambda }_{{\sigma }_2\to {\sigma }_1}$, with a parameter ${\tau }_f$ which sets the timescale of the system kinetics,

$${\lambda }_{{\sigma }_1\to {\sigma }_2}=\frac{1.1}{{\tau }_f},{\lambda }_{{\sigma }_2\to {\sigma }_1}=\frac{1.6}{{\tau }_f}.$$ (40)

We simulate a case with ${\tau }_f=0.8$ s, which involves system kinetics that are slower than the data acquisition rate and a case with ${\tau }_f=0.067$ s, which involves system kinetics that are faster than the data acquisition rate.

Analysis with HMJPs

As a benchmark, we provide the results for the HMJP for those measurements shown on Fig. 1 a1 associated with slow switching rates. These results include estimates of the trajectory $\left(\overrightarrow{S},\overrightarrow{D},M\right)$ (see Fig. 3), state levels $\overline{\mu }$ (see Fig. 4), and the switching rates $\overline{\lambda }$ (see Fig. 4). To obtain these estimates, we generate samples from the posterior distribution $\mathbb{P}\left(\overline{\rho },\overline{\overline{\pi }},\overline{\lambda },\overline{\mu },\left(\overrightarrow{S},\overrightarrow{D},M\right)|\mathbf{x}\right)$ with the HMJP sampler of Computational Inference.

In Fig. 3 a1, the ground truth trajectory is shown in cyan, whereas the measurements are shown in gray. We showed the zoomed trajectory and observations in Fig. 3 b. We also provide the empirical histogram of the observation in Fig. 3 a2, highlighting the slow switching rates of the system. After determining the posteriors over the trajectories with HMJPs, for illustrative purposes, we only show the maximal a posteriori (MAP) trajectory in Fig. 3 b. We observe that the HMJP MAP trajectory (magenta) captures most of the fast switches, shown in Fig. 3 b, in the system trajectory. In Fig. 4, there are four panels. In these four panels, we provide the superposed posterior distributions over the two state levels and two rates estimated by the HMJP along with its associated $95\text{\%}$ confidence interval (sometimes called a credible interval in Bayesian analysis) and ground truths (dashed green lines).

In summary, HMJP performs well on this benchmark data. The same is true of the simpler HMM (as would be expected) whose results are shown in Figs. S7 and S8. An important bring-home message for the HMJP, however, is the fact that even if state transitions occur midway through the integration time, the HMJP can discern when these occurred. The same is not true of the HMM that, as mentioned earlier, assumes by construction that state transitions must occur at the end of the data acquisition period.

Comparison of HMJPs with HMMs

We now present a comparison of HMJPs and HMMs on the analysis of the simulated measurements shown in Fig. 1 b1. We expect the HMJP to outperform the HMM as we are now operating in a regime, with switching rates 2.5 times faster than earlier, where the HMM requirement spelled out in Eq. 10 breaks down.

We used these measurements to estimate the posterior distribution over the trajectory $\mathcal{T}(\cdot )$, initial and switching probabilities $\overline{\rho }$ and $\overline{\overline{\pi }}$ or $\overline{\overline{P}}$, state levels $\overline{\mu }$, and escape rates, $\overline{\lambda }$. To accomplish this, we generate samples from the posterior distributions $\mathbb{P}\left(\overline{\rho },\overline{\overline{\pi }},\overline{\lambda },\overline{\mu },\left(\overrightarrow{S},\overrightarrow{D},M\right)|\mathbf{x}\right)$ and $\mathbb{P}\left(\overline{\rho },\overline{\overline{P}},\overline{\mu },\overrightarrow{c}|\mathbf{x}\right)$ using the HMJP and HMM samplers of Computational Inference, respectively.

Following the pattern from the previous section on slow kinetics, we first show the trajectories inferred by HMJPs and HMMs in Fig. 5, then we show estimates of the state levels and transition probabilities in Fig. 6. For clarity, to compare apples to apples, as HMMs infer transition probabilities but not rates, we compared the transition probabilities computed from the rates obtained by HMJPs to the transition probabilities inferred by HMMs. Here, we emphasize that we obtain a unique transition probability matrix from the rate matrix; however, because of the multivalued nature of the logarithm function, the rate matrix cannot be “uniquely” inferred from the transition probability matrix.

In particular, the escape rates estimated from HMJPs are used in Eq. 19 to yield transition probabilities that we subsequently compared with the transition probabilities inferred by HMMs.

Predictably, the HMM performs poorly. For example, we see in Fig. 5 a4 that the HMJP MAP trajectory captures many of the fast switches occurring during integration times. The HMM MAP trajectory is severely constrained to allowing switches at the end of the time period and, as such, cannot accommodate fast kinetics. Although the trajectory inferred by HMJPs is not perfect, this ability to tease out many correct state switches in its MAP trajectory is sufficient for HMJPs to obtain estimates of the transition probabilities and state levels that lie within the 95% confidence interval; see panels Fig. 6, a1, a2, and b1–b4. The same does not hold for HMMs where their inability to detect state switches now percolates down to the quality of their estimates for the state levels and transition probabilities. To wit, from panels Fig. 6, a1 and a2 we see that the HMM grossly overestimates (by about 90%) ${\mu }_{{\sigma }_1}$ and underestimates (about 30%) ${\mu }_{{\sigma }_2}$. What is more, as can be seen in Fig. 6, b1–b4, the HMM provides very wide posterior distributions over transition probabilities. This is in contrast to the much sharper posterior of the HMJP whose mode closely coincides with the ground truth; see Fig. 6, b1–b4.

An observation is warranted here. Because the HMM cannot accommodate fast kinetics, it must ascribe the apparent spread around the $P_{{\sigma }_k\to {\sigma }_{k^{\prime }}}$ histogram (see Fig. 6, b1–b4) to an increased variance in the posterior distribution of transition probabilities. So, although the breadth of the posteriors of the HMJP are primarily ascribed to the fact that finite data inform the posterior, the origin of the breadth of the histogram of the HMM is an artifact of its inability to accommodate fast kinetics.

Later, in Figs. 7 and 8, we simulated three more data sets with fast dynamics as in Eq. 40 with ${\tau }_f=0.067$ s with duty cycles that are 50, 5, and 1% with integration period $\Delta t=0.1$ s. We emphasize that in Fig. 4, we presented the 90% duty cycle case. In Figs. 7 and 8, we show the posterior estimates for the HMJP state levels (see Fig. 7) and switching rates (see Fig. 8). In these figures, we see that the HMJP posterior estimates are centered around the ground truth values for both state levels and switching rates. However, posterior distributions for state level estimates of HMJP get narrower as in the case of HMM for smaller duty cycles (see Fig. 7, a1–d1 and a2–d2). Namely, because of the measurement model, as the duty cycle gets shorter and the region over which kinetics can be learned (the integration time) of the HMJP concomitantly shrinks. This smaller integration time results in narrower state level posterior estimates. In addition, the posterior distributions for switching rate estimates of HMJP get wider for shorter duty cycles (see Fig. 8, a1–d1 and a2–d2). This result can be attributed to the fact that as the duty cycle gets shorter then the integration period does not provide extra information about the kinetics of the physical system. This leads a highly varying time grid for the dynamics of the physical system. Therefore, the HMJP ends up providing wider posterior-switching rate estimates. In summary, longer duty cycle gives rise to wider posterior state level estimates and narrower posterior-switching rate estimates of HMJP.

This agreement is not surprising. For instance, as $\tau \to 0$, we find that $\frac{1}{\tau }{\int }_{t_n-\tau }^{t_n}dt{\mu }_{\mathcal{T}\left(t\right)}\approx {\mu }_{\mathcal{T}\left(t_n\right)}={\mu }_{c_n}$ and so Eq. 4, used in the HMJP, reduces to Eq. 11, used in the HMM. This provides the analytical proof that the HMJP “measurement model” simplifies to the measurement model in HMM framework for very small integration time τ.

In the appendix, we first probe the effect of duty cycles (90, 50, 5, and 1% where the integration period is $\Delta t=0.1$ s) on the posterior state level and switching rate estimates of the HMJP for fast kinetics set by Eq. 40 with ${\tau }_f=0.04$ s (see Figs. S1 and S2). We see that the HMJP provides poor posterior-switching rate estimates as the duty cycle decreases in terms of the increased width of the switching rate posterior distributions. However, HMJP’s posterior-switching rate and state level estimates are still centered around the ground truths unlike what HMM can do, as presented in Fig. 6. That is attributed to the random time grid used in HMJP framework unlike what is inherited from the HMM.

We subsequently investigated the model selection problem in Figs. S5 and S6 by analyzing two state fast dynamics with HMJP as though we had a three-state system. We find that our framework can identify the redundant state. This is observed based on the middle histogram of the diagonal panels of Fig. S6 in which we see that the final posterior distribution revert to the prior distribution. This tells us that the data do not warrant a third state. Next, we looked into the effect of measurement noise on the HMJP posterior state level and -switching rate estimates by analyzing three more data sets with different detector FWHM values (0.75, 1.1, and 1.3 au). We found that the posterior distribution widths for state level and switching rates increased though the HMJP estimates remain robust with respect to various FWHM values.

Afterwards, we revisited finer details of the effect of finiteness of data on the HMM posterior distributions over transition probabilities for fast switching rates in Supporting Materials and Methods Section A; see Fig. S16. In particular, we analyzed a sequence of three data sets using the HMM framework with the same fast switching rates as in Fig. 5 a1 but with differing data set lengths. The amount of data in Fig. S17 beyond∼400 data points (where the data acquisition period is $\Delta t=0.1$ s) seems to have a limited effect on the posterior distribution over state levels. Further analysis on the performance of both the HMJP and the HMM is relegated to the Supporting Materials and Methods Section A, in particular, cases in which the rate from one state to another is fast and the other is slow. We also provide HMJP rate estimates in Fig. S15, a1 and a2 for the data set given in Fig. 1 b1 as well as a comparison of the HMJP posterior transition probability estimates associated with the data provided in Fig. 1 a1 with and without learning the trajectory $\mathcal{T}(\cdot )$ simultaneously in Fig. S18. Finally, we compare the posterior trajectory estimates of the HMJP and the HMM based on a metric that is the enclosed area under the learned trajectories in Fig. S19.

Discussion

HMMs have been a hallmark of time series analysis in single-molecule biophysics (10,11,39, 40, 41,45,46,49, 50, 51, 52, 53, 54, 55, 56, 57), but they have a critical limitation: HMMs apply only provided the temporal resolution of the experimental apparatus is faster than the system kinetics under study (32,97, 98, 99). Otherwise, HMMs mistakenly ascribe the signal generated by fast dynamics to misassignments of signal levels. Fundamentally, this limitation arises because HMM detection models link the measurements with the “instantaneous” state of the system that dynamically evolves. Of course, the HMM framework holds provided that measurements are not obtained with an integrative detector as in the case of stroboscopic fluorescence microcopy experiments (100,101) and fast shuttering systems (102).

By contrast, the HMJP we describe here can deal with rapid dynamics and integrative detectors. This is because the HMJP is the continuous-time analog of the HMM. As compared to the HMM, the main novelty underlying the HMJP framework is primarily in the emission model, which accounts for realistic detectors operating in integrative rather than counting mode. Such detectors are common to modern biophysical experiments (35,103, 104, 105).

Other methods have also attempted to tackle the challenge presented by fast dynamics. One such example is the ${\text{H}}^2\text{MM}$, although it tackles data derived from a different type of experiment as the HMJP. In particular, the ${\text{H}}^2\text{MM}$ assumes the data are available as single-photon trajectories while we focus on the fundamental challenge of unraveling processes on timescales faster than those of detectors with finite exposure time.

We briefly highlight two more examples, although these differ from ours in that they hold for very specific cases. For example, (20) provides a method to extract fast kinetics obscured by the detector integration time. However, the analysis differs from what we introduce in our method in two ways: 1) (20) introduces a method to deal with physical systems that only have two conformational states; therefore, the physical system either populates one or the other conformational state, and 2) the method in (20) operates within the Bayesian framework; however, it requires the marginalization of fractional occupancies of the single molecule. Thereby, the framework presented in (20) does not provide a posterior distribution over the entire system trajectory that can be simultaneously estimated alongside the kinetic rates and model parameters. On the other hand, with HMJPs, we can provide a recipe to sample from a full joint posterior over all unknowns simultaneously and self-consistently following Bayesian paradigm. These unknowns include kinetic parameters, model parameters as well as the single-molecule trajectories modeled as MJPs.

As a second and final example, Lee (10) demonstrates a way to address the problem of extracting kinetic information when there is asynchronous switching (namely the conformational state of the physical state does not change at the same time as the time of data acquisition, instead it happens during the integration period) from one conformational state to the other or alternatively when one conformational state is short lived. Here, the main objective was not to extract kinetics faster than the data acquisition rate (and for this reason HMMs were employed therein). Rather, the main goal was to improve the estimation of switching rates based on transition probabilities within HMM framework. As such, their methodology relied on fitting mixtures of Gaussians until fits to FRET efficiency histograms satisfied a predetermined optimality criterion.

The HMJP does have limitations. In the limit that state switching rate grows, the amount of data needed to ascertain a meaningful posterior over the transition kinetics also grows. In the trivial limit that the state switching is extremely fast, no method, whether HMJP or otherwise, would be able to tease out information on the transition kinetics from what appears as a noisy but otherwise horizontal time trace with no discernible transitions. Although, the quality of the data is not a fundamental limitation for the HMJP, it is clear that the duration of the detector dead time affects the performance of all methods of inference. Specifically, the longer the dead time, the worse the HMJP will perform. In particular, when the dead time is as long as the exposure itself, the HMJP reduces to the HMM. A deeper question relates to whether, at fast enough timescales, it even makes sense to speak of discrete states and whether we should be speaking of continuous space and time. At the moment, these questions lie beyond the scope of this study.

Of equal interest, within the discrete state space paradigm, is the possibility to learn the number of states within an HMJP framework. That is, to repitch the HMJP within a Bayesian nonparametric paradigm following the footsteps of the HMM and its nonparametric realization, the infinite HMM (38, 39, 40, 41,106, 107, 108). Methods have been developed to report on point statistics as they pertain to infinite MJPs (70,109). A natural extension for us would be to propose a way to construct and sample from a joint posterior over all unknowns already discussed in this study as well as the state number.

Author Contributions

Z.K. analyzed data and developed analysis tools. Z.K. and I.S. developed computational tools. Z.K., I.S., and S.P. conceived research. S.P. oversaw all aspects of the projects.

Editor: Anatoly Kolomeisky.