An Introduction to Infinite HMMs for Single-Molecule Data Analysis

Abstract

The hidden Markov model (HMM) has been a workhorse of single-molecule data analysis and is now commonly used as a stand-alone tool in time series analysis or in conjunction with other analysis methods such as tracking. Here, we provide a conceptual introduction to an important generalization of the HMM, which is poised to have a deep impact across the field of biophysics: the infinite HMM (iHMM). As a modeling tool, iHMMs can analyze sequential data without a priori setting a specific number of states as required for the traditional (finite) HMM. Although the current literature on the iHMM is primarily intended for audiences in statistics, the idea is powerful and the iHMM’s breadth in applicability outside machine learning and data science warrants a careful exposition. Here, we explain the key ideas underlying the iHMM, with a special emphasis on implementation, and provide a description of a code we are making freely available. In a companion article, we provide an important extension of the iHMM to accommodate complications such as drift.

Main Text

Hidden Markov models (HMMs) (1, 2) provide a method for analyzing sequential time series data and have been a workhorse across fields including biology (3, 4), physics (5, 6, 7), and engineering (8, 9, 10).

The power of HMMs has been heavily exploited in biophysics (11, 12) in the interpretation of single-molecule experiments such as fluorescence resonance energy transfer (FRET) (13), force spectroscopy (14), atomic force microscopy (15), and ion-channel patch clamp (16). The data from these disparate techniques can be analyzed using HMMs because biomolecules or collections of biomolecules are seen as visiting discrete states and the signal emitted from each state is corrupted by noise (see Fig. 1).

Figure 1.

A synthetic time trace illustrating measurements of a hypothetical biomolecule that undergoes conformational transitions. (Left) The state space consists of conformations depicted discretely as ${\sigma }_1,{\sigma }_2,\dots$. (Middle) Time series of noisy observations, $x_n$, produced by the biomolecule (blue) and the corresponding noiseless trace (red). Over the time course of the measurements, the biomolecule attains only conformations ${\sigma }_1$–${\sigma }_5$, though additional conformations might be visited at subsequent times. For the sake of concreteness only, we label these states in order of appearance from 1 through 5. (Right) Binning the collected observations reveals “emission distributions,” $F_{{\sigma }_k}$, associated with each conformation. These distributions are highlighted with red lines. The centers (mean values) of the emission distributions are used to obtain the noiseless trace in the middle panel. The illustration on the left is created using data from (47) (PDB: 2N4G). To see this figure in color, go online.

Although HMMs have been hugely successful, the method has encountered a fundamental limitation, which is that the number of different states visited over the course of an experiment must be specified in advance (13). This limitation is too restrictive for complex biomolecules, where state numbers may often be difficult to assess a priori. It also presents a problem when the number of states appears to vary from trace to trace, as would be expected if some states are only rarely visited. As a work-around, problem- and user-dependent pre- or post-processing steps have therefore been suggested in the biophysical literature to winnow down the number of candidate models or to eliminate problematic traces altogether. Popular choices include model-selection tools such as information criteria (e.g., Bayesian and Akaike information criteria (11, 17)) or maximum evidence methods (e.g., (12)). However, thanks to recent mathematics—Bayesian nonparametrics, first introduced in 1973 (18)—an elegant solution is available that largely circumvents these model-selection steps.

In this perspectives article, we provide a description of the concepts and implementation of an important, and to our knowledge new, computational tool that exploits Bayesian nonparametrics: the infinite HMM (iHMM), which was heuristically described in (19) but only fully realized in 2012 (20).

It has recently been suggested that Bayesian nonparametrics are poised to have a deep impact in biophysics (21), and in this field, they have just begun to be exploited (22, 23, 24). However, there is, to our knowledge, no single resource yet available that describes the iHMM, its concepts, or its implementation, as would be required to bring the power of Bayesian nonparametrics to bear on biophysics and accelerate its inevitable widespread adoption. Indeed, although the iHMM tackles a conceptually simple problem, it relies on mathematics, whose literature is inaccessible outside a rarified community of statisticians and computer scientists.

It is for this reason that we have organized our perspective article as follows: Formulation of the iHMM describes the structure of the HMM in its finite (traditional) and infinite (nonparametric) realizations; Inference on the iHMM presents a computational algorithm to perform inference with the iHMM that we make freely available; Results shows results from sample time traces; and Perspectives discusses the potential for further applications to biophysics.

Methods

Formulation of the iHMM

Here, we introduce the HMM and its generalization, the iHMM. To facilitate the presentation, we initially describe the structure of the space within which the biomolecule evolves and subsequently formulate the dynamics. This discussion applies to both HMMs and iHMMs.

In the HMM framework, a system of interest is assumed to alternate successively between different states labeled ${\sigma }_k$, where k takes integer values from 1 to some L. We use L to denote the total number of states available to the system, no matter whether all these states are visited or some remain unvisited during the time course of the measurements. For instance, for an experiment on a single protein, the protein is the “system” and the “states” are conformations, such as open/closed conformations of an ion channel (16) or low/high-efficiency states in FRET experiments (25). For an illustration see Fig. 1 (left).

In the standard HMM, we assume that the system’s transitions are governed by Markovian dynamics (1). This means that the system jumps from a state ${\sigma }_k$ to a state ${\sigma }_j$, for example, from one FRET value to another, in a stochastic manner that depends exclusively on ${\sigma }_k$ and not any other state visited in the past. For this reason, all transitions out of state ${\sigma }_k$ are fully described by a probability vector ${\tilde{\pi }}_{{\sigma }_k}=\left({\pi }_{{\sigma }_k\to {\sigma }_1},{\pi }_{{\sigma }_k\to {\sigma }_2},\dots \right)$, where ${\pi }_{{\sigma }_k\to {\sigma }_j}$ is the probability of departing from state ${\sigma }_k$ and arriving at ${\sigma }_j$.

A note on our notation is appropriate here. Throughout this survey we adopt the tilde to denote vectors with components over $\mathbb{S}$, the system’s state space $\left\{{\sigma }_1,{\sigma }_2,\dots \right\}$, such as ${\tilde{\pi }}_{{\sigma }_k}$. Shortly, we will adopt bars to denote vectors with components over time.

Once the system reaches a state ${\sigma }_k$, observations are emitted stochastically according to a probability distribution unique to ${\sigma }_k$ (see Fig. 1, right). We call this the “emission distribution,” $F_{{\sigma }_k}$. It is often practical to model the emission distributions by a general family and use state-specific parameters, ${\varphi }_{{\sigma }_k}$, to distinguish its members. For example, to model Gaussian emissions x, as in Fig. 1, we may choose

$$F_{{\sigma }_k}\left(x\right)=\frac{1}{{\sigma }_{{\sigma }_k}\sqrt{2\pi }}\text{exp}\left(-\frac{{\left(x-{\mu }_{{\sigma }_k}\right)}^2}{2{\sigma }_{{\sigma }_k}^2}\right),$$ (1)

where ${\varphi }_{{\sigma }_k}=\left({\mu }_{{\sigma }_k},{\sigma }_{{\sigma }_k}\right)$ stands for the mean, ${\mu }_{{\sigma }_k}$, and standard deviation (width), ${\sigma }_{{\sigma }_k}$, of the observations produced by state ${\sigma }_k$. Therefore, for example, we define ${\varphi }_{{\sigma }_1}=\left({\mu }_{{\sigma }_1},{\sigma }_{{\sigma }_1}\right)$, ${\varphi }_{{\sigma }_2}=\left({\mu }_{{\sigma }_2},{\sigma }_{{\sigma }_2}\right)$, etc., where $({\mu }_{{\sigma }_1}$, ${\mu }_{{\sigma }_2},\dots )$ are the state FRET efficiencies and $({\sigma }_{{\sigma }_2}$, ${\sigma }_{{\sigma }_2}$, …) are the corresponding spreads attributed to noise.

Next, we let $s_n$ denote the state of the system at the nth time step of the experiment and $x_n$ its corresponding observation. Thus, we label with n the states of the system as it evolves through time, forming a sequence $s_1\to s_2\to \dots \to s_N$, with each $s_n$ being equal to some state ${\sigma }_k$ chosen from $\mathbb{S}$. For completeness, we can also assume that, before the first measurement, the system is at a default state, which we denote as $s_0$. Again, we mention that $\mathbb{S}$ might contain states that do not appear in this sequence, i.e., states that remain unvisited throughout the experimental time course.

As is common in the statistical literature—and especially since we will borrow from this notation to introduce the iHMM—we express the HMM compactly using the scheme

$$s_n|s_{n-1}\sim Ca{t}_{\mathbb{S}}\left({\tilde{\pi }}_{s_{n-1}}\right),$$ (2)

$$x_n|s_n\sim F_{s_n},$$ (3)

and depict it schematically in Fig. 2. The ∼ symbol used above indicates that the random variables on the left side of the scheme are “distributed according to”, and therefore sampled from, the probability distributions shown on the right side (26). For example, Eq. 2 denotes a sampling from a categorical probability distribution that is supported on $\mathbb{S}$, i.e., $s_n$ equals a state ${\sigma }_k$ that is taken from $\mathbb{S}$ with probability ${\pi }_{s_{n-1}\to {\sigma }_k}$.

Figure 2.

Graphical representation of the HMM. In the HMM, a biomolecule of interest transitions between unobserved states $s_n$ according to the probability vectors ${\tilde{\pi }}_{s_n}$ and generates observations $x_n$ according to the probability distributions $F_{s_n}$ that depend on the parameter ${\varphi }_{s_n}$. Here, following convention, the $x_n$ values are shaded to denote that these quantities are observed, whereas the $s_n$ values are hidden. Arrows denote the dependences among the model variables and red lines denote the model parameters. To see this figure in color, go online.

As can be seen from Eqs. 2 and 3, the HMM models the experimental output in a doubly stochastic manner: 1) the state of the system $s_n$ evolves stochastically within $\mathbb{S}$, as expressed by Eq. 2; and 2) the observations $x_n$ from each $s_n$ are also emitted stochastically, as expressed by Eq. 3. In particular, for single-molecule experiments, these two characteristics allow the HMM to elegantly capture 1) the seemingly random biomolecular state switching; and 2) the noise corrupting the measurements (13, 27).

From the experimental measurements, we gain access only to the observations $x_n$, which take the form of a time series, $\overline{x}=\left(x_1,x_2,\dots ,x_N\right)$, over some regularly spaced time intervals $t_n$. In the most general case, the goals of the HMM are to estimate 1) the underlying state sequence, $\overline{s}=\left(s_1,s_2,\dots ,s_N\right)$, which is unobserved during the measurements; and 2) the model parameters, which include ${\varphi }_{{\sigma }_k}$ of the emission distributions and the state transition probabilities, ${\pi }_{{\sigma }_k\to {\sigma }_j}$, associated with the states in $\mathbb{S}$. Of course, in practice, these goals are feasible only for those states that are visited during the experimental time course.

Normally, this is the extent of the HMM. That is, one fixes L, i.e., the size of the state space $\mathbb{S}$, writes down the likelihood of observing the sequence of observations, $\overline{x}=\left(x_1,x_2,\dots ,x_N\right)$, and maximizes this likelihood to estimate the quantities of interest. Extensive literature—that has made its way into standard textbooks, for example (26)—explains each of these steps in detail.

However, in preparation for the iHMM, we take a Bayesian route to accomplish the goals of the HMM (21). This is a computational overkill for some applications but otherwise essential to its nonparametric generalization.

Following the Bayesian paradigm (21, 28), we assign prior probabilities to model parameters including the emission parameters, ${\varphi }_{{\sigma }_k}$, and transition probability vectors, ${\tilde{\pi }}_{{\sigma }_k}$. For instance, we may assume a prior ${\varphi }_{{\sigma }_k}\sim H$ given by a common distribution for all states. However, assigning a prior ${\tilde{\pi }}_{{\sigma }_k}$ is more subtle, as any choice must ensure that the predicted transitions stay within the system’s state space, $\mathbb{S}$. It is specifically the formulation of this prior that fundamentally distinguishes the finite from the infinite variants of the HMM (20), both of which we describe next.

In the finite variant of the HMM (1, 2), before setting the prior on ${\tilde{\pi }}_{{\sigma }_k}$, we must fix L, the total number of states in $\mathbb{S}$, or, in the language of single-molecule biophysics, conformations available to the biomolecule. Once L is fixed, the symmetric Dirichlet distribution is a common choice:

$${\tilde{\pi }}_{{\sigma }_k}\sim Di{r}_{\mathbb{S}}\left(\frac{\alpha }{L},\dots ,\frac{\alpha }{L}\right).$$ (4)

Here, $Di{r}_{\mathbb{S}}\left(\alpha /L,\dots ,\alpha /L\right)$ denotes the Dirichlet distribution supported in $\mathbb{S}$ with concentration parameter α (for an explicit definition of the Dirichlet distribution, see the Supporting Material). Basically, this prior asserts that lacking any information on the system’s kinetics, the model is likely to place on average equal probability on any possible transition ${\sigma }_k\to {\sigma }_j$. In this prior, the value of α can be used to influence the transitions departing each state. For example, with $\alpha \gg 1$, the prior tends to favor uniform transitions, whereas with $\alpha \ll 1$, the prior tends to favor sparse transitions. In other words, the resulting sequence, $\overline{s}$, tends to contain several different states for $\alpha \gg 1$, whereas it tends to contain fewer for $\alpha \ll 1$.

The Dirichlet prior in Eq. 4 offers two key advantages: 1) it provides a noninformative prior, since no specific transitions between the L states are preferentially selected; and 2) it combines well with (i.e., is conjugate to) the categorical distribution of Eq. 2, which greatly simplifies model parameter estimation.

Although such priors help lessen the computational burden, pre-setting L ignores the data, and thus, the arbitrary choice of L may cause under- or over-fitting, which, in turn, has far-reaching consequences in estimating state kinetics. Pre-setting L also does not allow the model’s complexity to grow in response to newly available data (e.g., a rare state visited later or in another time series). Resolving the latter issue in a principled fashion can also help avoid cherry-picking data sets that behave more closely to one’s expected or preferred value of L. It is to resolve such issues that the iHMM was developed in the first place.

Now, in the infinite HMM, the key difference is that $\mathbb{S}$ is assumed to be infinite in size (19). To avoid any confusion, we point out that this assumption is different from forcing the system to visit an infinite number of states, as it might appear at first. Regardless of the size of $\mathbb{S}$, the system visits only K different states, where K cannot exceed, for example, the number of steps, N, in the collected time series. In practice, of course, we expect K to be considerably smaller than N.

As we will see, with an infinite state space, the iHMM can recruit as many states as necessary and, in doing so, avoids underfitting by growing the state space as new states are visited (through the effect of the prior), but also avoids overfitting by placing more weight on states already visited (through the effect of the likelihood).

The generalization of the Dirichlet distribution appropriate for the infinite sized $\mathbb{S}$ is the Dirichlet process (DP),

$${\tilde{\pi }}_{{\sigma }_k}\sim D{P}_{\mathbb{S}}\left(\alpha ,\tilde{\beta }\right),$$ (5)

where $\tilde{\beta }$ is the “base distribution” of the DP, which determines how ${\tilde{\pi }}_{{\sigma }_k}$ are distributed on average. The notion of a base is only required for the infinite DP and has no direct analog in the finite Dirichlet distribution. By contrast, α is similar to the concentration of the finite HMM, since it controls the sparsity of the transition probability vectors, ${\tilde{\pi }}_{{\sigma }_k}$. For instance, large values of α make the model more likely to choose ${\tilde{\pi }}_{{\sigma }_k}$ values that are similar for all states, whereas low values of α make the model more likely to choose ${\tilde{\pi }}_{{\sigma }_k}$ values that differ considerably.

The DP is at the basis of much of nonparametric Bayesian inference. It was only in 2012 that a hierarchical DP (HDP) was invoked to construct the iHMM (20). The basic idea is to select a random—albeit discrete—base distribution $\tilde{\beta }=\left({\beta }_{{\sigma }_1},{\beta }_{{\sigma }_2},\dots \right)$ with members ${\beta }_{{\sigma }_k}$ that 1) are infinite in number, 2) attain nonnegative values, and 3) sum to 1. In the HDP, it is common to sample such a base from a Griffiths-Engen-McCloskey distribution realized through a stick-breaking construction (20, 29, 30),

$$\tilde{\beta }\sim GE{M}_{\mathbb{S}}\left(\gamma \right),$$ (6)

which ensures all these properties. Intuitively, the stick-breaking above is obtained as follows. We start with a stick of length 1. We break the stick at some location $v_1$ and use the broken fraction for the probability of visiting ${\sigma }_1$; thus, ${\beta }_{{\sigma }_1}=v_1$. Subsequently, we break the remaining stick at $v_2$ and use that new fraction for the probability of visiting ${\sigma }_2$; thus, ${\beta }_{{\sigma }_2}=v_2\left(1-{\beta }_{{\sigma }_1}\right)$, and so on. In this stick-breaking, the locations $v_k$ are sampled from a β-distribution, $\mathcal{B}\left(1,\gamma \right)$, so γ determines the relative location of the break point. Thus, γ can be used to set how rapidly ${\beta }_{{\sigma }_k}$ decreases toward zero and ultimately controls the number of states in $\mathbb{S}$ with an appreciable likelihood of being visited at least once during the experiment.

In summary, the iHMM—together with its priors illustrated in Fig. 3—takes the form

$$\tilde{\beta }\sim GE{M}_{\mathbb{S}}\left(\gamma \right)$$ (7)

$${\tilde{\pi }}_{{\sigma }_k}|\tilde{\beta }\sim D{P}_{\mathbb{S}}\left(\alpha ,\tilde{\beta }\right)$$ (8)

$${\varphi }_{{\sigma }_k}\sim H$$ (9)

$$s_n|s_{n-1},\tilde{\tilde{\pi }}\sim Ca{t}_{\mathbb{S}}\left({\tilde{\pi }}_{s_{n-1}}\right)$$ (10)

$$x_n|s_n,\tilde{\varphi }\sim F_{s_n},$$ (11)

where, for notational simplicity, we use $\tilde{\tilde{\pi }}=\left\{{\tilde{\pi }}_{{\sigma }_1},{\tilde{\pi }}_{{\sigma }_2},\dots \right\}$ and $\tilde{\varphi }=\left\{{\varphi }_{{\sigma }_1},{\varphi }_{{\sigma }_2},\dots \right\}$ to gather the transition probability vectors and emission parameters associated with all states in $\mathbb{S}$. Here, $\tilde{\tilde{\pi }}$ can be seen as a bi-infinite matrix whose elements are the probabilities for each possible transition ${\sigma }_k\to {\sigma }_j$, and it is directly analogous to the transition matrix of the finite HMM.

Figure 3.

Graphical representation of the iHMM. The hidden Markov model that formulates the observations to be analyzed (black lines) is shown together with its priors (red lines). For completeness, we also show the concentration parameters α and γ and the prior probability distribution on the emission parameters, H, that fully characterize the iHMM. The key difference from the HMM shown in Fig. 2 is that now the model parameters ${\tilde{\pi }}_{{\sigma }_k}$ and ${\varphi }_{{\sigma }_k}$ are treated as random variables similar to the hidden states, $s_n$, and observations, $x_n$. For details, see the main text. To see this figure in color, go online.

Unlike the finite HMM, where we need to be specific about the size of the state space, the iHMM gives us greater flexibility through the hyperparameters α and γ. For instance, larger concentrations are more appropriate for biomolecules with roughly similar transition rates and several conformations, whereas low concentrations are more appropriate for biomolecules with dissimilar transition rates and few conformations. In general, it is possible to sum over these concentrations if we are truly ignorant of the properties of the biomolecule at hand (20, 31).

As we will see shortly, this powerful formalism allows us to infer state numbers robustly, and even if the number of states is apparent—something that is rarely the case for more complex problems (32)—it avoids the critical shortfall of cherry-picking data sets to be analyzed that appear to have similar state numbers.

Inference on the iHMM

Before describing the analysis that iHMMs may offer, we show how to infer quantities such as the hidden state sequence and the parameters from the experimental traces. A working implementation of the algorithm described in this section, equipped with a user interface, can be found in the Supporting Materials. (This code can also be found on the authors’ web site, as well as on GitHub.)

To be clear, on account of the infinite size of $\mathbb{S}$, common methods used in finite-state HMMs, such as expectation maximization (1), are inapplicable. Instead, we focus our discussion on a specialized method, the beam sampler (33). Other methods are also described elsewhere (20, 31).

The beam sampler is a special instance of the Gibbs sampler (34), and although it is beyond the scope of this survey, a brief description about its usage might be beneficial here. Similar to all samplers of this family, we use the beam sampler to generate (pseudo) random sequences of the model variables we are interested in inferring. Specifically, in our case these variables consist of the hidden-state sequence $\overline{s}$ and the model parameters $\tilde{\beta },\tilde{\tilde{\pi }},\tilde{\varphi }$. When generating the sequences, we use (7), (8), (9), (10), (11) and the data $\overline{x}$. As a result, the generated sequences have the same statistical properties as if they consisted of samples from the posterior probability distribution, $\mathbb{P}\left(\overline{s},\tilde{\beta },\tilde{\tilde{\pi }},\tilde{\varphi }|\overline{x}\right)$. Thus, we may use them to compute averages, confidence intervals, and maximum a posteriori estimates or simply to produce histograms that resemble $\mathbb{P}\left(\overline{s},\tilde{\beta },\tilde{\tilde{\pi }},\tilde{\varphi }|\overline{x}\right)$, as we show in the next section.

Overall, Gibbs and related samplers provide a more general approach than, for example, expectation maximization, which provides only the maximum of $\mathbb{P}\left(\overline{s},\tilde{\beta },\tilde{\tilde{\pi }},\tilde{\varphi }|\overline{x}\right)$. Instead, with Gibbs sampling, we fully characterize $\mathbb{P}\left(\overline{s},\tilde{\beta },\tilde{\tilde{\pi }},\tilde{\varphi }|\overline{x}\right)$ over its whole domain. For an introduction to the general methodology underlying Gibbs sampling, we refer the interested reader to (35), and from now on, we focus on the iHMM.

Suppose we have a time series of experimental observations $\overline{x}$. Here, we describe the steps involved in generating samples (superscript r) of our model variables ${\overline{s}}^{\left(r\right)},{\tilde{\beta }}^{\left(r\right)},{\tilde{\tilde{\pi }}}^{\left(r\right)},{\tilde{\varphi }}^{\left(r\right)}$. Given a computed sample $\left(r-1\right)$, the naive approach for computing the next sample would be to update each of the involved variables conditional upon the other variables and the data (35). For example, we could generate ${\overline{s}}^{\left(r\right)}$ by sampling from $\mathbb{P}\left(\overline{s}|{\tilde{\beta }}^{\left(r-1\right)},{\tilde{\tilde{\pi }}}^{\left(r-1\right)},{\tilde{\varphi }}^{\left(r-1\right)},\overline{x}\right)$. Then, we could generate ${\tilde{\beta }}^{\left(r\right)}$ by sampling from $\mathbb{P}\left(\tilde{\beta }|{\overline{s}}^{\left(r\right)},{\tilde{\tilde{\pi }}}^{\left(r-1\right)},{\tilde{\varphi }}^{\left(r-1\right)},\overline{x}\right)$, and so on. Based on the theory of Markov-chain Monte Carlo sampling (see, for example, (34)), this approach is guaranteed to produce samples from the full posterior $\mathbb{P}\left(\overline{s},\tilde{\beta },\tilde{\tilde{\pi }},\tilde{\varphi }|\overline{x}\right)$, as we wish. However, the infinite size of $\mathbb{S}$ makes this approach impractical, since in each iteration we need to sample infinite-sized $\tilde{\beta },\tilde{\tilde{\pi }},\tilde{\varphi }$, which is computationally infeasible.

It is fundamentally because of this difficulty that we need the beam sampler, which introduces a set of auxiliary variables (slicers) that in each iteration effectively truncate $\mathbb{S}$ to a finite portion (33, 36). In particular, for each time step n in the data set, consider an auxiliary variable, $u_n$, and let $\overline{u}=\left(u_1,u_2,\dots ,u_N\right)$ gather all of them. Since $\overline{u}$ consists of auxiliary variables only, we have

$$\mathbb{P}\left(\overline{s},\tilde{\beta },\tilde{\tilde{\pi }},\tilde{\varphi }|\overline{x}\right)=\sum _{\overline{u}}\mathbb{P}\left(\overline{u},\overline{s},\tilde{\beta },\tilde{\tilde{\pi }},\tilde{\varphi }|\overline{x}\right),$$ (12)

where the sum on the righthand side is considered over all possible values $\overline{u}$ may take. Now, we can use the equality in Eq. 12 to sample from $\mathbb{P}\left(\overline{u},\overline{s},\tilde{\beta },\tilde{\tilde{\pi }},\tilde{\varphi }|\overline{x}\right)$ instead of from $\mathbb{P}\left(\overline{s},\tilde{\beta },\tilde{\tilde{\pi }},\tilde{\varphi }|\overline{x}\right)$. The benefit of doing so is that when updating $\tilde{\beta },\tilde{\tilde{\pi }},\tilde{\varphi }$, we also have to sample conditional upon $\overline{u}$. Thus, by properly choosing the auxiliaries $u_n$, we make these updates consider only a finite selection of the states ${\sigma }_k$. For instance, by having $u_n$ uniformly distributed over the interval $\left(0,{\pi }_{s_{n-1}\to s_n}\right)$, it is sufficient to consider only those ${\sigma }_k$ states in $\mathbb{S}$ that have a probability of being visited at least once that is >$u_n$. Therefore, we can generate the desired samples while maintaining a finite $\mathbb{S}$ that we expand and compress dynamically according to $\overline{u}$.

In summary, given ${\overline{s}}^{\left(r-1\right)}$, ${\tilde{\beta }}^{\left(r-1\right)}$, ${\tilde{\tilde{\pi }}}^{\left(r-1\right)}$, and ${\tilde{\varphi }}^{\left(r-1\right)}$ the beam sampler generates a new set of samples through the following steps:

• 1.Generate ${\overline{u}}^{\left(r\right)}$ by sampling from $\mathbb{P}\left(\overline{u}|{\overline{s}}^{\left(r-1\right)},{\tilde{\tilde{\pi }}}^{\left(r-1\right)}\right),$
• 2.Expand $\mathbb{S}$ according to ${\overline{u}}^{\left(r\right)},$
• 3.Generate ${\overline{s}}^{\left(r\right)}$ by sampling from $\mathbb{P}\left(\overline{s}|{\overline{u}}^{\left(r\right)},{\tilde{\tilde{\pi }}}^{\left(r-1\right)},{\tilde{\varphi }}^{\left(r-1\right)},\overline{x}\right),$
• 4.Compress $\mathbb{S}$ according to ${\overline{s}}^{\left(r\right)},$
• 5.Generate ${\tilde{\beta }}^{\left(r\right)}$ by sampling from $\mathbb{P}\left(\tilde{\beta }|{\tilde{\beta }}^{\left(r-1\right)},{\overline{s}}^{\left(r\right)}\right),$
• 6.Generate ${\tilde{\tilde{\pi }}}^{\left(r\right)}$ by sampling from $\mathbb{P}\left(\tilde{\tilde{\pi }}|{\overline{s}}^{\left(r\right)},{\tilde{\beta }}^{\left(r\right)}\right),$
• 7.Generate ${\tilde{\varphi }}^{\left(r\right)}$ by sampling from $\mathbb{P}\left(\tilde{\varphi }|{\overline{s}}^{\left(r\right)},\overline{x}\right),$

where, for simplicity, we have dropped additional dependencies in the conditional probability distributions above.

In principle, $\tilde{\beta }$, $\tilde{\tilde{\pi }}$, and $\tilde{\varphi }$ are infinite dimensional vectors. However, as mentioned above, the sampler requires the computation of only those components that correspond to the states $\overline{u}$ allows to be visited. Therefore, in step 4 of the above algorithm we can safely discard those components that are absent from the state sequence ${\overline{s}}^{\left(r\right)}$. By doing so, in each time step, we only need to store vectors of finite dimension, ${\tilde{\beta }}^{\left(r\right)}$, ${\tilde{\tilde{\pi }}}^{\left(r\right)}$, and ${\tilde{\varphi }}^{\left(r\right)}$. Subsequently, in step 2 of the following iteration, we can supplement ${\tilde{\beta }}^{\left(r\right)}$, ${\tilde{\tilde{\pi }}}^{\left(r\right)}$, and ${\tilde{\varphi }}^{\left(r\right)}$ with states as necessary. The discarded states, since they are unvisited, do not produce any of the observations in $\overline{x}$. Thus, they do not provide any information that could be lost. Similarly, the supplemented states are also not associated with any of the observations in $\overline{x}$. Thus, when we generate them, we simply have to draw samples from their priors. Consequently, the supplemented states neither require nor introduce extra information, as might first appear to be the case. In summary, expanding and compressing $\mathbb{S}$ in steps 2 and 4 leaves our inference unaffected.

The Supporting Material provides a pseudocode implementation of the algorithm with further details regarding each of the sampler’s steps for the interested reader.

Results

In this section, we demonstrate the use of the iHMM in the analysis of time-series data. To benchmark the method, we use synthetic time series where the “ground truth” is known. In a companion research article that draws heavily from the formulation described in this article, we show the performance of the method on actual time traces that exhibit further complications, such as drift (37). For such time traces, we need an additional important generalization to the iHMM: the iHMM must be coupled to a continuous process. For now, our synthetic traces are meant to illustrate realistic data with noise, but from which drift is otherwise absent or minimal.

Below, we present the results of the analysis on two types of time trace that are specifically problematic for the HMM: 1) data set 1, consisting of time traces where the visited states are in high number and their emissions show significant overlap; and 2) data set 2, consisting of time traces where some states are rarely visited and the number of states appears to change as more data become available.

Details about the generation of these data sets, as well as more in-depth analyses than those discussed here, can be found in the Supporting Material.

The data sets we analyze are shown in Fig. 4. Specifically, we used these time series to estimate the posterior distributions over 1) the number, K, of different conformations visited by the biomolecule, and 2) the parameters describing the emission distributions, such as the mean value for each conformation ${\mu }_{{\sigma }_k}$, for those states. To accomplish this, we generate samples from $\mathbb{P}\left(\overline{u},\overline{s},\tilde{\beta },\tilde{\tilde{\pi }},\tilde{\varphi }|\overline{x}\right)$ using the beam sampler described earlier.

Figure 4.

Synthetic data sets resembling a hypothetical biomolecule undergoing transitions between discrete states that we analyzed with the iHMM. (Left) Time series $\overline{x}=\left(x_1,\dots ,x_N\right)$ of noisy observations. During the measuring period, the biomolecule attains five conformations, ${\sigma }_1,\dots ,{\sigma }_5$. The number of conformations are a priori unknown and the iHMM seeks to determine the probability over the number of states, as well as their properties, given the data available. In data set 1, the biomolecule transitions often through every state. By contrast, in data set 2, transitions to some states are rare. As a result, all states in data set 1 are almost equally visited throughout the experiment time course, whereas in data set 2, higher states are visited, by chance, only toward the end of the trace. (Right) The corresponding emission distributions, $F_{{\sigma }_k}$, as obtained by simply binning the observations (blue) and plotting the exact ones used for the simulations (red). For both data sets, the emission distributions show significant overlap. In all panels, dotted lines indicate the exact mean values, ${\mu }_{{\sigma }_k}$, of the emission distributions. To see this figure in color, go online.

Fig. 5 shows how the number of different states visited in each sampled state sequence ${\overline{s}}^{\left(r\right)}$ for data set 1, which we denote with $K^{\left(r\right)}$, changes through the sampler’s iterations. Again, we mention that K counts only those states that are visited within the given trace. Initially, $K^{\left(1\right)}$ equals the number of states in the state space as specified in the initialization of our method. After ∼50 iterations, $K^{\left(r\right)}$ drops to 5, which is the correct number of states attained by our hypothetical biomolecule.

Figure 5.

After some iterations, the sampler used in the iHMM to analyze data set 1 of Fig. 4 eventually converges to the correct number of states. The number of visited states, $K^{\left(r\right)}$ (top), and the means of the emission distributions, ${\mu }_{{\sigma }_k}^{\left(r\right)}$ (bottom), change throughout the sampler’s iterations. Unlike the HMM, which uses a finite and fixed state space, the iHMM learns the number of available states and grows/shrinks the state space as required by the data.

Subsequently, $K^{\left(r\right)}$ stays at 5 and occasionally adds extra states that are rapidly eliminated. This behavior is characteristic of our algorithm (34). In the initial iterations, known as “burn-in,” the sampler randomly explores possible choices for the model variables that eventually evolve closer to the ground truth. Once they approach the ground truth, the sampler begins generating them based on the corresponding posterior distribution, $\mathbb{P}\left(K|\overline{x}\right)$. So, after burn-in, values with high $\mathbb{P}\left(K|\overline{x}\right)$ are sampled more often than those with low $\mathbb{P}\left(K|\overline{x}\right)$.

The same is true of the other variables we try to estimate. For example, in Fig. 5, we also show how the sampled means of the emission distributions, ${\mu }_{{\sigma }_k}^{\left(r\right)}$, change through the iterations. As with $K^{\left(r\right)}$, during burn-in, the sampler explores different choices that eventually converge to the ground truth. Afterward, the sampler occasionally explores new emissions, for example, around iterations 150–250, that quickly disappear.

To obtain more quantitative results, we may drop the burn-in samples from the generated sequences and use the remainder of the samples to produce histograms. For example, in Fig. 6, we show histograms of $K^{\left(r\right)}$ and ${\mu }_{{\sigma }_k}^{\left(r\right)}$. These histograms approximate the corresponding posterior probability distributions and thus provide a full characterization of the estimated variables. In these cases, the breadth of the histograms arises because the data we analyze are limited and noisy, so the estimates are associated with some uncertainty that is reflected in the broadening of the posterior distributions.

Figure 6.

We may use samples from the iHMM posterior probability to infer the size of the state space and the location of each state. In particular, we illustrate histograms for $\mathbb{P}\left(K|\overline{x}\right)$ (top) and $\mathbb{P}\left({\mu }_{{\sigma }_k}|\overline{x}\right)$ (bottom) using data set 1 of Fig. 4. In both panels, dashed lines indicate the exact (ground-truth) values used to produce the data in Fig. 4. To see this figure in color, go online.

As can be seen, data set 1—although it contains a large number of states that are not well separated—can be successfully analyzed by means of the iHMM. However, data set 2 poses another difficulty. Specifically, the biomolecule visits some states only rarely. In experiments, this would give rise to traces with an unequal number of states. One may naively suggest that longer traces should be collected, but due to experimental limitations (e.g., early photobleaching in FRET experiments), such long traces might not always be available. Instead, a large number of shorter traces, each containing an unknown portion of the complete state space, may need to be analyzed separately and the results combined (38). For this reason, it is crucial to treat all traces on an equal footing without a priori assuming a different number of states individually for each one, as the HMM would require.

To illustrate how the iHMM handles such cases, we use the iHMM to analyze the emission data of Fig. 4 twice, first using only the initial 300 time steps, where only two of the five states are visited, and subsequently using the complete trace, which contains all five states.

Fig. 7 shows the results of the analyses. Specifically, we show the estimated noiseless traces (Fig. 7, upper) and the estimated numbers of states (Fig. 7, lower). As can be seen, the iHMM successfully estimates the correct number in each trace. Further, as can be seen from the de-noised traces, the portion of the state space, i.e., states ${\sigma }_1$ and ${\sigma }_2$, that is common is estimated accurately from both traces.

Figure 7.

We may use the iHMM to estimate portions of the complete state space such as those contained in different segments of data set 2 provided in Fig. 4. (Upper) Estimated noiseless traces for two cases: 1) using a limited segment of the full trace; and 2) using the full trace. Although only the latter case allows an estimate of all five states, both cases provide similar estimates over those states that they mutually visit. (Lower) Corresponding estimates of the number of states contained in each trace. To see this figure in color, go online.

Perspectives

In this survey, we presented the iHMM (19, 20), a relatively recent development in statistics that has already found diverse applicability in, among other fields, genomics (39), genetics (40), finance (41, 42), tracking (43, 44), machine learning (45, 46), and biophysics (22, 23, 24).

As a modeling tool, the iHMM inherits the characteristics of its predecessor—the finite HMM—but offers greater flexibility in the modeling and analysis of the experimental data. An important advantage is that it does not require the underlying system to have a limited state space as would its predecessor. This feature is of particular importance to biophysics, where it is often the case that little information is available to fix the number of biomolecular states a priori.

Despite the difficulty in prespecifying the complexity of the state space, the finite HMM is a routine choice for the analysis of single-molecule time series (11, 12, 13). Nevertheless, as the experimental techniques improve and data from complex biomolecules are becoming available, the shortcomings of the finite HMM have become more apparent. To this end, methods to circumvent the limitations of HMM have been developed independently of the progress made on the same problems that have exploited Bayesian nonparametrics.

Although it has previously been suggested that single-molecule analysis may benefit from Bayesian nonparametrics (22), to date only limited applications of these methods have been explored. This is in part because models like the iHMM and its implementation are scattered over various sources, but also because of the important language barrier between fields where Bayesian nonparametrics have changed the course of data analysis and the physical sciences. We believe that it is in part for this reason that methods continue to be developed to address consequences of the finiteness of the state space of the HMM in the physical sciences.

The iHMM, described here, presents great advantages in the analysis of single-molecule observations. For one thing, it elegantly addresses those challenges that are presented by the HMM, namely, the problem of having to select a number of states when the emission distributions are unclear, but also to address the problem—currently often addressed in a user-dependent fashion—of how to proceed when different traces present a seemingly different number of states. The key idea is that, just as the HMM performs de-noising while parametrizing transition kinetics, the iHMM takes this logic one step further by additionally learning the number of states in a self-consistent manner.

Although greatly advantageous, the iHMM itself presents an important challenge for experimental data with drift, a common feature of biophysical time traces (13, 25). Indeed, without its explicit consideration, drift would be interpreted as the population of artifact states by a method designed to recruit extra states to accommodate the data. This is a key challenge that has so far not been considered anywhere. Without careful consideration, drift would render the power of the iHMM into a key disadvantage. Although our main emphasis here has been to feature the power of the iHMM to biophysics, our companion research article (37) now tackles the problem of extending iHMMs to deal with drift.

Author Contributions

I.S. and S.P. performed research; I.S. contributed analytic tools and analyzed data; I.S. and S.P. wrote the article.

Editor: Brian Salzberg.