A tiered hidden Markov model characterizes multi-scale chromatin states

Abstract

Precise characterization of chromatin states is an important but difficult task for understanding the regulatory role of chromatin. A number of computational methods have been developed with varying levels of success. However, a remaining challenge is to model epigenomic patterns over multi-scales, as each histone mark is distributed with its own characteristic length scale. We developed a tiered hidden Markov model and applied it to analyze a ChIP-seq dataset in human embryonic stem cells. We identified a two-tier structure containing 15 distinct bin-level chromatin states grouped into three domain-level states. Whereas the bin-level states capture the local variation of histone marks, the domain-level states detect large-scale variations. Compared to bin-level states, the domain-level states are more robust and coherent. We also found active regions in intergenic regions that upon closer examination were expressed non-coding RNAs and pseudogenes. These results provide insights into an additional layer of complexity in chromatin organization.

Introduction

In a multi-cellular organism, virtually all the cells share the same genome, but each cell-type has a distinct gene expression pattern. Chromatin provides an important layer of cell-type specific transcriptional control [1,2]. The basic unit of chromatin is the nucleosome, which wraps a 147 bp sequence of the genome. The nucleosome contains two copies each of four core histone proteins: H2A, H2B, H3 and H4 [3]. Each histone has an N-terminal tail that can be covalently modified at multiple positions. Distinct combinatorial patterns (also known as chromatin states) play important roles in transcriptional regulation [1, 2]. As genome-wide histone modification data are being generated in a rapid speed [4–13], there has been a growing interest in developing computational methods to precisely define chromatin states [10, 14–18]. Previous methods have mainly focused on detecting local chromatin state variation, whereas large-scale patterns (also known as domains) remain poorly characterized. Nevertheless, epigenetic domains have been identified in various data-types [19–25]. To systematically identify domain patterns from multiple histone marks, we recently developed a hidden Markov model, treating each gene as a separate unit [26]. By applying this method to analyze a collection of ChIP-seq datasets in 27 human cell lines, we found that chromatin states can be used to classify cell-types with high accuracy [27].

Rather than focusing on each length scale separately, it is desirable to characterize multi-scale chromatin states in a single computational framework. To this end, we present a new approach called tiered hidden Markov model (THMM). We tested this approach by analyzing a publicly available ChIP-seq dataset from the Roadmap Epigenome Project [8]. Our analysis identified a two-tiered structure of chromatin states, which we call the bin- and domain-level states. Whereas bin-level states can effectively capture local (200 bp) variation of histone modification patterns, the domain-level state detects large-scale (> 1Kb) variations. We show that this two-tier characterization is useful for better understanding the regulatory role of chromatin.

Results

Dataset Collection and Pre-processing

ChIP-seq data from the H1 human embryonic stem (ES) cell line was obtained from the Roadmap Epigenome Project [8] (http://www.epigenomebrowser.org/). Five modifications (H3K4mel, H3K4me3, H3K9me3, H3K27me3 and H3K36me3) with well-known biological functions were chosen for analysis. Raw sequence reads were mapped to non-overlapping 200bp bins via BEDTools [28] and normalized to have the unit of reads per million reads (RPM). Bins that overlapped 50% or more with known repetitive regions [29] were removed due to possible alignment issues. After removing these highly repetitive regions, the remaining 99.97% bins were analyzed further.

For simplicity, we focused on chromatin state organization around genie regions, and truncated the genome by keeping only the promoter and transcribed regions of protein-coding genes according to Refseq [30]. To be precise, in this study we use the term ‘promoter’ to include the region 2KB upstream from the transcription start site, whereas the ‘gene body’ is defined as the region from transcription start site to transcription end site. We thus excluded most intergenic regions, which consist of the majority of the genome, from our initial analysis. This truncated genome contains a total of 6,332,441 bins (1.27 Gb).

Tiered Chromatin States in Human ES Cells

We applied our THMM approach to characterize the chromatin states on the truncated genome in human ES cells based on the five histone modification marks mentioned above. Because of its smaller size, we first determined the optimal number of bin-level states by using the data on chromosome 22. Since the log-likelihood of the model increases monotonically with model complexity, we used permuted data as a control, and evaluated the difference of log-likelihood for observed and permuted data, which was generated by randomly reordering all bin locations on chromosome 22 without changing the corresponding sequence reads. This strategy is similar to the gap-statistic commonly used for k-means clustering [31]. We varied the number of bin-level states from three to twenty eight, and found that the log-likelihood differences between the observed and permuted data plateaus around K = 15 (Supplemental Figure 1), suggesting that the optimal number of bin-level states is around 15. As an additional validation, we found that 94% of truncated genome falls into one of the 15 most abundant combinatorial patterns (Supplemental Figure 2). We compared three non-degenerative tiered structures that are consistent with this constraint, corresponding to a “3 × 5” model (that is, three domains with five bin-level states per domain), a “4 × 4 “ model, and a “5 × 3 “ model, respectively. The “3 × 5” model has the best performance but quite similar to the “5 × 3 “ model (Supplemental Figure 3). For simplicity and interpretability, we selected the “3 × 5” model as the final model. We then refined the parameter value estimate by fitting the entire truncated genome (Table 1) and used it for the rest of the analysis in this paper (see Material and Methods for details).

Table 1.

Mean-level ChIPseq counts (RPM) for each chromatin state in the final THMM

domain-level	non-active					null					active
bin-level	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15
H3K4me1	88.3	137.1	131.9	103.4	71.0	24.8	4.0	2.6	3.7	6.6	53.9	4.3	27.3	244.7	102.3
H3K4me3	23.1	194.4	1921.4	35.7	7138.4	16.9	15.0	14.8	15.1	15.2	22.1	15.3	16797.0	5273.4	2017.1
H3K9me3	9.2	148.4	7.5	5.5	5.4	5.3	5.0	2.8	13.2	4.6	77.7	6.0	6.1	398.0	6.3
H3K27me3	51.9	9151.6	1044.3	7.6	36.1	6.4	4.7	4.4	6.0	9.7	7.7	4.3	7.0	173.1	7.6
H3K36me3	6.2	44.5	4.4	7.6	2.4	9.2	23.8	3.3	6.0	2.7	80.2	138.8	4.6	614.1	15.2

We found certain similarities among the bin-level states associated with a common domain-level state; most bin-level states within a domain share similar histone modification patterns. The bin-level states associated with Domain 1 (States 1–5) are generally associated with high levels of H3K27me3; Domain 2 (States 6–10) is generally absent of all histone marks; while Domain 3 (States 11–15) is enriched with H3K4me3 and H3K36me3 and depleted of H3K27me3 (Table 1 and Figure 2). Following our previous work [26, 27], we annotated Domains 1–3 as non-active, null, and active, respectively.

Figure 2.

The combinatorial histone modification patterns associated with different chromatin states. The sequence counts are normalized by converting to Z-scores defined as follows:

$${\text{Z}}_{\text{ik}}=({\text{X}}_{\text{ik}}-{\mu }_{\text{i}})/{\sigma }_{\text{i}},$$

where X_ik is the average count for histone modification i=1, 2, 3, 4, 5 within state k=l, 2,…15, μ_i is the average modification level in the truncated genome, and σ_i is the standard deviation of histone modification i in the truncated genome.

Next we examined the overall distribution of the domain-level states. The majority (95.6%) of the truncated genome is assigned to the null domain (Figure 3, Supplemental Figure 4), which is also the largest on average, with a mean length of 53.9 bins (10.8 Kb), but the domain size is highly variable with a standard deviation (SD) of 79.3 bins (15.9 Kb). In comparison, the active (average length ± SD: 5.4 ± 8.3 Kb) and non-active (average length ± SD: 2.3 ± 2.6 KB) domains are smaller on average, and also have less absolute variability (though the relative variability is comparable). The null domains are primarily associated with introns, whereas the non-active domains are enriched in the promoter regions (Figure 4).

Figure 3.

Proportion of the truncated genome in each domain. Domain colors are the same as in Figure 1.

Figure 4.

The proportion of each (A) domain-level and (B) bin-level state found in promoter (green), introns (purple) and exons (black) regions.

While chromatin states are defined based on histone modification data alone, they are useful only if the resulting annotations are also functionally meaningful. It is well known that chromatin plays an important role in gene regulation, and previous studies have shown that active and inactive genes are associated with different sets of histone marks [5]. For example, while H3K36me3 is enriched in highly transcribed genes, the H3K27me3 mark is associated with transcriptionally inactive genes. To test whether our unsupervised chromatin state annotation methods can recapitulate such differences, we analyzed an ES RNA-seq dataset [32], focusing on domain-level states. Raw sequence reads were processed as for the ChIP-seq data and scaled to reads per million reads (RPM). The active domain (States 11–15) is indeed enriched with significantly higher expression levels (average RNA-seq level ± SD: 1.1E4 ± 8.3E4 RPM) compared to other domains (two sample t-test versus null and non-active domain p-values < 0.0001) (Figure 5A), followed by the non-active domains (average RNA-seq level ± SD: 9.8E2 ± 1.7E4 RPM), and the null states have the lowest transcription level (average RNA-seq level ± SD: 3.9E3 ± 7.8E3 RPM). These transcription associated changes are consistent with a role of H3K27me3 in gene silencing [33]. Taken together, these results have provided a functional validation of our method.

Figure 5.

Functional characterization of the chromatin states. (A) The average RNA-seq values for each bin-level state. Vertical lines represent the mean value for each domain. The bottom track is the truncated genome-wide average. (B) Representative enriched functions and pathways identified using DAVID. P-values for enrichment in the corresponding DAVID category are determined by the Fisher exact test and adjusted for multiple testing by Benjamini-Hochberg procedure. Small p-values indicate evidence of an association with the corresponding DAVID category. Colors of each state correspond to domain membership as in Figure 1.

In addition, we applied the DAVID software to identify enriched functional categories that are associated with each domain type [34]. To avoid overlapping annotations, a gene striding multiple domain-level states was assigned to the most enriched one along that gene. As a result, we obtained 9,484 non-active, 11,703 null and 2,130 active genes for ES cells. The active genes are enriched with cell maintenance related functions such as the zinc finger region, ribosome, RNA binding, and cell cycle (Figure 5B), which are all known to be active in ES cells. Genes characterized by the null domains are involved in functions related to fully differentiated tissues such as olfactory receptor, defense response, taste and vision. In contrast, the non-active genes are related to development and cell growth such as the regulation of transcription, homeobox gene clusters, mitochondrion and, the negative regulation of cell death.

Bin-level state differences within each domain

While the bin-level states within each domain share similar properties, there still exist critical differences between them. For example, while all five bin-level states within the active domain are enriched in active genes, two (States 13 and 15) are more enriched in promoter regions (Figure 4B). State 13 is especially enriched with H3K4me3 and tends to colocalize with transcription start sites. In contrast, the other three states (States 11, 12, 14) are mainly located inside gene-bodies (Figure 4B). State 14 is especially enriched with the H3K36me3 mark (Figure 2) and, as expected, corresponds to the most actively transcribed regions (Figure 5A). These bin-level differences strongly suggest the utility of our two-tiered approach in detecting complex chromatin patterns associated with a common functional element.

Similarly, the bin-level states within the non-active domain are also different, with States 1, 3 and 5 more strongly enriched in promoter regions than the other two states (Figure 4B). Interestingly, State 3 has the signature of bivalent domains [35], enriched with both H3K4me3 and H3K27me3 marks. State 2 is highly enriched with H3K27me3 but devoid of H3K4me3 (Figure 2), corresponding to strong repression (Figure 5A). The null states are similar in general and are associated with low transcriptional activity, although State 7 is more enriched with H3K36me3 and transcribed at a higher level.

Robustness of domain-level states

Noise in experimental data leads to uncertainty of chromatin state annotations. Since the sensitivity of chromatin states on measurement error should decrease with respect to length scale, we expect that the domain-level states are more robust than the bin-level states. To test this hypothesis, we fit a THMM model independently on each of two replicates and quantified the accordance with Cohen’s κ [36] (see Methods for details). Indeed, we found that the domain-level states (κ = 0.81, p-value<0.0001; Supplemental Table 2) are more robust than the bin-level states (κ = 0.47, p-value<0.0001; Figure 6A). Compared to a single-scale approach, our THMM has provided the flexibility to prioritize either spatial-resolution or robustness, depending on the specific biological questions at hand.

Figure 6.

Comparison between the (A) robustness and (B) coherency of the bin-level and domain-level states. (A) The robustness of bin-level (green) or domain-level (yellow) chromatin states are evaluated by the agreement between assignment obtained independently from two replicates, and compared with random background (black curve). (B) The spatial coherence of bin-level (green) and domain-level (yellow) chromatin states are evaluated by the agreement between bin-pairs separated by a fixed lag n, where n is the number of 200bp bins.

Though each of our domains has an average length larger than 2 Kb, we wanted to compare the spatial coherency of the bin-level states to those of the domain-level states. To this end, we quantified κ from comparing the chromatin state of each bin with another one that is n bins away in the genome. We found that the domain-level states are more coherent than the bin-level states (Figure 6B). Even at a distance of 2 Kb, the domain-level states still retain a κ of 0.58, compared to 0.33 for bin-level states (and zero expected by chance).

Chromatin States in Intergenic Regions

We extended our investigation by applying the THMM to predict genome-wide chromatin states, including all the intergenic regions. In total, our analysis covered 15,863,683 bins (corresponding to 3.17 Gb) (Supplemental Table 3). As expected, the vast majority (98.8%) of intergenic bins were assigned to one of the null domain associated states (Figure 7, Supplemental Figure 5). The intergenic null domains (mean length ± SD = 26.4 ± 486.1 Kb) are typically larger (two-sample t-statistic p-value < 0.0001) than those in the truncated genome (mean length ± SD = 10.8 ± 15.9 Kb).

Figure 7.

The distribution of domain assignment for the intergenic genome. 98.8% of the truncated genome is assigned to the null domain, 0.83% to the non-active domain, and 0.36% to the active domain. Domain colors are the same as in Figure 1.

Previous studies have identified large domains that are associated with lamina proteins[19]. These lamina-associated domains (LADs) are generally associated gene silencing. Interestingly, we found that the vast majority of LAD-associated bins are assigned to the null domain (Supplemental Figure 6), suggesting that the histone defined chromatin states are closely associated with the higher-order chromatin structure.

Of note, there are 34,024 intergenic bins that fall into the active domains. These domains are much shorter on average than in the truncated genome (mean size = 2.8 Kb and 5.4 Kb, respectively). We selected the RNA sequence reads that are mapped to intergenic regions, and then compared those mapped to the active domains with the intergenic background. We found that on average the expression level at the active domains is 25 times higher (mean value = 1.6E3 RPM and 66.4 RPM, respectively) (Supplemental Figure 7). For reference, the expression level in active domains in the truncated genome is much higher (mean value = 1.1E4 RPM, two-sample t-test p-value<0.0001).

One important class of non-coding RNA is long intergenic non-coding RNAs (lincRNAs), which have been increasingly recognized as key regulators of diverse cellular processes [37–39]. We mapped the above RNA-seq with known lincRNA annotations [40] to identify actively transcribed lincRNA in ES cells and found that they are highly enriched in active domains (χ² = 2.8E2, df = 2, p-value < 0.0001). Moreover, the expression levels of lincRNAs are relatively high (mean value = 4.6E3 RPM) (Supplemental Figure 8).

Another interesting class of features in intergenic regions is pseudogenes, which has traditionally been thought as dysfunctional fossils of coding genes [41]. However, recent studies have suggested that a subset of pseudogenes still have functional roles, for example, by regulating the expression level of its parental allele [42, 43]. It remains unclear whether there are distinct epigenetic signatures associated with different classes of pseudogenes. While the majority of the pseudogenes (as annotated in http://pseudogene.org) are embedded in the null domain, a larger portion (2.54%) than the truncated genome (1.2%, two-sample test of proportions p-value < 0.0001) is mapped to the active domains. We annotated each pseudogene as active, null, or non-active, in the same way as for coding genes, and calculated the enrichment score of each subtype relative to the whole population of pseudogenes. Interestingly, we found that three immunoglobulin-related subtypes are highly enriched with active domains (Supplemental Figure 9, Supplemental Table 4). A functional role for immunoglobin pseudogenes has been proposed for more than a decade. These pesudogenes are highly conserved, have open reading frames, and retain canonical structure sequence patterns [42]. Our results suggest that active chromatin states may play a role in maintenance of their functional capacity. Taken together, the above results suggest that the domain-level states may be useful for detecting functional elements in intergenic regions.

Discussion

We have developed a THMM approach to systematically detect chromatin states over multiple length scales. There are a number of advantages of this two-tiered approach compared to the regular HMM approach used in previous studies: (1) the two-tiered chromatin states can be naturally interpreted as large- and small-scale chromatin structures, respectively, facilitating future biological investigations of the chromatin structures or interactions; (2) compared to a coarse-grained HMM model, the two-tiered approach preserves spatial resolution at the bin-level; and (3) the two-tiered approach is more robust than using a single-tier. Similar to our previous work [26, 27], we identified three domain-level chromatin states in ES cells, corresponding to active, non-active, and null domains, respectively. Furthermore, this new approach also allowed us to identify refined bin-level patterns within each domain, thereby increasing the spatial resolution. As such, our THMM approach has provided insights into an additional layer of complexity in chromatin state organizations.

While our analysis was mainly focused on the truncated genome containing only the protein-coding genes, we also found interesting patterns in intergenic regions by applying the THMM to predict whole-genome chromatin states. Specifically, we found that a large number of bins are characterized by a chromatin state within active domains. Closer examination suggests that a certain fraction of these domains are associated with either noncoding RNAs or pseudogenes. While similar results have been obtained by other methods in recent literature [38, 43], our two-tiered approach provided more refined description of the chromatin states. Our results suggest that chromatin state information may serve as a useful guide for detecting functional elements in intergenic regions.

Material and Methods

THMM Model

We have developed a THMM approach to model chromatin states at multiple length scales. THMM is a special kind of hidden Markov models (HMM) that contains a two-tiered hidden-state structure, which is suitable for detecting both bin- and domain-level states.

We started by fitting a fully connected HMM with K hidden states. Following previous work [18, 44], we modeled the emission probabilities as multivariate normal emission distributions; that is, the probability of observing the M-length vector Y_j at the t^th bin is:

$$\text{P}({\text{Y}}_{\text{j}}\mid {\text{H}}_{\text{t}}=\text{k})~\text{MVN}({\mu }_k,{\mathrm{\sum }}_k),$$

where H_t is the hidden state, and μ_k and Σ_k are the mean and covariance of emission probability distribution for state k, respectively. We recognize that the sequence reads data may be better modeled with a Poisson or negative binomial distribution, but it is mathematically challenging to extend to multivariable setting and to incorporate into the HMM framework.

Let Y_k be a matrix of the observed Y_j’s assigned to state k; the log likelihood of the K means and covariances given the data is the sum of the corresponding log-likelihoods for each of the K states:

$$l({\mu }_1,{\mu }_2,\dots ,{\mu }_K,{\mathrm{\sum }}_1,{\mathrm{\sum }}_2,\dots ,{\mathrm{\sum }}_K\mid Y)=-\frac{1}{2}{\mathrm{\sum }}_{k=1}^K[\mathit{Mln}(2\pi )+ln\mid {\mathrm{\sum }}_k\mid +{(Y_k-{\mu }_k)}^{-1}{\mathrm{\sum }}_k^{-1}{(Y_k-{\mu }_k)}^T].$$

The model parameters were estimated by using the Baum-Welch algorithm [45]; the Viterbi algorithm [45] was used to obtain the maximum-likelihood estimate of the chromatin state at each bin location. In order to account for the trend of better fitting with more complex models, we determined the number of hidden states, K, by the log-likelihood difference between the observed and permutated data, as motivated by the gap-statistic for K-means clustering [31]. Specifically, for each value of K chosen between three and twenty eight, we computed the log-likelihood difference between the models for fitting the observed and permuted data. The permuted data were generated by permuting the bin locations across the truncated genome. While that the value of log-likelihood increases monotonically with K, saturation occurs around K = 15. As a compromise between model performance and complexity, we selected K = 15 or 16 in our final model.

As discussed in the main text, one limitation of this initial model is that it only models histone modification on a single length-scale (i.e., 200bp). To overcome this limitation, we divided the hidden states into clusters with the properties that intra-cluster states are fully connected, whereas inter-cluster transitions are confined to a specific state selected from each cluster. For clarification, we call these clusters the domain-level states, whereas the individual hidden states are referred to as the bin-level states. As shown in Figure 1, the resulting model is a special case of HMM whose topology is characterized by a tiered structure, hence called the tiered hidden Markov model (HMM).

Figure 1.

The topology of our THMM. Each bin-level state is represented by a circled color-coded according to its corresponding domain-level state (represented by a box). Note that transitions between different domain-level states can only occur via a special bin-level state from each domain. States within the null domain are represented by the color yellow; states within the active domain are shown in red; states within the non-active domain are in blue.

In order to infer the optimal THMM structure, we initialized the emission probability of each bin-level state based on our fully connected HMM. To determine how many bin-level states should be included in each domain, we made the simplifying assumption that the number of bin-level states in each domain is identical. Under this assumption, there are only three non-trivial possibilities, corresponding to three, four, or five domains, respectively. To determine which configuration is the best, we fitted each model to the data and evaluated the corresponding log-likelihood values. The configuration corresponding to the highest log-likelihood value was selected as the final model.

Quantification of agreement between different annotations

We used Cohen’s κ–statistic [36] to quantify the agreement between a pair of annotations along the genome, which may correspond to either the chromatin states identified from our computational model or functional annotations from curated databases. The κ–statistic is defined as

$$\kappa =\frac{P_a-P_e}{1-P_e},$$

where P_a is the proportion of agreement between the two annotations, and P_e is the expected agreement due to chance as estimated by random permutation. Typically (but not always), the value of κ is between zero (no agreement) to one (perfect agreement). Values over 0.75 are often considered excellent, whereas those below 0.40 are considered poor. P-values are not typically calculated as small values of κ can be significantly different from zero [46]. To account for differences of the prevalence of different annotations, κ was further normalized by dividing by the maximum value κ_max. We applied this approach to evaluate the reproducibility of chromatin state assignment to compare with known functional annotations.

Highlights.

• We present a first computational model for multi-scale chromatin state identification.

• We have identified a two-tier chromatin state structure in human embryonic stem cells.

• We found that different domain types are associated with distinct biological functions.

• Domain-levels states are more robust and functional coherent than bin-level states.

• LincRNAs and Immunoglobin-related pseudogenes are highly enriched with active domains.