CatLearning: highly accurate gene expression prediction from histone mark

Abstract

Histone modifications, known as histone marks, are pivotal in regulating gene expression within cells. The vast array of potential combinations of histone marks presents a considerable challenge in decoding the regulatory mechanisms solely through biological experimental approaches. To overcome this challenge, we have developed a method called CatLearning. It utilizes a modified convolutional neural network architecture with a specialized adaptation Residual Network to quantitatively interpret histone marks and predict gene expression. This architecture integrates long-range histone information up to 500Kb and learns chromatin interaction features without 3D information. By using only one histone mark, CatLearning achieves a high level of accuracy. Furthermore, CatLearning predicts gene expression by simulating changes in histone modifications at enhancers and throughout the genome. These findings help comprehend the architecture of histone marks and develop diagnostic and therapeutic targets for diseases with epigenetic changes.

Introduction

Transcription is the first stage of gene expression, wherein genetic information is transferred from DNA to RNA. It has been observed that a single set of genomic DNA can express distinct sets of genes, modulated by epigenetic information [1]. The key epigenetic information is stored in histone marks, DNA modifications, RNA, and RNA modifications [2]. Histone marks are post-translational modifications of histone proteins that combine with genomic DNA to form chromatin [3, 4].

Epigenetic alterations of histone marks are believed to play a crucial role in various cell fate changes, including tumorigenesis, drug resistance, stem cell generation, directed differentiation in stem cell therapy, aging, and more [5, 6]. Therefore, epigenetic inhibitors, such as EZH2 and HDAC inhibitors, which target key enzymes involved in histone modification, are undergoing clinical trials [7, 8]. By inhibiting these enzymes, these drugs aim to disrupt histone marks and alter gene expression patterns. Understanding how gene expression changes when histone marks are disrupted is crucial in the context of epigenetic regulation and its role in various diseases. This knowledge can help identify appropriate histone mark targets for clinical purposes. The regulation of gene expression by histone marks depends on the type of post-translational modifications, the locations of modification, the level of modification, and the interplay among different histone marks [9, 10]. Experimentally demonstrating the regulation of gene expression by histone marks is highly challenging due to the unlimited synergy of these conditions. The advancement of machine learning and deep learning has provided new insights into the role of histone marks in regulating gene expression [11, 12]. Algorithms predicting gene expression from histone marks can significantly expedite the development and utilization of epigenetic inhibitors in clinical applications.

It is known that histone marks are present in high concentrations around the transcription starting site (TSS), and regulatory elements such as enhancers can impact gene expression even from a far distance [6]. Connecting the proximal and far distal information from TSS has been a challenge. Traditional machine learning methods, such as Linear Regression (LR) [13], Support Vector Machines (SVMs) [14], and random forests (RF) [15], have been used to predict gene expression based on histone marks. However, these methods employ the best-bin strategy, in which only bins mostly correlated with gene expression are selected, for better performance. Therefore, information from other bins is not utilized. In addition, these methods are challenging to train on large amounts of data sets. To overcome these limitations, deep learning was widely used in studies of large sample sizes. LSTM (Long Short-Term Memory) is an advanced form of RNN (Recurrent Neural Network). Based on this architecture, AttentiveChrome [16] is designed for the prediction of gene expression using histone mark information within narrow windows around TSS. However, learning the larger-scale interaction is challenging for this approach. The Convolutional Neural Network (CNN) can automatically capture both local and global features in input samples [17]. Additionally, by implementing large-scale networks, CNN is suitable for learning the relationship between histone modification and gene expression across a wide range of genomes. One such tool is DeepChrome, which classifies whether the expression level of genes is high or low using histone modification information within 10Kb regions around TSS [18]. Applying a graph-based framework, GC-MERGE incorporates 3D genomic long-range interactions and local histone marks to predict gene expression [19]. To increase prediction accuracy, Chromoformer integrates the transformer, a model originally developed for natural language processing, and 3D chromatin conformation in the deep learning architecture [20]. Chromoformer can capture long-range dependencies between histone marks at multiple distal regulatory regions associated with wider genomic windows near TSS. Previous attempts have relied on 3D chromatin conformation interactions from Hi-C data to improve performance [19, 20]. Unfortunately, Hi-C data are not widely accessible in different cell lines and the most available datasets have relatively low resolution (ranging from 40Kb to 1Mb) [21].

Histone marks are mostly concentrated near the TSS, but some marks are also found far from the TSS [22]. In the mammalian genome, noncoding regions make up the majority of the genome, outnumbering coding regions. These noncoding regions consist of cis-regulatory elements as well as long-range regulatory elements [23]. So, histone marks far from the TSS are also crucial for regulating gene expression. Previous methods have separated the training and testing data from the same sequencing result for cross-validation [18–20]. This approach limits the prediction to detect only a subset of genes and restricts the amount of training data available. Moreover, a regression model that can predict an exact gene expression value is needed for practical applications, rather than a simple classification of highly or lowly expressed genes.

Here, we present an algorithm called CatLearning to predict gene expression from histone marks. CatLearning uses a CNN architecture with a specialized adaptation Residual Network (ResNet). By coupling the advantage of separating genes with different expression levels before training, CatLearning captures features from histone marks with varying enrichment scales in a large genomic region. CatLearning attains a high level of accuracy using just one histone mark. Moreover, CatLearning learns the chromatin architecture with only histone marks and predicts gene expression by simulating histone modification changes at enhancers and genomewide. CatLearning is useful in developing diagnostic and therapeutic targets for diseases involving epigenetic changes.

Materials and Methods

Data collection and processing

RNA-seq and five major histone marks (HMs), H3K4me3, H3K9me3, H3K27me3, H3K36me3, and H3K27ac, ChIP-seq data were downloaded from the ENCODE [24] (Encyclopedia of DNA Elements) and GEO [25] (Gene Expression Omnibus) database (Supplementary Table S1). Reads were trimmed using Trim Galore (version 0.6.4) (https://www.bioinformatics.babraham.ac.uk/projects/trim_galore) to remove adaptors and low-quality reads. Trimmed reads were mapped to the human reference genome hg19 by Bowtie2 [26] and STAR [27] for ChIP-seq and RNA-seq, respectively. To normalize epigenomic signals among the genome, BEDtools [28] (version 2.92.2) and bedGraphToBigWig (version 4) (https://www.encodeproject.org/software/bedgraphtobigwig/) were used with the following parameters ‘genomecov --scaleFactor 10^7/(the number of mapped reads)’. deepTools [29] (version 3.4.3) was used to count normalized signals of ChIP-seq with 100 bp bin size. Peaks were called by MACS2 [30] (version 2.2.7.1). StringTie [31] (version 2.2.1) was applied to count and normalize gene expression into FPKM (fragments per kilobase of exon model per million mapped fragments).

For data quality control, peak number (>5000), peak fold change (>1), FRiP (the fraction of reads in peaks, >0.1), mapping ratio (>45%), and GC content ratio (35%–65%) were used to clean ChIP-seq data and gene number (>5000) was used to clean RNA-seq data. In addition, ChIP-seq data were further cleaned by the Pearson coefficient of replicates in the same cell line. Saturation of ChIP-seq reads was performed by Preseq (version 2.0.0) (https://smithlabresearch.org/software/preseq/), which filtered out 10% of the low-saturation samples.

Training and test data preparation

For regression and classification model training, epigenomic signals around TSS 500Kb were assigned to the corresponding gene expression. Epigenomic signals were scaled using the Z-score on each chromosome. FPKM values were used for regression training. To obtain the binary label of all genes in each cell line, genes were divided into two classes: those with gene expression more than the median were labeled as 1 (highly expressed), otherwise labeled as 0 (lowly expressed). In one particular cell line, ChIP-seq and RNA-seq samples were randomly combined into nonrepetitive combinations consisting of five HMs and one transcriptome as one full dataset. Twenty combination datasets were generated for each one cell line. Seventeen datasets were used as the training data and the remaining three datasets were used as the test data. The genes were divided into subgroups based on their FPKM values: genes with FPKM values between 0.1 and 1 and genes with FPKM values between 1 and 100. Each subgroup was trained separately using five histone marks data near the 500Kb of TSS. The hyperparameters for training remained unchanged when using subgrouped genes.

Model architecture

Motivated by the concept of the multi-scale ResNet (https://github.com/geekfeiw/Multi-Scale-1D-ResNet), we proposed a specialized adaptation ResNet for handling 1D inputs to solve our problem. ResNet is moduled with two kinds of convolution kernel (one big and one small). Type I convolution kernel is large, so every time we look at large-scale features, and Type II convolution kernel is small, we look at small-scale features. Our feature extraction process can be seen as finding coarse features first and then finding fine features from these coarse features. This specialized ResNet could capture a broad range of patterns in the data. Key components and functionalities are illustrated as follows (Fig. 1).

Figure 1.

The model architecture of CatLearning. The model consisted of three blocks: CNN, ResNet Type I, and ResNet Type II. The CNN was used to capture local features. To enhance the performance of the model, two blocks of ResNet with different convolution kernels were used. The Type I convolution kernel was large and used to capture large-scale features, while the Type II convolution kernel was small and used to capture small-scale features.

• (1) Multi-types input handling: The model was designed to process several separate 1D input channels, allowing it to accommodate multiple data sources of various histone modifications simultaneously. This capability was particularly useful in our application where different histone modification data needed to be analyzed together.

• (2) Initial feature extraction: Each input channel first went through a shared initial convolution layer with a kernel size of 7, followed by batch normalization (BN), a ReLU activation function, and a max-pooling operation. This stage was designed to extract basic features from each input channel while reducing its dimensionality, making the subsequent processing more efficient.

• (3) Customizable residual blocks: The core framework of the model was built with two kinds of residual blocks; one ResNet block (Type I) was designed to capture wider contextual information, which employed larger convolutions with a kernel size of 10. In contrast, the other ResNet block (Type II) used a convolution layer with a kernel size of 3, which was suitable for capturing local dependencies. This distinction allowed the network to learn from a diverse range of spatial scales within the data. To identify the importance of the blocks in the model, we performed an ablation experiment on the ResNet Type I and Type II blocks, respectively, for training. When Type I melts, the Type II structure was used to replace it, and when Type II melts, the Type I structure was used to replace it. The pooling size of the average pooling layer was adjusted accordingly to keep the depth and other structures of the network structure unchanged. In addition, the training parameters remained unchanged.

• (4) Layer stacking and downsampling: The blocks were organized into layers, with each layer increasing the number of feature maps and applying downsampling (using a stride of 2). This downsampling operation was designed to progressively increase the receptive field and reduce the dimensionality of the feature maps. This hierarchical structure enabled the model to learn increasingly complex and abstract representations of the input data.

• (5) Average pooling and concatenation: After processing through the residual blocks, an average pooling operation was applied to reduce features of various inputs. Afterward, feature maps of multiple inputs were concatenated. This step merged the learned features from all active input channels, integrating the information captured from different sources.

• (6) Output layer: The pooled features were then passed through a dropout layer for regularization, followed by a fully connected layer that mapped the learned features to the desired output classes. This final layer was responsible for the regression task.

Model training and evaluation

All CatLearning models were trained for 80 epochs using AdamW optimizer [32] and the model resulting from the best test epoch was chosen as the final model. Both the initial learning rate and weight decay were set as 3 × 10⁻³. Batch size was chosen as 256. Because the ChIP-seq and RNA-seq data were not from the same experiment, it was difficult to directly use the expression level of a batch of RNA-seq as the observed value. Hence, the median value of gene expression in each cell type was used as the observed value. Twenty cell types were chosen for training individually since they have more experimental repeats. For regression training, the mean squared error (MSE) between the predicted scalar and target values was used as a loss function. To correct the sequencing variations, 15% of the top variance genes were filtered. Unfiltered genes were then divided into two expression subgroups according to FPKM: 0.1–1 and 1–100. To evaluate the performance of models, The ACC (accuracy) value, Pearson coefficient, R² (R-squared), MAE (mean absolute error), and Gene cor were calculated. ACC was defined as the percentage of genes with a predicted value within a two-fold change (up or down-regulation) of the observed value. The Person coefficient was calculated by the raw FPKM between prediction and observation for individual samples across genes. MAE was defined as the mean absolute error. R² was defined as the proportion of explainable variance to total variance. MAE and R² were calculated between prediction and observation for individual samples across genes. Gene cor was defined as the Pearson coefficient for an individual gene across samples. For classification training, cross-entropy between the predicted probability and binary gene expression label was used as a loss function. The AUC, ACC value, AUPR, F1 score, Precision, and Recall were used for evaluating the performance. AUC was defined as the area under the receiver operating characteristic curve. The ACC of classification was defined as the percentage of correctly classified gene numbers. AUPR was defined as the area under the precision–recall curve. F1 score was defined as the harmonic mean of precision and recall. Precision was defined as the ratio of true positives to total positives predicted. Recall was defined as the ratio of true positives to total actual positives. AUC, AUPR, F1 score, Precision, and Recall were calculated between prediction probability and observed binary gene expression label. The FPKM standard for classification was a two-step classification: genes were sorted by expression levels above or below 1 FPKM and then grouped by values above or below 100 or 0.1 FPKM, respectively.

DeepChrome [18], AttentiveChrome [16], GCMERGE [19], and Chromoformer [20] models and data preprocessing scripts were available from the official code repositories provided in the published studies. The optimal hyperparameters previously set for each benchmark model were used. Five cell types (A549, GM12878, H1, HepG2, and Liver) were selected for benchmarking since they have processed 3D genome data from the Chromoformer repositories.

Analysis of filtered genes

For each cell type, the filtered genes were identified as the top 15% of genes with the highest variant expression. R package pheatmap (https://cran.r-project.org/web/packages/pheatmap/index.html) was used to profile the distribution of filtered genes in different cell types. The housekeeping genes list was downloaded from the previous study [33]. Gene set enrichment analysis (GSEA) was performed using ClusterProfiler [34] (version 4.8.2) with an equal number of filtered and unfiltered genes. In this analysis, unfiltered genes were denoted as −1, while filtered genes were labeled as 1. GO (Gene ontology) biological process terms with P-values <.05 were selected as significantly enriched terms. Genomic annotation bed files were downloaded from the University of California, Santa Cruz (UCSC) table browser [35]. The enrichment signals in 100 bp bins around TSS were calculated by the computeMatrix function in deepTools. The variance of histone mark signal for each gene was calculated across all samples in the particular cell type and normalized by the average enrichment of all replicates.

Histone marks combination analysis

All combinations of one to five histone modifications were used for the analysis. To keep the learning condition consistent, the used model structure and optimal hyperparameters were unchanged. Average histone mark signals around TSS were calculated by the computeMatrix function in deepTools. To compare the fold change of different histone marks, narrow peaks were called for all histone marks. The coefficient of variation (CV) was used to evaluate the stability of prediction. To further identify if the sharp peaks were more predictive than the broad peaks, H3K4me1, H3K4me2, H3K9ac, and H3K79me2 data were downloaded from ENCODE and GEO databases (Supplementary Table S2). These histone mark data were used for individual histone mark training by CatLearning.

Model training by different regions around transcription starting site

HM signals at different regions around the TSS were used as training and test datasets. To keep the learning condition consistent, the used model structure and optimal hyperparameters were unchanged. Because the amount of input data was increased by expanding the regions, we increased the model kernel size to fit the calculation. The variance of histone modifications was normalized by the average enrichment in this region. The list of paired enhancers and genes was available from the UCSC GeneHancer database [36].

The interaction intensity

RNA Pol II ChIA-PET processed loops bedpe files, containing genomic interaction information, were utilized from the ENCODE database (ENCFF421KYP, ENCFF913VWM, ENCFF753NSM, ENCFF002YGU, ENCFF125JGW, ENCFF364UNM, ENCFF511QFN, and ENCFF597SQA). The coordinates of assembly were converted to hg19 by the liftOver tool (https://genome.ucsc.edu/cgi-bin/hgLiftOver). The promoter region was defined as 3Kb upstream and 500 bp downstream of TSS.

The interaction loop with one end located in the promoter and the other end located in the analyzed region was considered as a positive hit. The total intensity of all interactions within each gene was used as the interaction intensity for this gene.

Prediction of gene expression based on perturbation of histone mark

ChIP-seq and RNA-seq data for the perturbation of histone modification enzymes were downloaded from the GEO database. The ChIP-seq dataset showing low correlation was removed as it did not pass the quality control (Supplementary Table S3). To simulate the depletion of histone marks, the signals of targeted histone marks were set as 0.01.

To predict gene expression after enhancer inactivation, we trained models using raw histone mark values and then reduced the H3K27ac signals at the screened positive enhancer regions [37] with different fold changes. The expression values of genes within 500Kb of each enhancer were predicted. Among these enhancer and gene pairs, the top 2000 gene–enhancer pairs with the highest fold change and q-value below 0.1 were classified as positive pairs, while the rest pairs containing the same enhancers were classified as negative ones. To calculate the ratios of positive to negative pairs, the number of enhancer–gene pairs was counted. The 500Kb regions around TSS were partitioned into 50Kb windows. Within each window, the numbers of positive or negative pairs were counted by 1000 bp sliding bins. Then, the positive and negative counts were normalized by the total number of pairs, respectively. The processed contact domains bedpe files, which contained information about topologically associated domains (TADs), were downloaded from the ENCODE database (ENCFF126GED, ENCFF318QPG, ENCFF336WPU, ENCFF395LRK, and ENCFF531LSJ).

Analysis of transcription factors

Five transcription factors (JUND, CEBPB, MYC, NRF1, and SRF) ChIP-seq data of five cell types (GM12878, H1, MCF7, HepG2, and K562) were downloaded from ENCODE database (Supplementary Table S4). The correlation between transcription factors and histone marks was calculated across the whole genome with a 1000 bp bin size.

Results

CatLearning accurately predicts gene expression based on histone marks

We developed a new model architecture named CatLearning (Chromatin and transcription relation learning) to predict gene expression from histone marks (Fig. 2A). CatLearning utilized a modified CNN with a specialized version of ResNet for learning features from histone marks with varying distributions and enrichment scales. By utilizing CatLearning, we were able to effectively integrate histone mark information from distal regulatory regions up to 500Kb from TSS. Five histone marks, including H3K4me3, H3K9me3, H3K27me3, H3K36me3, and H3K27ac, were used for analysis based on their high availability of sequencing results (Supplemental Fig. S1A–C). To evaluate the performance of CatLearning, we defined a prediction result as correct if it was within a two-fold difference from the observed value, which generated the ACC applicable to biologists. Moreover, to learn genome-wide gene regulation, we utilized all genes present in different biological replicates of sequencing data for both training and prediction purposes. This approach differs from previous algorithms that only used one sequencing result to split genes into different sets for cross-validation (Fig. 2B). Our approach could increase the amount of data used for learning and more importantly, the number of genes predicted for downstream usage.

Figure 2.

CatLearning can accurately predict gene expression from histone marks.

(A) Illustration showing the architecture of CatLearning. Signals of five histone marks spanning 500Kb upstream and downstream to TSS were counted and normalized. The counted region of each gene was divided into 10 000 bins. Genes were then divided into two subgroups by expression level: FPKM [0.1, 1) and FPKM [1, 100) for training and prediction. The model also provided a classification function to divide genes into subgroups before a detailed value was predicted. A residual learning module was introduced during learning. After getting the weight parameter, the output of the five histone marks was combined to predict the gene expression. FPKM, fragments per kilobase of exon model per million mapped fragments. HM, histone mark. (B) Illustration showing how the training data set and the testing data set were divided. Pred, predicted. Obs, observed. (C) The number density of genes with different expression values across various cell types. (D) The ACC increased when genes were subgrouped for training. The boxes in violins were drawn from the lower quartile (Q1) to the upper quartile (Q3) with the middle dots denoting the mean. All 20 cell types were used. ACC_Subgroup, prediction ACC using subgrouped genes as indicated. ACC_Total, prediction ACC using all genes. (E) Scatter plots showing the observed FPKM and predicted FPKM in three representative cell lines. R, correlation coefficients that were assessed by Pearson product–moment correlation. R², R-square. MAE, mean absolute error. The two dashed lines represented a two-fold change in observed FPKM. Pred, predicted. Obs, observed. (F) The ACC of prediction in different models. Boxes showed the average accuracy of testing sets of prediction in five cell types. The five cell types were selected from the benchmark of Chromoformer data. The boxes were drawn from the lower quartile (Q1) to the upper quartile (Q3) with the middle line denoting the median, and whiskers with a maximum of 1.5 IQR (interquartile range). n = 5 in each box. P values were calculated by Student’s t-test, two-sided. ACC values in the same cell line were lined up. (G) Same as (F), except the Pearson coefficient between predicted and observed gene expression was used.

Based on our hypothesis that the vast range of gene expression values, often posing challenges for machine learning algorithms, might adversely affect model performance, we conducted a preliminary experiment. The results revealed a discrepancy between our predictions and the actual datasets (Supplemental Fig. S2A–F), further supporting our hypothesis. To mitigate this issue, we aimed to categorize genes based on their gene expression levels. Further analysis of gene expression in different cell lines revealed that most expression values were concentrated around 0.1 and 10 FPKM (Fig. 2C). We then divided genes into different subgroups based on their expression levels. Genes with different RPKM ranges were grouped for training and prediction while keeping the algorithm unchanged. As expected, subgrouping of genes based on their expression values significantly increased the accuracy (Supplemental Fig. S2G). We found that genes with expression levels outside the range of 0.1–100 FPKM were rare and not required for our predictions regarding histone mark changes. Genes with expression levels below 0.1 FPKM could not be detected well by experiments, with observed ACC below 50%, so their prediction accuracy was questionable (Supplemental Fig. S2G). On the other hand, genes with expression levels over 100 FPKM were mainly housekeeping genes that were stably expressed (Supplemental Fig. S3A). Therefore, we built two algorithms for prediction using genes with expression levels between 0.1–1 and 1–100 FPKM. This subgrouping method significantly improved the prediction accuracy, resulting in a mean ACC increase from 23.64% to 77.81% (Fig. 2D and Supplemental Fig. S3B–E).

We noticed there were genes with high expression variance and histone mark variance across various experimental repeats (Supplemental Fig. S3F and G). These genes were enriched in pathways that were associated with fast alternations, including metabolic, response, and signaling processes (Supplemental Fig. S3H). This implies that sequencing experiments may not be able to reliably detect such dynamic genes. Therefore, predicting the expression levels based on unstable histone mark sequencing results was unnecessary and inaccurate to direct histone mark targets for diseases involving epigenetic changes. The depletion of these genes will not affect the application of our model. The overall accuracy of CatLearning increased by selectively removing genes with high variation (Supplemental Fig. S4A–P). We noticed that the accuracy reached an inflection point when we gradually increased the number of depleted high-variation genes and removed the top 15% of genes (Supplemental Fig. S5A–E). To maintain high accuracy while keeping a substantial number of genes, we eliminated the top 15% of genes displaying high variations (Supplemental Fig. S5F). Thereafter, CatLearning achieved an average accuracy of 86.89% and a mean Pearson correlation of 0.93 (Fig. 2E). In addition, we performed ablation experiments to evaluate the ResNet Type I and Type II blocks. Deletion of either block reduced the performance of Catlearning (Supplemental Fig. S5G–K).

We divided genes into two subgroups in our model for prediction. To predict gene expression without predefined subgroups, it is important to classify genes into different subgroups before a detailed expression value is predicted. We modified the data loading and loss function to enable a two-step classification using CatLearning. Genes were sorted by expression levels above or below 1 FPKM and then grouped by values above or below 100 or 0.1 FPKM, respectively. CatLearning yielded high accuracy in this classification task (mean ACC = 88.73%) (Supplemental Fig. S6A–F). By combining the classification and regression algorithms, CatLearning was able to predict gene expression from histone marks.

We conducted a comparison of CatLearning with other regression prediction algorithms, AttentiveChrome [16], DeepChrome [18], GCMERGE [19], and Chromoformer [20], using the same dataset (Table 1). Our results showed that CatLearning demonstrated significantly higher regression evaluation metrics as compared to other regression prediction algorithms (Fig. 2F and G and Supplemental Fig. S7A–H). We also observed that the performance of all these algorithms was improved when we subgrouped genes by their expression levels. Even when categorizing genes as either highly or lowly expressed, CatLearning consistently outperformed the previous models (Supplemental Fig. S8A–F). Overall, these results confirm that the CatLearning architecture achieves high prediction accuracy for gene expression from histone marks, and the subgrouping of genes based on expression levels is crucial for preparing training data.

Table 1.

Summary of computational models.

Computational models	Advantage	Hi-C data requirement	Regression output	Utilize long-range histone information
DeepChrome	Local feature learning
AttentiveChrome	Capture long-term dependencies
GC-MERGE	Captures the local neighbor relationship	√	√
Chromoformer	Self-attention and long-term dependency modeling	√	√
CatLearning	Multi-scale feature capture		√	√

Catlearning achieves high accuracy by utilizing only one histone mark

We used five histone marks in our current model application. In actual experiments, the available histone mark data for training may not be sufficient to cover all these five types. To explore the benefits of using multiple histone marks, we investigated how different types of histone marks affect CatLearning’s performance. Using all potential combinations of five histone marks as input data, we found that the ACC and Pearson correlation increased as we used more types of histone marks for training (Fig. 3A and B and Supplemental Fig. S9A and B). Additionally, incorporating more types of histone marks as input training data also made CatLearning’s prediction performance more stable (Fig. 3C and Supplemental Fig. S9C).

Figure 3.

Training CatLearning with one histone mark is sufficient to achieve a high level of accuracy. (A) The accuracy of models trained by different combinations of histone marks in 20 cell types. The boxes in violins were drawn from the lower quartile (Q1) to the upper quartile (Q3) with the middle dots denoting the mean. K4, H3K4me3. K9, H3K9me3. K27, H3K27me3. K36, H3K36me3. K27ac, H3K27ac. HM, histone mark. Combinations were sorted by the descending order of mean ACC. (B) The accuracy of models trained by different numbers of histone marks in 20 cell types. HM, histone mark. (C) The CV (coefficient of variation) of prediction accuracy for models trained by different numbers of histone marks. The boxes were drawn from the lower quartile (Q1) to the upper quartile (Q3) with the middle line denoting the median, and whiskers with a maximum of 1.5 IQR. P values were calculated by Student’s t-test, two-sided. n = 20 (1 HM), 20 (2 HMs), 20 (3 HMs), 20 (4 HMs), and 20 (5 HMs). HM, histone mark. (D) The mean accuracy of models trained by one histone mark in 20 cell types. Marks were sorted by the descending order of mean ACC and divided into two groups: narrow and broad. HM, histone mark. (E) Change of accuracy after incorporating individual histone marks for training models in 20 cell types. Three replicates in each training were calculated. The x-axis represented the total number of histone marks used for training. The boxes were drawn from the lower quartile (Q1) to the upper quartile (Q3) with the middle line denoting the median, and whiskers with a maximum of 1.5 IQR. P values were calculated by Student’s t-test, two-sided. n = 3600 (2 HMs), 5400 (3HMs), 3600 (4HMs), and 900 (5HMs) in each box. (F) The percentage of extracted contributions of histone marks to the models in 20 cell types. The average contribution was calculated from the testing sets of prediction in each cell type. (G) The average signal density of histone marks in the 500Kb upstream and downstream of TSS. P values were calculated by Student’s t-test, two-sided. n = 277 939 in each violin. (H) Same as (F), except peak fold change was calculated.

We then investigated the impact of specific histone marks on the performance of CatLearning. Using H3K4me3 and H3K27ac in the input data resulted in better CatLearning performance than other combinations with the same number of histone marks. In particular, using only H3K4me3 and H3K27ac resulted in high average accuracies of 84.26% and 83.55%, respectively, which were close to the accuracy of 86.89% using five histone marks. (Fig. 3D). We analyzed additional histone marks such as H3K9ac and H3K4me2, which had narrow peaks, as well as H3K4me1 and H3K79me2, which had broad peaks. Our predictions based on the histone marks with narrow peaks were more accurate compared to those with broad peaks. Additionally, we observed that the addition of H3K4me3 or H3K27ac to the training dataset led to a greater increase in performance as compared to other histone marks (Fig. 3E and Supplemental Fig. S10A). To better understand how histone marks were utilized by CatLearning for predictions, we extracted the contributions of each histone mark. We found that although different types of histone marks had varying contributions in distinct cell lines, H3K4me3 and H3K27ac had significantly higher contributions than other histone marks (Fig. 3F). These findings suggest that H3K4me3 and H3K27ac are major factors in the prediction model.

We observed that H3K4me3 and H3K27ac, which were enriched as sharp peaks on the chromatin, displayed a higher signal density and enrichment level compared to other histone marks (Fig. 3G and H and Supplemental Fig. S10B). The architecture of CatLearning can capture features at different scales and is not more sensitive to local features at certain scales. These unique characteristics make it easier for CatLearning to learn features by utilizing the locally enriched histone mark signals that match the distribution of H3K4me3 and H3K27ac. Together, these data suggest that using more histone marks improves the stability of CatLearning, while H3K4me3 or H3K27ac, which displays locally rich features, is sufficient to yield high accuracy.

CatLearning learns chromatin interaction features and predicts gene expression by simulated histone changes at enhancers

CatLearning utilized histone marks spanning 500Kb from the TSS to predict gene expression. We then analyzed how the size of training regions affected the performance of CatLearning. Generally, the accuracy of predictions improved when we used larger regions of histone mark signals. However, we were surprised to find that the accuracy decreased when we expanded the regions of histone mark signals from 20Kb to 50Kb and from 150Kb to 200Kb (Fig. 4A and Supplemental Fig. S11A–C). We then asked why the predictive accuracy was decreased when more signals from this region were used. We split accurately predicted genes in the 20Kb region into TT (also predicted correctly in the 50Kb region) and TF (only predicted correctly in the 20Kb region) groups (Fig. 4B). A similar definition was used for 150Kb to 200Kb. The histone marks at these genes showed similar variations and enrichment levels (Supplemental Fig. S11D–G), indicating that the decrease in predictive accuracy was not due to histone mark dynamics.

Figure 4.

CatLearning can predict gene expression by simulated changes in histone modifications at enhancers. (A) The accuracy of trained models using different regions around TSS in 20 cell types. Error bars were drawn with SD. (B) The illustration showing how TF and TT genes were defined. (C) The interaction intensity of Pol II between promoters and 20Kb–50Kb regions around TSS. Pol II and promoter interaction intensity were from ChIA-PET data in ENCODE database. The violins were drawn with middle dots denoting the mean. P values were calculated by Student’s t-test, two-sided. n = 2533 (A549 TF), 13 897 (A549 TT), 3483 (H1 TF), 18 249 (H1 TT), 3409 (HCT116 TF), 11 991 (HCT116 TT), 1755 (HEK293T TF), 13 268 (HEK293T TT), 2041 (HepG2 TF), 12 495 (HepG2 TT), 678 (GM12878 TF), 19 744 (GM12878 TT), 1433 (K562 TF), 18 978 (K562 TT), 1768 (MCF7 TF), and 18 086 (MCF7 TT). (D) The interaction intensity of Pol II between promoters and 150Kb–200Kb regions around TSS. Pol II and promoter interaction intensity were from ChIA-PET data in ENCODE database. The violins were drawn with middle dots denoting the mean. P values were calculated by Student’s t-test, two-sided. n = 929 (A549 TF), 17 796 (A549 TT), 775 (H1 TF), 24 903 (H1 TT), 1813 (HCT116 TF), 16 169 (HCT116 TT), 2696 (HEK293T TF), 16 717 (HEK293T TT), 1727 (HepG2 TF), 14 570 (HepG2 TT), 1077 (GM12878 TF), 20 142 (GM12878 TT), 1103 (K562 TF), 21 162 (K562 TT), 1475 (MCF7 TF), and 19 382 (MCF7 TT). (E) The ratio of enhancer–gene pairs located in the same TAD. Values from the same cell line were lined up. P values were calculated by Student’s t-test, two-sided. n = 5 in each box. Pred positive, enhancer–gene pairs where genes were predicted to be regulated by corresponding enhancers. Pred negative, enhancer–gene pairs where genes were predicted to not be regulated by corresponding enhancers. (F–J) The distribution of distance between enhancer–gene pairs and TAD size in K562 (F), A549 (G), GM12878 (H), gastrocnemius medialis (I), and transverse colon (J) cells. The top 2000 enhancer–gene pairs with high fold change and q-value below 0.1 were selected as positive pairs, and the remaining pairs containing the same enhancers were considered negative ones. (K) The illustration showing the CatLearning revealed the regulation of enhancer–gene pairs.

We then hypothesized that the reduction in precision could be attributed to the interactions of chromatin. The most available Hi-C datasets have relatively low resolution (40Kb to 1Mb) [21, 38], so we utilized Pol II ChIA-PET data [39] to calculate chromatin interactions between gene promoters and regions from 20Kb to 50Kb and from 150Kb to 200Kb around TSS. The TT genes showed higher interactions than the TF genes in almost all cell lines with publicly available Pol II ChIA-PET data (Fig. 4C and D). Consistent with these observations, enhancers were highly enriched around 20Kb to 50Kb from the TSS and were reported to regulate only proximal and far distal regions [37, 40] (Supplemental Fig. S12A). To further validate how CatLearning learned chromatin interaction features, we used it to predict gene expression upon disrupting enhancers. In previous research, enhancers and their regulated genes were detected through enhancer inactivation [37]. H3K27ac is a well-known indicator of enhancer activity, so we used a series of reduction values to simulate its decrease and mimic enhancer inactivation. The inactivation of enhancers is localized to the enhancers themselves, with no alteration in the surrounding enrichment. Setting the value of H3K27ac to specific levels does not adequately reflect the relationship between enhancers and the surrounding region. Therefore, we reduced the enrichment as fold change to more accurately simulate the inactivation of enhancers. The prediction results showed that around 85% of previously identified enhancers and their regulated genes were also recognized as regulatory pairs by CatLearning (Supplemental Fig. S12B). Moreover, the expression of enhancer-regulated genes was predicted to decrease when enhancers were simulated to be inactivated (Supplemental Fig. S12C). The observed reduction in gene expression was less than the predicted decrease, likely due to incomplete inactivation of enhancers during the biological experiments.

When we simulated H3K27ac to 0.1 fold of that in wild-type (WT) cells, the decrease in enhancer-associated genes reached an inflection point (Supplemental Fig. S12C). We then used 0.1 fold for the subsequent simulation of enhancer inactivation. We expanded our prediction to additional cell lines with available Hi-C seq data [38]. Hi-C seq helps identify topologically associating domains (TADs), which are self-interacting genomic regions. Inside TADs, DNA sequences have a higher frequency of physical interactions compared to those outside of TADs. We found that the regulated enhancer–gene pairs identified by CatLearning were mainly located within one TAD (Fig. 4E). Additionally, the distance distribution between predicted enhancer–gene pairs was similar to the distribution of TAD length (Fig. 4F–J). The size of enhancer–gene pairs was slightly smaller than the size of TADs. Furthermore, the ratio of positive to negative enhancer numbers was similarly distributed as TAD length distributions (Supplemental Fig. S12D). These data indicate the enhancer–gene pairs are located at the boundaries of TADs. When chromatin loops to form TADs, the boundaries fold together to facilitate close interaction, allowing enhancers to regulate genes from a far distance (Fig. 4K). Supporting this hypothesis, the interaction intensity of the positive enhancer-gene pairs was higher than that of the negative pairs (Supplemental Fig. S12E). Together, these findings suggest that CatLearning learns about chromatin interaction features and can predict the expression of genes by simulated changes in histone modifications at enhancers.

CatLearning utilizes simulated histone changes throughout the genome to predict gene expression

Since alterations in histone marks can lead to changes in cell fate, we aim to predict how gene expression will be affected when there are disturbances in global histone marks. Previous studies have conducted several experiments to profile the alternations of gene expression and histone marks by depleting or knocking out enzymes responsible for histone modification. Disrupting the enzyme would eliminate corresponding histone marks across the genome, resulting in a low enrichment of histone marks throughout the genome. To simulate epigenome data after enzyme disruption, we manually reduced target histone mark levels, such as H3K36me3 in SETD2 depletion, to 0.01. We first utilized CatLearning to predict whether genes were up- or down-regulated due to the changes in histone marks. Our results showed that CatLearning yielded high predictive accuracy using both experimental sequencing data (mean ACC = 86.86%; mean AUC = 0.94) and our simulated data (mean ACC = 84.19%; mean AUC = 0.94) (Fig. 5A and Supplemental Fig. S13A). We then applied CatLearning to predict gene expression values in the regression model. The accuracy of the predictions was slightly lower but less variable compared to the observed experiment replicates (Fig. 5B–D and Supplemental Fig. S13B). Importantly, CatLearning showed comparable performance when tested on both sequencing data from biological experiments and computationally simulated data. Furthermore, the predicted gene expression changes, whether from experiment data or simulated data, were strongly correlated to experimental changes, with mean Pearson correlation R = 0.62 and 0.60, respectively (Fig. 5E–G and Supplemental Fig. S13C).

Figure 5.

CatLearning predicts gene expression with simulated histone changes throughout the genome. (A) Box blot showing the binary ACC of observed and predicted gene expression changes. The observed or predicted FPKM in histone mark disturbed cells compared to the observed FPKM in WT cells were calculated. Genes were then classified as increased or decreased. Obs_rep, the AUC was calculated by the observed expression between different replicates. Pred, the AUC was calculated by the predicted gene expression from observed histone marks. Pred simulation, the AUC was calculated by the predicted gene expression from the simulated depletion of histone marks. The boxes were drawn from lower quartile (Q1) to upper quartile (Q3) with the middle line denoting the median, and whiskers with maximum 1.5 IQR. P values were calculated by Student’s t-test, two-sided. n = 22 (Obs rep), 6 (Pred), and 8 (Pred simulation). (B) Same as (A), except the ACC in the regression model was calculated. n = 22 (Obs rep), 15 (Pred), and 16 (Pred simulation). (C) Scatter plot showing the observed and predicted FPKM in SETD2-depleted HepG2 cells. The prediction was conducted using the experimental sequencing results. R, correlation coefficients that were assessed by Pearson product–moment correlation. The two dashed lines represented a two-fold change in observed FPKM. (D) Same as (C), except simulated histone mark (H3K36me3) signals were used. In SETD2-depleted cells, H3K36me3 was set to 0.01. (E) The fold changes of gene expression between observed values. R, correlation coefficients that were assessed by Pearson product–moment correlation. The color of the dots represented the density of the dots. (F) Same as (E), except the fold changes of predicted gene expression from experimental histone mark signals were plotted. (G) Same as (E), except the fold changes of predicted values from simulated histone mark signals were plotted. In SETD2-depleted cells, H3K36me3 was set to 0.01. (H) Violin plot showing the average fold changes of the predicted FPKM in histone mark disturbed cells and the predicted FPKM in WT cells. Twenty cell types were used for prediction. The boxes in violins were drawn from the lower quartile (Q1) to the upper quartile (Q3) with the middle dots denoting the median value. K9, H3K9me3. K27, H3K27me3. K4, H3K4me3. K27ac, H3K27ac. K36, H3K36me3. n = 20 in each violin plot.

To further demonstrate the function of different histone marks, we simulated the removal of various histone marks and predicted gene expression changes using CatLearning. Additionally, we analyzed how gene expression was affected by simultaneously disturbing an active histone mark and an inactive histone mark (Fig. 5H). Consistent with previous findings, the removal of active histone marks resulted in decreased gene expression. Specifically, the depletion of H3K27ac had the most significant impact on reducing gene expression, while the loss of H3K4me3 had little effect on gene expression. Furthermore, the removal of inactive histone marks led to increased gene expression. When combined with H3K27ac, the additional removal of inactive histone marks such as H3K9me3 and H3K27me3 had a minimal effect. These findings indicate that H3K27ac plays a crucial role in active gene expression, while H3K4me3 serves as an indicator of active gene expression rather than a regulator of gene expression. Together, our findings suggest that CatLearning can predict gene expression by computationally simulated histone modification changes and decode the orchestration of histone marks.

Discussion

A persistent challenge in the field of epigenetics is predicting gene expression based on histone marks. Histone mark is a type of “code,” so-called histone code, that helps regulate gene expression [9, 41]. Understanding how countless combinations of histone marks determine gene expression remains a major unsolved challenge. Accurately predicting gene expression from histone marks is crucial to advance research and develop treatments for human diseases that involve epigenetic changes. To overcome this challenge, we have developed a method called CatLearning that uses a modified CNN with a specialized adaptation ResNet to quantitatively interpret histone codes and predict gene expression. Before training, genes with different expression levels are separated and the model architecture ensures the capture of features from histone marks with varying enrichment scales in a large genomic region, such as enhancer and their regulated genes. This is achieved despite not utilizing 3D information. Our approach has led to successful performance in predicting gene expression from histone marks.

CatLearning is an outstanding model for predicting gene expression in two significant biological issues: enhancer alteration and histone mark depletion. These two applications involve the alteration of histone marks either locally or genome-wide. Our model is capable of predicting gene expression from changes in histone marks, regardless of whether they are site-specific or genome-wide. We observed that CatLearning accurately predicted gene expression by paying attention to enhancers and 3D chromatin structures, which suggests that it has learned enhancer–gene distal regulation patterns. Hi-C sequencing data are unnecessary for learning chromatin architecture and are hindered by low resolution and availability. Previous methods separate the training and testing data from the same sequencing result, so the prediction can only detect a subset of genes and the training data are largely limited. We used the ACC metric to measure the accuracy of our predictions. Unlike Pearson correlations, which can be influenced by data with high values, ACC is less affected by outliers. Additionally, we calculated correlations without log conversions to ensure strict quality control measures. Together, these new advances and benefits provide fresh opportunities for investigating the expanding range of histone marks associated with diseases and development.

The genome’s 3D interactions are mainly organized into TADs [42]. Enhancer–gene interactions occur within the same TAD [43–45], with enhancers influencing distant genes by creating chromatin loops that bring them close to their target promoters [46]. TAD boundaries help to isolate these domains. However, chromosome conformation studies have shown that a significant fraction of these interactions also occur across TAD boundaries. Up to a third of all long-range promoter interactions identified by Promoter Capture Hi-C in mice and humans are established across TAD boundaries [47, 48]. Our study aligns with these results, revealing that ~65% of the regulated enhancer–gene pairs identified by CatLearning were mainly located within one TAD (Fig. 4E), while ~35% of enhancer–gene pairs are established across TAD boundaries. Notably, enhancers are also found close to TAD boundaries [49]. Furthermore, TAD boundaries display enhancer-like features [50, 51]. Factors associated with active promoters and gene bodies, such as H3K4me3, H3K36me3, H3K27ac, and RNA Pol II, were enriched around boundary regions in both mice and humans [52, 53]. Moreover, analysis of promoter capture-seq data in HeLa cells revealed that boundaries were captured by promoters far more than by random regions. Genes interacting with boundaries were relatively more active than genes interacting with other genomic regions [54]. Together, these observations imply a connection between TAD boundaries and enhancer–gene interaction. TAD boundaries indicate the presence of active regulatory elements and transcription [55]. Consistent with these findings, our results also found the enhancer–gene pairs tend to be located at the boundaries of TADs. The distance distribution between predicted enhancer–gene pairs is similar to TAD length. Similar observations have been documented in Drosophila and mice. Hi-C studies in Drosophila have identified ~1300 TADs with an average size of around 100Kb [56]. In addition, the median length of the enhancer–gene inter-TAD interactions was 77.5Kb, which is slightly less than the average TAD size [57]. In the high-resolution Hi-C map of mouse cortex data, TADs with a median size of 185Kb were detected [56]. Additionally, 49% of the predicted enhancers were located within 120Kb of their target promoters, which is slightly less than the median TAD size [58]. The specifics of how enhancers and genes interact within the boundaries of TADs have not been thoroughly studied. However, our results are consistent with previous research and offer new insights into the role of TAD boundaries in promoting interactions that regulate genes. These findings encourage further investigation into the intricate mechanisms that govern long-range enhancer–promoter interactions and how they regulate gene expression.

CatLearning has several advantages in treating the input data to improve performance: (i) During the training process, genes are classified into different subgroups based on their expression levels. The deep learning model’s ability to learn is greatly diminished when the input data have a wide range. To overcome this challenge, genes are grouped based on their similar expression levels. This approach helps reduce the range of data and improve the learning efficiency of the model. Additionally, this categorization technique can be applied to other prediction algorithms to improve accuracy. (ii) Genes that exhibit high variability across sequencing experiments are depleted. These genes are not stably detected in either gene expression or histone mark sequencing, suggesting that they are very dynamic. Therefore, it is not appropriate to predict their expression based solely on the detected histone marks. (iii) Histone marks in a large range (500Kb) around the TSS are used for training. Although most of the histone marks are found near the TSS, some enrichments are also found in regions far from the TSS, such as enhancers in intergenic regions. (iv) Individual input modeling: The model is designed to individually process each input, ensuring that the significance of each component feature is preserved. This approach, which is often overlooked by other models, allows the deep learning model to capture the unique characteristics and nuances of each input before integrating them. As a result, it leads to a more accurate and comprehensive prediction by maintaining the distinct contributions of each feature. (v) The CNN with a specialized adaptation ResNet is used to build the model. The advantage of our model lies in the use of CNN, which can capture both local and global input data features automatically [17]. The CNN also shares the weight feature, which reduces the number of trainable network parameters, thereby improving generalization and preventing overfitting. Furthermore, our model makes use of multi-scale ResNet, which allows it to learn features from different histone marks that exhibit diverse distributions and enrichment scales. The model employs a multi-scale feature extraction method for each input. This technique enables the model to capture features at various scales, ensuring that both fine-grained and coarse-grained details are effectively extracted. The model can better understand and represent the complex patterns within the data, enhancing its overall predictive performance.

There are various ways to enhance the accuracy of the model. Machine learning and deep learning success depends on the training data. Increasing the sequencing quality of the histone marks, and curating data from additional histone marks, would likely boost the performance. A limitation of the current approach is that we can model and predict only for training cells and the model is not built ubiquitously for every cell. The prediction accuracy decreases when CatLearning is applied to novel cell types not used for training (Supplemental Fig. S14A and B), indicating the need for additional information to enhance its applicability. To further analyze how transcription factors are involved in tissue-specific gene expression, we add transcription factors to train the model. Five transcription factors (JUND, CEBPB, MYC, NRF1, and SRF) in five cell types (GM12878, H1, MCF7, HepG2, and K562) are used. During training, the inclusion of transcription factors decreases prediction accuracy, and this impact is specific to different tissue types (Supplemental Fig. S15A). This observation suggests that the binding of transcription factors is regulated by factors other than histone marks and is specific to each tissue, as different transcription factors exhibit similar correlations with histone marks across various cell lines (Supplemental Fig. S15B). CatLearning may learn the chromatin architecture that was different among various cell lines and tissues. Recent works demonstrate several strategies to improve the resolution of chromatin structure capture, and more 3D chromatin information is profiled in different cell lines [59]. By utilizing high-resolution 3D information and combining learning models like the transformer, it’s possible to create a universal model for all cell types. In addition, integrating other epigenetic information may aid in constructing this universal model. The model’s sensitivity to histone perturbation could be enhanced by incorporating the growing number of epigenetic sequencing results, such as histone modifier knockout, depletion, and inhibition. By making use of the learning architecture in the future, our model builds a foundation for the decoding of histone marks.

In the future, CatLearning could be systematically applied to determine which histone marks at specific loci are critical for cell fate determination, such as tumorigenesis, drug resistance, Induced pluripotent stem cells (iPSC) generation, and directed differentiation. By understanding the role of histone marks in gene expression, valuable time and effort can be saved for actual experiments. To facilitate downstream applications, we have made the pretrained model openly available and included code examples that demonstrate how to use it. We aim to improve the understanding of histone mark architecture and facilitate the development of therapeutic tools for diseases characterized by epigenetic changes.

Key Points

• CatLearning predicts gene expression changes by simulated histone changes at enhancers and throughout the genome. This is very useful when researchers want to know how gene expression will change upon altering histone marks.

• Catlearning shows superior performance in predicting expression value.

• Catlearning achieves high accuracy by utilizing only one histone mark.

• Catlearning learns chromatin interaction features without utilizing 3D information.

Funding

This research was partly supported by grants from the National Key R&D Program of China (No. 2022YFA1302800), the National Natural Science Foundation of China (Grant Nos. 32361133547, 32370613, 32222017, and 81874153), and the Fundamental Research Funds for the Central Universities (Grant No. 2019QN81005). This research was supported by Zhejiang Provincial Natural Science Foundation of China under Grant No. LZ24C060001, and the Opening Research Fund from Shanghai Key Laboratory of Stomatology, Shanghai Ninth People’s Hospital, College of Stomatology, Shanghai Jiao Tong University School of Medicine (Grant No. 2022SKLS-KFKT002).

Code Availability

The source code for CatLearning was available at the GitHub repository (https://github.com/Fanglab-zju/CatLearning). Pretrained weights for CatLearning were available at https://drive.google.com/file/d/1957CCAjaHfp6_bJ_yiZ_6ebqhhleWW5F/view?usp=sharing. Code implementations for benchmark models were downloaded from the respective code repositories: chromoformer (https://github.com/dohlee/chromoformer) GC-MERGE (https://github.com/rsinghlab/GC-MERGE), DeepChrome (https://github.com/QData/DeepChrome), and AttentiveChrome (https://github.com/QData/AttentiveChrome).

Data Availability

Raw histone ChIP-seq and RNA-seq data were collected from both of ENCODE and GEO database (Supplementary Tables S1 and S2). NCBI RefSeq gene annotations were downloaded from UCSC Table Browser (https://genome.ucsc.edu/cgi-bin/hgTables). Normalized interaction frequencies for promoter-centered Hi-C experiments were obtained from the preprocessing script of Dohoon et al. [20]. (https://dohlee-bioinfo.sgp1.digitaloceanspaces.com/chromoformer-data/). The accession code under ENCODE database of RNAPII ChIA-PET data were ENCFF421KYP, ENCFF913VWM, ENCFF753NSM, ENCFF002YGU, ENCFF125JGW, ENCFF364UNM, ENCFF511QFN, and ENCFF597SQA. TAD data were also downloaded from ENCODE database with the accession code ENCFF126GED, ENCFF318QPG, ENCFF336WPU, ENCFF395LRK, and ENCFF531LSJ. #Source data are provided with this paper.

Author Contributions

W.L., Y.T., Y.L., Y.C., and D.F. conceived the project. W.L. built the model. Y.T. and Y.L. performed the bioinformatics analysis. B.L. and R.Z. consulted the project. Y.T., Y.L., Q.S., and D.F. analyzed the data. Y.T., Y.L., S.L., and D.F. prepared the first draft and revision.