Human exons and introns classification using pre-trained Resnet-50 and GoogleNet models and 13-layers CNN model

Abstract

Background:

Examining functions and characteristics of DNA sequences is a highly challenging task. When it comes to the human genome, which is made up of exons and introns, this task is more challenging. Human exons and introns contain millions to billions of nucleotides, which contributes to the complexity observed in this sequences. Considering how complicated the subject of genomics is, it is obvious that using signal processing techniques and deep learning tools to build a strong predictive model can be very helpful for the development of the research of the human genome.

Results:

After representing human exons and introns with color images using Frequency Chaos Game Representation, two pre-trained convolutional neural network models (Resnet-50 and GoogleNet) and a proposed CNN model having 13 hidden layers were used to classify our obtained images. We have reached a value of 92% for the accuracy rate for Resnet-50 model in about 7 h for the execution time, a value of 91.5% for the accuracy rate for the GoogleNet model in 2 h and a half for the execution time. For our proposed CNN model, we have reached 91.6% for the accuracy rate in 2 h and 37 min.

Conclusions:

Our proposed CNN model is faster than the Resnet-50 model in terms of execution time. It was able to slightly exceed the GoogleNet model for the accuracy rate value.

1. Background

The study of all living organisms’ DNA sequences is known as genomics, which is a branch of biology.¹ A cell’s genome serves as a representation of every genetic piece the cell contains. Genes linked to diseases can be found thanks to genomics, which also helps us better understand the variety of living things.² It examines the organization and structure of genomes (or genes), as well as how they operate.

The human cell’s genome is made up of the mitochondrial DNA and the 23 pairs of chromosomes that constitute the nucleus.³ The four nucleotides (or letters) that represent human DNA are A for adenine, C for cytosine, G for guanine, and T for thymine.⁴ A segment of this DNA called a gene is a unit of genetic information that codes for a single protein. Coding sections (also known as exons), non-coding regions (also known as introns), intergenic regions, repetitive sequences, and others compose human genes.⁵

Coding sequences or exons are DNA sequences that are translated into mRNA and subsequently transformed by the proper mRNA molecules into a polypeptide chain. A codon, often known as a set of three nucleotides, in a coding sequence codes for one amino acid in a polypeptide chain. Only about 2% of DNA is composed of coding genes; the other 98 percent is made up of non-coding regions.⁶

According to research published within the context of the ENCODE project,⁷ which aims to identify the activities of every component of our genome, non-coding DNA or introns, which was wrongly referred to in the past as junk DNA, is important in the regulation of gene activity and the emergence of disorders.⁸

Genome mapping, finding new genes, and examining how genes are expressed and controlled are just a few of the analyses that form genomics.⁹ Computers are essential in these analyses because specialized software makes it easier to classify genes based on how similar their sequences are.¹⁰ Given the vast volume of data, genomics stores and analyzes data using bioinformatics and deep learning methods.¹¹ It is in this context that our work is situated.

In this work, we will represent human exons and introns with color images. Then, we will use two pre-trained convolutional neural network (CNN) models (resnet50 and googleNet) and we will propose a simplified CNN model for the classification of images representing coding and non-coding regions of human genes. In Section 2, we will introduce deep learning and convolutional neural network models and describe the architecture of the used pre-trained CNN models. Then, in Section 3, we will describe our methodology on representing coding and non-coding regions of human genes with RGB images and classifying them using 3 CNN models. In Section 4, we will apply ResNet50 model, googleNet model and our proposed CNN model to our constructed images and expose obtained classification results. Finally, in Section 5, we will conclude our work.

2. Deep learning and convolutional neural networks

Deep learning algorithms build a model based on the features they extract from big datasets, such as a collection of photos or genomes. Algorithms can use the model once it has been created to analyze different data sets.¹²

Deep learning is applied in a variety of fields, including bioinformatics,¹³ genomics,¹⁴ robotics,¹⁵ language processing,¹⁶ image recognition,¹⁷ voice recognition,¹⁸ and technology for diagnostic support in the medical field.¹⁹.

2.1. Convolutional neural network (CNN)

Convolutional neural networks (CNNs) are a subset of artificial neural networks used in image processing and recognition. A multilayered perceptron-like system designed for low processing demands is used by CNNs. Several convolutional layers, grouping layers, fully connected layers, and normalization layers make up the input, output, and hidden layers of a CNN. By removing constraints and enhancing image processing efficiency, a system for image processing and natural language processing is created that is significantly specialized, more efficient and easier to train.²⁰

2.1.1. Convolution layer

The first layer of a convolutional neural network is always the convolution layer. The main objective of this layer is to recognize a set of features in incoming images that are represented (Fig.1). Convolution filtering is employed in this process; the goal is to form a window across the image designating the feature and compute the convolution product between the feature and each segment of the scanned image,²¹ as shown in the following equation.

$$Y_j^l=f\left[{\displaystyle \sum _{i\in I_j}}(Y_j^{l-1}*K_{\mathit{ij}}^l+b_j^l)\right]$$ (1)

Where:

• •$I^j$ represents the input image set.
• •$Y_j^l$ represents the $j^{\mathit{th}}$ feature map of the l layer.
• •The operator ”*” represents the operation of convolution.
• •$Y_i^{l-1}$ is the $i^{\mathit{th}}$ feature map of the $l-1$ layer.
• •$K_{\mathit{ij}}^l$ represents the filter connecting the $j^{\mathit{th}}$ feature map of the l layer and the $i^{\mathit{th}}$ feature map of the $l-1$ layer.
• •$b_j^l$ is the bias.
• •$f[.]$ represents the nonlinear activation function capable of resolving issues that linear models are unable to.

Fig. 1.

Convolution layer.

Common activation methods include sigmoid, tanh, ReLU, etc. The corresponding calculation equations are as follows:

$$\mathrm{Sigmoid}:f(x)=\frac{1}{1+e^{-x}}$$ (2)

$$\mathrm{Tanh}:f(x)=\frac{e^x-e^{-x}}{e^x+e^{-x}}$$ (3)

$$\mathrm{ReLU}:f(x)=\mathit{max}(0,x)$$ (4)

Convolution layer volume is sized using three hyper parameters: layer depth, pitch, which regulates receptive field overlap, and zero padding.

2.1.2. Pooling layer

Usually placed between two convolution layers, this type of layer performs the pooling function on each feature map it receives as input.²² In the pooling process, images are reduced in size while retaining their essential elements (Fig.2).

Fig. 2.

Pooling layer.

The pooling layer can be used to minimize the amount of parameters and calculations in the network. By doing this, overfitting is avoided and the network’s effectiveness is increased.

2.1.3. ReLU layer

A layer that will apply an activation function to the output signals is introduced between the processing layers to speed up processing. Rectified Linear Unit (ReLU), a nonlinear activation layer used in convolution maps, converts the negative integers in the filtered images to zeroes.²³

2.1.4. Fully connected layer

A neural network’s top layer is always the fully connected layer, which accepts a vector as input and produces another vector. This is accomplished by applying an activation function and may be a linear combination to the input data (Fig.3).

Fig. 3.

Fully connected layers.

2.2. Pre-trained Convolutional Neural Networks

A pre-trained model is a model or a saved network developed by another party and trained on a sizable dataset to solve a related problem. Instead of creating a model from scratch, data scientists can employ a pre-trained model as a starting point. There are many pre-trained CNN models such as VGG16, VGG19, Alexnet, EfficientNet, GoogleNet (or inceptionv3) and resnet50.²⁴

2.2.1. Resnet50 architecture

Resnet, short for residual network, is a particular variety of convolutional neural network that He.K. et al. first introduced in 2015. ²⁵ resnet50 is a 50-layer CNN, it builds networks by stacking residual blocks.

The resnet architecture follows two fundamental design principles. First, regardless of the size of the output feature map, there is the same number of filters in each layer. Second, to maintain the time complexity of each layer when the size of the feature map is halved, it has twice as many filters. The building block for resnet employs a bottleneck design, which lowers the amount of parameters and matrix multiplications. This makes training each layer considerably faster. Instead of using two layers, it employs a stack of three layers (Fig.4.

Fig. 4.

The Resnet-50 architecture.

The pretrained network is capable of classifying photos into 1,000 different object categories, including various animals, a keyboard, a mouse, and a pencil. Hence, the network has picked up detailed feature representations for a variety of images. The network can accept images up to 224 by 224 pixels.

2.2.2. GoogleNet (Inception V1) architecture

Google Research suggested googlenet in 2014 (with the assistance of several institutions).²⁶ This architecture employs methods like global mean pooling and $1\times 1$ convolutions in the middle of the architecture. There are 22 layers in the entire architecture. It can be used on individual devices even with limited processing capabilities because it was built with computational economy in consideration (Fig.5.

Fig. 5.

The GoogleNet architecture.

This architecture uses an image with RGB color channels that is $224\times 224$ in size. This architecture uses rectified linear units as the activation function for all convolutions. The network was trained on two different image datasets: imageNet database and Places365 database.

3. Methods

3.1. Methodology

Our methodology consists of five steps; first, to construct our images dataset, we convert nucleotide sequences of exons and introns into images using frequency chaos game representation. Second, we classify obtained images using Resnet-50 model, then we classify them using GooleNet model. After that, we propose our simplified CNN model and execute it to classify our images. Finally, we compare the results of the three applied CNN models (Fig.6).

Fig. 6.

Methodology Steps.

3.2. Dataset Construction

To construct our dataset, we have randomly downloaded 110 biological sequences of human genes, from the National Center for Biotechnology Information (NCBI) database [https://www.ncbi.nlm.nih.gov/]. Knowing the positions of exons in each gene, we have divided human genes to two classes: 1860 exons and 1631 introns and we have chosen only 1600 for each class for the image representation and classification (Table 1), then we have represented those exons and introns by RGB images.

Table 1.

Human exons and introns dataset details.

Genes’ number	110
Exons’ number	1860
Exons’ medium length	500 bp
Introns’ number	1631
Introns’ medium length	2000 bp

We have decided to represent our exons and introns by the Frequency Chaos Game Representation (FCGR) since the lengths of exons and introns are different. This representation allows us to visually group the images of words of the same suffix in dials. We note that by adding one more letter, the points of the images of any word are found in the same quarter of the figure (Fig. 7).

Fig. 7.

Frequency Chaos Game Rrepresentation by dials.

Our constructed images are respectively the 4th, 5th and 6th orders of Frequency Chaos Game Representation (FCGR) used in our previous work²⁷ shown in Fig. 8.

Fig. 8.

Constructed images, (a): FCG Representation of the first exon of ACE human gene, (b): FCG Representation of the first intron of ACE human gene.

Images in Fig. 8 represent human gene ACE. (a): represents the 4^th, 5^th and 6^th orders of the first exon. (b): represents the 4^th, 5^th and 6^th orders of the first intron.

3.3. Dataset

Our final dataset contains three sets of images; each set is split into 2 classes: exons and introns.

Set A: this set holds images representing the $4^{\mathit{th}}$ order of the FCGR. There are 3200 images divided into 1600 images representing exons and 1600 images representing introns. The size of each image is $224\times 224\times 3$.

Set B: this set includes images representing the $5^{\mathit{th}}$ order of the FCGR. There are 3200 images divided into 1600 images representing exons and 1600 images representing introns. The size of each image is $224\times 224\times 3$.

Set C: this set comprises images representing the $6^{\mathit{th}}$ order of the FCGR. There are 3200 images divided into 1600 images representing exons and 1600 images representing introns. The size of each image is $224\times 224\times 3$.

4. Results

In this section, we will expose classification results for the Resnet-50 model, the GoogleNet model and our proposed CNN model.

4.1. Classification results for Resnet-50 model

We remind that the Resnet-50 model is a pre-trained convolutional neural network with 50 layers depth and can classify up to 1000 class (object image). To adapt this model to our classification problem, we have frozen the weights of the ten first layers and we have added one fully connected layer at the end, having 2 neurons since we have a 2-class classification problem. For the classification, we have divided each set into 70% for the training phase and 30% for the test phase and we have trained the Resnet-50 model for only 10 epochs.

In Table 2, we illustrate training rate, accuracy rate values and the execution time of the three sets classified by Resnet-50 model:

Table 2.

Classification results of Resnet-50 model.

	Training rate	Accuracy	Execution Time
Set A	90.21%	90.9%	7 h 07 min 24 s
Set B	90.42%	92%	6 h 54 min 13 s
Set C	89.58%	89.4%	6 h 55 min 03 s

When we classified our images with Resnet-50 model, we have obtained: for set A (the $4^{\mathit{th}}$ FCGR order), a value of 90.21% for the training rate, a value of 90.9% for the accuracy rate and 7 h 07 min 24 s for the execution time. For set B (the $5^{\mathit{th}}$ FCGR order),a value of 90.42% for the training rate, a value of 92% for the accuracy rate and 6 h 54 min 13 s for the execution time. For set C (the $6^{\mathit{th}}$ FCGR order),a value of 89.58% for the training rate, a value of 89.4% for the accuracy rate and 6 h 55 min 03 s for the execution time.

When we visualize Fig. 9, we can notice that the Resnet-50 model, when it was applied on set A, it was capable of predicting 87.3% of exons and 94.6% of introns, for set B, 90% of exons were correctly identified and 94% of introns were correctly identified and for set C 84.8% of exons were correctly recognized and 94% of introns were identified correctly.

Fig. 9.

Confusion matrices for Resnet-50 model. (a): set A, (b): set B, (c): set C.

4.2. Classification results for GoogleNet model

We remind that the GoogleNet model is a pre-trained convolutional neural network with 22 layers depth and can classify up to 1000 class (object image). To adapt this model to our classification problem, we have frozen the weights of the ten first layers and we have added one fully connected layer at the end, having 2 neurons since we have a 2-class classification problem. For the classification, we have divided each set to 70% for the training phase and 30% for the test phase and we have trained the GoogleNet model for only 10 epochs.

In Table 3, we illustrate training rate, accuracy rate values and the execution time of the three sets classified by GoogleNet model:

Table 3.

Classification results of GoogleNet model.

	Training rate	Accuracy	Execution Time
Set A	91.46%	91.5%	2 h 28 min 23 s
Set B	89.06%	89.1%	2 h 23 min 37 s
Set C	89.27%	89.3%	2 h 26 min 59 s

When we classified our images using GoogleNet model, we have reached: for set A (the $4^{\mathit{th}}$ FCGR order), a value of 91.46% for the training rate, a value of 91.5% for the accuracy rate and 2 h 28 min 23 s for the execution time. For set B (the $5^{\mathit{th}}$ FCGR order),a value of 89.06% for the training rate, a value of 89.1% for the accuracy rate and 2 h 23 min 37 s for the execution time. For set C (the $6^{\mathit{th}}$ FCGR order),a value of 89.27% for the training rate, a value of 89.3% for the accuracy rate and 2 h 26 min 59 s for the execution time.

When we observe Fig. 10, we can notice that the GoogleNet model, when it was applied on set A, it was able to predict 86.7% of exons and 96.3% of introns, for set B, 82.7% of exons were correctly identified and 95.4% of introns were correctly identified and for set C 90.4% of exons were correctly recognized and 88.1% of introns were identified correctly.

Fig. 10.

Confusion matrices for GoogleNet model. (a): set A, (b): set B, (c): set C.

4.3. Proposed model

With the aim of improving classification rates, we have tested many CNN architectures and we ended up with this model. Our proposed CNN model is a 13 layers depth model having 2 convolution layers, 2 MaxPooling layers, 2 batch Normalization layers, 1 relu layer, 4 fully connected layers, 1 softMax layer and 1 classification layer as shown in Table 4:

Table 4.

Our proposed CNN model architecture.

Input Layer
$1^{\mathit{st}}$Convolution	Filters number	32
Layer	Filters size	$7\times 7$
	Padding	7
$1^{\mathit{st}}$MaxPooling	Pool size	$7\times 7$
Layer	Padding	7
$2^{\mathit{nd}}$Convolution	Filters number	16
Layer	Filters size	$5\times 5$
	Padding	5
$2^{\mathit{nd}}$MaxPooling	Pool size	$5\times 5$
Layer	Padding	5
Fully	$1^{\mathit{st}}$ layer neurons’ number	512
Connected	$2^{\mathit{nd}}$ layer neurons’ number	256
Layers	$3^{\mathit{rd}}$ layer neurons’ number	218
	$4^{\mathit{th}}$ layer neurons’ number	2
Classification Layer
Epochs Number		10
Training Dataset		70%
Test Dataset		30%

We have chosen to do classification without overlap, we have divided our images into 70% for the training phase and 30% for the test phase and we have trained our model during 10 epochs.

4.4. Classification results for our proposed CNN model

In Table 5, we represent training rate, accuracy rate values and the execution time of the three sets classified by our proposed CNN model:

Table 5.

Classification results of our proposed CNN model.

	Training rate	Accuracy	Execution Time
Set A	91.46%	91.6%	2 h 37 min 55 s
Set B	87.81%	89.7%	2 h 35 min 53 s
Set C	89.06%	89.3%	2 h 38 min 19 s

When we used our proposed CNN model to classify our images, we have reached: for set A (the $4^{\mathit{th}}$ FCGR order), a value of 91.46% for the training rate, a value of 91.6% for the accuracy rate and 2 h 37 min 55 s for the execution time. For set B (the $5^{\mathit{th}}$ FCGR order),a value of 87.81% for the training rate, a value of 89.7% for the accuracy rate and 2 h 35 min 53 s for the execution time. For set C (the $6^{\mathit{th}}$ FCGR order),a value of 89.06% for the training rate, a value of 89.3% for the accuracy rate and 2 h 38 min 19 s for the execution time.

When we observe Fig. 11, we can notice that our proposed CNN model with only 13 layers depth, when it was applied on set A, it was able to predict 90.6% of exons and 92.5% of introns, for set B, 94.8% of exons were correctly identified and 84.6% of introns were correctly identified and for set C 87.3% of exons were correctly recognized and 91.3% of introns were identified correctly. (see Fig. 11).

Fig. 11.

Confusion matrices for our proposed CNN model. (a): set A, (b): set B, (c): set C.

5. Discussion

When classifying our images with Resnet-50 model, we have obtained a maximum training rate for the set B with a value of 90.42% and an accuracy value of 92% for the same set and the minimum execution time is about 7 hours and 30 minutes.

This Resnet-50 model was able to correctly classify 94% of introns for the three sets of images and 90% of exons for set B.

When classifying our images with GoogleNet model, we have obtained a maximum training rate for the set A with a value of 91.46% and an accuracy value of 91.5% for the same set and for the minimum execution time, it is about 2 hours 38 minutes and 37 seconds for the set B.

The GoogleNet model was capble of predicting correctly 96.3% of introns for set A and 90.4% of exons for set C.

When classifying our images with our proposed CNN model, we have obtained a maximum training rate for the set A with a value of 91.46% and an accuracy value of 91.6% for the same set and for the minimum execution time, it is about 2 hours 58 minutes and 53 seconds for the set B.

Our 13 layers depth proposed CNN model was able to predict 92.5% of introns for set A and 94.8% of exons for set B.

Our obtained results are comparable with results in²⁸ where, based on a simplified version of ResNet, they suggested a novel model (S-ResNet). S-ResNet differs from Resnet in that it places a shortcut link at each layer of the convolution process rather than after a block made up of two convolution layers. They did classification of acceptor splice sites and donor splice sites using S-Resnet and compared it with other CNN models such as: AlexNet, VGG-Net and other CNN models. They reached an accuracy rate value of 93.5% for their proposed S-Resnet model. The results are also comparable with results in²⁹ where they preprocessed their exon and intron dataset using clustering technique and they proposed ANFIS model (a combination between an artificial neural network and a fuzzy inference system) for the classification process. They obtained a value of 88.88% for the accuracy rate on the test phase.

As shown in Table 6, our proposed CNN model has a high value of specificity, a value of 94.2%, this means that our model is capable of classifying introns correctly. While, the GoogleNet model was able to correctly classify exons for set A with a value of 95.9% of sensitivity. The S-Resnet model outperformed the other four models with a value of 93.5% of accuracy rate.

Table 6.

Comparison of results.

	Training rate	Accuracy	Execution Time
Model	Acc. rate	Sens.	Spec.
Resnet-50	92%	94.2%	90.4%
GoogleNet	91.5%	95.9%	90.2%
13-layers CNN	91.6%	92.4%	94.2%
S-Resnet28	93.5%	92.9%	93.8%
ANFIS29	88.88%	75%	100%

6. Conclusion

In this work, we have used CNN models to classify color images representing human exons and introns. We have constructed 3 datasets, each set contains 3200 images divided equally into 2 classes. Then, we have used two pre-trained models: Resnet-50 and GoogleNet to classify those images, we have obtained 92% for the accuracy rate for the resnet-50 model and an accuracy rate of 91.5% for the GoogleNet model. After that, we have proposed our CNN model with 13 layers, we have reached promising results, we have obtained a value of 91.6% for the accuracy rate.

On a future work, we want to enlarge our dataset and we want to apply other pre-trained models to classify human exons and introns.

7. Declarations

List of abbreviations

DNA: Deoxyribonucleic Acid.

FCGR: Frequency Chaos Game Representation.

CNN: Convolutional Neural Network.

Ethics approval and consent to participate

All human genes are provided by the National Center of Biotechnology Information.

Consent for publication

Our manuscript does not any individual personal information.

Availability of data and materials

Our data will not be shared because it’s just the application of coding technique on gene sequences. But human gene sequences are available on https://www.ncbi.nlm.nih.gov/

Funding

There is no source funding for this research paper.

Authors’ contributions

All authors have equally contributed to this research paper

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.