Gene mutation estimations via mutual information and Ewens sampling based CNN & machine learning algorithms

Abstract

We conduct gene mutation rate estimations via developing mutual information and Ewens sampling based convolutional neural network (CNN) and machine learning algorithms. More precisely, we develop a systematic methodology through constructing a CNN. Meanwhile, we develop two machine learning algorithms to study protein production with target gene sequences and protein structures. The core of the CNN and machine learning approach is to address a two-stage optimization problem to balance gene mutation rates during protein production. To wit, we try to optimally coordinate the consistency between the given input DNA sequences and the given (or optimally computed) target ones through controlling their intermediate gene mutation rates. The purposes in doing so are aimed to conduct gene editing and protein structure prediction. For example, after the gene mutation rates are estimated, the computing complexity of protein structure prediction will be reduced to a reasonable degree. Our developed CNN numerical optimization scheme consists of two newly designed machine learning algorithms. The stochastic gradients for the two algorithms are designed according to the Kuhn-Tucker conditions with boundary constraints and with the support of Ewens sampling, multi-input multi-output (MIMO) mutual information, and codon optimization techniques. The associated learning rate bounds are explicitly derived from the method and the two algorithms are numerically implemented. The convergence and optimality of the algorithms are mathematically proved. To illustrate the usage of our study, we also conduct a real-world data implementation.

1. Introduction

This study is motivated by applications in healthcare, biomedicine, and biomedical engineering (see, e.g. Dai [12], Ng et al. [38], Pardi et al. [39], Tu [48], and Yang et al. [54]). In this study, we address an optimally balanced gene mutation rate control problem during protein production, approached through a two-stage minimization-maximization process. Furthermore, the mutation rate can also be a scaled rate in terms of a scaled function (see, e.g. Section 3.1). After the gene mutation rates are estimated (or trained), the computing complexity of protein structure prediction will be reduced to a reasonable degree. The ‘optimal balance’ here means that we try to optimally coordinate the consistency between the given input DNA sequences and the given (or optimally computed) target ones through controlling their intermediate gene mutation rates. In other words, our stated problem is an optimization problem for a given target with unknown parameters (i.e. the gene mutation rates). Since our system for this problem is complicated (see the details in Sections 2 and 3), it is difficult to apply an analytical method directly to solve this problem. Thus, as an alternative, we use a functional data regression based method to solve this optimization problem via training the optimal parameters. During the training process, the functional dataset can be pre-given in a real-world situation. Nevertheless, in this paper, our required functional dataset is generated by random number generators that are frequently used in Monte Carlo simulations. In Dai [14,17], the functional data regression is realized through a conditional expectation projection for a given target over the functional dataset. Due to the Kolmogorov conditional expectation theorem (see Theorem 5.1 in Kallenberg [29]) and the study in Yosida [55], the conditional expectation is a self-adjoint operator if the functional dataset is from a $L^2$-integrable space. Hence, it is a projective operator. Then, based on these properties, we can generalize the conventional linear or nonlinear regression to the so-called big model regression (see Dai [14,17] for more details). More precisely, the projection can be denoted or approximated by a big model constructed through convolutional neural networks (CNNs, see, e.g. Dai [14,17] and Yamashitza et al. [53]), Kolmogorov-Arnold Networks (KANs, see, e.g. Braun and Griebel [4], Kolmogorov [29], and Liu et al. [33,34]), etc. According to the needs of different applications, the big model regressions can be constructed in different forms. The current hot large language model and generative video model ‘Transformer and Diffusion Transformer’ are among our mentioned constructions (see, e.g. Brizuela and Merchan [5], Peluchetti [41], and Vaswani et al. [49]). The studies in justifying the reasonability of these big model regressions through minimizing their corresponding projection errors are also the current hot topics (see, e.g. Dai [14,17], Gonon et al. [23], and Kratsios et al. [30] for more details).

Therefore, according to the natural flow chart of DNA to RNA to protein as shown in Figure 2 and the big model regression construction idea as introduced above, we will develop a systematic methodology to study protein production. More precisely, the protein has the target gene sequences and protein structures as shown in Figure 1. In other words, we present a flowchart in Figure 1 to summarize our methodology visually, which is a diagram corresponding to Figure 2 to show the workflow from input DNA sequences through the CNN layers to the final protein production. Furthermore, this methodology is established through constructing a CNN and developing two machine learning algorithms for a two-stage minimization-maximization optimization problem (see, e.g. Callaway [6], Dai [15], Dauparas et al. [18], Jumper et al. [26], LeChun et al. [32], Wang et al. [50], and Wicky et al. [52]). In addition, we try to make our designed CNN interpretable with respect to the natural flow chart as shown in Figure 2. More precisely, in the CNN presented in Figure 1, it consists of two layers: the convolutional layer and the max-pooling layer. The convolutional layer consists of three sublayers for controlling DNA/RNA gene mutation rates through multi-input multi-output (MIMO) mutual information and Ewens sampling formulas respectively. The max-pooling layer is to match the growing amino acid chain in Figure 2 and the input optimized codon sequences from the convolution layer. The pooling rule can be either a naturally (e.g. randomly) selected one or an optimally computed one by conducting codon optimization. More precisely, we will develop a CNN based numerical optimization scheme consisting of two newly designed iterative algorithms to estimate the optimal gene mutation rates. Furthermore, this scheme is theoretically analyzed and implemented. Besides the interests of studying the protein production itself in the areas of healthcare and biomedical diagnosis (see, e.g. Jumper et al. [26]), our study here is also aimed to make new contributions to the study of bionics. For example, we are with the hopes to improve the cross-layer protocol designs in quantum computing, metaverse, 6G/6G⁺ (sixth generation and beyond) wireless communication, etc. (see, e.g. Akyildiz et al. [1], Dai [11,15], and Ghafoor et al. [22]).

Figure 2.

The flow chart of DNA to RNA to protein.

Figure 1.

The CNN aided production flow chart to produce the target proteins. In this CNN, it consists of two layers: the convolutional layer and the max-pooling layer. The convolutional layer consists of three sublayers for controlling DNA/RNA gene mutation rates through MIMO mutual information and Ewens sampling formulas respectively. The max-pooling layer is to match the growing amino acid chain in Figure 2 and the input optimized codon sequences from the convolution layer. The pooling rule can be either a naturally (e.g. randomly) selected one or an optimally computed one by conducting codon optimization. Finally, the Algorithms 3.1 and 3.2 are designed in Section 3.4.

Specifically, our designed CNN in Figure 1 is to make its updated parameters (i.e. gene mutation rates) more closely match those of the target proteins in the sense as follows: the parameterized probability of taking the (pre-designed or called) target codon sequences or the optimized codon sequences is maximized. Concerning the design of a CNN, the readers are referred to the existing literatures (see, e.g. Dai [14], Dauparas et al. [18], Jumper et al. [26], LeChun et al. [32], Silver et al. [44], Wang et al. [50], and Wicky et al. [52]) for further references. In this paper, we will make an effort to integrate the Ewens sampling formula (see, e.g. Ewens [19], Gan and Ross [21], Tsukuda [47], and Watterson [51]) and newly developed MIMO mutual information formula into our system for the purpose of determining stochastic gradients of our CNN based algorithms. Meanwhile, we will also integrate the codon optimization technique (see, e.g. Arbib et al. [2], Pham et al. [42], Sen et al. [43], and Taneda and Asai [45]) into our study for the purpose of optimally ordering the amino acid sequence into the framework of our developed CNN as shown in Figure 1. More precisely, this integration process corresponds to the protein synthetic procedure as shown in Figure 2. Note that, since the amino acid sequence in a protein determines the three-dimensional shape of the protein (see, e.g. Dauparas et al. [18], Wang et al. [50], and Wicky et al. [52]), our study presented in this paper can also be used to predict a protein structure to some degree.

Furthermore, the protein production procedure can be summarized into two major steps (see, e.g. Nelson and Cox [37]): (1) from DNA (DeoxyriboNucleic Acid) to mRNA (message Ribonucleic Acid) through transcription; (2) from mRNA to protein through translation, which are shown as in Figure 2.

Nevertheless, prior to and during the process of transcription in step 1 as shown in Figure 2, gene mutation can happen due to different reasons (e.g. environmental factors such as radiation, certain chemical factors, spontaneous mutation, and other errors). Currently, how to control the gene mutation rates through different techniques (e.g. gene editing in reference such as Bailey and Maus [3], Dai [12], Hoeck et al. [24], Lattanzi and Maddalo [31], Zhou et al. [57], and Zhou et al. [58]) in order to meet the requirements of the target proteins is an interesting topic (concerning this point, see also Figure 3 for an illustration). Since the gene mutations can happen prior to and during the process of transcription, we design three sublayers in the convolutional layer of the CNN in Figure 1 to map the mutation rates into this CNN as its system parameters. Between the input sublayer and inner sublayer, we try to control gene mutations that directly happen in DNA itself. Between the inner sublayer and output sublayer, we try to control the gene mutations that happen in the transcription. Furthermore, we estimate these rates by selecting suitable stochastic gradients corresponding to a minimization-maximization two-stage optimization problem. More precisely, the estimation is through the well-known Evens sampling formula (see, e.g. Ewens [19], Gan and Ross [21], Tsukuda [47], and Watterson [51]) and a newly established MIMO mutual information formula.

Figure 3.

An Illustration of Gene Editing and Healthcare. The graph in this figure is adapted from Wanyang Dai's Plenary Speech of Biomedical Engineering-2019, Las Vegas, USA, July 8-9, 2019.

In the process of translation in step 2 as shown in Figure 2, a target protein (a sequence of amino acids) will be produced through a ribosome and the codon information carried by a mRNA (see, e.g. Arbib et al. [2], Pham et al. [42], Sen et al. [43], and Taneda and Asai [45]). Each codon is a sequence of three nucleotides encoding a particular amino acid into the target protein during the protein synthesis. However, there are only 20 different amino acids available to match 61 distinct codons (64 minus three stop codons as shown in Figure 4 and as discussed in Karasan et al. [28] and Taneda and Asai [45]), which leads to many possible choices in encoding the target proteins. In real-world situations, the matches between amino acids and codons can be naturally (or randomly) realized, which are defined as natural matches in Figure 1. However, if we want to produce proteins with targets, this kind of matches can be implemented via an optimization method, which are defined as optimal matches in Figure 1. Therefore, how to find such an optimal choice through codon optimization is a hot topic along this research direction (see, e.g. Arbib et al. [2], Pham et al. [42], Sen et al. [43], and Taneda and Asai [45]). By chance, this codon optimization can be integrated into the design of our max-pooling layer in Figure 1.

Figure 4.

Amino Acid and Codon Table.

Comparing with existing studies and AlphaFold, we develop a new systematic methodology through constructing a CNN to study protein production with the target gene sequences and protein structures. The core of the CNN and machine learning approach is to address a two-stage optimization problem to balance gene mutation rates during protein production. To the best of our knowledge, our system model and its related two developed CNN based iterative algorithms are new. Currently, we have not found any existing studies to compare with them. Furthermore, the stochastic gradients of the two algorithms are designed according to the well-known Kuhn-Tucker conditions with boundary constraints (see, e.g. Luenberger [36]) besides the Ewens sampling and mutual information formulas. The associated learning rate bounds are also explicitly derived. The algorithms are numerically implemented. Their convergence and optimality are mathematically proved. The limit points of the two algorithms satisfy the corresponding optimality equations for their corresponding constrained optimization problem, which imply that they are estimated optimal (or at least local optimal) gene (or scaled gene) mutation rates. Since our optimization problem can be a multi-objective oriented one, their corresponding system can be used to enhance the quality of our required proteins while to reduce the toxicity of viruses.

Concerning this balancing or equilibrium issue, the corresponding objective functions can be selected through different score functions including codon adaptation index (CAI), codon pair bias (CPB), and hidden stop codon (HSC) for single protein production (see, e.g. Sen et al. [43] and Taneda and Asai [45]). Interested readers can find more score functions recently summarized in Paremskaia et al. [40]. To be convenient for further discussions concerning the chemical reaction among different protein production, we formulate our current system in a more general multi-protein production way. Therefore, our computed optimal gene mutation rates may provide some insights about how to find suitable enzymes, vaccines, and antibodies to promote the protein production.

Concerning this point, we make some comparison between our current study and AlphaFold as follows.

In the published paper of Jumper et al. [26], the authors present the achievement concerning the highly accurate protein structure prediction with AlphaFold. Furthermore, Google DeepMind just recently enhances the technology in Jumper et al. [26] to the more powerful case that gives scientists the capability to predict the structures of proteins during interactions with other molecules (see Callaway [6] for details). The structures of proteins (especially the three-dimensional shapes of proteins) are determined by amino acid sequences (see, e.g. Chen et al. [7], Dauparas et al. [18], Wang et al. [50], and Wicky et al. [52]). Hence, the study conducted in our current paper concerning the optimization of gene mutation rates and amino acid sequences has the natural connection with AlphaFold. Based on the announcements concerning AlphaFold and to the best of our knowledge, our current study with two developed algorithms is new in comparing with AlphaFold. Our study in this paper has the different purpose from the existing studies in AlphaFold. More precisely, our current study is to estimate the gene mutation rates if the target gene sequence is given. Meanwhile, the existing studies in AlphaFold is to predict the gene sequence and protein structure if the gene mutation is given. Of course, after the gene mutation rates are estimated (or trained) through our method, the computing complexity of protein structure prediction will be reduced to a reasonable degree for an AlphaFold based method. However, based on the recent announcement concerning AlphaFold (see Callaway [6] for details), the current AlphaFold is a diffusion transformer based generative AI system (see, e.g. Brizuela and Merchan [5], Dai [14,17], Peluchetti [41], and Vaswani et al. [49]). Thus, our optimized prediction concerning amino acid sequences (presented in this paper) can be integrated into the AlphaFold system. Even more, the unification of our developed algorithms with AlphaFold can be further integrated into the quantum-cloud computing platform for more efficient usages in real-world situations (see Curioni [8], Dai [9,11–13,16], Fox et al. [20], and Figure A1 in Appendix 2 of this paper for details).

The rest of the paper is organized as follows. In Section 2, we develop our system model. In Section 3, we study optimally balanced gene (or scaled gene) mutation rate computing, which consists of four subsections: Ewens sampling and its related stochastic gradient are presented in Subsection 3.1, a new mutual information formula and its related stochastic search direction are derived in Subsection 3.2, codon optimization and max-pooling are presented in Subsection 3.3, a CNN based computing scheme with two iterative algorithms and its convergence theorem are designed and stated in Section 3.4. In Section 4, we present the numerical case studies to illustrate the effectiveness of our CNN based computing scheme. In Section 5, we present a real-world data implementation to illustrate the usage of our study, where we also illustrate that the gene mutation rate computing is essentially the computation of a gene mutation rate-oriented attention function in a quantum transformer. In Appendix 1 of this paper, we provide a mathematical proof of our CNN based algorithm convergence theorem. In Appendix 2 of this paper, we give a brief introduction concerning quantum computing for gene computation. The related quantum transformer is also briefly introduced, which can be used to guide the selection of medicines, anti-bodies, and vaccines. In Appendix 3 of this paper, we introduce some score functions for the illustration of codon optimization.

2. From DNA to protein: modeling

As pointed out in Introduction of this paper. we will develop a systematic methodology to study protein production as shown in Figure 1 where we present a flowchart summarizing our methodology visually. More precisely, this flowchart is a diagram corresponding to Figure 2 to show the work flow from input DNA sequences through the CNN layers to the final protein production. Furthermore, this methodology is established through constructing a CNN and developing two machine learning algorithms for a two-stage minimization-maximization optimization problem.

To reach this purpose, in this section, we develop a quantitative system corresponding to the methodology flowchart in Figure 1 to model the protein production procedure from the input DNA sequences to the produced proteins as shown in Figure 2. The aim of studying this production procedure is to maximize the parameterized probability $P_{{\mathrm{\text{β}}}^2}$ of taking the (pre-designed or called) target codon sequences or the optimized codon sequences as presented in (6). Meanwhile, we are aimed to minimize the parametric probability $P_{{\mathrm{\text{β}}}^1}$ of gene mutations between the original gene sequences and the target gene sequences. The computation is through a CNN based two-stage optimization problem (see, e.g. Dai [15]), which consists of a convolutional network layer and a max-pooling layer as shown in Figure 1. The convolutional network layer corresponds to the DNA transcription and the RNA transport with a gene mutation rate vector $\mathrm{\text{β}}=({\mathrm{\text{β}}}^1,{\mathrm{\text{β}}}^2)$. The max-pooling layer is designed to offer the target codon sequences or compute the optimized codon sequences corresponding to the input amino acid sequences through codon optimization.

More precisely, we will consider J input proteins with their corresponding target proteins. Since a single codon is a nucleotide triple, we suppose that the lengths of the carried DNA nucleotides for each input protein and its target protein are n=3L for some nonnegative integer L. For each $j\in \{1,..,J\}$, we use $D_j^{\mathrm{target}}$ and $H_j^{\mathrm{target}}$ to denote the corresponding codon and amino acid sequences respectively. Similarly, we use $D_j^{\mathrm{in}}$ and $H_j^{\mathrm{in}}$ to denote the corresponding input codon and amino acid sequences respectively. Furthermore, we use $D_j^{\mathrm{out}}\left(\mathrm{\text{β}}\right)$ and $H_j^{\mathrm{out}}\left(\mathrm{\text{β}}\right)$ to denote the codon and amino acid sequences corresponding to the mutation rate vector β, which is produced from the convolutional network layer as shown in Figures 1 and 2. For the purpose of quantitative modeling, these sequences can be expressed by

$$\begin{array}{rl}{D}_j^{\text{target}}& =(d_{j1}^{\text{target}},\dots ,d_{\mathrm{jL}}^{\text{target}}),H_j^{\text{target}}=(h_{j1}^{\text{target}},\dots ,h_{\mathrm{jL}}^{\text{target}}),\end{array}$$ (1)

$$\begin{array}{rl}{D}_j^{\text{in}}& =(d_{j1}^{\text{in}},\dots ,d_{\mathrm{jL}}^{\text{in}}),H_j^{\text{in}}=(h_{j1}^{\text{in}},\dots ,h_{\mathrm{jL}}^{\text{in}}),\end{array}$$ (2)

$$\begin{array}{rl}{D}_j^{\text{out}}\left(\mathrm{\text{β}}\right)& =\left(d_{j1}^{\text{out}}\right(\mathrm{\text{β}}),\dots ,d_{\mathrm{jL}}^{\text{out}}(\mathrm{\text{β}}\left)\right),H_j^{\text{out}}\left(\mathrm{\text{β}}\right)=\left(h_{j1}^{\text{out}}\right(\mathrm{\text{β}}),\dots ,h_{\mathrm{jL}}^{\text{out}}(\mathrm{\text{β}}\left)\right).\end{array}$$ (3)

Note that, all the notations appeared in (1)–(3) can be mapped into real number oriented ones and we will elaborate this remark in the subsequent discussion. In the special case that we try to reproduce the input proteins, we have that

$$\begin{array}{r}{D}_j^{\text{target}}=D_j^{\text{in}}\left(\text{or }{D}_j^{\text{opt}}\right)\text{while}{H}_j^{\text{target}}=H_j^{\mathrm{in}}\text{for each}j\in \{1,\dots ,J\},\end{array}$$ (4)

where, $D_j^{\text{opt}}$ for all $j\in \{1,\dots ,J\}$ are obtained through codon optimization with the input amino acid sequences $\{H_j^{\text{target}},j\in \{1,\dots ,J\left\}\right\}$. In general, we can suitably design the target proteins with the purpose to increase the quality of our required proteins and/or decrease the toxicity of a virus.

Now, let ${\mathrm{\text{β}}}_j^1$ be the mutation rate between the input sublayer and the inner sublayer while let ${\mathrm{\text{β}}}_j^2$ be a scaled mutation rate (that can be transformed into a conventional mutation rate) between the inner sublayer and the output sublayer. Furthermore, let $S_n^j$ be the input nucleotide sequence and $S_n^j\left({\mathrm{\text{β}}}_j^1\right)$ be the output nucleotide sequence between the first two layers as on the left-hand side of Figure 1, i.e.

$$\begin{array}{r}{S}_n^j=\{s_1^j,s_2^j,\dots ,s_n^j\},S_n^j\left({\mathrm{\text{β}}}_j^1\right)=\{s_1^j\left({\mathrm{\text{β}}}_j^1\right),s_2^j\left({\mathrm{\text{β}}}_j^1\right),\dots ,s_n^j\left({\mathrm{\text{β}}}_j^1\right)\},\end{array}$$ (5)

where, $s_i^j$ and $s_i^j\left({\mathrm{\text{β}}}_j^1\right)\in \{A,C,G,U\}$. Then, the aim of our current study can be summarized to solve the following multi-objective constrained optimization problem in terms of the gene mutation rate vector ${\mathrm{\text{β}}}_j=({\mathrm{\text{β}}}_j^1,{\mathrm{\text{β}}}_j^2)$,

$$\begin{array}{r}\underset{{\mathrm{\text{β}}}_j^1\in [\delta ,\xi ]}{min}\underset{{\mathrm{\text{β}}}_j^2\in [\delta ,\mathrm{\infty })}{max}(\text{log}(P_{{\mathrm{\text{β}}}_j^2}\{D_j^{\text{out}}\left({\mathrm{\text{β}}}_j^2\right)=D_j^{\text{target}}\}P\{s_{i_l}^j\ne s_{i_l}^j\left({\mathrm{\text{β}}}_j^1\right),l\in \{1,\dots ,i\}\}))\end{array}$$ (6)

for each $j\in \{1,\dots ,2\}$, where, $0<\delta <\xi <1$ are two constants and the integer i is the number of mutated genes between the two gene sequences in (5). Furthermore, $P_{{\mathrm{\text{β}}}_j^2}$ and $P_{{\mathrm{\text{β}}}_j^1}$ are the probabilities with gene mutation rates ${\mathrm{\text{β}}}_j^2$ and ${\mathrm{\text{β}}}_j^1$ in terms of the sample vectors in (5). In addition, the ‘min max’ in (6) means ‘minimization-maximization’.

Note that, the problem in (6) can also be formulated in a more general sense (i.e. a game-theoretic problem as studied in Dai [9,11]). For the convenience of our current discussion, we focus the study of this paper on the problem in (6). More precisely, in the convolutional network layer, we want to train the gene mutation rate vector β through maximum likelihood and mutual information methods with constraints. Furthermore, the well-known Ewens sampling formula (see, e.g. Ewens [19], Gan and Ross [21], Tsukuda [47], and Watterson [51]) is employed to design the related stochastic gradient to maximize the likelihood. Meanwhile, a new mutual information formula is established to design the stochastic gradient for the corresponding probability minimization problem. In addition, in the max-pooling layer, we try to find the optimal gene sequential combination with its associated amino acid chain through codon optimization (see, e.g. Arbib et al. [2], Pham et al. [42], Sen et al. [43], and Taneda and Asai [45]). In solving such a problem, big data issue is frequently relevant. Currently, we use Lenovo Thinkpad X1 LAPTOP-C1HOOFBH to numerically implement our designed algorithms just for experimental purposes in small-scale cases, whose central processing unit (CPU) is 11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80 GHz 2.80 GHz and whose random access memory (RAM) is 16.0 GB. In the future, to improve the efficiency and accuracy of these algorithms in solving real-world problems, the quantum computing (or more generally, quantum-cloud computing) can be introduced to tackle the involved heavy computation problem (see, e.g. Figure A1 in Appendix 2 of this paper, Curioni [8], Dai [9,11–13,16], and Fox et al. [20] for more details).

3. Optimally balanced gene mutation rate computing

In this section, we give detailed study concerning optimally balanced gene mutation rate computing, which corresponds to the methodology flowchart in Figure 1 to model the protein production procedure from the input DNA sequences to the produced proteins as shown in Figure 2. This section consists of four subsections: Ewens sampling and stochastic gradient are presented in Subsection 3.1, a new MIMO mutual information formula and its related stochastic search direction are derived in Subsection 3.2, codon optimization and max-pooling are presented in Subsection 3.3, a CNN based computing scheme consisting of two iterative algorithms with their convergence is designed and stated in Subsection 3.4.

3.1. Ewens sampling and stochastic gradient

In this subsection, we first introduce the well-known Ewens sampling formula (see, e.g. Ewens [19], Tsukuda [47], and Watterson [51]). Then, we use this formula to determine the parameters between the output sublayer and the inner sublayer of the convolutional network layer by introducing a stochastic gradient as shown on the left-half side of Figure 1. More precisely, let $\mathcal{N}$ be the set of positive integer set (i.e. $\mathcal{N}\equiv \{1,2,\dots \}$), and let $\{c_i^n,i\in \mathcal{N}\}$ be a sequence of nonnegative integer-valued random variables for each $n\in \mathcal{N}$, which satisfies

$$\begin{array}{r}\sum _{i=1}^{n}i{c}_i^n=n,c_i^n=0\text{for}i>n\end{array}$$ (7)

and corresponds to a nucleotide sequence $S_n=\{s_1,s_2,\dots ,s_n\}$ with $s_i\in \{A,C,G,U\}$ (see, e.g. Nelson and Cox [37], Taneda and Asai [45]) for each $i\in \{1,2,\dots ,n\}$. In the later reference, the integer n will be called the length of an associated nucleotide sequence. For each positive integer $b\in \{1,2,\dots ,n\}$ and nonnegative integer $a_i$ with $i\in \{1,2,\dots ,b\}$, we use $C_b^n$ and $A_b$ to denote the following two vectors

$$\begin{array}{r}{C}_b^n=(c_1^n,\dots ,c_b^n),A_b=(a_1,\dots ,a_b),\end{array}$$ (8)

where, $C_n^n$ is a random vector corresponding to a given random population. Furthermore, it denotes the component counts in a random combinatorial structure of a sample with size $n\in \mathcal{N}$. In other words, it is the allelic partition in a sample of size n. When the random population follows the stationary distribution of the infinitely-many neutral allele model with scaled mutation rate $\theta >0$, the distribution of $C_n^n$ is given by the well-known Ewens sampling formula (see, e.g. Ewens [19] and Tsukuda [47]). There are different ways to interpret the mutation rate θ (see, e.g. Tsukuda [47]). Here, we wish to compute the mutation rate θ in terms of a scaled function $\psi \left(n\right)$ (e.g. $\psi \left(n\right)\in \{n,n\text{log}(n\left)\right\}$). Therefore, in this case, the Evens sampling formula can be expressed with respect to $\theta \in [0,\mathrm{\infty })$ as follows,

$$\begin{array}{r}{P}_{\theta }\{C_n^n=A_n\left(\theta \right)\}=\frac{n!}{\mathrm{\Gamma }(n,\theta )}\prod _{i=1}^n{\left(\frac{\psi \left(n\right)\theta }{i}\right)}^{a_i\left(\theta \right)}\frac{1}{\left(a_i\right(\theta \left)\right)!}I\left\{\sum _{i=1}^{n}i{a}_i\right(\theta )=n\},\end{array}$$ (9)

where, $I\{\cdot \}$ is the indicator function and $\mathrm{\Gamma }(n,\theta )=\left(\psi \right(n\left)\theta \right)\left(\psi \right(n)\theta +1)\cdot \cdot \cdot \left(\psi \right(n)\theta +n-1)$ is the rising factorial function. To estimate the mutation rate θ from the given sample $A_n\left(\theta \right)$, a maximum likelihood estimation method (or a Stein estimator) can be applied (see, e.g. Lu and Dai [35]). For convenience, we here use the maximum likelihood estimation method with constraint to estimate the mutation rate θ. In this case, the stochastic gradient can be calculated by the following formula,

$$\begin{array}{r}\mathrm{\Delta \theta }=\frac{\mathrm{\partial }\text{log}{P}_{\theta }\{C_n^n=A_n\left(\theta \right)\}}{\mathrm{\partial }\theta }.\end{array}$$ (10)

Furthermore, we can apply the stochastic gradient $\mathrm{\Delta \theta }$ in (10) to the computation between the output sublayer and the inner sublayer of the convolutional network layer on the left-half side of Figure 1.

More precisely, for each $j\in \{1,2,\dots ,\}$, the stochastic gradient $\mathrm{\Delta }{\mathrm{\text{β}}}_j^2$ between the output sublayer and the inner sublayer can be derived if we use ${\mathrm{\text{β}}}_j^2$ to replace θ and use $A_n^j\left({\mathrm{\text{β}}}_j^2\right)=\left(a_1^j\right({\mathrm{\text{β}}}_j^2),\dots ,a_n^j({\mathrm{\text{β}}}_j^2\left)\right)$ to replace $A_n\left(\theta \right)$ in (10), i.e.

$$\begin{array}{r}\mathrm{\Delta }{\mathrm{\text{β}}}_j^2=\frac{\mathrm{\partial }\text{log}{P}_{{\mathrm{\text{β}}}_j^2}\{C_n^n=A_n^j\left({\mathrm{\text{β}}}_j^2\right)\}}{\mathrm{\partial }{\mathrm{\text{β}}}_j^2},\end{array}$$ (11)

where, $A_n^j\left({\mathrm{\text{β}}}_j^2\right)$ corresponding to the one introduced in (8) for each $j\in \{1,2,\dots ,J\}$ can be calculated from the target codon sequence $D_j^{\text{target}}$ in (1). Furthermore, each component $s_i^j\left({\mathrm{\text{β}}}_j^1\right)$ of the nucleotide sequence $S_n^j\left({\mathrm{\text{β}}}_j^1\right)$ corresponding to the inner sublayer in (5) can be computed as follows,

$$\begin{array}{r}{s}_i^j\left({\mathrm{\text{β}}}_j^1\right)=\{\begin{array}{ll}{s}_i^j& \text{with probability}1-{\mathrm{\text{β}}}_j^2\text{for an index}i\in \{1,\dots ,n\},\\ s& \text{uniformly over}\{A,C,G,U\}\setminus s_i^j\text{with probability}{\mathrm{\text{β}}}_j^2.\end{array}\end{array}$$ (12)

Note that, the mutation rate ${\mathrm{\text{β}}}_j^2$ in (12) is assumed to be a conventional mutation rate that is already transformed from its counterpart of the scaled one in (9) (we here employ the same notation just for the convenience of presentation). Hence, the associated $A_n^j\left({\mathrm{\text{β}}}_j^1\right)=\left(a_1^j\right({\mathrm{\text{β}}}_j^1),\dots ,a_n^j({\mathrm{\text{β}}}_j^1\left)\right)$ corresponding to the definition in (8) can also be calculated.

3.2. Mutual information and stochastic search direction

In this subsection, we determine the gene mutation rate ${\mathrm{\text{β}}}_j^1$ between the input sublayer and inner sublayer by introducing a stochastic search direction as shown on the left-half side of Figure 1. In doing so, consider the relationship between $S_n^j$ and $S_n^j\left({\mathrm{\text{β}}}_j^1\right)$ in (5) as a MIMO channel with n parallel inputs and n parallel outputs. Instead of using the well-known entropy based mutual information formula (see, e.g. Dai [10] and references therein), we here explicitly derive a MIMO mutual information based formula with respect to the parameter ${\mathrm{\text{β}}}_j^1$ to solve the second stage minimization problem in (6) as follows,

$$\begin{array}{rl}& P_{{\mathrm{\text{β}}}_j^1}\{s_{i_l}^j\ne s_{i_l}^j\left({\mathrm{\text{β}}}_j^1\right),l\in \{1,\dots ,i\}\}\\ & =P_{{\mathrm{\text{β}}}_j^1}\{s_{i_1}^j\ne s_{i_1}^j\left({\mathrm{\text{β}}}_j^1\right),\dots ,s_{i_k}^j\ne s_{i_k}^j\left({\mathrm{\text{β}}}_j^1\right),s_u^j=s_u^j\left({\mathrm{\text{β}}}_j^1\right),u\notin \{i_1,\dots ,i_i\}\}\\ & ={\left({\mathrm{\text{β}}}_j^1\right)}^i{(1-{\mathrm{\text{β}}}_j^1)}^{n-i}.\end{array}$$ (13)

Then, the corresponding stochastic search direction can be presented by the following stochastic gradient $\mathrm{\Delta }{\mathrm{\text{β}}}_j^1$ between the input sublayer and the inner sublayer,

$$\begin{array}{r}\mathrm{\Delta }{\mathrm{\text{β}}}_j^1=\frac{\mathrm{\partial }{P}_{{\mathrm{\text{β}}}_j^1}\{s_{i_l}^j\ne s_{i_l}^j\left({\mathrm{\text{β}}}_j^1\right),l\in \{1,\dots ,i\}\}}{\mathrm{\partial }{\mathrm{\text{β}}}_j^1}.\end{array}$$ (14)

3.3. Codon optimization and max-pooling

Since a single codon is a nucleotide triplet, we suppose that the length n of the jth nucleotide sequence with $j\in \{1,..,J\}$ is $3L$ (i.e. n=3L) with corresponding codon sequence $D_j=(d_{j1},\dots ,d_{\mathrm{jL}})$. The purpose in doing so is to produce the target protein from an input amino acid sequence $H_j=\{h_{j1},\dots ,h_{\mathrm{jL}}\}$ with length $L\in \mathcal{N}$ as shown in Figures 2 and 4. Let $\mathcal{H}\left(h_{\mathrm{ji}}\right)$ with $j\in \{1,\dots ,n\}$ and $i\in \{1,\dots ,L\}$ be the set of codons coding for amino acid $h_{\mathrm{ji}}$ with $d_{\mathrm{ji}}\in \mathcal{H}\left(h_{\mathrm{ji}}\right)$. Then, by the design in Figure 5, we can formulate a multi-criteria codon optimization problem with mutation parameter ${\theta }_j$ as follows,

$$\begin{array}{r}\{\begin{array}{l}\text{Max }{F}_m(D_j,{\theta }_j)\text{for}j\in \{1,\dots ,J\},m\in \{1,\dots ,M\}\\ \text{subject to}{D}_j\in {\mathcal{D}}_j,\end{array}\end{array}$$ (15)

where $F_m(D_j,{\theta }_j)$ is the mth objective function for a sequence $D_j$ of codons with mutation parameter ${\theta }_j$. The set ${\mathcal{D}}_j$ is the corresponding feasible decision set that consists of all possible sequences of codons that code for the input amino acid sequence $H_j$ with no forbidden motifs. Note that, the objective functions in (15) can be designed in a more general fashion as in Dai [9,11]. Here, to be convenient for our illustration, we use the current format in (15). Furthermore, our discussions will based on the solutions to the multi-criteria codon optimization problem in (15) from the respective of pareto-optimality and game-theoretic Nash-equilibrium. The corresponding objective functions can be induced from the well-known score functions including CAI, CPB, and HSC (see, e.g. Sen et al. [43] and Taneda and Asai [45], and references therein). For the reader's convenience, we summarize some of these functions in Appendix 3 of this paper. Furthermore, under certain conditions, all the pareto-optimal codon sequences in the feasible decision space can be computed via a mathematical programming algorithm (see, e.g. Sen et al. [43] and Taneda and Asai [45]) or a neural network based algorithm (see, e.g. Jain et al. [25]).

Figure 5.

Codon optimization. The graph in this figure is initially designed for the keynote talk in Dai [13].

As pointed out in the introduction of this paper, there are only 20 different amino acids available to match 61 distinct codons (64 minus three stop codons as shown in Figure 4 and as discussed in Karasan et al. [28] and Taneda and Asai [45]), which leads to many possible choices in encoding the target proteins. In real-world situations, the matches between amino acids and codons can be naturally (or randomly) realized, which are called natural matches corresponding to the first choice in Figure 1. In this case, the experimental input data generated from the two gene mutation rate optimization algorithms in Subsection 3.4, which correspond to Ewens sampling and the mutual information formulas in Subsections 3.1 and 3.2 respectively, does not have an impact on the codon optimization problem as presented in (15). However, if the optimal matches corresponding to the second choice in Figure 1 are selected, the experimental input data generated from the two gene mutation algorithms in Subsection 3.4 will have an impact on the codon optimization problem in (15). As indicated in Figures 1 and 5, the experimental input data from the two gene mutation algorithms in Subsection 3.4 to the optimization problem in (15) is the codon sequences with optimized gene mutations. Nevertheless, in this case, the computation of the optimized gene mutation rates will be more complicated. Besides the objective function in (6), we need add the score functions as objective functions to solve a more complex multi-objective optimization problem as shown in Figure 1 with the choice of optimal matches, where the gene mutation rates are considered as unknown parameters. As mentioned previously, the score functions can be taken to be CAI, CPB, and HSC as summarized in Appendix 3 of this paper but with mutation rates as unknown parameters shown in Figure 4. Based on existing studies (see, e.g. Jain et al. [25], Karasan et al. [28], Paremskaia et al. [40], Sen et al. [43], and Taneda and Asai [45]), codon optimization indeed promotes the target gene expressions and this optimization heavily depends on the input gene sequences corresponding to various organisms. Thus, if the input gene sequences are replaced by our experimental input data generated from the two gene mutation algorithms in Subsection 3.4, the gene expressions will have impact on our computational outcomes. Conducting an integrated simulation by unifying our optimal gene mutation rate estimation algorithms and existing codon optimization algorithms is our future research problem.

3.4. CNN and machine learning based algorithms

This subsection consists of two subsubsections. First, in Subsubsection 3.4.1, we develop two CNN and machine learning based iterative algorithms to compute the optimal gene mutation rate vector ${\mathrm{\text{β}}}_j$ with $j\in \{1,2,\dots ,J\}$ for the optimization problem in (6). The computed gene mutation rate vector from the first algorithm (i.e. Algorithm 3.1) can be used to generate gene sequences to be used as sample gene sequences for the second algorithm (i.e. Algorithm 3.2). Meanwhile, the second algorithm can also be used as an independent mutation rate estimation algorithm for general sample gene sequences. Second, in Subsubsection 3.4.2, we study the convergence of the proposed parameter estimation algorithms.

3.4.1. Two algorithms

The first algorithm presented in Figure 6 and Algorithm 3.1 is based on the Ewens sampling formula in (9) and its corresponding stochastic gradient in (10), which is a maximum likelihood oriented probability optimization algorithm with a single-side boundary constraint.

Figure 6.

The flow chart of Algorithm 3.1.

Algorithm 3.1

This algorithm consists of the following steps:

• 1.Given the target codon sequence $D_j^{\text{target}}$ in (1) for each $j\in \{1,2,\dots ,J\}$ or compute it by solving the codon optimization problem in (15);
• 2.
  For each $j\in \{1,2,\dots ,J\}$ and a constant $\delta \in [0,\mathrm{\infty })$, select an initial parametric vector (initial gene mutation rate vector) $({\mathrm{\text{β}}}^2{)}_0=({\mathrm{\text{β}}}_{10}^2,{\mathrm{\text{β}}}_{20}^2,\dots ,{\mathrm{\text{β}}}_{J0}^2)\in [\delta ,\mathrm{\infty })\times \cdot \cdot \cdot \times [\delta ,\mathrm{\infty })$ with initial stochastic gradient as

  $$\begin{array}{r}\mathrm{\Delta }{\mathrm{\text{β}}}_{j0}^2=-\sum _{i=1}^n\frac{n}{n{\mathrm{\text{β}}}_{j0}^2+i-1}+\sum _{i=1}^n\frac{a_i\left({\mathrm{\text{β}}}_{j0}^2\right)}{{\mathrm{\text{β}}}_{j0}^2}{I}_{\left\{a_i\right({\mathrm{\text{β}}}_{j0}^2\ge 1\}};\end{array}$$
• 3.
  For each $j\in \{1,2,\dots ,J\}$, if the index m=1, set

  $$\begin{array}{r}{u}_{(m-1)}^j=\{\begin{array}{ll}0& \text{if}{\mathrm{\text{β}}}_{j(m-1)}^2>\delta ,\\ -\mathrm{\Delta }{\mathrm{\text{β}}}_{j(m-1)}^2& \text{if}{\mathrm{\text{β}}}_{j(m-1)}^2<=\delta (\text{take}{\mathrm{\text{β}}}_{j(m-1)}^2=\delta ),\end{array}\end{array}$$

  where, the stochastic gradient $\mathrm{\Delta }{\mathrm{\text{β}}}_{j(m-1)}^2$ corresponds to (11). Otherwise, if the index $m\in \{2,3,\dots \}$, set

  $$\begin{array}{r}{u}_{(m-1)}^j=\{\begin{array}{ll}0& \text{if}{\mathrm{\text{β}}}_{j(m-1)}^2>\delta ,\\ -\mathrm{\Delta }{\mathrm{\text{β}}}_{j(m-2)}^2& \text{if}{\mathrm{\text{β}}}_{j(m-1)}^2<=\delta (\text{take}{\mathrm{\text{β}}}_{j(m-1)}^2=\delta );\end{array}\end{array}$$
• 4.
  For each $j\in \{1,2,\dots ,J\}$ and $m\in \{1,2,\dots \}$, set

  $$\begin{array}{rl}{\mathrm{\text{β}}}_{\mathrm{jm}}^2& =\{\begin{array}{ll}{\mathrm{\text{β}}}_{j(m-1)}^2+{\alpha }_{j(m-1)}^2(-\mathrm{\Delta }{\mathrm{\text{β}}}_{j(m-1)}^2)& \text{if}{\mathrm{\text{β}}}_{j(m-1)}^2>\delta ,\\ {\mathrm{\text{β}}}_{j(m-1)}^2+{\alpha }_{j(m-1)}^2(-\mathrm{\Delta }{\mathrm{\text{β}}}_{j(m-1)}^2-u_{(m-1)}^j)& \text{if}{\mathrm{\text{β}}}_{j(m-1)}^2<=\delta ,\end{array}\\ {\psi }_{\left(m\right)}^j& =\{\begin{array}{ll}n\text{log(n)}& \text{if}\mathrm{\Delta }{\mathrm{\text{β}}}_{j(m-1)}^2>=0,\\ n& \text{if}\mathrm{\Delta }{\mathrm{\text{β}}}_{j(m-1)}^2<0,\end{array}\\ \mathrm{\Delta }{\mathrm{\text{β}}}_{\mathrm{jm}}^2& =-\sum _{i=1}^n\frac{{\psi }_m^j\left(n\right)}{{\psi }_m^j\left(n\right){\mathrm{\text{β}}}_{\mathrm{jm}}^2+i-1}+\sum _{i=1}^n\frac{a_i\left({\mathrm{\text{β}}}_{j0}^2\right)}{{\mathrm{\text{β}}}_{\mathrm{jm}}^2}{I}_{\left\{a_i\right({\mathrm{\text{β}}}_{j0}^2\ge 1\}},\end{array}$$

  where, ${\alpha }_{j(m-1)}^2\in (0,1)$ is an adaptively given learning rate corresponding to each m;
• 5.For pre-given sufficiently small real number $ϵ\ge 0$, if $\Vert ({\mathrm{\text{β}}}^2{)}_m-({\mathrm{\text{β}}}^2{)}_{m-1}\Vert \le ϵ$, terminate the computation procedure and record the computed values, where, $\Vert \cdot \Vert$ denotes the Euclidean norm, and $({\mathrm{\text{β}}}^2{)}_m=({\mathrm{\text{β}}}_{1m}^2,{\mathrm{\text{β}}}_{2m}^2,\dots ,{\mathrm{\text{β}}}_{\mathrm{Jm}}^2)$. Otherwise, if $\Vert ({\mathrm{\text{β}}}^2{)}_m-({\mathrm{\text{β}}}^2{)}_{m-1}\Vert >ϵ$, set m=m+1 and repeat steps 3–4.

Here, in Algorithm 3.1, we have used the fact that $a_i\left({\mathrm{\text{β}}}_{\mathrm{jm}}^2\right)$ is a given number. Thus, it does not change with the index m (i.e. $a_i\left({\mathrm{\text{β}}}_{j(m-1)}^2\right)=a_i\left({\mathrm{\text{β}}}_{j(m-2)}^2\right)$) for all $m\in \{1,2,\dots \}$. The second algorithm presented in Figure 7 and Algorithm 3.2 is based on the MIMO mutual information formula in (13) and its corresponding stochastic gradient in (14), which is also a probability optimization oriented algorithm but with double-side boundary constraints.

Figure 7.

The flow chart of Algorithm 3.2.

Algorithm 3.2

This algorithm consists of the following steps:

• 1.Given the target codon sequence $D_j^{\text{target}}$ in (1) for each $j\in \{1,2,\dots ,J\}$, or compute it by solving the codon optimization problem in (15), or generate it by use of the gene mutation rate $\left({\mathrm{\text{β}}}^2\right)$ obtained from Algorithm 3.1;
• 2.
  For two constants δ and ξ satisfying $0<\delta <\xi <1$, select an initial parametric vector (initial gene mutation rate vector) $({\mathrm{\text{β}}}^1{)}_0=({\mathrm{\text{β}}}_{10}^1,{\mathrm{\text{β}}}_{20}^1,\dots ,{\mathrm{\text{β}}}_{J0}^1)\in [\delta ,\xi ]\times \cdot \cdot \cdot \times [\delta ,\xi ]$, and take

  $$\begin{array}{r}\mathrm{\Delta }{\mathrm{\text{β}}}_{j0}^1=\frac{i}{{\mathrm{\text{β}}}_{j0}^1}-\frac{n-i}{1-{\mathrm{\text{β}}}_{j0}^1},\end{array}$$

  where, the right-hand side of the stochastic gradient $\mathrm{\Delta }{\mathrm{\text{β}}}_{j0}^1$ is obtained from (14) and the integer i is the number of mutated genes in the target codon sequence;
• 3.
  For each $j\in \{1,2,\dots ,J\}$, if the index m=1, set

  $$\begin{array}{rl}{v}_{2(m-1)}^j& =\{\begin{array}{ll}0& \text{if}\delta <{\mathrm{\text{β}}}_{j(m-1)}^1<\xi ,\\ -\mathrm{\Delta }{\mathrm{\text{β}}}_{j(m-1)}^1& \text{if}{\mathrm{\text{β}}}_{j(m-1)}^1>=\xi (\text{take}{\mathrm{\text{β}}}_{j(m-1)}^1=\xi ),\\ 0& \text{if}{\mathrm{\text{β}}}_{j(m-1)}^1<=\delta (\text{take}{\mathrm{\text{β}}}_{j(m-1)}^1=\delta ),\end{array}\\ v_{1(m-1)}^j& =\{\begin{array}{ll}0& \text{if}\delta <{\mathrm{\text{β}}}_{j(m-1)}^1<\xi ,\\ 0& \text{if}{\mathrm{\text{β}}}_{j(m-1)}^1>=\xi (\text{take}{\mathrm{\text{β}}}_{j(m-1)}^1=\xi ),\\ \mathrm{\Delta }{\mathrm{\text{β}}}_{j(m-1)}^1& \text{if}{\mathrm{\text{β}}}_{j(m-1)}^1<=\delta (\text{take}{\mathrm{\text{β}}}_{j(m-1)}^1=\delta ).\end{array}\end{array}$$

  Otherwise, if the index $m\in \{2,3,\dots \}$, set

  $$\begin{array}{rl}{v}_{2(m-1)}^j& =\{\begin{array}{ll}0& \text{if}\delta <{\mathrm{\text{β}}}_{j(m-1)}^1<\xi ,\\ -\mathrm{\Delta }{\mathrm{\text{β}}}_{j(m-2)}^1& \text{if}{\mathrm{\text{β}}}_{j(m-1)}^1>=\xi (\text{take}{\mathrm{\text{β}}}_{j(m-1)}^1=\xi ),\\ 0& \text{if}{\mathrm{\text{β}}}_{j(m-1)}^1<=\delta (\text{take}{\mathrm{\text{β}}}_{j(m-1)}^1=\delta ),\end{array}\\ v_{1(m-1)}^j& =\{\begin{array}{ll}0& \text{if}\delta <{\mathrm{\text{β}}}_{j(m-1)}^1<\xi ,\\ 0& \text{if}{\mathrm{\text{β}}}_{j(m-1)}^1>=\xi (\text{take}{\mathrm{\text{β}}}_{j(m-1)}^1=\xi ),\\ \mathrm{\Delta }{\mathrm{\text{β}}}_{j(m-2)}^1& \text{if}{\mathrm{\text{β}}}_{j(m-1)}^1<=\delta (\text{take}{\mathrm{\text{β}}}_{j(m-1)}^1=\delta );\end{array}\end{array}$$
• 4.
  For each $j\in \{1,2,\dots ,J\}$ and $m\in \{1,2,\dots \}$, set

  $$\begin{array}{r}{\mathrm{\text{β}}}_{\mathrm{jm}}^1=\{\begin{array}{ll}{\mathrm{\text{β}}}_{j(m-1)}^1+{\alpha }_{j(m-1)}^1\left(\mathrm{\Delta }{\mathrm{\text{β}}}_{j(m-1)}^1\right)& \text{if}\delta <{\mathrm{\text{β}}}_{j(m-1)}^1<\xi ,\\ {\mathrm{\text{β}}}_{j(m-1)}^1+{\alpha }_{j(m-1)}^1(\mathrm{\Delta }{\mathrm{\text{β}}}_{j(m-1)}^1+v_{2(m-1)}^j)& \text{if}{\mathrm{\text{β}}}_{j(m-1)}^1>=\xi ,\\ {\mathrm{\text{β}}}_{j(m-1)}^1+{\alpha }_{j(m-1)}^1(\mathrm{\Delta }{\mathrm{\text{β}}}_{j(m-1)}^1-v_{1(m-1)}^j)& \text{if}{\mathrm{\text{β}}}_{j(m-1)}^1<=\delta ,\end{array}\end{array}$$

  where, the stochastic gradient $\mathrm{\Delta }{\mathrm{\text{β}}}_{j(m-1)}^1$ is corresponding to (14) and the learning rate ${\alpha }_{j(m-1)}^1$ $\in (0,1)$ is adaptively given;
• 5.For pre-given sufficiently small real number $ϵ\ge 0$, if $\Vert ({\mathrm{\text{β}}}^1{)}_m-({\mathrm{\text{β}}}^1{)}_{m-1}\Vert \le ϵ$, terminate the computation procedure and record the computed values, where, $\Vert \cdot \Vert$ denotes the Euclidean norm, and $({\mathrm{\text{β}}}^1{)}_m=({\mathrm{\text{β}}}_{1m}^1,{\mathrm{\text{β}}}_{2m}^1,\dots ,{\mathrm{\text{β}}}_{\mathrm{Jm}}^1)$. Otherwise, if $\Vert ({\mathrm{\text{β}}}^1{)}_m-({\mathrm{\text{β}}}^1{)}_{m-1}\Vert >ϵ$, set m=m+1 and repeat steps 3–4.

3.4.2. Main convergence theorem

Before we present the convergence theorem for Algorithms 3.1 and 3.2, we define two positive constants $K_2$ and $K_1$ as follows,

$$\begin{array}{rl}{K}_2& =max\{K_{21},K_{22},K_{23},K_{24}\},\end{array}$$ (16)

$$\begin{array}{rl}{K}_1& =\frac{i}{{\delta }^2}+\frac{n-i}{(1-\xi {)}^2},\end{array}$$ (17)

where, as in (6), the integer i appeared in (17) is the number of mutated genes between the two gene sequences in (5) (hopefully, we wish that this index i used here will not cause confusion with other index i used in this paper). Furthermore, the two positive constants $K_{21}$, $K_{22}$, $K_{23}$, and $K_{24}$ appeared in (16) are given by

$$\begin{array}{rl}{K}_{21}& =\sum _{i=1}^n\frac{n\text{log}\left(n\right)}{{\left(n\text{log}\right(n)\delta +i-1)}^2}+\sum _{i=1}^n\frac{a_i\left({\mathrm{\text{β}}}_{j(m-1)}^2\right)}{{\delta }^2}{I}_{\left\{a_i\right({\mathrm{\text{β}}}_{j(m-1)}^2)\ge 1\}},\\ K_{22}& =\sum _{i=1}^n\frac{n}{{(\mathrm{n\delta }+i-1)}^2}+\sum _{i=1}^n\frac{a_i\left({\mathrm{\text{β}}}_{j(m-1)}^2\right)}{{\delta }^2}{I}_{\left\{a_i\right({\mathrm{\text{β}}}_{j(m-1)}^2)\ge 1\}},\\ K_{23}& =\sum _{i=1}^n\frac{n\text{log}\left(n\right)}{n\text{log}\left(n\right)\delta +i-1}+\sum _{i=1}^n\frac{a_i\left({\mathrm{\text{β}}}_{j\left(0\right)}^2\right)}{\delta }{I}_{\left\{a_i\right({\mathrm{\text{β}}}_{j\left(0\right)}^2)\ge 1\}},\\ K_{24}& =\sum _{i=1}^n\frac{n}{\mathrm{n\delta }+i-1}+\sum _{i=1}^n\frac{a_i\left({\mathrm{\text{β}}}_{j\left(0\right)}\right)}{\delta }{I}_{\left\{a_i\right({\mathrm{\text{β}}}_{j\left(0\right)})\ge 1\}}.\end{array}$$

Furthermore, define

$$\begin{array}{r}\mu \left(m\right)=1+\sum _{r_1=2}^{m}1+\sum _{r_1=2}^m\sum _{r_2=2}^{r_1}1+\cdots +\sum _{r_1=2}^m\sum _{r_2=2}^{r_1}\cdots \sum _{r_{m-2}=2}^{3}1.\end{array}$$ (18)

Then, we have the following theorem.

Theorem 3.1

This theorem consists of two claims corresponding to Algorithms 3.1 and 3.2 respectively:

Claim One: Concerning the parametric sequence $\left\{\right({\mathrm{\text{β}}}^2{)}_m,m\in \{1,2,\dots \}\}$ computed from Algorithm 3.1, if the learning rate sequence satisfies

$$\begin{array}{r}\mu \left(m\right){\alpha }_{j(m-1)}^2{K}_2\left|{\mathrm{\text{β}}}_{j\left(0\right)}^2\right|<1/m^2\text{with}{\alpha }_{j(m-1)}^2{K}_2<1\end{array}$$ (19)

for each $j\in \{1,\dots ,J\}$, we have the following convergence,

$$\begin{array}{r}({\mathrm{\text{β}}}^2{)}_m\to ({\mathrm{\text{β}}}^2)\text{along}m\in \{1,2,\dots \},\end{array}$$ (20)

where, the limit point $\left({\mathrm{\text{β}}}^2\right)\in [\delta ,\xi ]\times \cdot \cdot \cdot \times [\delta ,\xi ]$ is a J-dimensional constant vector.

Claim Two: Concerning the parametric sequence $\left\{\right({\mathrm{\text{β}}}^1{)}_m,m\in \{1,2,\dots \}\}$ computed from Algorithm 3.2, if the learning rate sequence satisfies

$$\begin{array}{r}\mu \left(m\right){\alpha }_{j(m-1)}^1{K}_1\left|{\mathrm{\text{β}}}_{j\left(0\right)}^1\right|<1/m^2\text{with}{\alpha }_{j(m-1)}^1{K}_1<1\end{array}$$ (21)

for each $j\in \{1,\dots ,J\}$, we have the following convergence,

$$\begin{array}{r}({\mathrm{\text{β}}}^1{)}_m\to ({\mathrm{\text{β}}}^1)\text{along}m\in \{1,2,\dots \},\end{array}$$ (22)

where, the limit point $\left({\mathrm{\text{β}}}^1\right)\in [\delta ,\mathrm{\infty })\times \cdot \cdot \cdot \times [\delta ,\mathrm{\infty })$ is a J-dimensional constant vector.

The proof of Theorem 3.1 will be provided in Appendix 1 of this paper.

4. Simulation case studies

In this section, we conduct Algorithms 3.2 and 3.1 based simulation case studies. They are presented in the following two subsections. To be simple, we take J=1 and omit the related index j appeared in the notations used in this section. As mentioned in Section 2, we currently use Lenovo Thinkpad X1 LAPTOP-C1HOOFBH to numerically implement our designed algorithms just for experimental purposes in small-scale cases, whose central processing unit (CPU) is 11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80 GHz 2.80 GHz and whose random access memory (RAM) is 16.0 GB. In our tested cases presented in this section, the implementations are all done within minutes.

4.1. Examples based on Algorithm 3.2

In this subsection, we present three Algorithm 3.2 based simulation examples corresponding to the nucleotide sample sizes n=60, n=600, and n=3000 respectively. From the simulations presented and displayed in Examples 4.1–4.3 and Figures 8–10, we can see that the algorithm computation performance and accuracy increase as the used nucleotide sample size n increases.

Figure 8.

Optimal mutation rate estimation via Algorithm 3.2 with learning rate ${\alpha }^1=1/\left(1.5K_1\right)$, where the used nucleotide sample size is n=60 with the lower boundary parameter $\delta =0.01$ and the upper boundary parameter $\xi =0.9$. The pre-given mutation rate is $\theta =0.1$ and the iteration number of the algorithm is M=3000.

Figure 10.

Optimal mutation rate estimation via Algorithm 3.2 with learning rate ${\alpha }^1=1/\left(1.5K_1\right)$, where the used nucleotide sample size is n=3000 with the lower boundary parameter $\delta =0.01$ and the upper boundary parameter $\xi =0.9$. The pre-given mutation rate is $\theta =0.1$ and the iteration number of the algorithm is M=3000.

Example 4.1

In this example, we iteratively estimate an optimal gene mutation rate through Algorithm 3.2 over an interval $[\delta ,\xi ]$ with $0<\delta <\xi <1$. In the upper-left graph of Figure 8, we display the input DNA nucleotide sample sequence uniformly generated from the set $\{1,2,3,4\}=\{A,C,G,U\}$. In the upper-right graph, we display the target DNA nucleotide sample sequence generated from the input DNA sequence with a pre-given mutation rate θ. Furthermore, each nucleotide (say, 1) will become a nucleotide other than itself (say, one of the nucleotides over the set $\{2,3,4\}$) with the mutation rate $\theta /3$. In the lower-left graph, we display the mutated genes that are denoted in magenta color. In the lower-right graph, we display the computed gene mutation rate curve (in red color) with respect to the iteration number. For the purpose of comparisons, we also display the pre-given mutation rate line (in blue color) and the counted frequency curve (in green color). Furthermore, with the increase of iteration number, the red curve (or Algorithm 3.2) converges to a constant close to the pre-given mutation rate.

Example 4.2

In this example, everything is set the same as for Example 4.1 except the sample size is changed to n=600 from n=60. From the lower-right graph of Figure 9, we can see that the computed gene mutation rate is more closer to the pre-given one than its counterpart in Figure 8 of Example 4.1. Therefore, Algorithm 3.2 with n=600 has the better simulation performance than its counterpart with n=60 in Example 4.1.

Figure 9.

Optimal mutation rate estimation via Algorithm 3.2 with learning rate ${\alpha }^1=1/\left(1.5K_1\right)$, where the used nucleotide sample size is n=600 with the lower boundary parameter $\delta =0.01$ and the upper boundary parameter $\xi =0.9$. The pre-given mutation rate is $\theta =0.1$ and the iteration number of the algorithm is M=3000.

Example 4.3

In this example, everything is set the same as for Example 4.1 except the sample size is changed to n=3000 from n=60. From the lower-right graph of Figure 10, we can see that the computed gene mutation rate accurately matches the pre-given one. Therefore, Algorithm 3.2 with n=3000 has the much better simulation performance than its counterparts with n=60 and n=600 respectively in Figure 8 of Example 4.1 and Figure 9 of Example 4.2.

4.2. Examples based on Algorithm 3.1

In this subsection, we present an Algorithm 3.1 based simulation example with the nucleotide sample size n=3000 and the initial value ${\mathrm{\text{β}}}_0^2=0.9$.

Example 4.4

In this example, we iteratively estimate an optimal scaled gene mutation rate through Algorithm 3.1 over an interval $[\delta ,\mathrm{\infty })$ with $0<\delta <\mathrm{\infty }$. The explanations to the upper-left and the upper-right graphs in Figure 11 are the same as those for the corresponding graphs in Example 4.3. In the lower-left graph, we display the component count evolution with respect to the sample data number, which is corresponding to the vectors in (8). In the lower-right graph, we display the computed optimal scaled gene mutation rate evolution in terms of the iteration number of Algorithm 3.1. The convergence of the algorithm with the initial value ${\mathrm{\text{β}}}_0^2=0.9$ and the sample size n=3000 is displayed in the graph. Furthermore, from our numerical implementation experiences, we have the conclusion that the algorithm convergence with n=3000 can be guaranteed if the initial value ${\mathrm{\text{β}}}_0^2\in [0.7,1.3]$. When n becomes larger, the interval to support the initial value becomes wider. Comparing with Algorithm 3.2 based computation, Algorithm 3.1 needs the relatively large iteration number to reach the algorithm convergence. Here, in the lower-right graph of Figure 11, we take the iteration number $M=$3,000,000 for the computation.

Figure 11.

Optimal scaled mutation rate estimation via Algorithm 3.1 with learning rate ${\alpha }^2=1/\left(15K_2\right)$, where the used nucleotide sample size is n=3000 with the lower boundary parameter $\delta =0.01$, and the iteration number of the algorithm is $M=$3,000,000.

5. A real-world data implementation

In the previous section (Section 4), we have presented four simulation case studies, where the used data sets are generated from a pseudo-random number generator. In this section, we employ our MIMO mutual information based Algorithm 3.2 to conduct a real-world data implementation, where the data sets are collected from The Cancer Genome Atlas (TCGA) with its website at [46]. Here, we note that TCGA is a project initiated by National Cancer Institute (NCI) and National-Human-Genome-Research-Institute (NHGRI) to discover the gene evolutions of various human cancels. Both of NCI and NHGRI are under National Institutes of Health (NIH). Moreover, the TCGA project is based on the large scale gene sequence measurement technology.

For the current purpose of our real-world data implementation, we take the corresponding data sets from Zhihu Website [56], which correspond to TCGA original data sets of human normal gene expression counts and colon cancel gene expression counts respectively (see, e.g. Figures 12 and 13). In the table of Figure 12, there are 41 tissues listed in the first row and each of them is associated with 59,427 normal genes listed in its corresponding column. In the table of Figure 13, there are 41 tissues listed in the first row and each of them is associated with 59,427 corresponding tumor genes listed in its corresponding column.

Figure 12.

Normal tissue gene expression count number table: there are 41 normal tissues listed in the first row and each of them is associated with 59,427 normal tissue genes listed in its corresponding column. Each block beginning at the second row and the second column displays the corresponding normal tissue gene expression count number.

Figure 13.

Tumor tissue gene expression count number table: there are 41 tumor tissues listed in the first row and each of them is associated with 59,427 tumor tissue genes listed in its corresponding column. Each block beginning at the second row and the second column displays the corresponding tumor tissue gene expression count number.

Example 5.1

In this example, we take 10,000 genes with their corresponding gene expression counts from 59,427 normal tissue genes listed in the first column as shown in Figure 12. We also take 10,000 genes with their corresponding gene expression counts from 59,427 tumor tissue genes listed in the first column as shown in Figure 13. In the two types of 10,000 genes, there are 4598 genes that are mutated during the process of 10,000 normal tissue genes evolving to 10,000 tumor tissue genes. There are 5402 genes that are not mutated. Note that, we here assume that a gene is considered as a mutated gene if its gene expression count number changes at least one unit. In the upper-left graph of Figure 14, we classify the normal tissue genes into four classes, which are based on four thresholds: 1/8000, 2/400, 3/100, and 1. If a gene expression count number divided by the total number of gene expression counts of the 10,000 normal tissue genes is in the interval [0,1/8000), it belongs to class 1. If a gene expression count number divided by the total number of gene expression counts of the 10,000 normal tissue genes is in the interval [1/8000, 2/400), it belongs to class 2. If a gene expression count number divided by the total number of gene expression counts of the 10,000 normal tissue genes is in the interval [2/400,3/100), it belongs to class 3. If a gene expression count number divided by the total number of gene expression counts of the 10,000 normal tissue genes is in the interval [3/100,1], it belongs to class 4. In the upper-right graph of Figure 14, we classify the tumor tissue genes into four classes through the same way as used for the normal tissue genes. In the lower-left graph of Figure 14, we display the distribution of mutated genes from 10,000 normal tissue genes to 10,000 tumor tissue genes. In the lower-right graph of Figure 14, we display the computed gene mutation rate curve (in red color) with respect to the iteration number. The bottom line of the lower-right graph of Figure 14 also represents the real-world mutation rate 0.4598. From this graph, we can see that the red curve (or Algorithm 3.2) converges to a constant close to the real-world gene mutation rate with the increase of iteration number.

Figure 14.

Gene classifications of normal tissue genes and tumor tissue genes, gene mutation distribution, and optimal gene mutation rate estimation of a real-world data implementation.

Finally, we remark that the gene mutation rate computing presented in Example 5.1 is essentially the computation of a gene mutation rate-oriented attention function as presented in the quantum transformer (see, Figure A2 in Appendix 2 of this paper for more details), which can be used to guide the selection of medicines, anti-bodies, and vaccines.

6. Conclusion

This research is motivated from enhancing the quality of our required proteins while weakening the toxicity of viruses. More precisely, we develop a systematic methodology through constructing a CNN to study protein production with the target gene sequences and protein structures. The core of the CNN and machine learning approach is to address a two-stage optimization problem to balance gene mutation rates during protein production. To wit, we try to optimally coordinate the consistency between the given input DNA sequences and the given (or optimally computed) target ones through controlling their intermediate gene mutation rates. The purposes in doing so are aimed to conduct gene editing and protein structure prediction. Our developed CNN numerical optimization scheme consists of two newly designed iterative algorithms. The stochastic gradients for the two algorithms are designed according to the Kuhn-Tucker conditions with boundary constraints and with the support of Ewens sampling, MIMO mutual information, and codon optimization techniques. The associated learning rate bounds are explicitly derived and the two algorithms are numerically implemented. The convergence and optimality of the algorithms are also mathematically proved. To the best of our knowledge, our system model and its related two developed CNN based iterative algorithms are new. Currently, we have not found any existing studies to compare with them. Therefore, our newly developed CNN and algorithms will advance the current state of knowledge in gene mutation rate control and protein production. For example, after the gene mutation rates are estimated, the computing complexity of protein structure prediction will be reduced to a reasonable degree. Our new model and algorithms can also interact with the existing studies to promote protein structure prediction. To illustrate the usage of our study, we have conducted both simulation oriented case studies and a real-world data implementation.

Concerning our future experimental validation efforts, we will conduct an integrated simulation by unifying our optimal gene mutation rate estimation algorithms and existing codon optimization algorithms. Due to the predicted heavy computation, we have also provided a quantum computing based roadmap for these efforts, which is presented in Appendix 2 of this paper. We hope that these efforts will be helpful in developing a systematic platform for the usage of real-world problems. Finally, in this research of our paper, we have displayed our simulation results in Figures 6–9, which correspond to real (or pre-given) mutation rates. The determination of gene sequence such as the orders of codon and amino acid is an important step to determine protein structure, which is the focal point of our current study. How to determine the actual protein structures including 3-dimensional shapes will be our future studies as pointed out in our paper.

Appendix 1. Proof of Theorem 3.1

In this section, we provide a proof for Theorem 3.1, which consists of three parts respectively presented in three subsections as follows.

A.1. Designing rational of Algorithms 3.1 and 3.2

We divide this subsection into two subsubsections concerning maximization rational for Algorithm 3.1 and minimization rational for Algorithm 3.2 respectively.

A.1.1. Maximization rational for Algorithm 3.1

First, concerning the steps 3–4 in Algorithm 3.1, we have the following illustration. Since ${\mathrm{\text{β}}}_j^2\in [\delta ,\mathrm{\infty })$ for sufficiently small $\delta >0$, we can introduce the corresponding inequality constraint as follows,

$$\begin{array}{r}g\left({\mathrm{\text{β}}}_j^2\right)=-{\mathrm{\text{β}}}_j^2+\delta \le 0.\end{array}$$

Then, by the Kuhn-Tucker condition for the minimization problem with equality and inequality constraints (see, e.g. pages 314–315 of Luenberger [36]), we can write the associated optimality equations as follows,

$$\begin{array}{r}-\mathrm{\Delta }{\mathrm{\text{β}}}_j^2+u\mathrm{\nabla g}\left({\mathrm{\text{β}}}_j^2\right)=0\text{with}ug\left({\mathrm{\text{β}}}_j^2\right)=0,\end{array}$$ (A1)

where, the negative sign ‘−’ before $\mathrm{\Delta }{\mathrm{\text{β}}}_j^2$ is due to the fact that our optimization problem with respect to ${\mathrm{\text{β}}}_j^2$ in (6) is a maximization problem. Furthermore, u is a constant and $\mathrm{\nabla g}\left({\mathrm{\text{β}}}_j^2\right)=-1$ denotes the gradient of the function g. From the second and first equations in (A1), we can obtain that

$$\begin{array}{r}u(-{\mathrm{\text{β}}}_j^2+\delta )=0\text{which implies that}u=\{\begin{array}{ll}0& \text{if}{\mathrm{\text{β}}}_j^2>\delta ,\\ -\mathrm{\Delta }{\mathrm{\text{β}}}_j^2& \text{if}{\mathrm{\text{β}}}_j^2=\delta .\end{array}\end{array}$$ (A2)

Hence, based on the expression of u, the steps 3–4 for the ${\mathrm{\text{β}}}_j^2$ based iteration in Algorithm 3.1 can be designed.

A.1.2. Minimization rational for Algorithm 3.2

The argument similar to the discussion in the previous subsubsection can also be developed to explain the ${\mathrm{\text{β}}}_j^1$ based iteration in the steps 3–4 of Algorithm 3.1. More precisely, since ${\mathrm{\text{β}}}_j^1\in [\delta ,\xi ]$ for $0<\delta <\xi <1$, we can introduce the corresponding inequality constraint as follows,

$$\begin{array}{r}h\left({\mathrm{\text{β}}}_j^1\right)=\left(h_1\right({\mathrm{\text{β}}}_j^1),h_2({\mathrm{\text{β}}}_j^1\left)\right)\le 0,\end{array}$$

where,

$$\begin{array}{r}{h}_1\left({\mathrm{\text{β}}}_j^2\right)=-{\mathrm{\text{β}}}_j^1+\delta ,h_2\left({\mathrm{\text{β}}}_j^1\right)={\mathrm{\text{β}}}_j^1-\xi .\end{array}$$

Then, by the Kuhn-Tucker condition for the minimization problem with equality and inequality constraints (see, e.g. pages 314–315 of Luenberger [36]), we can write the associated optimality equations as follows,

$$\begin{array}{r}\mathrm{\Delta }{\mathrm{\text{β}}}_j^1+v{\mathrm{\nabla }}^{T}h\left({\mathrm{\text{β}}}_j^1\right)=0\text{with}v{h}^T\left({\mathrm{\text{β}}}_j^1\right)=0,\end{array}$$ (A3)

where, $v=(v_1,v_2)$ is a constant vector and $\mathrm{\nabla h}\left({\mathrm{\text{β}}}_j^1\right)=(-1,1)$ denotes the gradient vector of the function h. Furthermore, the notation ‘T’ in (A3) denotes the transpose of a vector. From the second equation in (A3), we can obtain that

$$\begin{array}{r}{v}_1(-{\mathrm{\text{β}}}_j^1+\delta )+v_2({\mathrm{\text{β}}}_j^1-\xi )=0.\end{array}$$ (A4)

Then, we have the following three cases for our discussion.

Case One is corresponding to the situation that $\delta <{\mathrm{\text{β}}}_j^1<\xi$. In this case, we can simply take that

$$\begin{array}{r}{v}_1=v_2=0.\end{array}$$ (A5)

Case Two is corresponding to the situation that ${\mathrm{\text{β}}}_j^1=\delta$. In this case, we can conclude that ${\mathrm{\text{β}}}_j^1\ne \xi$ and hence $v_2=0$. Thus, we have that

$$\begin{array}{r}{v}_1=\mathrm{\Delta }{\mathrm{\text{β}}}_j^1.\end{array}$$ (A6)

Case Three is corresponding to the situation that ${\mathrm{\text{β}}}_j^1=\xi$. In this case, we can conclude that ${\mathrm{\text{β}}}_j^1\ne \delta$ and hence $v_1=0$. Thus, we have that

$$\begin{array}{r}{v}_2=-\mathrm{\Delta }{\mathrm{\text{β}}}_j^1.\end{array}$$ (A7)

Finally, based on the expressions in (A5)–(A7), it follows from the first equation in (A3) that the steps 3–4 for the ${\mathrm{\text{β}}}_j^1$ based iteration in Algorithm 3.1 can be designed.

A.2. The convergence of Algorithms 3.1 and 3.2

We divide this section into two subsubsections concerning computing stochastic gradients and convergence analysis respectively.

A.2.1. The stochastic gradients

For each $j\in \{1,2,\dots ,J\}$, consider the expression in (9). Without loss of generality, we suppose that

$$\begin{array}{r}I\{\sum _{i=1}^{n}i{a}_i\left({\mathrm{\text{β}}}_j^2\right)=n\}=1.\end{array}$$ (A8)

Then, the stochastic gradient in (11) can be further expressed as

$$\begin{array}{rl}\mathrm{\Delta }{\mathrm{\text{β}}}_j^2& =\frac{\mathrm{\partial }}{\mathrm{\partial }{\mathrm{\text{β}}}_j^2}(\text{log}(n!)+\sum _{i=1}^n\text{log}\left(\frac{1}{\left(a_i\right({\mathrm{\text{β}}}_j^2\left)\right)!}\right)\\ & +\sum _{i=1}^n\text{log}\left(\frac{\psi \left(n\right)}{\psi \left(n\right){\mathrm{\text{β}}}_j^2+i-1}\right)+\sum _{i=1}^n\text{log}\left({\left(\frac{\psi \left(n\right){\mathrm{\text{β}}}_j^2}{i}\right)}^{a_i\left({\mathrm{\text{β}}}_j^2\right)}\right))\\ & =-\sum _{i=1}^n\frac{\psi \left(n\right)}{\psi \left(n\right){\mathrm{\text{β}}}_j^2+i-1}+\sum _{i=1}^n\frac{a_i\left({\mathrm{\text{β}}}_j^2\right)\left(\frac{\psi \left(n\right)}{i}\right)}{{\mathrm{\text{β}}}_j^2\left(\frac{\psi \left(n\right)}{i}\right)}{I}_{\left\{a_i\right({\mathrm{\text{β}}}_j^2)\ge 1\}}\\ & =-\sum _{i=1}^n\frac{\psi \left(n\right)}{\psi \left(n\right){\mathrm{\text{β}}}_j^2+i-1}+\sum _{i=1}^n\frac{a_i\left({\mathrm{\text{β}}}_j^2\right)}{{\mathrm{\text{β}}}_j^2}{I}_{\left\{a_i\right({\mathrm{\text{β}}}_j^2)\ge 1\}}.\end{array}$$ (A9)

Furthermore, the error probability with respect to the parameter ${\mathrm{\text{β}}}_j^1$ in (6) can be explicitly calculated by (13). Thus, it follows from (13) that the stochastic gradient in (14) can be calculated as

$$\begin{array}{r}\mathrm{\Delta }{\mathrm{\text{β}}}_j^1=f_1\left({\mathrm{\text{β}}}_j^1\right)-f_2\left({\mathrm{\text{β}}}_j^1\right),\end{array}$$ (A10)

where,

$$\begin{array}{r}{f}_1\left({\mathrm{\text{β}}}_j^1\right)=\frac{i}{{\mathrm{\text{β}}}_j^1},f_2\left({\mathrm{\text{β}}}_j^1\right)=\frac{n-i}{1-{\mathrm{\text{β}}}_j^1}.\end{array}$$

A.2.2. The convergence analysis

Due to the stochastic gradient derived in (A9), it follows from Algorithm 3.1 that, for each $m\in \{3,4,\dots \}$, we have

$$\begin{array}{rl}& |{\mathrm{\text{β}}}_{\mathrm{jm}}^2-{\mathrm{\text{β}}}_{j(m-1)}^2|\\ & \le {\alpha }_{j(m-1)}^2\sum _{r_1=2}^m(\sum _{i=1}^n\frac{\psi \left(n\right)|{\mathrm{\text{β}}}_{j(r_1-1)}^2-{\mathrm{\text{β}}}_{j(r_1-2)}^2|}{\left(\psi \right(n){\mathrm{\text{β}}}_{j(r_1-1)}^2+i-1)\left(\psi \right(n){\mathrm{\text{β}}}_{j(r_1-2)}^2+i-1)}\\ & +\sum _{i=1}^n\frac{|{\mathrm{\text{β}}}_{j(r_1-1)}^2-{\mathrm{\text{β}}}_{j(r_1-2)}^2|a_i\left({\mathrm{\text{β}}}_{j(r_1-1)}^2\right)}{{\mathrm{\text{β}}}_{j(r_1-1)}^2{\mathrm{\text{β}}}_{j(r_1-2)}^2}{I}_{\left\{a_i\right({\mathrm{\text{β}}}_{j(r_1-1)}^2)\ge 1\}})\\ & +{\alpha }_{j(m-1)}^2(\sum _{i=1}^n\frac{\psi \left(n\right)\left|{\mathrm{\text{β}}}_{j\left(0\right)}^2\right|}{\psi \left(n\right){\mathrm{\text{β}}}_{j\left(0\right)}^2+i-1}+\sum _{i=1}^n\frac{\left|{\mathrm{\text{β}}}_{j\left(0\right)}^2\right|a_i\left({\mathrm{\text{β}}}_{j\left(0\right)}^2\right)}{{\mathrm{\text{β}}}_{j\left(0\right)}^2}{I}_{\left\{a_i\right({\mathrm{\text{β}}}_{j\left(0\right)}^2)\ge 1\}})\\ & \le {\alpha }_{j(m-1)}^2{K}_2\sum _{r_1=2}^m|{\mathrm{\text{β}}}_{j(r_1-1)}^2-{\mathrm{\text{β}}}_{j(r_1-2)}^2|+{\alpha }_{j(m-1)}^2{K}_2\left|{\mathrm{\text{β}}}_{j\left(0\right)}^2\right|\\ & \le {\alpha }_{j(m-1)}^2{K}_2\sum _{r_1=2}^m({\alpha }_{j(r_1-2)}^2{K}_2\sum _{r_2=2}^{r_1}|{\mathrm{\text{β}}}_{j(r_2-1)}^2-{\mathrm{\text{β}}}_{j(r_2-2)}^2|\\ & \phantom{{\alpha }_{j(r_1-2)}^2{K}_2\sum _{r_2=2}^{r_1}|{\mathrm{\text{β}}}_{j(r_2-1)}^2-{\mathrm{\text{β}}}_{j(r_2-2)}^2|}+\left({\alpha }_{j(m-1)}^2{K}_2\right)\left({\alpha }_{j(r_1-2)}^2{K}_2\right)\left|{\mathrm{\text{β}}}_{j\left(0\right)}^2\right|)+{\alpha }_{j(m-1)}^2{K}_2\left|{\mathrm{\text{β}}}_{j\left(0\right)}^2\right|\\ & \cdots \\ & \cdots \\ & \cdots \\ & \le \left({\alpha }_{j(m-1)}^2{K}_2\right)(1+\sum _{r_1=2}^{m}1+\sum _{r_1=2}^m\sum _{r_2=2}^{r_1}1+\cdots +\sum _{r_1=2}^m\sum _{r_2=2}^{r_1}\cdots \sum _{r_{m-2}=2}^{3}1)\left|{\mathrm{\text{β}}}_{j\left(0\right)}^2\right|\\ & \le \mu \left(m\right)\left({\alpha }_{j(m-1)}^2{K}_2\right)\left|{\mathrm{\text{β}}}_{j\left(0\right)}^2\right|\\ & \le \frac{1}{m^2},\end{array}$$ (A11)

where, $K_2$ is a constant given as in (16) and ${\mu }^2\left(m\right)$ is given by (18). Thus, it follows from (A11) that $\{{\mathrm{\text{β}}}_{\mathrm{jm}}^2,m\in \{1,2,\dots \left\}\right\}$ is a Cauchy sequence over the interval $[\delta ,\mathrm{\infty })$. Hence, there exists a constant ${\mathrm{\text{β}}}_j^2\in [\delta ,\mathrm{\infty })$ such that ${\mathrm{\text{β}}}_{\mathrm{jm}}^2\to {\mathrm{\text{β}}}_j^2$ as $m\to \mathrm{\infty }$. Furthermore, since ${\alpha }^{2(m-1)}$ for each m is positive, we can conclude that ${\mathrm{\text{β}}}_j^2$ is an optimal (at least a local optimal) point over $[\delta ,\mathrm{\infty })$ such that the Kuhn-Tucker conditions in (A1) hold.

Similarly, for each integer index $m\in \{3,4,\dots \}$, it follows from Algorithm 3.2 and the stochastic gradient in (A10) that

$$\begin{array}{rl}& |{\mathrm{\text{β}}}_{\mathrm{jm}}^1-{\mathrm{\text{β}}}_{j(m-1)}^1|\\ & \le {\alpha }_{j(m-1)}^1\sum _{r_1=2}^m\sum _{k=1}^2\left|f_k\right({\mathrm{\text{β}}}_{j(r_1-1}^1)-f_k({\mathrm{\text{β}}}_{j(r_1-2}^1\left)\right|+{\alpha }_{j(m-1)}^1\sum _{k=1}^2\left|f_k\right({\mathrm{\text{β}}}_{j\left(0\right)}^1\left)\right|\\ & \le {\alpha }_{j(m-1)}^1\sum _{r_1=2}^m\sum _{k=1}^2\left|f_k^{\prime }\right({\xi }_{j(r_1-1)}^k\left)\right||{\mathrm{\text{β}}}_{j(r_1-1)}^1-{\mathrm{\text{β}}}_{j(r_1-2)}^1|+{\alpha }_{j(m-1)}^1\left|{\mathrm{\text{β}}}_{j\left(0\right)}^1\right|\\ & \cdots \\ & \cdots \\ & \cdots \\ & \le \mu \left(m\right)\left({\alpha }_{j(m-1)}^1{K}_1\right)\left|{\mathrm{\text{β}}}_{j\left(0\right)}^1\right|\\ & \le \frac{1}{m^2},\end{array}$$ (A12)

where, $K_1$ is a constant given as in (17). Furthermore, due to the mean value theorem, ${\xi }_{j(r_1-1)}^1$ and ${\xi }_{j(r_1-1)}^2$ are between ${\mathrm{\text{β}}}_{j(r_1-1)}^1$ and ${\mathrm{\text{β}}}_{j(r_1-2)}^1$. In addition, the corresponding derivatives are given by

$$\begin{array}{r}{f}_1^{\prime }\left({\xi }_{j(r_1-1)}^1\right)=-\frac{i}{{\left({\xi }_{j(r_1-1)}^1\right)}^2},f_2^{\prime }\left({\xi }_{j(r_1-1)}^2\right)=\frac{n-i}{{(1-{\xi }_{j(r_1-1)}^2)}^2}.\end{array}$$

Thus, it follows from (A12) that $\{{\mathrm{\text{β}}}_{\mathrm{jm}}^1,m\in \{1,2,\dots \left\}\right\}$ is a Cauchy sequence over the interval $[\delta ,\xi ]$. Hence, there exists a constant ${\mathrm{\text{β}}}_j^1\in [\delta ,\xi ]$ such that ${\mathrm{\text{β}}}_{\mathrm{jm}}^1\to {\mathrm{\text{β}}}_j^1$ as $m\to \mathrm{\infty }$. Furthermore, since ${\alpha }^1$ is a positive constant, we can conclude that ${\mathrm{\text{β}}}_j^1$ is an optimal (at least a local optimal) point over $[\delta ,\xi ]$ such that the Kuhn-Tucker conditions in (A3) hold.

A.3. Final proof of Theorem 3.1

By combining the justifications presented in Subsubsections A.1 and A.2, we can reach the proofs for Claim One and Claim Two of Theorem 3.1. $◻$

Appendix 2. Quantum computing for gene computation

As mentioned in Section 2, to improve the efficiency and accuracy of our designed algorithms, the quantum computing (or more generally, quantum-cloud computing) can be introduced to tackle the heavy computation problem (see, e.g. Curioni [8], Dai [9,11–13,16], and Fox et al. [20] for more details). For reader's convenience, we present the quantum cloud computing flow chart for gene sequence related computations as in Figure A1. Inside the main quantum server presented in Figure A1, a quantum transformer can be designed to predict an optimal tissue gene expression count matrix in order to evolve tumor tissues back to normal tissues as mentioned in Section 5. During this procedure, the required gene mutation rates can be computed to guide the selection of medicines, anti-bodies, and vaccines. The operations for the designed quantum transformer may be angle-oriented quantum operations as developed in Dai [16]. For the reader's convenience, we present the encode part of the quantum transformer as in Figure A2.

Figure A1.

Quantum cloud computing flow chart for gene sequence related computations.

Figure A2.

A flow chart of encode to decode in a quantum transformer for predicting optimal tissue gene expression count matrix to evolve tumor tissues back to normal tissues.

Appendix 3. Score functions for codon optimization

In this subsection, we recall the definitions of some well-known score functions including CAI, CPB, and HSC (see, e.g. Sen et al. [43], Taneda and Asai [45], and references therein). Interested readers can find more score functions recently summarized in Paremskaia et al. [40].

CAI for the given sequence $D_j$ of codons is defined as follows, which is a measure for quantifying the use of fitted codons in given genes,

$$\begin{array}{rl}\text{CAI}\left(D_j\right)& ={(\prod W^{\text{CAI}}(d_{\mathrm{ji}}\left)\right)}^{\frac{1}{L}},\\ W^{\text{CAI}}\left(d_{\mathrm{ji}}\right)& =\frac{f\left(d_{\mathrm{ji}}\right)}{\underset{d\in \mathcal{H}\left(h_{\mathrm{ji}}\right)}{max}f\left(d\right)},\end{array}$$ (A13)

where, $W^{\text{CAI}}\left(d_{\mathrm{ji}}\right)$ is the relative adaptiveness of codon $d_{\mathrm{ji}}$ and $f\left(d\right)$ indicates the frequency of codon d in a given set of genes.

CPB is defined to be the log-odds ratio as follows, which is concerning how frequently each codon pair is observed in comparing with the expected value,

$$\begin{array}{rl}\text{CPB}\left(D_j\right)& =\frac{1}{L-1}\sum _{i=2}^L\text{CPS}(d_{j(i-1)},d_{\mathrm{ji}}),\\ \text{CPS}(d_{j(i-1)},d_{\mathrm{ji}})& =\text{log}\left(\left(\frac{f\left(h_{j(i-1)}\right)f\left(h_{\mathrm{ji}}\right)}{f\left(d_{j(i-1)}\right)f\left(d_{\mathrm{ji}}\right)}\right)\left(\frac{f(d_{j(i-1)},d_{\mathrm{ji}})}{f(h_{j(i-1)},h_{\mathrm{ji}})}\right)\right),\end{array}$$ (A14)

where, $\text{CPS}(d_{j(i-1)},d_{\mathrm{ji}})$ for $i\in \{2,\dots ,L\}$ is the codon pair score of a codon pair $(d_{j(i-1)},d_{\mathrm{ji}})$. Furthermore, the values of $f(d_{j(i-1)},d_{\mathrm{ji}})$, $f\left(h_{\mathrm{ji}}\right)$, and $f(h_{j(i-1)},h_{\mathrm{ji}})$ denote the corresponding codon pair, amino acid, and amino acid pair frequencies respectively.

HSC is defined as follows, which is expected to prevent erroneous translational frame shift by facilitating early termination,

$$\begin{array}{rl}\text{HSC}\left(D_j\right)& =\sum _{i=2}^L\text{hsc}(d_{j(i-1)},d_{\mathrm{ji}}),\\ \text{hsc}(d_{j(i-1)},d_{\mathrm{ji}})& =\sum _{\sigma \in {\sigma }^{\text{stop}}}\sum _{u\in \{1,2\}}{I}_{\text{end}}^{\sigma ,u}(d_{j(i-1)},d_{\mathrm{ji}}),\\ I_{\text{end}}^{\sigma ,u}\left(i\right)& =\{\begin{array}{ll}1& \text{if}\left(d_{j(3i-2-|\sigma |+u}\right),\dots ,d_{j(3i-3+u)})=\sigma ,\\ 0& \text{otherwise},\end{array}\end{array}$$ (A15)

where, σ is a forbidden motif and ${\sigma }^{\text{stop}}$ is a set of stop codons. Furthermore, $I_{\text{end}}^{\sigma ,u}\left(i\right)$ for an amino acid position index i indicates whether or not the codon sequence $(d_{j(i-k+1)},\dots ,d_{\mathrm{ji}})$ for a nonnegative integer k has the nucleotide sequence motif σ ending at the uth nucleotide position of $d_{\mathrm{ji}}$.

Funding Statement

The project is funded by the National Natural Science Foundation of China [grant number 11771006].

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

All data included in this study are available upon request by contact with the corresponding author.