Abstract
Developing control theory of gene regulatory networks is one of the significant topics in the field of systems biology, and it is expected to apply the obtained results to gene therapy technologies in the future. In this paper, a control method using a Boolean network (BN) is studied. A BN is widely used as a model of gene regulatory networks, and gene expression is expressed by a binary value (0 or 1). In the control problem, we assume that the concentration level of a part of genes is arbitrarily determined as the control input. However, there are cases that no gene satisfying this assumption exists, and it is important to consider structural control via external stimuli. Furthermore, these controls are realized by multiple drugs, and it is also important to consider multiple effects such as duration of effect and side effects. In this paper, we propose a BN model with two types of the control inputs and an optimal control method with duration of drug effectiveness. First, a BN model and duration of drug effectiveness are discussed. Next, the optimal control problem is formulated and is reduced to an integer linear programming problem. Finally, numerical simulations are shown.
1. Introduction
In the field of systems biology, there have been a lot of studies on modeling, analysis, and control of gene regulatory networks. Especially, control of gene regulatory networks corresponds to therapeutic interventions, which are realized by radiation, chemotherapy, and so on. In order to develop gene therapy technologies (see, e.g., [1]) in the future, developing control theory of gene regulatory networks is important. Furthermore, in recent years, the important result on control of gene regulatory networks has been obtained in [2]. That is, feedback control of synthetic biological circuits has been implemented, and the experimental result in which cellular behavior is regulated by control has been obtained. This result suggests that control methods of gene regulatory networks can be realized. Motivated by the above background, we study control methods of gene regulatory networks.
Gene regulatory networks are in general expressed by ordinary/partial differential equations with high nonlinearity and high dimensionality. In order to deal with such a system, it is important to consider a simple model, and various models such as Bayesian networks, Boolean networks (BNs) [3], hybrid systems (piecewise affine models), and Petri nets have been developed so far (see, e.g., [4] for further details). In control problems, BNs and hybrid systems are frequently used [5–8]. In the hybrid systems-based approach, the class of gene regulatory networks are limited to low-dimensional systems, because the computation time to solve the control problem is too long. In a BN, dynamics such as interactions between genes are expressed by the Boolean functions [3]; that is, gene expression is expressed by a binary value (0 or 1). There is a criticism that a Boolean network is too simple as a model of gene regulatory networks (see, e.g., [9]), but this model can be relatively applied to large-scale systems. In addition, since the behavior of gene regulatory networks is stochastic by the effects of noise, it is appropriate that a Boolean function is randomly decided at each time among the candidates of the Boolean functions. From this viewpoint, a probabilistic Boolean network (PBN) has been proposed in [10]. Furthermore, a context-sensitive PBN (CS-PBN) in which the deciding time is randomly selected has been proposed as a general form of PBNs [11, 12].
Furthermore, in the control theory of gene regulatory networks, the control input is given by the concentration level of a part of genes; that is, we assume that the concentration level of a part of genes can be arbitrarily determined. However, in the case where this assumption is not satisfied, it is important to consider structural control via external stimuli [13, 14]. These controls are realized by multiple drugs, and it is also important to consider multiple effects such as duration of effect and side effects [15]. To our knowledge, a unified method considering these properties has not been proposed so far.
Thus, in this paper, we propose a BN model with two types of the control inputs and an optimal control method with duration of drug effectiveness. The first control input is the control input satisfying the assumption that the binary value is arbitrarily determined. The second control input is called a structural control input herein, and the dynamics, that is, the Boolean functions, are selected among the candidates of the dynamics. However, it is difficult to uniquely select one Boolean function. Hence, we suppose that one Boolean function is selected probabilistically, and the probability distribution is switched by using the structural control input. A structural control method has been discussed in [13, 14], but the notion of the structural control input defined in this paper is different from that in those existing methods. Since the proposed BN model has a switch of the probability distribution, it may be regarded as a generalized version of PBNs.
In optimal control of PBNs and CS-PBNs, many results have been obtained so far (see, e.g., [11, 12, 16–21]). In many existing results, state transition diagrams with 2n nodes (i.e., 2n × 2n transition probability matrices) must be computed for a PBN with n states. As a result, in order to compute state transition diagrams, several issues such as memory consumption must be considered in implementation, and it is desirable to directly use a given Boolean function. The authors have proposed in [22] a control method in which state transition diagrams are not computed. In many existing results, we consider finding a control input such that the expected value of the cost function is minimized. In [22], we consider finding a control input such that the lower bound of the cost function is minimized under a certain constraint condition. Owing to this difference, state transition diagrams are not computed in the method in [22], and the optimal control problem is reduced to an integer linear programming (ILP) problem. Also in [16], ILP-based methods were proposed for other optimal control problems, and in those methods, solving multiple ILP problems is required.
In this paper, based on our previously proposed method [22], the optimal control problem with duration of drug effectiveness is reduced to an ILP problem. Since a given Boolean function is directly used, duration of drug effectiveness can be easily described as a linear inequality constraint. The proposed method provides us with a basic in control theory of gene regulatory networks. The conference paper [23] is a preliminary version of this paper. In this paper, we provide improved formulations and explanations, a discussion on duration of drug effectiveness, and a numerical simulation using the large-scale BN.
This paper is organized as follows. In Section 2.1, the Boolean networks with two kinds of the control inputs are proposed. In Section 2.2, duration of drug effectiveness is introduced. In Section 2.3, the optimal control problem is formulated. In Section 2.4, its solution method is proposed. In Section 3, two numerical examples are presented. In Section 4, we conclude this paper.
Notation 1 —
Let ℛ denote the set of real numbers. Let {0,1}n denote the set of n-dimensional vectors, which consists of elements 0 and 1. Let I n and 0m×n denote the n × n identity matrix and the m × n zero matrix, respectively. For simplicity, we sometimes use the symbol 0 instead of 0m×n and and the symbol I instead of I n. For a matrix M, lnM denotes the matrix such that the (i, j)th element is given as the natural logarithm of the (i, j)th element in M. For a matrix M, M T denotes the transpose of M.
2. Materials and Methods
2.1. The Boolean Networks with Control Inputs
A Boolean network (BN) with n states is given by
(1) |
where x ∈ {0,1}n is the state (e.g., the concentration of genes) and k = 0,1, 2,… is the discrete time. The function f a : {0,1}n → {0,1}n is a given Boolean function with logical operators such as AND (∧), OR (∨), and NOT (¬). If the BN (1) is deterministic, then the next state x(k + 1) is uniquely determined for a given x(k). See also Example 1 for an example.
Next, the control inputs are added to a BN (1). For the BN (1) with n state, consider two types of the control inputs. First, in a similar way to that of the conventional control method, the control input is added to the BN (1) as follows:
(2) |
where u ∈ {0,1}m is the control input; that is, the value of u (e.g., the concentration of genes) can be arbitrarily given, and f : {0,1}n × {0,1}m → {0,1}n is a given Boolean function. The ith element of the state x and the ith element of the control input u are denoted by x i and u i, respectively. In the BN (2), x(k + 1) is uniquely determined for the given x(k) and u(k).
Then, consider the structural control input. Suppose that the candidates of f are given by f i, i = 1,2,…, l. It will be difficult to select one Boolean function uniquely. In this paper, we assume that one discrete probability distribution is selected among m s discrete probability distributions. Probabilistic distributions are derived from experimental results, but details are one of the future works. Then, a method for inferring a probabilistic Boolean network will be useful (see, e.g., [24]). Let r i,j denote the probability that the Boolean function f j is selected in the ith discrete probability distribution. Then,
(3) |
hold. In addition, m s-dimensional binary variables u s ∈ {0,1}ms are assigned to m s discrete probability distributions, and let u i s denote the ith element of u s. The structural control input u s corresponds to m s kinds of external stimuli. Then, the equality constraint
(4) |
is imposed. Here, we show a simple example.
Example 1 —
As a simple example, consider the simplified model of an apoptosis network in Figure 1 [25]. Then, the Boolean network model expressing this apoptosis network is given by
(5) where the concentration level (high or low) of the inhibitor of apoptosis proteins (IAPs) is denoted by x 1, the concentration level of the active caspase 3 (C3a) is denoted by x 2, and the concentration level of the active caspase 8 (C8a) is denoted by x 3. The concentration level of the tumor necrosis factor (TNF, a stimulus) is denoted by u and is regarded as the control input. Since the caspase C3a is responsible for cleaving or breaking many other proteins, a high level of the C3a concentration, that is, x 2 = 1, implies cell near-death, otherwise, cell survival. As seen in (5), if the concentration of IAP is high (x 1 = 1) or the concentration of the caspase C8a is low (x 3 = 0), then the concentration of C3a becomes low; that is, x 2 = 0. On the other hand, x 1 and x 3 at the next time depend on the value of x 2 as well as u. In this way, some dynamical interactions exist. See [25, 26] for further details.
Suppose that l = 2 and m s = 2. Then, as an example of the candidates of the Boolean functions, we consider the following:
(6) We suppose that the Boolean function f 1 expresses the situation that the dynamics of an apoptosis network are selected with high probability and that the Boolean function f 2 expresses the situation that the state is not changed with high probability. By using u 1 s and u 2 s, one of the two discrete probability distributions {r 1,1, r 1,2} and {r 2,1, r 2,2} is selected at each time.
A BN with two types of the control inputs includes the probabilistic behavior, and we assume that the probability distribution can be controlled. From these facts, a BN studied in this paper can be regarded as a generalized form of a probabilistic Boolean network (PBN). To explain the relation between the proposed BN model and a PBN, we show a simple example.
Example 2 —
As a simple example, consider the PBN with three states and one control input. Suppose that the Boolean functions are given as follows:
(7) This PBN corresponds to the cases of l = 4 and m s = 1. The candidates of the Boolean functions f i, i = 1,2, 3,4, and the probabilities r 1,j, j = 1,2, 3,4, are obtained as follows:
(8) Next, consider the state orbit of this PBN. In PBNs, one Boolean function is probabilistically selected at each time. Then, for and u(0) = 0, we obtain the following:
(9) In this example, the cardinality of the finite state set {0,1}3 is given by 23 = 8, and we obtain the state transition diagram of Figure 2 by computing the transition from each value of the state. In Figure 2, the number assigned to each node denotes x 1, x 2, and x 3 (each element of the state), and the number assigned to each arc denotes the transition probability from some state to another state. For simplicity of illustration, the state transitions from are illustrated in Figure 2. In the existing solution methods for optimal control of PBNs, the optimal control input is computed using dynamic programming with state transition diagrams.
As shown in this example, computing state transition diagrams with 2n nodes (n is the number of the state) is required in the existing solution methods for optimal control of PBNs with n states, and this computation is hard for large-scale systems (see also Section 3.2). Thus, it is important to consider a new solution method. In this paper, for BNs with two types of the control inputs, a solution method using integer programming is proposed based on our previously proposed work in [22]. In the proposed method, computation of state transition diagrams such as that in Figure 2 is not needed.
Remark 3 —
By adding the candidates of the Boolean functions, BNs with two types of the control inputs can be transformed into BNs with only the structural control input. That is, the control input u can be eliminated from (2) by fixing the value of u in (2). Then, the number of the candidates of Boolean functions is 2m l, and 2m combinations for u must be computed in advance. To avoid this computation, we consider two types of the control inputs.
2.2. Duration of Drug Effectiveness
The control input u and the structural control input u s are realized by using multiple drugs. Then, we must consider the multiple effects such as duration of effect and the side effects. In this paper, we focus on the duration of drug effectiveness. In, for example, chemotherapy, therapeutic intervention is generally applied to the target cell in a cyclic manner [15]. Each therapeutic window is started by delivering the drug. The drug delivered is effective on the target cell for some period of time. This is followed by a recovery phase. However, when the drug is not delivered, the drug may be delivered in the timing that is faster than the next time in a cyclic [15]. Therefore, it is necessary to model several situations on duration of drug effectiveness. To model the duration of effect, three parameters L ui, W ui 1, and W ui 0 are defined for each input u i (or u i s). The parameters L ui and W ui 1 have been already defined in [15].
The parameter L ui is the length of the drug effectiveness period. That is, if u i(k) = 1, then u i(k + 1) = u i(k + 2) = ⋯ = u i(k + L ui) = 1 holds. Next, W ui 1(>L ui) is explained. If u i(k) = 1, then u i(k + 1), u i(k + 2),…, u i(k + W ui 1 − 1) is uniquely determined depending on L ui, and u i(k + W ui 1) is a decision variable. Then, W ui 1 − L ui − 1 corresponds to the length of a recovery phase. Finally, W ui 0 is explained. If u i(k) = 0, then u i(k + 1) = u i(k + 2) = ⋯ = u i(k + W ui 0 − 1) = 0 holds, and u i(k + W ui 0) is a decision variable. By using L ui, W ui 1, W ui 0, we can consider several situations, and we show two typical examples.
Example 4 —
First, suppose that, for the control input u ∈ {0,1}1, L u, W u 1, and W u 0 are given as L u = 1, W u 1 = 3, and W u 0 = 2, respectively. Consider the case of u(k) = 1. Then, u(k + 1) and u(k + 2) are uniquely determined as u(k + 1) = 1 and u(k + 2) = 0, respectively, and u(k + 3) is a decision variable. Then, u(k + 2) = 0 is the recovery phase, and W u 1 − L u − 1 = 1 is its length. In the case of u(k) = 0, the relation u(k + 1) = 0 holds, and u(k + 2) is a decision variable.
Another example is shown. Suppose that, for the control input u ∈ {0,1}1, L u, W u 1, and W u 0 are given as L u = 0, W u 1 = 3, and W u 0 = 3, respectively. In both, the case of u(k) = 1 and the case of u(k) = 0, u(k + 1) = u(k + 2) = 0 holds, and u(k + 3) is a decision variable. In this case, u(k + 1) = u(k + 2) = 0 is the recovery phase, and W u 1 − L u − 1 = 2 is its length.
By using the three parameters L ui, W ui 1, and W ui 0, several situations on the duration of effect can be modeled (see also [15]). In addition, since these parameters can be given for each u i (or u i s), effectiveness of multiple drugs can be evaluated. Thus, in this paper, we consider not only two types of the control inputs but also duration of drug effectiveness.
2.3. Optimal Control Problem
First, the following two notations are defined. Let π i(k) denote the probability that some Boolean function f i is selected at time k. In addition, the probability that some time sequence of theBoolean functions f i(k1), f i(k1+1),…, f i(k2) is selected at time interval [k 1, k 2] is denoted by
(10) |
For simplicity of notation, i(k 1), i(k 1 + 1),…, i(k 2) are omitted in π(k 1, k 2).
Next, for the Boolean networks with n states and two types of the control inputs, consider the following optimal control problem.
Problem 1 —
Suppose that, for the Boolean network with n states and two types of the control inputs, the initial state x(0) = x 0, ρ satisfying 0 ≤ ρ ≤ 1, the control time N, the parameters on duration of drug effectiveness L ui(uis), W ui(uis) 1, and W ui(uis) 0 are given. Then, for all combinations of the Boolean functions satisfying the constraint
(11) find two control input sequences u(0), u(1),…, u(N − 1) and u s(0), u s(1),…, u s(N − 1) minimizing the lower bound of the cost function
(12) subject to the constraint on duration of drug effectiveness, where Q, Q f ∈ ℛ 1×n, R ∈ ℛ 1×m, and R ∈ ℛ 1×ms are weighting vectors whose elements are nonnegative real numbers.
For simplicity of discussion, a linear function with respect to x, u, and u s is considered as a cost function. We consider that a linear cost function is appropriate from the following two reasons.
For a binary variable δ ∈ {0,1}, the relation δ 2 = δ holds. That is, in the cost function, the quadratic term such as x i 2(k) is not necessary.
In control of gene regulatory networks, the expression of a certain gene is frequently focused (see, e.g., [18]). For example, in the gene regulatory network related to melanoma, it important to inhibit the concentration level of the gene WNT5A [27]. In this case, it is enough to consider the cost function (12).
Furthermore, in many existing methods on optimal control of PBNs, the expected value of a nonnegative function is frequently used as a cost function (see, e.g., [11, 12, 17–21]). However, the expected value of the state must be computed from all combinations of the Boolean functions, and this computation is hard for large-scale systems. To avoid this computation, in this paper, we evaluate the control performance by using the lower bound. If the constraint (11) is not included in Problem 1, then the behaviors are regarded as uncertain (nondeterministic) behaviors, and the best performance is derived in Problem 1. Since the combinations of the Boolean functions selected with low probability are included, performance evaluation is not appropriate. In order to exclude such combinations, we impose the constraint (11). Similar problem formulations have been considered in optimal control of stochastic hybrid systems (see, e.g., [28–30]). Thus, since the performance index in this paper is different from that in existing methods, it is difficult to directly compare the performance of the proposed method with those of existing methods. On the other hand, in [22], we discussed this topic from the qualitative viewpoint. In [22], the upper bound is also computed by using the control input such that the lower bound is minimized. If the lower bound and the upper bound are not improved by control, then the expected value will not be improved. Then, it is important to suitably set ρ in the constraint (11). See [22] for further details.
We show an example for setting weighting vectors from the biological viewpoint.
Example 5 —
Consider the Boolean network expressing an apoptosis network in Example 1 again. For this system, we consider finding a control strategy such that a stimulus u is not applied as much as possible, and cell survival is achieved; u = 0 implies that a stimulus is not applied to the system, and x 1 = 1 and x 2 = 0 express cell survival [25]. Then, as one of the appropriate cost functions, we can consider the following cost function:
(13) By the coordinate transformation of x 1 into 1 − x 1, this cost function can be rewritten as the form of (12).
2.4. Solution Method
We propose a solution method for Problem 1. First, two lemmas are introduced as preparations. Next, Problem 1 is reduced to an integer linear programming (ILP) problem.
As preparations, two lemmas are introduced. To reduce Problem 1 to an ILP problem, it is necessary to transform a Boolean function into a polynomial on the real number field. First, the following lemma [31] is used.
Lemma 6 —
Consider the two binary variables δ 1, and δ 2. Then, the following relations hold:
¬δ 1 is equivalent to 1 − δ 1,
δ 1∨δ 2 is equivalent to δ 1 + δ 2 − δ 1 δ 2,
δ 1∧δ 2 is equivalent to δ 1 δ 2.
For example, δ 1∨¬δ 2 is equivalently transformed into δ 1 + (1 − δ 2) − δ 1(1 − δ 2) = 1 − δ 2 + δ 1 δ 2. Furthermore, the product of binary variables such as δ 1 δ 2 can be linearized by using the following lemma [32].
Lemma 7 —
Suppose that the binary variables δ j ∈ {0,1} and j ∈ 𝒥 are given, where 𝒥 is some index set. Then, z = ∏j∈𝒥 δ j is equivalent to the following linear inequalities:
(14) where |𝒥| is the cardinality of 𝒥.
From Lemmas 6 and 7, we see that any Boolean function can be equivalently transformed into a pair of some linear function and some linear inequality. See [31, 32] for further details. For example, δ 1∨¬δ 2 is equivalent to a pair of 1 − δ 2 + z and z = δ 1 δ 2. By using Lemma 7, z = δ 1 δ 2 can be expressed as a set of linear inequalities.
Now, we consider reducing Problem 1 to an ILP problem.
By using Lemma 6, the candidates of the Boolean functions f i(x(k), u(k)), i = 1,2,…, l, are transformed into a polynomial on the real number field. Let denote the polynomial obtained. Then, consider the following system using :
(15) |
where δ 1(k), δ 2(k),…, δ l(k) are binary variables satisfying
(16) |
The binary vector is used to select the polynomial and to express (11) as a linear form. Here, we define the following vector:
(17) |
Then, by using the natural logarithm, π(0, N − 1) in (11) is expressed as
(18) |
In this expression, one probability distribution is selected by using u i s(k), and the probability that a certain Boolean function is selected is determined by δ(k). Then, Problem 1 is equivalent to the following problem.
(19) |
By using Lemma 7, the system (15) and (∑i=1 mslnS i δ i s(k))δ(k) can be equivalently expressed in the following linear form:
(20) |
(21) |
(22) |
where z i s(k) : = δ i s(k)δ(k), and (22) is the linear inequality obtained by applying Lemma 7 to (15). The vector z b(k)∈{0,1}b is an auxiliary binary variable obtained by using Lemma 7, and the dimension of z b(k), that is, b, is determined depending on the form of the given Boolean functions. In addition, z(k) is defined as
(23) |
Here in after, for simplicity of notation, B b z b(k) is rewritten as B z z(k), , and ∑i=1 mslnS i z i s(k) is rewritten as Cz(k), .
Now, we consider transforming Problem A by using (20), (21), and (22). By using
(24) |
obtained from the state equation in (20), we can obtain
(25) |
where
(26) |
Next, the inequality constraint ∑k=0 N−1(∑i=1 mslnS i δ i s(k))δ(k) ≥ lnρ in Problem A is equivalent to
(27) |
where . Furthermore, from (22), we can obtain
(28) |
where
(29) |
Next, consider the constraint on duration of drug effectiveness. This constraint can be expressed as a Boolean function. Then, by using Lemmas 6 and 7, it can be transformed into the following form:
(30) |
(31) |
where and are binary decision variables with certain dimensions. Deriving a general form of coefficient matrices will be difficult, but for the given L ui, W ui 1, and W ui 0, deriving coefficient matrices is easy.
We show two examples.
Example 8 —
Consider Example 4 again. First, consider the case of L u = 1, W u 1 = 3, and W u 0 = 2. Then, noting explanations in Example 4, we can obtain
(32) and in this case, these are equivalent to
(33) We explain u(2) = v 2, and v 2 ≤ 1 − v 0 as an example. If v 0 = 1, then v 2 ≤ 0 holds. Since v 2 is binary, we can obtain v 2 = 0; that is, u(2) = 0. If v 0 = 0, then v 2 ≤ 1 holds, and we can obtain u(2) = v 2. That is, u(2) can take on either 0 or 1. From the previous discussion, we see that a pair of u(2) = v 2 and v 2 ≤ 1 − v 0 is equivalent to u(2) = (1 − v 0)v 2. Thus, we can obtain the forms of (30) and (31). In the case of N = 5 (N is the control time in Problem 1), (30) can be obtained as
(34) where V 2 = 0. We remark that, in a general case, the product such as z = v 0 v 2 must be transformed into linear inequalities by using Lemma 7.
Next, consider the case of L u = 0, W u 1 = 3, and W u 0 = 3. Then, we can obtain the following:
(35) and this case is one of the simplest cases. In the case of N = 5 (N is the control time in Problem 1), (30) can be obtained as
(36) where V 2 = 0, W 1 = 0, and W 2 = 0.
Finally, the cost function (12) is rewritten as
(37) |
where , , and . By substituting (25), (30), and (31) into (28) and (37), Problem A is equivalent to the following ILP problem.
(38) |
Problem B can be solved by a suitable solver such as IBM ILOG CPLEX [33].
3. Results and Discussion
In this section, we show numerical simulations. First, we consider the WNT5A network [14]. Next, in order to evaluate the proposed method from the viewpoint of the computation time, we consider an artificial example.
3.1. WNT5A Network
The gene regulatory network with the gene WNT5A is related to melanoma, and it has been extensively studied (see, e.g., [27]). The BN model x(k + 1) = f a(x(k)) of the WNT5A network is given by the following:
(39) |
where the concentration level (high or low) of the gene WNT5A is denoted by x 1, the concentration level of the gene pirin by x 2, the concentration level of the gene S100P is denoted by x 3, the concentration level of the gene RET1 is denoted by x 4, the concentration level of the gene MART1 is denoted by x 5, the concentration level of the gene HADHB is denoted by x 6, and the concentration level of the gene STC2 is denoted by x 7. See [14] for further details. In a WNT5A network, it is important to inhibit the concentration level of the gene WNT5A [27].
The optimal control problem is formulated. For simplicity, we consider only the structural control input. Then, suppose that the number of the structural control inputs is two. If u 1 s(k) = 1, then the system is given as:
(40) |
If u 2 s(k) = 1, then the system is given as:
(41) |
The case of u 1 s(k) = 1 corresponds to the situation such that the dynamics of the WNT5A network, that is, x(k + 1) = f a(x(k)), are selected with high probability. The case of u 2 s(k) = 1 corresponds to the situation such that the state is not changed; that is, x(k + 1) = x(k) is selected with high probability. From the previous setting, r 1,1 = 0.8, r 1,2 = 0.2, r 2,1 = 0.1, and r 2,2 = 0.9. For this WNT5A network with structural control inputs, consider solving Problem 1. Q, Q f, and R s in Problem 1 are given as , , and , respectively. The initial state is given as . In addition, the control time N in Problem 1 is given by N = 5. Finally, the constraint on duration of drug effectiveness is imposed for only u 2 s(k). The parameters L u2s, W u2s 1, and W u2s 0 are given as L u2s = 1, W u2s 1 = 3, and W u2s 0 = 2, respectively. Thus, we can obtain the ILP problem (Problem B), where the dimension of binary variables is 130 and the number of inequalities is 264.
We show the computational result. Let denote the optimal value of the lower bound of a given cost function in Problem 1. Let denote the upper bound of the cost function derived by using the optimal control input. First, consider the case of ρ = 10−5. Then, we can obtain and , and u s(k) is obtained as
(42) |
Noting that r 2,1 = 0.1 and ρ = 10−5( = 0.15), all combinations of the Boolean functions are considered, and the value of ρ is not appropriate. In particular, implies that x 1(k) = 1, k = 0,1,…, 5, and is the trivial upper bound.
Next, consider the case of ρ = 0.2. Then, we can obtain , and u s(k) is obtained as
(43) |
From the obtained inputs, we see that the system is controlled by switching two discrete probability distributions, and the obtained inputs satisfy the constraint on duration of drug effectiveness. Noting that the trivial value of is 15, we see that in this case the effectiveness of control synthesis is clear.
Finally, we discuss the computation time for solving Problem 1. The computation time of the ILP problem was less than 20 [msec], where we used IBM ILOG CPLEX 11.0 as an ILP solver on the computer with Windows Vista 32-bit, the Intel Core 2 Duo CPU 3.0 GHz, and the 4 GB memory. Since the WNT5A network considered here is small size, Problem 1 can be solved fast.
3.2. Artificial Example
In order to evaluate the computation time for solving Problem 1, we consider one artificial example of a BN with 15 states and 3 control inputs. We stress that the existing method [11, 12, 17–19, 21] cannot be applied to such a BN. This is because it is necessary to compute the state transition diagram such as that in Figure 2, that is, the transition probability matrix with 2n × 2n. In naive implementation using MATLAB [34], matrices with 215 × 215 cannot be created due to memory consumption, where we used the computer described previously.
The optimal control problem is formulated. In this example, we consider 3 control inputs and 2 structural control inputs. If u 1 s(k) = 1, then the system is given as follows:
(44) |
If u 2 s(k) = 1, then the system is given as follows:
(45) |
The Boolean function f 1 is given by the following:
(46) |
The Boolean function f 2 is given by the following:
(47) |
From the previous setting, r 1,1 = 0.8, r 1,2 = 0.2, r 2,1 = 0.2, r 2,2 = 0.8 hold. In Problem 1, Q, Q f, R, and R s are given as , , , and , respectively. The initial state and the parameter ρ are given as and ρ = 10−4, respectively. The constraint on duration of drug effectiveness is imposed for u 1(k) and u 2 s(k). For u 1, the parameters L u1, W u1 1, and W u1 0 are given as L u1 = 0, W u1 1 = 3, and W u1 0 = 3, respectively. For u 2 s, the parameters L u2s, W u2s 1, and W u2s 0 are given as L u2s = 1, W u2s 1 = 3, and W u2s 0 = 2, respectively.
Next, we discuss the computation time. Consider the two cases of N = 10 and N = 20. Then, in the ILP problem (Problem B) obtained, the dimension of binary variables is 1420 for N = 10 and 2840 for N = 20, and the number of inequalities is 3381 for N = 10 and 6731 for N = 20. In the case of N = 10, the computation time of the ILP problem was 96 [sec], where we used the computer described previously. In the case of N = 20, the computation time of the ILP problem was 238 [sec]. We remark that BNs with such a size are large scale in control problems of gene regulatory networks. Thus, we conclude that Problem 1 can be solved within the practical computation time.
4. Conclusions
In this paper, we have proposed a Boolean network (BN) model with two types of the control inputs and an optimal control method. By using this model, several situations in control of gene regulatory networks can be modeled. To model more realistic situations, duration of drug effectiveness has also been introduced. Since duration is given for each control input, effectiveness of multiple drugs can be evaluated. Furthermore, for this BN model, the optimal control problem has been formulated, and this problem is reduced to an integer linear programming problem. Finally, numerical simulations have been shown. The proposed method provides us with a basic in the control theory of gene regulatory networks.
Recently, to simplify state transition diagrams such as that in Figure 2, a stochastic Boolean network has been proposed in [35]. The authors proposed in [36] a similar method using polynomial optimization. In addition, to simplify a given Boolean network, the Karnaugh map realization of a Boolean network has been proposed in [37]. These methods are useful for reducing the computational burden. It is one of the future works to consider the control problem with duration of drug effectiveness based on these methods.
Acknowledgment
This work was partially supported by the Grant-in-Aid for Young Scientists (B) no. 23760387.
References
- 1.Santos-Rosa H, Caldas C. Chromatin modifier enzymes, the histone code and cancer. European Journal of Cancer. 2005;41(16):2381–2402. doi: 10.1016/j.ejca.2005.08.010. [DOI] [PubMed] [Google Scholar]
- 2.Milias-Argeitis A, Summers S, Stewart-Ornstein J, et al. In silico feedback for in vivo regulation of a gene expression circuit. Nature Biotechnology. 2011;29(12):1114–1116. doi: 10.1038/nbt.2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Kauffman SA. Metabolic stability and epigenesis in randomly constructed genetic nets. Journal of Theoretical Biology. 1969;22(3):437–467. doi: 10.1016/0022-5193(69)90015-0. [DOI] [PubMed] [Google Scholar]
- 4.Jong HD. Modeling and simulation of genetic regulatory systems: a literature review. Journal of Computational Biology. 2002;9(1):67–103. doi: 10.1089/10665270252833208. [DOI] [PubMed] [Google Scholar]
- 5.Akutsu T, Hayashida M, Ching W, Ng MK. Control of Boolean networks: hardness results and algorithms for tree structured networks. Journal of Theoretical Biology. 2007;244(4):670–679. doi: 10.1016/j.jtbi.2006.09.023. [DOI] [PubMed] [Google Scholar]
- 6.Azuma S, Yanagisawa E, Imura J. Controllability analysis of biosystems based on piecewise-affine systems approach. IEEE Transactions on Automatic Control. 2008;53:139–152. [Google Scholar]
- 7.Kobayashi K, Imura J, Hiraishi K. Polynomial-time algorithm for controllability test of a class of Boolean biological networks. Eurasip Journal on Bioinformatics and Systems Biology. 2010;2010:12 pages. doi: 10.1155/2010/210685.210685 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Langmead CJ, Jha SK. Symbolic approaches for finding control strategies in boolean networks. Journal of Bioinformatics and Computational Biology. 2009;7(2):323–338. doi: 10.1142/s0219720009004084. [DOI] [PubMed] [Google Scholar]
- 9.Mochizuki A. An analytical study of the number of steady states in gene regulatory networks. Journal of Theoretical Biology. 2005;236(3):291–310. doi: 10.1016/j.jtbi.2005.03.015. [DOI] [PubMed] [Google Scholar]
- 10.Shmulevich I, Dougherty ER, Kim S, Zhang W. Probabilistic Boolean networks: a rule-based uncertainty model for gene regulatory networks. Bioinformatics. 2002;18(2):261–274. doi: 10.1093/bioinformatics/18.2.261. [DOI] [PubMed] [Google Scholar]
- 11.Faryabi B, Vahedi G, Chamberland JF, Datta A, Dougherty ER. Intervention in context-sensitive probabilistic boolean networks revisited. Eurasip Journal on Bioinformatics and Systems Biology. 2009;2009:13 pages. doi: 10.1155/2009/360864.360864 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Pal R, Datta A, Bittner ML, Dougherty ER. Intervention in context-sensitive probabilistic Boolean networks. Bioinformatics. 2005;21(7):1211–1218. doi: 10.1093/bioinformatics/bti131. [DOI] [PubMed] [Google Scholar]
- 13.Shmulevich I, Dougherty ER, Zhang W. Control of stationary behavior in probabilistic Boolean networks by means of structural intervention. Journal of Biological Systems. 2002;10(4):431–445. [Google Scholar]
- 14.Xiao Y, Dougherty ER. The impact of function perturbations in Boolean networks. Bioinformatics. 2007;23(10):1265–1273. doi: 10.1093/bioinformatics/btm093. [DOI] [PubMed] [Google Scholar]
- 15.Yousefi MR, Datta A, Dougherty ER. Optimal intervention strategies for therapeutic methods with fixed-length duration of drug effectiveness. IEEE Transactions on Signal Processing. 2012;60(9):4930–4944. [Google Scholar]
- 16.Chen X, Akutsu T, Tamura T, Ching WK. Finding optimal control policy in probabilistic Boolean networks with hard constraints by using integer programming and dynamic programming. International Journal of Data Mining and Bioinformatics. 2013;7(3):321–343. doi: 10.1504/ijdmb.2013.053306. [DOI] [PubMed] [Google Scholar]
- 17.Ching WK, Zhang SQ, Jiao Y, Akutsu T, Tsing NK, Wong AS. Optimal control policy for probabilistic Boolean networks with hard constraints. IET Systems Biology. 2009;3(2):90–99. doi: 10.1049/iet-syb.2008.0120. [DOI] [PubMed] [Google Scholar]
- 18.Datta A, Choudhary A, Bittner ML, Dougherty ER. External control in Markovian genetic regulatory networks. Machine Learning. 2003;52(1-2):169–191. doi: 10.1093/bioinformatics/bth008. [DOI] [PubMed] [Google Scholar]
- 19.Datta A, Choudhary A, Bittner ML, Dougherty ER. External control in Markovian genetic regulatory networks: the imperfect information case. Bioinformatics. 2004;20(6):924–930. doi: 10.1093/bioinformatics/bth008. [DOI] [PubMed] [Google Scholar]
- 20.Denic SZ, Vasic B, Charalambous CD, Palanivelu R. Robust control of uncertain context-sensitive probabilistic Boolean networks. IET Systems Biology. 2009;3(4):279–295. doi: 10.1049/iet-syb.2008.0121. [DOI] [PubMed] [Google Scholar]
- 21.Pal R, Datta A, Dougherty ER. Optimal infinite-horizon control for probabilistic Boolean networks. IEEE Transactions on Signal Processing. 2006;54(6):2375–2387. [Google Scholar]
- 22.Kobayashi K, Hiraishi K. An integer programming approach to optimal control problems in context-sensitive probabilistic Boolean networks. Automatica. 2011;47(6):1260–1264. [Google Scholar]
- 23.Kobayashi K, Hiraishi K. Probabilistic control of Boolean networks with multiple dynamics: Towards control of gene regulatory. Proceedings of the 50th IEEE Conference on Decision and Control and European Control Conference (CDC-ECC); 2011; Orlando, Fla, USA. pp. 4371–4376. [Google Scholar]
- 24.Marshall S, Yu L, Xiao Y, Dougherty ER. Inference of a probabilistic boolean network from a single observed temporal sequence. Eurasip Journal on Bioinformatics and Systems Biology. 2007;2007:15 pages. doi: 10.1155/2007/32454.32454 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Chaves M. Methods for qualitative analysis of genetic networks. Proceedings of the European Control Conference; 2009; pp. 671–676. [Google Scholar]
- 26.Tournier L, Chaves M. Uncovering operational interactions in genetic networks using asynchronous Boolean dynamics. Journal of Theoretical Biology. 2009;260(2):196–209. doi: 10.1016/j.jtbi.2009.06.006. [DOI] [PubMed] [Google Scholar]
- 27.Weeraratna AT, Jiang Y, Hostetter G, et al. Wnt5a signaling directly affects cell motility and invasion of metastatic melanoma. Cancer Cell. 2002;1(3):279–288. doi: 10.1016/s1535-6108(02)00045-4. [DOI] [PubMed] [Google Scholar]
- 28.Adamek F, Sobotka M, Stursberg O. Stochastic optimal control for hybrid systems with uncertain discrete dynamics. Proceedings of the 4th IEEE Conference on Automation Science and Engineering (CASE ’08); August 2008; Arlington, Va, USA. pp. 23–28. [Google Scholar]
- 29.Bemporad A, Di Cairano S. Optimal control of discrete hybrid stochastic automata. (Lecture Notes in Computer Science).Hybrid Systems: Computation and Control. 2005;3414:417–432. [Google Scholar]
- 30.Kobayashi K, Matou K, Hiraishi K. Probabilistic-constrained optimal control of a class of stochastic hybrid systems. International Journal of Control, Automation and Systems. 2012;10(5):897–904. [Google Scholar]
- 31.Williams HP. Model Building in Mathematical Programming. 4th edition. New York, NY, USA: John Wiley & Sons; 1999. [Google Scholar]
- 32.Cavalier TM, Pardalos PM, Soyster AL. Modeling and integer programming techniques applied to propositional calculus. Computers and Operations Research. 1990;17(6):561–570. [Google Scholar]
- 33.IBM ILOG CPLEX Optimizer. http://www-01.ibm.com/software/commerce/optimization/cplex-optimizer/index.html.
- 34.MathWorks MATLAB. http://www.mathworks.com/
- 35.Liang J, Han J. Stochastic Boolean networks: an efficient approach to modeling gene regulatory networks. BMC Systems Biology. 2012;6(article 113):20 pages. doi: 10.1186/1752-0509-6-113. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Kobayashi K, Hiraishi K. Optimal control of probabilistic Boolean networks using polynomial optimization. IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences. 2012;95(9):1512–1517. [Google Scholar]
- 37.Layek RK, Datta A, Dougherty ER. From biological pathways to regulatory networks. Molecular BioSystems. 2011;7(3):843–851. doi: 10.1039/c0mb00263a. [DOI] [PubMed] [Google Scholar]