Using evolutionary computations to understand the design and evolution of gene and cell regulatory networks

Abstract

This paper surveys modeling approaches for studying the evolution of gene regulatory networks (GRNs). Modeling of the design or ‘wiring’ of GRNs has become increasingly common in developmental and medical biology, as a means of quantifying gene-gene interactions, the response to perturbations, and the overall dynamic motifs of networks. Drawing from developments in GRN ‘design’ modeling, a number of groups are now using simulations to study how GRNs evolve, both for comparative genomics and to uncover general principles of evolutionary processes. Such work can generally be termed evolution in silico. Complementary to these biologically-focused approaches, a now well-established field of computer science is Evolutionary Computations (EC), in which highly efficient optimization techniques are inspired from evolutionary principles. In surveying biological simulation approaches, we discuss the considerations that must be taken with respect to: a) the precision and completeness of the data (e.g. are the simulations for very close matches to anatomical data, or are they for more general exploration of evolutionary principles); b) the level of detail to model (we proceed from ‘coarse-grained’ evolution of simple gene-gene interactions to ‘fine-grained’ evolution at the DNA sequence level); c) to what degree is it important to include the genome’s cellular context; and d) the efficiency of computation. With respect to the latter, we argue that developments in computer science EC offer the means to perform more complete simulation searches, and will lead to more comprehensive biological predictions.

1. Introduction

A central challenge to understanding gene expression is to recreate the regulatory networks controlling expression; or at the least to generate models of these networks which capture essential characteristics of their connectivity and control, and which can be quantitatively analyzed. By developing such a quantitative theory of gene control, we can hope to develop far more powerful experimental tests and applications. A great deal of diverse work has gone into developing and testing models of gene regulatory networks (GRNs) in recent decades. As GRN models become more developed, allowing greater understanding of the particular ‘wiring’ or regulatory dynamics involved in particular cases (the ‘design’), it becomes natural to ask how these GRNs evolved, both for comparative genomics and to uncover general principles of evolutionary processes. This paper surveys approaches that have been developed for the simulation of GRN evolution. At the same time, considerable work has been done on the computing side, with ‘evolutionary computations’ (EC) now a well-developed branch of computer science. Here the focus is on optimization, inspired by the mechanisms of biological evolution, for a broad range of problems which are challenging for traditional computational methods. The biological and computer science approaches have operated somewhat independently, but have much to offer each other. We feel, especially, that simulations of GRN evolution, which are computationally intensive, could benefit from increased use of EC optimization techniques and the analytical tools that come with them. At the same time, the computer science stands to benefit greatly from increased appreciation and use of the complexity of biological evolutionary dynamics.

We will focus on GRNs, but more recent modeling developments consider the cellular context of gene regulation, e.g. including post-transcriptional dynamics, protein-protein interactions, modification of transport properties, etc. We will discuss some of these developments in the context of ‘cell regulatory networks’ (CRNs), but with the emphasis on gene regulation (not, for example, on physiological or metabolic regulation).

Many GRN computational projects fit expression data for particular phenomena to particular models, in order to characterize the design of the network. In this, there is a wide array of choices regarding the way in which the network is represented and the level of detail modeled. The computational problem is largely one of parameter estimation (since, as a generality, expression data is incomplete and imprecise). Evolution of parameter sets can be a way in which to optimize the model, and is most in line with the computer science EC approach. While such parameter searches for specific models can shed light on possible evolutionary trajectories, the more that constraints on connectivity and types of interaction in the network are relaxed, the more that simulations represent broader evolutionary dynamics. We refer to the projects focusing on the design of a particular network or the generation of a particular function (e.g. bistable behavior) as the evolutionary design of GRNs (or CRNs); and projects which are mainly aimed at modeling biological evolution of regulatory networks as evolution in silico.

We will discuss some general applications in simulating GRN evolution, but a main theme will be on the evolution of spatially patterned gene expression, of fundamental importance in developmental biology. In particular, we will return frequently to the very well-studied case of anterior-posterior (AP) positioning in the early Drosophila (fruit fly) embryo. GRN models for such applications have the critical addition of spatial dependence, frequently through modeling of the transport properties which give this dependence (e.g. diffusion).

While the common goal of GRN modeling projects is to accurately represent the structure and dynamics of the networks, like all modeling approaches, the level of detail taken determines the types of questions answered. In this review, we will distinguish between coarse-grained models, in which genes are treated as ‘black boxes’, with only the between-gene connections and their strengths modeled, and finegrained models, in which the level of detail can include specific sequence data. Intermediate between these, we use ‘mid-grained’ to refer to models which include some information about cis-regulatory structure (i.e. are resolved at the cis-regulatory module, or CRM, level). We will discuss the types of questions that are being addressed by the different levels of model, ways in which these models are being extended, and computational considerations in choosing the appropriate level of modeling.

Whatever the level, evolutionary simulations and computations share the same overall approach (summarized in Fig. 1):

1. An initial population is chosen. In the simple case, individuals can simply be different parameter sets for a given GRN, but this can be extended to include cases where individuals represent different connectivities or member genes.

2. Individuals are tested for fitness against the test criteria. For example, for spatial expression problems, individuals are scored by how well they recreate experimental patterns (e.g. an individual’s parameters are used in a differential equations model of the patterning process, and the simulated pattern is scored against experimental data).

3. Low-scoring individuals are selected out of the population.

4. New individuals are introduced into the population to replace those just selected out. Generation of new individuals is specified by inheritance rules from parent individuals.

5. Mutation of parameters. This can occur at numerous levels, depending on the model. For example, gene-gene interactions can have mdified strength or be eliminated; transport properties can be altered; cis-regulatory elements can be modified; etc. Some of these options are illustrated in Fig. 2. For more detailed levels of modeling, the mechanisms of mutation become more diverse; for example, at the sequence level, one can distinguish a point mutation from a crossover operation involving an entire region of a sequence.

6. Repeat b–e) for some number of generations.

Figure 1.

Overview of evolutionary computation approach.

Figure 2.

Examples of how gene networks can be altered and become more complex. Left, alterations in cis-regulation; Right, alterations in protein interactions or transport properties. A) alterations in reaction strengths, for example increasing activation or dimerization rate. B1) development of a new dynamic, such as adding multiple activation sites or adding ability to dimerize. B2) modification of existing species, such as turning additive activation into cooperative activation; or enzymatic modification (e.g. phosphorylation). C) modification of protein transport properties.

Depending on constraints (computational, data level, etc.), the modeler should bear in mind the ways in which biological networks can become more complex during evolution. Fig. 2 suggests a few of these which may affect evolution of spatially patterning GRNs. The strength of modeling, however, is in being able to codify conceptual understanding of processes and test them. A modeler must be clear on the questions to be addressed: models which include all possible interactions ab initio run the large risk of producing nothing understandable. Simple models may more clearly identify dynamic principles, which can then be built up or extended into more complex models. One consideration in starting with fixed, simple models, however, is to not constrain the types of solutions, i.e. to not have the preconceptions of the model determine the answers obtained. Such outcomes can result from sticking too tightly to what is unambiguously known from experiment. Computations which allow for some freedom in generating alternatives can have better predictive power for the eventual structure of a network and allow one to analyze the contributions of different dynamic aspects to overall behavior.

2. Evolutionary computation of gene and cell regulatory networks

We organize this review according to the level of detail modeled. The ‘coarse-grained’ approach treats each gene as a black box, reducing complicated gene-gene interactions to single connections with signs (positive – activation, negative - repression). Such approaches oversimplify gene regulatory dynamics, but can be good as a first step in the mathematical description of a given gene network. Neglecting CRMs (which in many cases are experimentally separable and can carry their functions autonomously, independent of the rest of the regulatory region) is a crucial weakness of the coarse-grained approach. ‘Mid-grained’ approaches begin to incorporate CRM structure and regulation in order to address this weakness. ‘Fine-grained’ models, which incorporate specific binding site information (e.g. [1, 2, 3, 4, 5]), have been developed for specific cases, but can be computationally intensive for general use. The midgrained approach is ‘black box’ at the level of the CRM – it ignores specific binding site data (which can be vast, e.g. in Drosophila), but captures some of the dependence of gene regulation on DNA structure, unlike the coarse-grained approach. As a hierarchy of models, initial ideas regarding network dynamics can be approached with coarse-grained models; effects of CRM architecture can be approached at the mid-grained level; and this may proceed to fine-grained modelling to study regulation at the level of individual binding sites. A great deal of evolutionary work has used coarse-grained approaches (section 2.1); newer developments using mid- and fine-grained approaches will be covered in section 2.3. Extension of strict GRN models to CRNs will be discussed in section 2.2.

2.1 Coarse-grained modeling

The majority of network modeling has been at this level. While fast, a key consideration is the treatment of GRNs as signed networks – it does not account for the effects of DNA structure in gene regulation.

Approaches include (please see [6] for a recent review): regulation (or interaction) matrix, in which networks are modeled as systems of linear differential equations (e.g. [7, 8, 9]); S-systems, which add specific power-law nonlinearities to differential equation models (see [10], 11]); reaction-diffusion (RD) modeling, in which spatial dependencies in gene expression patterning are added through diffusion or transport terms (these involve partial differential equations (PDE) models, e.g. [12] and many others); artificial neural network models, in which linear interactions between genes are modulated nonlinearly through a sigmoidal function and weighting of terms can allow for the modeling of very complex behavior (see [13, 14, 15]; and finally, see [16] for a ‘connectionist’ model of Drosophila pattern formation which has been greatly extended [17, 18, 19, 20, 21, 22]. We address these in more detail below.

2.1.1 Regulation matrix approach

The majority of publications in GRN evolution use a regulation matrix approach (usually explicitly, sometimes implicitly), in which the gene-gene interactions are represented in a matrix, W_ij (discussed further below, see schematic in Fig. 5). The way information is encoded in the W matrix distinguishes several different levels of GRN modeling, summarized in Fig. 3. The simplest way, developed and explored by Andreas Wagner and co-workers, uses discrete (Boolean) modeling (see section 2.1.1.1). More complicated and computationally expensive methods have been developed for continuous modeling (ordinary differential or partial differential equations, ODE or PDE, respectively; see sections 2.1.1.2.3–2.1.1.2.5).

Figure 5.

Representation of a gene regulation network via the connectivity or regulation matrix W = W_ij=W_i←j (the last notation explicitly showing that a W element represents the action of the jth gene on the ith gene). A) Matrix representation of gene regulation by transcription factors (TFs) encoded by the genes. In biological terms, the w_ij ‘s represent both the enhancer binding constant and the transcriptional activation (repression) ability of the jth factor on the ith gene. w_ij represents the influence of the jth factor relative to all regulators of gene i. The ith row of W corresponds to the entire enhancer of gene G_i, with all regulatory DNA elements that affect the expression of G_i. The matrix W = (w_ij) corresponds to all regulatory DNA elements relevant to regulatory ^interactions among network genes. The more zero entries W has, the fewer regulatory interactions exist among network genes. B) Each gene (horizontal arrow) is regulated by the products of the other genes, via upstream enhancer elements (boxes). Strength and direction of regulation (depicted as different color saturation levels) are a function of both the regulatory element and the abundance of its corresponding gene product. Genotype is represented as the matrix W, and phenotype is the vector Ŝ, of geneproduct levels at equilibrium [26, 27]. C) Expression of the factor s_i depends on a linear sum of its regulatory factors.

Figure 3.

Classification of GRN modeling approaches discussed in this paper, from simplest (discrete, local) to most complicated (continuous, spatially-distributed).

S-systems (section 2.1.1.3) are a continuous, local extension of the W matrix approach; while continuous spatially distributed models of gene expression, the connectionist or gene circuit approaches [1,16,17,18,19,20], have been developed for simulating and analyzing GRNs responsible for pattern formation in embryo development (section 2.1.1.2.5). Newer models extend GRNs into CRNs (cell regulatory networks) by including more than just gene-gene interactions (e.g. including protein-protein interactions etc.; see section 2.2).

2.1.1.1 Discrete (Boolean) modeling

Discrete approaches represent gene states in the regulation matrix as Boolean on or off (In some cases, triple states (1, 0, −1) have been used.) These models are easy to implement and are not computationally expensive. This is a flourishing area of GRN computational evolution, with dozens of publications and a recent monograph (A. Wagner’s book [23]). This approach forms a standard against which to compare more complicated and computationally expensive continuous models. Figure 4 illustrates some of the key differences between discrete and continuous GRN modeling (on an example in early Drosophila development).

Figure 4.

Some of the key differences between discrete and continuous GRN models, on an example in early insect gene expression. Both, discrete (A) and continuous (B) models simulate patterning of four early fly genes (gt, giant; hb, hunchback; kni, knirps; Kr, Kruppel) in the posterior of the embryo. In A) the gt gene is on at t=1; while from t=2 to t=100, both gt and hb are on while Kr and kni remain off (after [29]; see also Fig. 6). The model is local, and only represents gene states at the 80% AP position. Gene states are either ON or OFF, without intermediate expression levels. While some discrete approaches have used several expression levels, continuous representation of expression levels is needed to capture important developmental dynamics. In (B), a reaction-diffusion model represents not only continuous expression levels, but also the spatial distribution of the expression. This model captures experimentally observed features such as the gradual rise of the Hb domain and the gradual anterior shift of all three domains (after [17]).

2.1.1.1.1 Wagner’s model

This approach is laid out in two pioneering publications of Andreas Wagner [24, 25]. A gene network is represented by the state of the network genes 1-N:

$$\overrightarrow{S}(t)≔(S_1(t),\dots ,S_N(t)),$$ (1)

S_i(t) is binary – the gene is either expressed or not. Cross- and auto-regulatory interactions causing the state to change are modeled by difference equations:

$$S_i(t+\tau )=\sigma [\sum _{j=1}^N{w}_{ij}{S}_j(t)]=\sigma [h_i(t)].$$ (2)

where the expression state of gene Gi at time t + τ, S_i (t + τ ), is a function of a weighted sum, hi(t), of the expression state of all network genes at time t. σ(x) is the sign function (σ(x) = −1 for x < 0, σ(x) = +1 for x > 0 and σ(0) = 0), and τ is a time constant whose value will depend on biochemical parameters such as the rate of transcription or the time necessary to export mRNA into the cytoplasm for translation. The real constants w_ij represent the ‘strength’ of regulatory interaction of the product of G_j with G_i , (activation, w_ij > 0 – repression, w_ij < 0). Fig. 5 is a schematic of the classic W matrix approach.

Use of discrete gene states certainly speeds up computations for finding network connectivity, and also likely makes the results more robust. It has been argued that continuous concentration kinetics can be ignored, since GRN dynamics must be robust (hence the kinetics are not tuned perfectly; e.g. see Wagner’s arguments in [28]). However, there are a number of cases where concentration-dependent kinetic effects have been shown to be critical in gene regulation and gene expression – the limits of the discrete modeling will be discussed further below (section 3.2).

Wagner’s ideas have been extended by a number of groups for studying evolutionary processes, for example Siegal and Bergman [26]; Azevedo et al. [29]; Gardner and Kalinka [30]; Misevic et al. [31]; Huerta-Sanchez and Durrett [32]; MacCarthy and Bergman [33] (next section). As summarized by Martin and Wagner [34+: “Despite being quite abstract, variants of this model have proven highly successful in explaining the regulatory dynamics of early developmental genes in the fruit fly Drosophila, as well as in predicting mutant phenotypes (Mjolsness et al. 1991 [16]; Sharp and Reinitz 1998 [35]; Jaeger et al. 2004 [17]). The model has also helped elucidate a number of fundamental evolutionary questions. Among them are the questions of why mutants often show a release of genetic variation that is cryptic in the wild type (Bergman and Siegal, 2003 [27]), how adaptive evolution of robustness occurs in genetic circuits (Wagner 1996; Siegal and Bergman 2002; Leclerc 2008) [25, 26, 36]), and whether recombination can influence epistasis (Azevedo et al. 2006.[29]).”

2.1.1.1.2 Generalizations of Wagner’s model

In this section, we give an overview of the main extensions to Wagner’s approach and some key results.

Siegal and Bergman [26, 27] introduced a sigmoidal function f(x) = 2 / (1 + e^−ax) −1 (Wagner’s [24] model is a special case of f(x) in the limit that a →∝ ). This allows for the biologically realistic feature of changing a repressive state into an activating state. This was later extended [33] to allow for modification via recombination (allele r or R assigned with equal probability). Evolutionary simulations with such models provided a basis for the observed low correlation between gene (node) connectivity and lethality of mutations in that gene [37].

Azevedo et al. [29] developed an application of a Wagner-type model to the evolution of the Drosophila gap gene network, albeit in a somewhat abstract form (Fig. 6). This gap gene network was used as a benchmark test for more general problems in systems biology, such as how to model GRN evolution. Their evolutionary computations suggested that recombination selects for robustness, which in turn supports the conditions for sexual reproduction (see also [38,39]).

Figure 6.

Application of a Wagner-type network model to the gap gene system of Drosophila (at 80% AP position, cf Fig. 4A). a, network representation of the regulatory interactions between four gap genes [17] (gt, hb, kni, Kr). Activations and repressions are denoted by arrows and bars, respectively. Numbers indicate relative interaction strengths. b, interaction matrix (W matrix) representing the network in a. c, graphical representation of the gene expression states of each gap gene over three successive time steps (to a stable configuration). Gene i is considered to be ON (filled box) if s_i(t)>0, and OFF (open box) if s_i(t)≤0. From [29].

Masel [40] altered the Wagner model (eqn. 2) by having S_i(t) take on values of 1 (expressed) or 0 (not expressed), with σ(x) = 1 if x≥0 and σ(x) = 0 if x<0. The 0, 1 mapping is more realistic biologically since genes that are off have no effect on anything, compared to the eq. (2) −1,1 formulation in which if gene i is off it has a positive effect on gene j with W_ij<0. Masel’s gene network simulations showed that genetic assimilation, whereby an acquired trait loses dependency on an environmental trigger and becomes an inherited trait, can occur in the absence of selection for the trait. This was a concept articulated earlier by Waddington [41] “…if selection was practiced for the readiness of a strain of organisms to respond to an environmental stimulus in a particular manner, genotypes might eventually be produced which would develop into the favoured phenotype even in the absence of the environmental stimulus. A character which had originally been an “acquired” one might be said to have become genetically assimilated.” See also [42, 43]. This process has also been modeled with classic W matrices: Espinosa-Soto et al. [44] found that “plasticity facilitates the origin of genotypes that produce a new phenotype in response to non-genetic perturbations… selection can then stabilize the new phenotype genetically, allowing it to become a circuit’s dominant gene expression phenotype.” See also [45].

Wagner [25] and Siegal and Bergman [26], simulating N-gene network evolution, have argued that networks evolve to be robust. Huerta and Sanchez-Durrett [32], by varying system size and convergence criteria, with and without recombination, reported that this evolution to robustness is itself a conserved trait, i.e. not dependent on the details of the model.

Leclerc [36] introduced measures for the cost of GRN complexity to the Wagner approach. He argued that spurious perturbations could increase robustness if the cost of network complexity was ignored. Accounting for these costs could favor sparser networks, which can have higher efficiency.

Martin and Wagner [34] added recombination and point mutation steps to the regulatory matrix (eqns. 1 and 2), and used these to simulate evolution of embryonic GRNs. Their computations suggest that recombination may be a much stronger factor in GRN evolution than point mutation; and when both factors are involved, genotype diversity is increased, the deleterious effect of point mutations is decreased, and cis-regulatory complexes (combinatorial regulatory interactions) emerge.

2.1.1.1.3 Concluding Remarks

Wagner’s approach has been developed as a means for approaching gene networks in the absence of precise knowledge about the continuous concentration kinetics of gene regulation – see discussion in [28]. It has led to impressive conclusions regarding general features of gene networks and their evolution, as detailed above. It has been argued [28] that quantitative knowledge of gene product activity is missing for many cases. However, in development especially, there are many processes which are concentration-dependent on levels of gene products, for example morphogen gradient reading for positional information (e.g. AP Drosophila patterning, Figs. 4 and 6). It is becoming increasingly precisely understood how gene regulation operates in these processes. For such cases, binary or even discrete gene product states are likely to be an oversimplification for understanding the network dynamics. One of the arguments for taking a discrete approach is that gene states are robust (not highly sensitive to parameter values) in the face of intrinsic biochemical fluctuations and extrinsic environmental or general regulatory changes [28]. While this may provide some support for modeling at the level of discrete gene states, the question of how this robustness arises within the reality of biochemical kinetics can only be answered through continuous ODE/PDE modeling. A very rich area for future exploration is the extension of the above-discussed discrete models to continuous treatments, to corroborate whether inclusion of the underlying kinetics corroborates the previous conclusions or introduces new dynamical behavior.

2.1.1.2 Extensions of the Wij approach

A series of recent publications show new steps towards more effective and realistic GRN evolutionary modeling. These approaches include spatial dependence (up to the level of PDE reaction-diffusion modeling), flexible genetic architecture, varied mutagenesis paths, and evolution with gene expression noise. As summarized in Fig. 3, some of the new developments are discrete; but we feel the extensions into continuous modeling are the most promising for addressing biological problems, especially in development. For instance, a number of teams are now focusing on the evolution in silico of the biological evolution of segmentation (Sole et al. [46], Francois et al. [47, 48]; Fujimoto et al. [49]; Cooper et al. [50]; Spirov and Holloway [51, 52]; ten Tusscher and Hogeweg [53]).

2.1.1.2.1 W_ijk matrix

One drawback of the W_ij matrix approach is that all interactions involve one gene on another. This does not account for the well documented and important aspects of co-activation and co-repression by two or more factors on transcriptional rates. One way to approach this is to extend the regulation matrix to a third-order W_ijk formulation, as shown schematically in Fig. 7 (see also Jaeger et al. [18]).

Figure 7.

W matrix interactions. M – spatial morphogen, A – activator, R – repressor. A) traditional Wagner-type approach: arrows show regulatory actions on gene X, each of which is encoded in matrix W_jk. Mutation acts on the W_jk elements. B) Regulatory co-actions of pairs of factors on target gene X. Co-actions (e.g. synergistic effects) are shown for the co-action of A or R with the morphogen M. These are encoded in the triple-index W_ijk; mutation acts on elements of this matrix.

With a W_ij approach, more complicated regulatory connections can arise via adding a new TF (a, below) or a new gene encoding the TF (b, below):

1. W_jk → W_(j+1)(k+1)

2. W_jk → W_j(k+1)

With the W_ijk approach, a new co-action of one TF with another TF (a, below) or even with a new transcription co-factor (b, below) can be added:

1. W_ljk → W_l(j+1)(k+1)

2. W_ljk → W_{(l+1)(j+1)(k+1)}

A related extension of the connectionist approach is the Sigma-Pi Neural Networks formalism [54]:

$$v_i^{l+1}=g(\sum _{jk}{T}_{ijk}^l{v}_j^l{v}_k^l+\sum _j{T}_{jk}^l{v}_j^l+h_i^l).$$

in which third-order (three-index) connections T_ijk allow for two input variables (j, k) to jointly influence the activity of the i-th output variable.

2.1.1.2.2 Flexible genetic architecture and diverse mutagenesis

ten Tusscher and Hogeweg [55] extended the basic W matrix approach to allow for multiple transcription factor binding sites (TFBS’s) in the regulatory region of each gene, allowing for different mutation types (TFBS mutations, gene mutations, macromutations on segments of the genome; see also [56]); and they also model sexual reproduction (see Fig. 8 for a summary of the approach). These network models are still Boolean, and do not incorporate diffusion or transport for modeling spatial pattern formation. Using ‘phenotype’ genes and TF (transcription factor) genes in this flexible genetic architecture with sexual reproduction, they found that phenotypic diversity could be attained with limited genotypic diversity. As they state “This process enables phenotypic divergence under random mating by reducing the impact of recombination and increasing hybrid fitness.” This model implicitly uses a W matrix approach, but also begins to treat some processes at a mid-grained level (cf. section 2.3).

Figure 8.

Overview of the ten Tusscher-Hogeweg model for flexible genetic architecture. A) Organisms reside on a 2D grid world in which they live, move, reproduce and die. They compete in a multi-niche environment, imposing selection for phenotypic differentiation. B) Individuals contain a genome which consists of a linear array of genes and their upstream TFBS. There are two types of genes, TF and phenotype genes. The genome codes for a gene regulatory network, with genes as nodes and TFBS as edges (either activating, green, or repressing, red). The birth state is the initial expression state of the genes. The network edges determine how gene states are updated as a function of the state of other genes. The phenotype of the individual is determined solely by the final state of the phenotype genes. C) Mutation events can affect individual TFBS, individual genes, or stretches of genome. D) Sexual reproduction is implemented as a crossing over between two parental genomes to create offspring genomes. Crossing over occurs between homologous locations, where the two parental genomes have the same gene type. From [55].

ten Tusscher and Hogeweg subsequently adapted their model to be spatially-dependent to study the evolutionary origin of body segmentation and developmental domain formation [53]. This involved defining organisms as one dimensional rows of cells and providing spatial information with a morphogen (Fig. 9; also compare approaches of Francois et al. [47]; Francois and Siggia [48]; Fujimoto et al. [49]). With this model, the authors could compare the different types of networks which evolved and compare them in terms of robustness, evolvability, modularity and the type of developmental mechanism produced. They found two evolutionary strategies that formed a segmented body plan: a first one in which segments formed first, followed by domains; a second in which segments and domains formed simultaneously (cf. discussion of short- and long-germ development in next subsection). ten Tusscher and Hogeweg’s work extended the W matrix method to developmental patterning, but further work is needed to clarify the strengths and weaknesses of Boolean TF states versus continuous ODE/PDE models with respect to matching developmental data.

Figure 9.

Overview of the ten Tusscher-Hogeweg model for the evolution of segmentation and domain formation. The in-silico embryos live in a two-dimensional grid world (left). Each individual consists of a one dimensional row of 100 cells over which a maternal morphogen provides initial spatial information (middle). As in Fig. 8, each individual has a genome, consisting of genes with TFBS’s. In this case, there are ‘cell type identity’ genes instead of ‘phenotype’ genes. The evolved network models the spatiotemporal gene expression pattern underlying body plan development (right). An individual’s fitness depends on the number of segments and domains (right). Mutations can occur in both genes and TFBS’s (middle). From [53].

2.1.1.2.3 Evolution of different modes of segmentation

Fujimoto et al. [49] have developed a continuous evolutionary model for insect segmentation (see Fig. 3). This offers the opportunity to investigate selection and regulation in concentration-dependent situations, such as developmental signaling. The major biological problem they were addressing was how the three major modes of insect segmentation could have arisen in developmental networks. These modes are: short germ band, in which segments form only a few at a time, with the entire body plan laid down sequentially; long germ band, in which segments develop simultaneously; and intermediate germ band, a combination of the previous two.

The expression level of gene i is represented by the concentration of its product, protein Pi:

$$\frac{\partial p_i}{\partial t}=f(P_j;K_{j\to i})-\gamma P_i+D_i\frac{{\partial }^2{P}_i}{\partial x^2}$$

where γ is the degradation rate constant, D_i is the diffusion coefficient, and x is the position along the AP axis in the embryo. These equations are similar to earlier connectionist models [16]. The architecture of the network is represented by a “connection” matrix c_{j → i} where c_{j → i}=1, −1 and 0 indicate positive, negative, and no regulation, respectively. The matrix C is a discrete analog of the matrix W (while P and x are continuous (PDE formulation)). The activation/repression function f involves a Hill equation with threshold K_j→i. Combinatorial (cooperative or competitive) interactions between genes are also modeled. A maternal morphogen (gene 0) establishes an initial positional information gradient; all other genes develop pattern dynamically.

Mutations (at rate μ) are introduced at three levels: (A) in the connection matrix c_j→i, (B) in the thresholds K_j→i, and (C) in the diffusion constants D_i . Fitness of the networks is evaluated by the difference between the observed number of stripes for gene i and a target number of stripes. The authors found that three modes of stripe formation tended to develop, characterized by Feed-Forward Loops (FFLs); Feed- Back Loops (FBLs); and a combination of the two. These corresponded to spatiotemporal expression patterns and knockout responses for long-, short- and intermediate-germ band development, respectively. They further report that network architectures with FFLs and/or negative FBLs have a trade-off between mutational robustness and developmental speed, and this may affect evolution of body plans.

This approach is a promising step in modeling the evolutionary design of networks at the ODE / PDE level, compared with the discrete (Boolean) models above. At the present stage, it has simplified (Hill) kinetics, a simple mutation operator (no crossover), and fitness is not based on the positioning of stripes. With extension into these areas, the approach could form a more comprehensive basis for modeling GRN and CRN evolution and design. Natural ways to extend the approach would be to use a real-valued W matrix (instead of the discrete C), more realistic kinetics for gene activity, and more complex operators for mutation and crossover.

2.1.1.2.4 Evolution of robustness to noise

Phenotypes need to be robust against mutation in order to sustain themselves between generations. The phenotype of an individual also needs to be robust against fluctuations from both internal and external sources encountered during growth and development. In [57] Kaneko addressed the question of whether stochasticity in gene expression might be relevant to the evolution of these types of robustness. He used a simple gene network with switch-like (i.e. Boolean) dynamics due to sigmoidal input-output (see also [26]):

$$d{x}_i/dt=\text{tanh}[\beta \sum _{j>k}^M{J}_{ij}{x}_j]-x_i+\sigma \eta (t),$$

where J_ijj=−1,1,0 (a discrete analog of the W matrix), and stochastic effects are modeled with the added η(t) term for Gaussian white noise. (x is continuous in this ODE approach, but x tends towards discrete values because of the tanh function.) Mutation changes the J_ij at mutation rate μ. Fitness is defined by a k number of genes, such that if k genes are ‘on’ after a particular evolutionary time fitness is maximized. Fitness is at a minimum if all k genes are ‘off. Kaneko found that networks acquired both mutational and noise robustness if the phenotypic variance induced by mutations was smaller than that observed in an isogenic population (where variance is only due to gene expression noise). This suggests that robustness evolves (both types) if noise in gene expression is above some threshold. To our knowledge, this is the first and only publication modeling expression noise in GRN evolution. We believe such an approach has substantial prospects for future elucidation of evolutionary trajectories. A natural extension would be to include more realistic and smoothly varying kinetics and spatial dependence (for developmental applications).

2.1.1.2.5 Evolution of reaction-diffusion models

A number of projects have implemented fully continuous modeling of gene expression dynamics (concentration dependent) and spatial dependence (usually through diffusion). For example, connectionist reaction-diffusion (RD) models have been used extensively in Drosophila AP patterning (e.g. [16, 58, 17]. Here, we outline several recent projects using RD models in an evolutionary context. While EC of RD systems can be much more computationally intensive than for discrete models, the payoff can be more accurate descriptions and understanding of the evolution of GRNs. Developmental networks, especially, are spatially and concentration dependent, and addressing such fundamental evo-devo problems as GRN robustness and evolvability needs to be done at this level of modeling.

2.1.1.2.5.1 Evolutionary design of robust GRNs

A major question in development is how GRN expression patterns resist external and internal variability and noise in order to reliably produce organisms. One aspect of this is to understand the dynamic (e.g. kinetic and transport) features which produce such robustness, another is to study how evolution might generate such robust GRNs. A number of workers [19, 20, 59] have started addressing this through the specific case of robustness of the hb gene product to variability in the maternal Bicoid (Bcd) transcription factor gradient. As shown in Fig. 10, Bcd shows quite high variability between embryos. If hb, its direct target, were non-robust it would be similarly variable (Fig. 10, bottom left), but in reality it shows very little positional variability (Fig. 10, bottom right).

Figure 10.

Positional variability in early Drosophila AP patterning. The maternal factor Bcd is highly variable (top). If the readout mechanism of its target, hb, were non-robust, expression should be similarly variable (left). Hb expression in reality is very precise, reflecting a robust mechanism (right).

Hardway et al. [59] used a connectionist model of two gap genes (hb and another factor) regulated by the Bcd gradient to study the selection for robustness. Using a variable Bcd input, they searched for conditions in which Hb displayed the experimentally observed precision. A cost function, C, was defined by the difference between the Hb solutions and a step function. C was calculated for 3 wide-ranging values of the Bcd decay constant, then averaged. Parameter sets that minimized the average C were most robust.

A genetic algorithm (GA; a type of EC) was used to search the parameter space (i.e. using a process as outlined in Fig. 1 to optimize parameters): the process was started with 100 random sets of parameters (and parameter constraints, e.g. that diffusivities and rate constants be positive); solutions (gene expression patterns) were solved numerically; C was evaluated for each solution; the 10 solutions with the lowest average C were retained, while 90 new random parameter sets were introduced; and the process was then repeated. Through long computations of this process to find minimal cost solutions, they concluded that experimental levels of Hb precision were only achieved when a 2^nd gap gene was active in the model.

The genetic ensemble studied is perhaps the simplest one, at a coarse-grained level, capable of precise gradient reading (cf [19, 20]). It is much smaller, in terms of number of genes, than the networks studied in previous sections. However, it is a very good example of how reverse engineering of a GRN-module via GA can be done at the RD level. This approach provided for conclusions regarding the type of dynamics necessary to buffer against continuous concentration fluctuations in an upstream signaling gradient.

2.1.1.2.5.2 Gene cooption in evolving GRNs

Early in metazoan evolution, gene networks specifying developmental events in embryos may have consisted of no more than two or three interacting genes. Over time, these were augmented by incorporating new genes and integrating originally distinct pathways [60]. While it may initially be thought that new functions require novel genes, whole genome sequencing has shown that apparent increases in developmental complexity do not correlate with increasing numbers of genes [61]. Therefore, evolution of developmental pathways may most commonly proceed by recruitment of preexisting external genes into preexisting networks, to create novel functions and novel developmental pathways [62]; developmental evolution may act primarily on genetic regulation [63, 64].

Specifically, gene recruitment may occur through mutational changes in the regulatory sequences of a gene in an established pathway, enabling a new transcriptional regulator (or regulators) to bind. This regulator may be from a newly evolved gene (say via duplication and subsequent change), in which case it simply adds to the existing pathway, or it may have already been part of a pre-existing pathway, in which case the two pathways become integrated. In either case, the developmental function of the pathway may be significantly altered [60, 65].

Spirov et al. approached this issue through a model of Drosophila AP segmentation. They extended a gene circuit model of gap gene pattern formation to allow for addition and removal of genes to the circuit (via new operators in the EC algorithm) [66, 67, 68]. As shown in Fig. 11, this adds (or removes) rows and columns to the W interaction matrix. A newly added row represents the actions of the other genes on the new gene (G_r1), and a corresponding new column represents the action of the product of the new gene (TF_r1) on the pre-existing network genes.

Figure 11.

Modeling co-option of a new gene into a network during evolution. A) Initial n-gene network (G₁, …, Gn), producing n transcription factors (TF₁,…, TFn). B) New gene (G_r1), producing new factor TF_r1, can be incorporated by chance and retained if it maintains or improves fitness. The initial n-gene network can become completely rewired by co-opting G_r1.

They found that recruitment occurred in all simulations, even for those in which only point mutations were operating [66, 67]. Crossover aided recruitment, but was not necessary. The recruited genes were either ubiquitous or formed spatial patterns, many of which were similar to real, biological gene patterns, including patterns for genes not in the core starting networks [66, 67, 68].

While these simulations were computationally expensive (they could benefit from a more effective GA; see next section), this work demonstrated that relatively simple evolutionary operators can account for network outgrowth (see also [69], with respect to gene duplication and redundancy), and that the evolved networks can display robustness to regulatory variability.

2.1.1.2.5.3 EC for connectionist models

The approaches in the previous two subsections used GA techniques for finding optimal solutions to segmentation patterning problems, and the earlier connectionist models used simulated annealing (SA; Reinitz et al.; [70]) and parallel simulated annealing (PSA; Chu et al., [71]) for parameter searches. Several newer projects are working on applying EC strategies to speed up parameter searches in reverse-engineering the gap network via coarse-grained connectionist models (i.e. fitting connectionist models to experimental data).

Fomekong-Nanfack et al., 2007 [72] developed an ES (Evolutionary Strategy) for finding parameters in segmentation models some 5–140 times faster than PSA. Their ‘island’ ES algorithm operates on N populations of individuals, each with a population size i. Initialization, selection, recombination and mutation are performed only within populations. Populations are linked via a migration operation (every m iterations) in which the best individual from each ‘island’ population is copied to another population.

Jostins and Jaeger [73] developed parallel island ES (piES) methods, which were faster and more reliable than the comparable PSA approach. The found that asynchronous communication, in which islands (each operating on a different processor) send migration information to a buffer which can be received by another population at a later time, showed significant speed-up, by minimizing information waiting times.

Kozlov and Samsonov [74] have recently developed a paralleltechnique called DEEP (Differential Evolution Entirely Parallel). This also uses parallel computation of populations each on its own computational node, with migration steps allowing for exchange between populations. They propose a new migration scheme in which the best member of one population substitutes for the oldest member of a target population, organized in a ring.

The EC approaches outlined by these three projects are leading to much improved methods for fitting gap-gene connectionist models to data (compared to SA). We believe this new efficiency will produce new developments in the area of RD modeling of embryo patterning.

2.1.1.3 S-systems

S-systems (synergetic and saturable; see the introductory paragraph of 2.1) have been developed by Irvine and Savageau [75] and Savageau [11] and used for modeling biochemical pathways, gene networks and immune networks. They are local, having no diffusion or transport. S-systems have been widely applied to reverse engineering of GRNs from DNA microarray experiments, and also in the development of EC techniques to GRN reverse engineering. The bulk of this work has been in computer science, and has not generally been picked up by biological modelers. S-systems are of the form:

$$\frac{d{x}_i}{dt}={\alpha }_i\prod _{j=1}^n{x}_j^{g_{ij}}-{\beta }_i\prod _{j=1}^n{x}_j^{h_{ij}}$$

The two terms correspond, respectively, to synthesis and degradation influences from other genes in the network; specifically, α_i and β_i are rate constants and represent basal synthesis and degradation rates, while g_ij and h_ij (kinetic orders) indicate the influence of gene j on the synthesis and degradation of the product of gene i (reviewed in [6, 76]).

2.1.1.3.1 Genetic Algorithms (GA)

S-systems have been used extensively for developing optimization algorithms, both Genetic Algorithms (GA; this section) and Genetic Programming (GP; next subsection). For an in depth review in this area, please see [77].

In early work using GA to reverse engineer a GRN, Tominaga et al. [78] used time series data to infer an S-system model for a very small network (2 genes; and using synthetic data). The convergence rate was very low [79], which is typical of simple GA (SGA), which tends to converge in initial stages of the search but evolutionarily stagnate in later stages. The approach was improved in Kikuchi et al. [80], by using a more robust real coded genetic algorithm (RCGA), which added a penalty term to the cost function to prune unlikely connections in the network, introduced a novel crossover method, and implemented a gradual optimization strategy. RCGA converged faster and more accurately than SGA, but was computationally very costly because of numerical integration of the entire system of S-system differential equations. Further improvements to the method have included: a hybrid algorithm of SGA with a Modified Powell method [81]; a hybrid algorithm of SGA for static Boolean networks applied to an S-system with steady state and temporal data [82]; and a combination of RCGAs with unimodal normal distribution crossover and minimal generation gap to optimize parameters in S-systems [83, 84, 85].

Subsequent work has explored a number of optimization techniques with S-systems: distributed GA with ‘scale-free’ properties [86]; an intelligent two-stage evolutionary algorithm (iTEA), in which an intelligent optimizer solves decomposed ODEs independently, then combines all solutions from each subtask [87]; a two-part memetic algorithm (MA) involving local search by an evolutionary strategy (ES) for parameter estimation with an embedded global GA search for structure identification [88, 89,90]; a Genetic Local Search with independent Diversity Control (GLSDC), which was successful in reconstructing mediumscale GRNs from noisy expression data [91,92]; the Network- Structure-Search Evolutionary Algorithm (NSS- EA) and Grid-Oriented Genetic Algorithm (GOGA) Framework [93, 94, 95]. MA techniques have been supplemented by using differential evolution (DE), hill-climbing local-search, and information criterionbased fitness evaluation instead of the conventional least-squared errors approach [96, 97, 98, 99, 100]; and hybrid DE has been used to find a starting point for gradient based optimization [101,102].

2.1.1.3.2 Genetic Programming (GP)

Instead of operating on encoded parameter ‘chromosomes’ for fixed models, like GA, Genetic Programming (GP) evolves mathematical expressions or computer programs. In GP, a mathematical expression or computer program is typically represented as a tree structure, in which every tree node has an operator function and every terminal node has an operand. The general process of GP includes: initialization (random generation of trees); evaluation (calculating the fitness of each individual tree); selection (of individuals, probabilistically); crossover (random selection of two individuals as parents, random swap of sub-trees between the parents); and mutation (such as insertion or deletion of terminal nodes. Please see [103] for further details. In contrast to GA, which usually requires defining equations before optimization, GP provides a general approach for finding arbitrary equations from time series data without specific knowledge of the equations.

A number of workers have used GP to infer S-systems or metabolic schemes (e.g. [104]). One issue in applications is that GP can be inefficient, due to relying on randomly generated constants. Sakamoto and Iba [105] introduced a least-mean-square (LMS) approach to improve this. Sugimoto and co-workers [106] developed a GP which predicted two equations of a metabolic reaction scheme for adenylate kinase and phosphofructokinase in a Michaelis–Menten format, a challenging task if the underlying mechanism is not known. Numerical integration can be very time consuming in GP; Kim et al. [107] introduced a symbolic preprocessing regression step to avoid this. Cho and co-workers [108] have proposed an S-tree framework for GP of S-systems. Their approach can intrinsically account for sparseness in biological networks.

2.1.1.3.3 Concluding remarks

A considerable number of groups are applying evolutionary algorithms to fit GRN models and parameters to gene expression data. So far no comprehensive comparison among these algorithms has characterized their relative efficiency, robustness, and accuracy. However, some more limited comparisons have been made. Moles et al. [109] compared several stochastic global optimization methods on the case study of a biochemical model consisting of 8 ODEs (36 parameters). The model was formulated in a Michaelis–Menten representation, which could not take advantage of the highly structured format of the Biochemical Systems Theory (BST) representation. Spieth et al. [110] compared six evolutionary algorithms in three model frameworks: linear weight matrices (W matrix of section 2.1.1), S-systems, and H-systems. They used one fitness function to evaluate the convergence of the algorithms. The computer science of EC for GRNs is a very active area of research, but a more comprehensive comparison of evolutionary algorithms is still needed, in order to help biological users evaluate relative strengths of approaches for particular GRN (or CRN) problems. Also, work is needed to extend the computer science results on test cases such as S-systems to more complicated biological cases, such as RD modeling. This will be addressed further below.

2.2 Cell regulatory network evolution

The previous section focused on gene networks, especially transcriptional regulation. Here we survey recent projects in extending such GRN techniques to broader dynamics occurring beyond the DNA, such as protein-protein regulation, dimerization, translational dynamics, etc.; i.e., using the GRN approach to move towards broader cell regulatory networks, or at the least to include more of the cellular context of gene regulation.

A major challenge of characterizing GRNs is to identify functional dynamic modules within large networks. Can the larger network be built from modular subnetworks [111, 112, 113], and can these subnetworks be identified? These problems increase as the cellular contexts of GRNs are considered. Frequently gene networks are only partially experimentally determined; and even with full knowledge of the connectivity in a network, it is frequently the quantitative interactions between components which determines dynamic behavior (e.g. [114, 115]). One approach to identifying modules has been to use evolutionary computations to generate potential networks that produce required functional dynamics, and then to evaluate the qualities of these networks.

2.2.1 Segment polarity network evolution

The segment polarity network lies downstream of the Drosophila AP segmentation network discussed above and has also received a great deal of attention with respect to GRN modeling. von Dassow et al. [116, 117, 118] developed an ODE (continuous) model of 5 genes and their products operating in a several cell system (Fig. 12). The kinetics in a given cell depends on inputs from neighboring cells, making the model spatially dependent. While not explicitly using a regulatory matrix, similar dynamics are implicit in the Hill kinetics used for the production of an X molecular species due to action of an activator A.

Figure 12.

Model of the Drosophila segment polarity network, adapted from [112]. The model incorporates regulatory interactions between 5 genes. mRNAs are indicated by lowercase ovals, proteins by uppercase squares. Solid lines indicate fluxes, dashed lines are regulatory interactions, activators end in arrowheads, inhibitors end in circles. Large rectangles indicate cell membranes. The model simulates a row of 4 cells, each with the network shown here.

2000 evolutionary generations of the segment polarity network were simulated under selection for sharpening the En and Wg border patterns. The model was coded in two copies (diploid), and sexual recombination was used for exchange of genetic material. In comparison with controls, diploidy and recombination both increased robustness, measured as minimal perturbation in response to gene mutation. Kim and Fernandes have more recently used this model to study the dependence of robustness on ploidy and recombination [119].

This project used a novel method for modeling macromolecule binding, which makes it somewhat of a mid-grained model: they represented binding sites between molecules (e.g. proteins) as binary strings, with binding affinity given by the relative complementarity between the strings of two interacting molecules.

Hoyos et al. [120] recently took a similar quasi-evolutionary approach to cell fate specification in vulval development in the nematode Caenorhabditis. An intercellular signaling network specifies three fates in a row of six precursor cells, yielding a quasi-invariant 3°3°2°1°2°3° cell fate pattern. The authors used a signaling model that kept the network architecture constant but varied parameters. They did not explicitly model evolution (selection, recombination, etc.), but they generated model solutions for 10⁷ parameter sets. This produced the diversity in fate outcomes seen across 4 species of Caenorhabditis, which could represent changes incurred over evolutionary divergence.

2.2.2 In silico evolution approach

Earlier work used evolutionary simulation to optimize parameters in a fixed model of a chemical circuit [121]. (See also Kobayashi et al. [122] [123] for optimization of oscillator parameters in a model with fixed components.) Francois and Hakim [124] generalized this approach to finding functional modules in gene regulatory models of unspecified topology, including post-transcriptional dynamics. Building blocks for DE models (see Fig. 13) were subject to mutation and selection. For test dynamics, fitness was evaluated on the capacity of the networks to produce bistable switches or oscillations.

Figure 13.

List of possible reactions with TFs (left) and associated rate equations (right). Only the rate term corresponding to the displayed reaction is shown; total change of a species sums all such terms. From [124].

Mutations occurred via five different pathways: (i) modification of the degradation rate of a protein; (ii) modification of the kinetic constant of a reaction; (iii) creation of a new gene, with reactions for production and degradation of its protein added to the network; (iv) introduction of a new interaction between a protein and a gene promoter, with reactions added to the network for protein binding/unbinding to the gene promoter (or existing complex) and modified production rate; (v) addition of a posttranscriptional reaction, which can involve a range of uni- or bi-molecular options. Starting from random kinetic constants, simulations evolved networks which met the dynamic criteria (switches or oscillators). Many of these networks required post-transcriptional interactions, and some had oscillations that occurred only at the protein level.

Paladugu et al. [125] broadened this approach to use evolutionary simulations to create a library of functional motifs, starting with modules forming oscillators, bistable switches, homeostatic systems and frequency filters. They relaxed the strict Hill kinetics used in [124], allowing also for simple mass action kinetics and Michaelis–Menten kinetics. Their aim is to use this library to identify functional modules in real networks.

François and Siggia [48] have subsequently worked on a more strict (quantitative) definition of the fitness function in terms of the evolution of biochemical adaptation (evolvability). Their metric is based on minimizing the difference between a time-independent input and the output of an adaptive circuit, such that persistent time-dependence is penalized. Adaptive networks should show little dependence on input state. They showed that there is always a path in parameter space of continuously improving fitness that leads to perfect adaptation, implying that the actual mutation rates used in simulations do not bias the results. They argue that this property enables one to make deductive predictions of real networks from models.

2.2.3 Metazoan segmentation by in silico evolution

Francois et al. [47] introduced spatial dependence into their original model and applied this to metazoan segmentation. They were interested in whether the use of similar genes in diverse phyla could be due to independent recruitment or rather reflected common ancestry. They found that in silico networks selected against a criterion of segmentation tended to involve cascades of repressors, as seen in natural segmentation networks.

Embryos were simulated as linear arrays of 100 cells, each containing the GRN. There was no intercellular communication; spatial dependence was imposed by a gradient function. Transcriptional regulation involved both activating and repressing TFs, acting through Michaelis kinetics. Mutation acted through creating or destroying genes, adding or removing transcriptional regulatory linkages, or altering parameters. Their fitness function produced a gradualist trajectory – segmentation evolved via incremental increases in fitness due to the selective advantage of body plan periodicity.

Body plan segmentation in arthropods has striking similarities to vertebrate segmentation of the neural system. Hox genes are involved in both cases, and vertebrate segmentation proceeds via somitogenesis, a sequential system with parallels to the short germ band mode of development in insects. Working from their earlier model, François and Siggia [126] focused on the properties of the fitness function which could account for segmentation across these cases. They argued that fitness should favor: (1) the diversity of genetic expression – e.g. with fitness improving monotonically with the number of ‘realizator’ genes (those specifying cell identity); and (2) a unique cell fate – one cell should express only a single master control gene for a given segmental identity. Only one combination satisfies these two contradictory factors, that is to maximize entropy at the global scale (total number of ‘realizator’ genes) and minimize conditional entropy locally (one local ‘realizator’). Using this goal, the authors modeled evolution of segment forming networks with both a moving morphogen gradient (for vertebrate and short germ insect modes of development) and a fixed morphogen gradient (for long germ insect development, such as Drosophila). Production of segmentation from these assumptions about the fitness function is very intriguing, but needs to be evaluated in light of comparative biology and the evolutionary biology of segmentation (see further in the Discussion).

2.2.4 Concluding remarks

To briefly summarize the coarse-grained treatments, we feel the W matrix approaches, both discrete and continuous, have been elaborated well enough to attack some real biological problems. However we feel the original Wagner type discrete evolutionary modeling may be approaching its limits (see further in the Discussion). This technique has excelled for speed and convenience in manipulating GRN metrics and topology. For example, point mutations, recombination, or even gene introduction and withdrawal are simple manipulations within the W matrix framework. However real biological regulatory networks are based on qualitatively more complex principles of organization and function. For instance, regulatory genes are usually under the control of multiple (semi-) autonomous regulatory modules. In the following section, we outline in silico evolution approaches that explore some of these more complex regulatory features.

2.3 Mid- and fine-grained modeling

While the some of the projects above begin to account for multiple binding of TFs and even cooperative/competitive effects between the TFs, they do not address the architecture of genes’ cis-regulatory regions. Beginning to model and understand the role of this architecture on gene regulation requires what we call a ‘mid-grained’ or ‘fine-grained’ level of approach. Mid-grained modeling involves treating functional regions of the cis-regulatory regions, the CRMs, as discrete units subject to evolution. A CRM, e.g. in Drosophila, can contain multiple binding sites (BSs), but act as a modular unit controlling particular aspects of expression – for example particular stripes or sets of stripes in AP segmentation. In mid-grained approaches, the CRMs are the ‘black box’ units at which modeling is done, compared to coarsegrained approaches, for which entire genes (regulatory and transcribing regions) are the ‘black box’ units. One approach to mid-grained GRN modeling is to represent sets of BSs in each CRM as sequences of symbols (one symbol – one BS) with defined rules of action and coaction for neighboring BSs. That is, a given BS (occupied by its TF) can be strengthened or repressed by a neighboring TF-bound BS ([52, 133], see Fig. 14). Stretches of “placeholders” (no BSs) serve as spacers, delimiting neighboring CRMs. ‘Fine-grained’ approaches operate at individual BS resolution. These models can be corroborated at the level of sequence data, but can be computationally intensive to solve. We describe below a recent set of projects which are beginning to develop an evolutionary approach to fine-grained modeling.

Figure 14.

Repesentation of CRMs as symbolic strings, and the method of calculating the strength of a given TFBS. A) Abstract representation of hb CRMs (Element 1/proximal, Element 2/oogenesis) as clusters of BSs for transcription factors, delimited by spacers (stretches of zeros=placeholders). Each position on the symbolic string can be occupied either by zero (no BS) or by a number from 1 to 7, representing a BS for one of seven transcription factors (Bcd, Cad, Tll, Hb, Gt, Kr or kni). Two values characterize each BS: its identifier and its strength. B) The algorithm to sum the activation strengths for a given activator BS, taking into account both repression via quenching and co-activation from neighboring BSs. For simplicity, we assume that both repression and co-activation are short-range, limited to three neighboring sites. First, local BS strengths are tallied, then neighboring activation is added (co-activation), and, finally, neighboring repression is added (quenching). A and R are activator and repressor BS, respectively.

2.3.1 Evolution of segmentation genes with multiple CRMs

Evidence suggests that the embryonic patterns of each of the Drosophila segmentation genes is regulated by multiple CRMs [e.g. 127]. Some pair-rule genes (the segmentation genes expressed after the gap genes) do show one-to-one correspondence between CRM and domain [128,129,130]. However, this is not universal. In some cases, a single CRM can control multiple domains: in the fushi-tarazu pair rule gene all seven stripe domains are regulated from a single CRM [131]. Other pair-rule genes show a mix, with some stripes one-to-one with a CRM, and other CRMs controlling multiple stripes [128,129,130]. There is also redundancy: many well-known genes for which CRMs have been known for decades are now being found to have ‘shadow’ elements. These distinct (newly discovered) CRMs functionally duplicate the expression controlled by the well-known (non-shadow) CRMs [132]. Many segmentation gene domains which are not fully redundant still show control from two (or even three) CRMs. The role of CRM-domain correspondence in biological development is a very open question.

Related to this is the question of order of domain appearance. For example, the even-skipped (eve) gene is expressed sequentially in short germ insects and relatively simultaneously in long germ insects, but even in this latter case (e.g. in Drosophila) there is a distinct order to the appearance of stripes [129–131]. Are there general rules to how evolutionary changes in eve (or other gene) CRMs correlate with these changes in domain formation? The approach presented in [52, 133], allows for the encoding of CRMs for multiple genes in evolutionary computations of segmentation patterning. Fig. 14 shows how this is done for the hb gap gene.

Symbolic strings (octal in this case) can represent random initial sequences, the wild-type hb regulatory sequences, and intermediate stages in between. GAs are used to perform crossover operations on the strings to evolve them. The strings are formal representations of the real functional connections controlling the hb gene via the network of transcription factors (including the Hb factor itself). At each in silico evolution generation, candidate strings are used to solve a reaction-diffusion model of hb gene expression (see [52,133] for details), accounting for TF strength, short-range co-activation and short-range repression (quenching). The fitness of the string is determined by how well its model pattern matches the experimental data (e.g. Hb expression).

It was shown that it can be roughly a hundred times faster to find one CRM governing formation of all three domains of the hb pattern, than to find three separate CRMs independently controlling separate hb domains one-to-one [133]. This suggests that genes which show multiple domain control by single CRMs may have evolved quite quickly. Constraints of one-to-one, or even one-to-a-few, CRM-domain correspondence may come with costs in terms of evolutionary speed, which would have to be balanced by selective advantage of those constraints (e.g. modularity of stripe formation by ‘swapping’ out sequences). EC helps to characterize the types of CRM-domain dependence that may have arisen, and their associated evolutionary costs.

2.3.2 Evolutionary modeling of feed forward loops

The feed forward loop (FFL; Fig. 15) is a 3-gene motif that is over-represented in real networks in comparison with random networks of the same connectivity (Conant and Wagner [134]; Milo et al. [135, 136]; Ishihara et al., 2005 [137]; Mangan and Alon [138]; Wall et al. [139]). Cooper et al. [140] used an EC approach to determine whether FFLs might have evolved to convert a particular input through X into a particular output Z.

Figure 15.

General scheme of the 3-gene feed forward loop (FFL).

The authors developed a model for the binding of TFs to individual targets on the promoter (the approach was applied to prokaryotic regulation, for E. coli). For C_i the concentration of TFi and Q_ij=c_ij/k_ij, the ratio of ‘on’ to ‘off constants for binding gene j, the steady-state probability of bound TF is P_ij =C_iQ_ij/(1 +C_iQ_ij). Each transcription factor i has an effect on transcription of gene j given by E_ij. The level of transcription of gene j produced by the interaction with factor i is P_ijE_ij. Multiple TFs can bind independently, cooperatively or competitively in the model. For each computation, 3300 gene expression steps are calculated, with continuously increasing X-dependent input. Mutation occurs in parameters E_ij and Q_ij.

Predictions made from the simulations regarding the form of FFL evolved for types of output gene patterns were corroborated against 36 real FFLs in E. coli. Their findings suggest that expression of the downstream Z gene depends on autoregulation in the FFL.

This project is a first step towards fine-grained modeling of a GRN with evolutionary computations. While Q_ij and E_ij are analogous to the regulation matrix W_ij, they break apart TF binding and transcriptional effects, so that evolution of each can be studied.

This approach was next applied to spatial gradient reading [50]. The model is still entirely local (no intercellular communication); spatial dependence stems from a gradient of an upstream TF (‘trigger’) which varies linearly across an array of 11 cells (Fig. 16). Small (two to four gene) networks were evolved to form ‘step function’ responses to the trigger (Fig. 16F). The authors found that competitive binding in networks is more likely to produce step function responses than networks with independent BSs.

Figure 16.

(E) Example of an evolved network with one intermediate: the trigger regulates both the target and intermediate genes, the target and intermediate may both autoregulate and cross-regulate. (F) Spatial expression patterns: the 11 cells of the model are shaded according to expression pattern. AT and RT have evolved opposite responses to the trigger gradient. From [50].

This approach of breaking the regulation matrix down to BS resolution is promising for studying the role of, for example, types of binding in evolution, and could provide a powerful technique for future work.

3. Discussion and Future Directions

The challenge for the work described in this review is to efficiently find the connectivity and dynamics regulating expression in gene networks, and to understand the implications for evolutionary theory. As stated by Francois et al. [47]: “The traditional strategy for modeling a biological system is to start with a network defined by genetics, obtain constants for the interactions (from diverse sources), and then hope it works. However, this strategy does not shed light on the invariant dynamical structure that a particular set of genes implement. This structure is important to understand since it can be implemented by different genes in different species.” Indeed, gene network modeling must not aim just at reconstructing particular connectivity sets, but at uncovering regulatory dynamics. The challenge is that important dynamics lie at a number of levels of gene regulation, and the more detailed the level modeled, the more expensive the computation. Here, we discuss some of the issues that arise, and future directions we find promising.

3.1 Coarse- vs. fine-grained levels of GRN modeling

As described in section 2, the majority of approaches in GRN evolutionary design and evolution in silico have been at the coarse-grained level, where genes are ‘black box’ nodes and gene control is represented by edges in the network. However, as discussed in section 2.3.1, separable CRMs for single genes are common in vertebrate and invertebrate development, and these can range from single CRM-multiple expression functions to very redundant multiple CRM-single expression functions (see review in [132]). The complexity of CRM activity for a gene includes stage-specific CRMs, tissue-specific CRMs, and shadow CRMs [127]. The same TF can act as an activator on one CRM and a repressor on another CRM of a given gene (and not bind the rest of its CRMs). These dynamics of CRMs affect the dynamics of gene expression, and are critical to how GRNs evolved. At a finer level, evolution and regulation operate at the BS and sequence level. CRM and sequence level information will be critical in GRN models and the evolutionary study of GRNs, but come at a computational cost compared to coarse-grained approaches. Systematic comparisons of the cost/benefit between coarse-grained and finer-grained approaches are needed for more test cases, to better understand the appropriate level for addressing particular questions. Our conclusion at this stage is that the mid-grained level of GRN modeling (CRM level) is the best tradeoff between highly expensive calculations (which impact the extent of computational evolution that can be performed) and biologically reasonable simplification of the gene regulatory organization.

3.2 Discrete vs. continuous approaches (Boolean vs. ODE / PDE models)

The choice between Boolean and ODE / PDE models is one of the most crucial questions in the application of evolutionary computations to GRNs. It is argued that with the degree of unknown kinetic constants in GRNs, a Boolean approach may offer the best way to characterize at least the qualitative aspects of a network [28]. For many problems, a Boolean approach may well be the best way to initially characterize a problem, and to gain insight into general evolutionary principles (section 2.1.1). However, there are several reasons to be cautious. First, there are general features of Boolean networks which do not correspond well to dynamics of real GRNs. These include cycle attractors (e.g. [141]) and garden-of-Eden states (e.g. [142]), which may bias our understanding of the modeled GRN’s function and evolution. Second, for reverse-engineered GRNs, where there is experimental model validation, evidence suggests that continuous models are more faithful to known interactions than Boolean models. For example, for AP segmentation in Drosophila, Perkins et al. [143] compared two discrete logical models with two continuous reaction-diffusion models of the corresponding dynamics. Both of the RD models fit the data better than the logical models. The authors expressed concern “that the strict on/off nature of the logical rules renders many regulatory inputs completely redundant, effectively eliminating them from the regulatory structure. … In a densely connected network like the gap gene system, it results in the elimination of many correct regulatory links.” This would apply in more exploratory modeling cases, where there is not such an experimental check, but where the Boolean approach could still bias the outcome. If the aim is to minimize networks and eliminate redundant connections, this technique may be preferred; but if the aim is regeneration of the biological control network, an ODE/PDE approach may be better. Finally, the evolutionary landscape of GRNs can be quite different depending on whether a discrete or continuous approach is used. Using a discrete approach, Ciliberti et al. [28, 144] suggested that the collection of GRNs which create a particular phenotype (e.g. expression pattern) form a neutral basin in the fitness landscape, such that drift within the basin allows for a neutral means of sampling different phenotypic variations (at the ‘borders’ of the basin). However, this discrete approach does not address the natural continuous variation of gene-gene interaction parameters (due, e.g., to tuning of enzymatic co-factors or complex coregulation by multiple transcription factors). Our experience in evolutionary searches indicates that very small differences in these parameters can produce very different phenotypes (e.g. robust vs. non-robust to maternal variability). This suggests that the achievement of robust GRNs in a continuous evolutionary search can be quite rare, and that such solutions can be quite isolated, reflecting a complex fitness landscape which is far from neutral. Continuous descriptions are needed to capture the size and complexity of the genotype space. Such complexity is also indicated by theoretical studies of continuous-GRN parameter spaces showing multistability (e.g. [145]).

3.3 Further directions of the W-matrix approach

The level of activity in modeling Drosophila segmentation (e.g. [1, 17, 18, 19, 21, 22]) provides a prime opportunity for evaluating and refining techniques for fitting expression data. Connectionist-type models have shown success with WT patterning, including temporal shifts [146], but have not been as successful in generating mutant phenotypes (see [21]). We believe extensions of the connection matrix, as in section 2.1.1.2.1, may prove useful for modeling mutants. Another route would be to use S-systems for the production parts of the model equations. The promising aspect of this is that the computer science of JEC with S-systems is very well developed (section 2.1.4), which could rapidly lead to applications with heavy evolutionary components. A good starting place might be to use the simplified S-system with diffusion of Furusawa and Kaneko [147, 148, 149].

3.4 More reactions for in silico evolution

More diversity in the types of reaction modeled may broaden the applicability of in silico evolution. For instance, Eldar and co-authors [150] used a numerical screen of RD models for a general morphogen system, with the constraint of finding robust networks (to fluctuations in morphogen production; see also [151]). For a system with a ligand (L) its receptor (R) and a protease (P), they analyzed the following features: self-enhanced morphogen degradation; degradation of free morphogen by protease; binding of protease to receptor; signaling-dependent feedback on receptor expression; and degradation of morphogen through receptor-mediated endocytosis. Inclusion of such features associated with robustness may make EC approaches more accurate for studying the evolution of robustness (and likewise for other dynamic properties of the network).

3.5 Fitness functions and crossover algorithms

The choice of fitness (or more accurately ‘fitness function’, since it assigns a number to an arbitrary genetic network) is crucial for GRN evolutionary design. As Francois and Siggia [48] stated “The topography of the cost [fitness] function and the way mutations sample it control the convergence rate to an optimum. If the topography is a funnel leading to a unique minimum, convergence for any reasonable mutation process is assured…The fitness function should … ideally provide cues that will direct an arbitrary genetic network along a path of continuous increasing fitness.” While this is ideal from a computational convergence perspective, many (if not the large majority of) evolutionary problems do not offer a smoothly ascending route to the optimum. Rather, many evolutionary searches are through a ‘rugged’ landscape [152] with significant valleys and hills between local optima; or can be characterized as ‘subbasin-portal’ landscapes, in which long periods of neutral evolution are punctuated by quick transits to a discretely different fitness level. Computation through such ‘hard’ landscapes is a major challenge for efficient EC projects.

There have been a number of works grading and classifying the difficulty of biological and artificial evolutionary searches (e.g. [152, 153, 154, 155]); and a number of methods have been proposed in computer science to address these. We discuss subbasin-portal architectures here.

Traditional point mutation methods in GA tend to be quite detrimental to searches in neutral fitness basins, because they tend to destroy forming or already-formed meaningful ‘words’ in a string (e.g. a CRM in the DNA) which may alter fitness (i.e. find a portal). An improvement is to use “building blocks” (BBs) [156, 157, 158, 159, 160, 161, 162], such that a solution can be decomposed into a number of BBs (e.g. CRMs), which can be searched for independently and afterwards be combined to obtain good or optimal solutions. Parallel work on the use of BB computations in experimental directed evolution may provide inspiration for developments in GRN modeling (e.g. [163, 164, 165, 166, 167]).

To implement BBs in GRN evolution, we have developed a crossover recombination operator which maintains meaningful BB sequences, such as CRMs [51,52,133]. We call this technique retroGA, since it is inspired by the mechanism of retroviral recombination. The technique converges much more quickly than traditional GA or other ECs on test functions with subbasin-portal architecture (Royal Road (RR) and Royal Staircase (RS)), and we have begun to use it on similarly-structured biological problems, such as the Drosophila segmentation network and evolution of the bacterial ribosomal RNA operon promoter (rrnP1).

A significant mathematical theory has been developed around test functions such as RR and RS [168, 169, 170, 171, 172, 173, 174, 175, 176]. With such approaches, the degeneracies in fitness basins can be understood from the framework of statistical mechanics, dynamical systems theory, evolutionary search theory, molecular evolution theory, and mathematical population genetics. This allows for prediction of optimal parameter settings for such searches, and for the understanding of emergent mechanisms in the dynamics of general evolutionary searches. One of our major aims with developing BB ECs for CRM evolution is to be able to bridge this analytical understanding of evolutionary trajectories in hard landscapes from test cases to real GRNs.

3.6 Conclusion

The use of modeling in understanding GRNs has become mainstream in the past decade, with applications in numerous areas. We have focused on the modeling of developmental networks, but evolutionary modeling has exploded across other areas of molecular biology as well (in vivo and in vitro). One aspect of ‘evolutionary’ in this review has been the biological side, in which modeling is being used to study the way in which GRNs have or may have evolved to solve particular functions (e.g. spatial or temporal periodicity of expression). Such modeling has also indicated general evolutionary principles, such as how networks might evolve robustness to various perturbations. The other aspect of ‘evolutionary’ covered here is the computer science side. GRN computations of evolution are very intensive, and require cutting-edge optimization methods. EC has developed over the past two decades as the cutting-edge methodology for efficient searches. As such, we feel that progress in modeling biological evolution will be greatly enhanced by incorporation of recent developments in EC optimization techniques, many of which have already been developed on ‘test case’ biological type problems. Such a convergence of approaches will greatly open up the type, level of detail, and complexity of gene regulatory and cell regulatory problems which can be modeled.