Evolutionary Dynamics of RNA-like Replicator Systems: A Bioinformatic Approach to the Origin of Life

Abstract

We review computational studies on prebiotic evolution, focusing on informatic processes in RNA-like replicator systems. In particular, we consider the following processes: the maintenance of information by replicators with and without interactions, the acquisition of information by replicators having a complex genotype-phenotype map, the generation of information by replicators having a complex genotype-phenotype-interaction map, and the storage of information by replicators serving as dedicated templates. Focusing on these informatic aspects, we review studies on quasi-species, error threshold, RNA-folding genotype-phenotype map, hypercycle, multilevel selection (including spatial self-organization, classical group selection, and compartmentalization), and the origin of DNA-like replicators. In conclusion, we pose a future question for theoretical studies on the origin of life.

1. Introduction

This paper is centered on one question: How can a system of simple RNA-like replicators increase its complexity through evolution? Our motivation is two fold. The question is crucial to the RNA world hypothesis as explained in the next section. Moreover, the simplicity of RNA-like replicator systems makes it easier to investigate evolution as a process of pattern formation operating at multiple levels of organization such as genotypes, phenotypes, interactions, and spatiotemporal distributions of individuals. We approach the above question from the view point of bioinformatics in a wide sense, namely, the study of informatic processes in living systems [1]. From this view point, we will review a variety of mathematical or computational models of RNA-like replicator systems. Emphasis will be placed on the use of models as a tool to discover the unforeseen rather than a tool to confirm the preconceptions.

1.1. Organization of this paper

In Section 2, we briefly review the RNA world hypothesis and motivate the central question of this paper.

In Section 3, we review the evolutionary dynamics of replicators that do not interact with each other. We first describe the quasi-species theory as an improvement on the “survival of the fittest” principle. We next show that there is a severe limit on the amount of information that can be maintained by evolution (the problem of information maintenance). In addition, we discuss error thresholds.

In Section 4, we review the evolutionary dynamics of replicators that have a complex genotype-phenotype map (no interactions assumed as in Section 3). We show that high redundancy in the genotype-phenotype map facilitates evolution toward a target phenotype. We also show that such high redundancy, however, does not solve the problem of information maintenance (phenotypic error threshold). In addition, we describe neutral evolution of mutational robustness.

In Section 5, we review the evolutionary dynamics of replicators that interact with each other. We first describe a replicator network known as a hypercycle and its limitation, namely, the problem of parasites. We then show how this problem can be solved by the consideration of spatial self-organization and discrete populations. We next introduce a simpler replicator network (one-replicase one-parasite system) and describe the phenomenon of multilevel evolution. In addition, we describe the effect of complex formation on the evolutionary dynamics of replicators.

In Section 6, we review the evolutionary dynamics of replicators that have a complex genotype-phenotype-interaction map. We show how complexity can evolve in an RNA-like replicator system through a positive feedback between the evolution of sequences and the evolution of ecosystems.

In Section 7, we review the evolutionary dynamics of compartmentalized replicators. We first describe the classical theory of group selection as applied to RNA-like replicator systems. We then describe a model of protocells and multilevel evolution that occurs in this model. We show that this multilevel evolution differs from that mentioned above and explain how this difference arises.

In Section 8, we review the evolutionary dynamics of DNA-like replicators (i.e., replicators that can serve as templates, but not as catalysts, of replication). We describe how the division of labor between templates and catalysts can emerge through the evolution of DNA-like replicators in RNA-like replicator systems.

In Section 9, after briefly summarizing the preceding sections, we suggest a possible direction for future research on the origin of life from the view point of bioinformatics.

2. The RNA world hypothesis

Living systems are amusingly diverse at a glance, beautifully sophisticated on inspection, and staggeringly complex on reflection. Yet, in thinking about the origin of life, one can ask a question: What would be the simplest system conceivable if one simplified current living systems as much as possible? Then, the conjugate question is, How could this simplest system evolve into systems as complex as life as we know it?

A basic unit of biological systems is the cell (ignoring viruses for a moment). Very roughly speaking, half the dry mass of an E. coli cell is proteins, a quarter RNA, an eighth phospholipid, and a sixteenth DNA [2]. Proteins catalyze various chemical reactions essential to cells including the synthesis of lipids, DNA, and RNA. Proteins are synthesized by the translation of mRNAs, which are, in turn, synthesized by the the transcription of DNA. DNA molecules are (nearly) always synthesized by the replication of already existing DNA molecules as templates (except, e.g., telomeres). In other words, information flows from DNA to proteins (or more precisely, from nucleic acids to proteins), but not vice versa—i.e., the central dogma of molecular biology [3, 4]. It, therefore, appears that proteins and DNA are the essential components of living systems.

However, a great surprise came from studies on ribosomes. These studies revealed that rRNAs, rather than ribosomal proteins, catalyze the synthesis of proteins (i.e., the polymerization of amino acids), discriminate between correct and incorrect codon-anticodon pairs, and prevent the premature hydrolysis of peptidyl-tRNAs (see, e.g., Ref. [5], for review). Therefore, “the ribosome is a ribozyme” [6]. These findings have two implications. First, it is conceivable that proteins can also catalyze the synthesis of proteins; in fact, proteins are the common catalysts of various chemical reactions occurring in the cell. Nevertheless, RNA molecules are the actual catalyst of protein synthesis, one of the most vital reactions for life. It seems as if this role of RNA is a historical contingency. The second implication is that not only proteins, but also RNA can function as an efficient catalyst. In addition, in vitro evolution experiments have shown that RNA molecules can catalyze a variety of chemical reactions relevant to biological processes such as RNA replication, nucleotide synthesis, thymidylate synthesis, lipid synthesis, and sugar metabolism (see Ref. [7], for pioneering work; see, e.g., Ref. [8, 9], for review). Therefore, RNA molecules can perform functions equivalent to those performed by proteins (at least partially).

Interestingly, a similar situation exists in RNA and DNA. RNA and DNA are chemically very similar to each other, the only difference being the presence or absence of one oxygen atom per nucleotide. Although RNA molecules are the only templates from which proteins are translated in the cell, DNA can also serve as such templates under suitable conditions in vitro [10]. Although DNA is the major carrier of genetic information in living systems, RNA can also carry genetic information as exemplified by RNA viruses. Moreover, in vitro evolution experiments have shown that not only RNA but also DNA can catalyze various chemical reactions including RNA ligation, RNA cleavage, and DNA ligation (see Ref. [11], for pioneering work; see, e.g., Ref. [12], for review). The range of reactions catalyzed by “deoxyribozymes” is currently limited in comparison with those catalyzed by ribozymes (22 reactions are catalyzed by deoxyribozymes, whereas 44 by ribozymes, according to Ref. [12]). However, there is currently no clear experimental evidence indicating that DNA is less competent than RNA at providing chemical catalysis [13].

Despite the chemical similarity, RNA and DNA are biologically distinct from each other. The function of DNA in living systems is essentially the storage of genetic information. RNA, in contrast, has various functions in many biological processes besides protein synthesis and information storage. For example, RNA appears in numerous cofactors essential for metabolism, such as ATP, nicotinamide adenine dinucleotide, and flavin adenine dinucleotide [14]; RNA serves as the precursor of DNA (i.e., DNA monomers are synthesized from RNA monomers [15]); the other functions of RNA include gene regulation, metabolite sensing, and defense against viruses [16].

The above observations can be summarized in two points: the functional equivalence (at least in principle) between RNA and proteins and between RNA and DNA; the involvement of RNA in various important processes of current living systems despite the aforementioned equivalence. These points each lead to a hypothesis (implication). First, a simpler form of “life” might be possible, in which both information storage and chemical catalysis are provided by a single type of molecules, RNA. Second, the ancestors of current living systems might have actually taken such a simpler form; and DNA and protein are the evolutionary “latecomers”, which took over most of the functions of RNA and beyond. These hypotheses (implications) are truly remarkable especially in view of the essentiality of proteins and DNA in current living systems.

These hypotheses are called the RNA world hypothesis [17] (see Ref. [18], for more extensive reviews). In its conceptually simplest form, an RNA world consists of RNA molecules that can replicate themselves, that is, RNA replicators [19–21]. Thus, the question naturally arises: How can a system of RNA replicators evolve into life as we know it? To put it in a more manageable form, How can a system of simple RNA-like replicators increase its complexity through evolution?

To consider this question, we analyze and compare a multitude of mathematical or computational models of RNA-like replicator systems. Before starting, however, let us first clarify what kind of insights we seek to obtain from such investigation. To this end, two facts are relevant. First, whether the ancestry of life can be traced back to RNA replicators is far from established, nor is it likely to be established in the near future. Second, the kinds of replicators we consider do not exist in reality though it might be a matter of time before they are synthesized in the laboratory [22–27]. Thus, our aim is neither the reconstruction of the history nor the theoretical reproduction of particular RNA replicator systems existing in reality. Rather, our aim is to investigate what one can (or cannot) expect from the evolutionary dynamics of RNA-like replicator systems conceived in the simplest forms. From this investigation, we also aim to learn what one should (or should not) conceive of RNA-like replicator systems if these systems are to display the evolution of complexity. Thereby, we seek to obtain general insights into the origin of biological complexity. To these ends, it is important to construct models without explicitly aiming at the production of preconceived output [28]. Instead, the behavior of models are to be explored and discovered [29]: “We want models that talk back to us, models that have a mind of their own” [30, p. 142]. We then compare different models to identify general principles and to determine causal relations.

Besides the above question, another question that is equally important must be mentioned: Can such RNA replicators actually exist (on early Earth)? Many scientists are striving to answer this question by laboratory experiments. Admittedly, however, the experimental work and theoretical work have been following a rather independent line of development, probably owing to this very difference in the questions.¹ Yet, the two questions are obviously related, and many of the theoretical studies are motivated by the experimental studies in one way or another. For the experimental studies, readers are referred to recent reviews and papers [18, 26, 27, 32].

3. Replicators without interactions

3.1. Essence of evolution

Evolution is based on the two types of variations:

1. In a population, variations are generated between individuals in their heritable characters.

2. There are variations between individuals in the number of descendants.

From these variations, it follows that the characters of a population change (evolve) over generations. In other words, evolution is accounted for by the conversion of “spatial” variations in a population (i.e., variations between individuals) into temporal variation of a population [33].

The above formulation of evolution does not by itself determine any particular evolutionary dynamics. Three points must be further considered: the nature of variations in heritable characters; the nature of variations in the number of descendants; and the relationship between the two. One must specify all three points, explicitly or implicitly, in order to construct a model of evolutionary processes. In the next section, we will consider the simplest such specification (assumption) in a model of RNA-like replicator systems.

In addition, evolution, as formulated above, includes what is sometimes called non-Darwinian evolution such as neutral evolution, inheritance of acquired characters, and evolution of evolvability. First, variations in the number of descendants can be uncorrelated with variations in heritable characters—hence neutral evolution (see also Section 4.1). Second, variations in heritable characters need not be produced randomly. Individuals may actively acquire heritable characters through interactions with the environments, as exemplified by the CRISPR-Cas.² Third, it is conceivable that the relationship (mapping) between the two kinds of variations (or more precisely, variability [35]) itself is a heritable character [36], variations of which can again be subject to evolution (see, e.g., Ref. [37, 38]). These processes enrich, rather than contradict, the framework of evolution.

3.2. Quasi-species theory

The simplest assumption for evolution is that heritable variations are the numbers of descendants. Under this assumption, we construct the simplest model of an RNA-like replicator system and analyze its evolutionary dynamics. This model will show that, contrary to intuition, “survival of the fittest” does not necessarily ensue.

We make the following assumptions about a replicator system [39–41]:

• Each replicator is represented by a sequence of 0s and 1s of length ν (which we call a genotype). (All possible genotypes compose a genotype space.)

• Each replicator replicates itself at a rate A_i (fitness). A_i is a function of a replicator’s genotype denoted by i (a genotype-fitness map).

• Mutations can happen during replication with a probability 1 − q per digit in a genotype. A mutation changes 0 to 1 or 1 to 0 (i.e., flips one bit). Let Q_ji be the probability that replication of genotype i produces genotype j. If j = i, Q_ii = q^ν. By definition, ∑_j Q_ji = 1. (For simplicity, we ignore insertion, deletion, and recombination.)

• The population size of replicators is infinitely large, and the system is well mixed.

• Replicators flow out of the system at a rate ϕ in such a way that the total concentration is kept constant. (This outflow introduces competition between replicators.)

From the above assumptions, we can construct the following ordinary differential equation (ODE) model:

$${\stackrel{.}{x}}_i=A_i{Q}_{ii}{x}_i+{\displaystyle \sum _{j\ne i}{A}_j{Q}_{ij}{x}_j-\varphi x_i}$$ (1)

where x_i denotes the concentration of replicators with genotype i (a dot above a variable indicates time derivative). Eq. 1 describes the population dynamics of replicators under mutation and selection. The first term on the right-hand side represents the multiplication by exact replication; the second term, mutation fluxes; and the third term, the outflow. To specify the expression of ϕ, we sum up Eq. 1 over i and obtain ċ = ∑_i A_ix_i − ϕc where c = ∑_i x_i (∑_j Q_ji = 1 is used). Based on this expression of ċ, we assume ϕ = ∑_i A_ix_i/c (i.e., ϕ is the population average of A_i) so that the total concentration c is kept constant.³ Eq. 1 is known as the quasi-species equation [41].

Before presenting the results of the model, let us add a few parenthetical remarks. To obtain Eq. 1 we did not consider decay of replicators because such consideration does not substantially modify the arguments described below.⁴ Moreover, replication in Eq. 1 can be either direct replication or complementary replication depending on a value of q. If 1/2 < q, replication is direct; if q < 1/2, replication is complementary. In what follows, we only consider the case of direct replication by assuming 1/2 ≤ q (see Ref. [42], for complementary replication).

We next analyze the equilibrium behavior of Eq. 1 to investigate the outcomes of evolution. Let us begin with the simplest case, in which replication is exact. If q = 1, Eq. 1 becomes

$${\stackrel{.}{x}}_i=(A_i-\varphi )x_i$$ (2)

The equation indicates that the genotypes whose replication rates (A_i) are higher than the population average (ϕ) will increase their concentrations; those that do not will decrease their concentrations. Consequently, ϕ will increase until the population eventually consists entirely of the genotype whose value of A_i is the greatest. Therefore, the survival of the fittest ensues.

We next consider the case in which q < 1. Eq. 1 can be written in a matrix form:

$$\stackrel{.}{\overrightarrow{x}}=\mathbf{\text{QA}}\overrightarrow{x}-\varphi \overrightarrow{x}$$ (3)

where A is a diagonal matrix whose diagonal elements are A_i, and Q is a matrix whose elements are Q_ij. This notation suggests the diagonalization of QA. We thus consider the transformation of variables with x⃗ = By⃗ where B is a matrix whose columns are the eigenvectors of QA (denoted by υ⃗_i). Using the variable y⃗, we can transform Eq. 3 into

$${\stackrel{.}{y}}_i=({\lambda }_i-\varphi )y_i$$ (4)

where λ_i is an eigenvalue of QA. Likewise, we can also transform ϕ = ∑_i A_ix_i/c into ϕ = ∑_i λ_iy_i/∑_i y_i. Eq. 4 has the same form as Eq. 2. In Eq. 4, x⃗ is decomposed into distributions (υ⃗_i) whose dynamics (ẏ_i) are independent of each other. Alternatively, Eq. 4 can be interpreted as describing the dynamics of competition between different stable distributions of x_i (viz. υ⃗_i) whose “concentrations” are y_i and whose “replication rates” are λ_i (ϕ is again the population average of replication rates). Eq. 4 shows that υ⃗_i whose λ_i value is greater than the population average (ϕ) will increase its concentration. Consequently, ϕ increases until the entire population consists of the “fittest” υ⃗_i. This result can be intuitively understood as follows. The normalization term −ϕx⃗ in Eq. 3 does not modify the direction of x⃗ because it is parallel to x⃗; therefore, the final state must be the scaled dominant eigenvector of QA.

Because QA is non-negative and primitive,⁵ the Perron-Frobenius theorem holds [44]. The theorem states that (1) there is unique υ⃗_M whose λ_M is real and satisfies λ_M > |λ_i| for all i ≠ M and that (2) the elements of υ⃗_M are all positive [43] (see Ref. [45], for a more special case). Using this theorem, we can infer that evolution transforms a population into a unique distribution (of genotypes) whose multiplication rate λ is the greatest. The theorem, however, also states that (3) υ⃗_M is the only eigenvector whose elements are all positive. This fact renders the biological meaning of υ⃗_i other than υ⃗_M less concrete (see Ref. [46], for how this is resolved by the finiteness of a population). Eigen et al. define υ⃗_M as the quasi-species and refer to it as the target of evolution [41].

Let us consider the following simplistic, yet illuminating example of a genotype-fitness map [47]. This genotype-fitness map defines a unique genotype whose A_i value is the greatest among all genotypes (the exact sequence pattern of the fittest genotype does not matter). The genotype differing from the fittest genotype by the greatest Hamming distance has the second greatest value of A_i (there is only one such genotype). The one-step mutants of the second fittest genotype—i.e., those differing from it by the Hamming distance of 1—have the third greatest value of A_i. All the other genotypes have the smallest value of A_i. For a pseudo-visual aid, it is illustrative to imagine a fitness landscape [48] in which there are two peaks in the “corners” of the genotype space and a valley between these peaks. Given that A_i is a function of the Hamming distance from the fittest genotype, it is convenient to consider a genotype class (denoted by G_d) that consists of the genotypes whose Hamming distance from the fittest genotype is d. Let z_d be the concentration of G_d (z_d = ∑_i∈Gd x_i). Then, we can transform Eq. 1 into

$${\stackrel{.}{z}}_d=A_{i\in G_d}{M}_{dd}{z}_d+{\displaystyle \sum _{e\ne d}{A}_{i\in G_e}{M}_{de}{z}_e-\varphi z_d}$$ (5)

which has the same form as that of the original equation except that M_de = ∑_i Q_ij where i ∈ G_d and j ∈ G_e [49].

The equilibrium values of z_d and x_d exhibit a sharp transition at some critical mutation rate 1 − q ≈ 0.036 (Fig. 1). If 1 − q exceeds this value, the dominant genotype switches from the fittest to the second fittest. After this switch, the concentration of the fittest genotype becomes nearly 0. This result shows that which genotype evolves depends not only on the fitness of individual genotypes, but also on the fitness of genotype neighborhoods (i.e., genotypes and their respective close mutants) and the mutation rate. (The other transition at a greater mutation rate will be discussed in Section 3.3.)

Figure 1. Survival of a non-fittest.

A: The equilibrium concentrations of genotype classes (z_d) are plotted against the mutation rate per digit (1 − q). The thick broken line represents the fittest genotype class (z₀); the thick solid line, the second fittest genotype class (z_ν); the thin solid line, the third fittest genotype class (z_ν−1); the dotted lines, the other genotype classes. The parameters were as follows: ν = 50; A_i∈G0 = 10; A_i∈Gν = 9.9; A_i∈Gν−1 = 2; A_i∈Gd = 1 where 0 < d < ν − 1. The results were obtained by numerically calculating the dominant eigenvector of the matrix (B_ij) where B_ij = A_k∈Gj M_ij. The total concentration is scaled to 1 (this is always the case unless otherwise stated; see also Footnote ³). B: The equilibrium concentrations of genotypes (x_i) are plotted against the mutation rate (1 − q). x_i is calculated as $x_{i\in G_d}=z_d/\left(\begin{array}{c}\nu \hfill \\ d\hfill \end{array}\right)\text{ where }\left(\begin{array}{c}\nu \\ d\end{array}\right)$ is the number of genotypes in a genotype class G_d. The parameters were the same as in A.

The above example indicates that survival of the fittest does not necessarily hold. Nevertheless, simplifying evolution as maximization of fitness is useful, so let us reformulate this maximization principle as follows (hereafter referred to as the quasi-species theory). Evolution operates not on individual genotypes, but on genotype neighborhoods (or more simply, quasi-species), that is, local subspaces of the genotype space.⁶ The outcome of evolution is determined by the (weighted average) fitness of genotype neighborhoods—hence survival of the fittest genotype neighborhood (which does not necessarily include the fittest genotype). The size of genotype neighborhoods on which evolution operates increases as the mutation rate or the sequence length increases. Stated differently, evolution depends (or “detects”) not only on an individual’s fitness, but also on an individual’s genotype, that is, on how an individual’s fitness (or phenotype) is “coded” in its genotype. This is because this coding influences the fitness of the individual’s close mutants (i.e., genotype neighborhood) [50]. The cases in which evolution depends on such coding are seen in extremely diverse types of evolutionary models [51–59] (often with a different terminology such as the evolution of mutational robustness). A few of these models will be reviewed later in this paper (Section 4.1 and 7.2.2).

3.3. Information threshold

We examine how much information can be maintained by evolution in the replicator system described above. To this end, let us consider the following genotype-fitness map (known as a sharply peaked fitness landscape): one genotype has the highest fitness A₀, and all the other genotypes have an identical fitness A_m (A_m < A₀) [39]. We categorize genotypes in two classes: the class G₀ consisting of the fittest genotype and the class G̃_m = {G_i | i > 0} consisting of all the other 2^ν − 1 genotypes (i.e., all mutants). Because of this asymmetric distribution of genotypes, a mutation occurring to every genotype in G̃_m is likely to produce another genotype in G̃_m. We thus ignore mutations from G̃_m to G₀ (called back mutations). Then, Eq. 5 is simplified into

$${\stackrel{.}{z}}_0=A_0{Q}_0{z}_0-z_0\varphi$$

$${\stackrel{.}{\tilde{z}}}_m=A_m{\tilde{z}}_m+A_0(1-Q_0)z_0-{\tilde{z}}_m\varphi$$ (6)

where z̃_m = ∑_i>0 z_i, Q₀ = q^ν, and ϕ = A₀z₀ + A_mz̃_m (z₀ + z̃_m is normalized to 1) [60]. Let us find under what condition G₀ survives. To this end, we use the trick called “invasion experiment”, which examines whether G₀ can invade the system that is occupied by G̃_m. For such invasion to be possible, the replication rate per unit amount of G₀ must be positive (i.e., ż₀/z₀ > 0) when z₀ ≈ 0, ${\stackrel{.}{\tilde{z}}}_m=0$, and z̃_m > 0. If this invasion criterion is fulfilled, z₀ > 0 is likely at equilibrium. ż₀/z₀ > 0 can be simplified into

$$A_0{Q}_0>A_m$$

that is, the effective multiplication rate of G₀ must be greater than the multiplication rate of G̃_m. Since Q₀ = q^ν ≈ e^{−ν(1 − q)}, we obtain

$$\nu <\frac{\text{ln}\sigma }{1-q}$$ (7)

where σ = A₀/A_m, which represents the relative fitness advantage of G₀. Importantly, σ appears as its logarithm in Eq. 7 and so has a minor effect as compared with 1 − q (mutation rate) and ν (length). If Eq. 7 is violated, G₀ cannot maintain itself through its own multiplication. Stated differently, if ν or 1 − q or both are too large, the system loses the “information” contained in G₀.

We next take back mutations into account. Mutations from G̃_m into G₀ keep z₀ positive even if the condition Eq. 7 is violated (this is indicated by the Perron-Frobenius theorem; see Section 3.2). However, the fact that z₀ > 0 does not necessarily mean that the information contained in G₀ is maintained by evolution as described below. First, if Eq. 7 is violated,⁷ the population distribution in the genotype space becomes almost uniform (Fig. 2A) [41]. That is, evolution hardly reflects any differences between genotypes. (If the population size is finite, the uniform distribution corresponds to genetic drift.) Second, let us consider a useful concept called the ancestor distribution [61]. Let $a_d^{\prime }(\tau ,t)$ be the fraction of the population at time t + τ (τ > 0) whose lineage is traced back to individuals existing at time t whose genotypes belong to G_d. Then, the ancestor distribution is defined as $a_d={\text{lim}}_{\tau \to \infty }^{t\to \infty }{a}_d^{\prime }(\tau ,t)$ [61]. If Eq. 7 is fulfilled, the lineage of almost an entire population can be traced back to ancestors whose genotypes belong to G₀, no matter how small the equilibrium value of z₀ may be (Fig. 2B). Stated differently, only the lineage of G₀ can continue in a long run. If, however, Eq. 7 is violated, the lineage of a population is traced back almost equally to ancestors of any genotype (Fig. 2B). That is, evolutionary history hardly reflects any differences between genotypes. (See Ref. [62], for how ancestor distributions can be defined for finite population systems.) Therefore, even if back mutations are taken into account, the information contained in G₀ is lost if Eq. 7 is violated.

Figure 2. Error threshold.

A: The equilibrium concentrations of genotype classes (z_d) are plotted against the mutation rate (1 − q). The thick line represents the fittest genotype (z₀); the thin lines, the mutant genotype classes (z_d>0). The inset shows the equilibrium concentrations of genotypes (x_i). In the inset, the thick line represents the fittest genotype; the thin lines, the mutant genotypes. x_i was calculated as $z_d/\left(\begin{array}{c}\nu \hfill \\ d\hfill \end{array}\right)\text{ where }\left(\begin{array}{c}\nu \hfill \\ d\hfill \end{array}\right)$ is the number of genotypes in a genotype class G_d. The parameters were as follows: ν = 50; A_i∈G0 = 10; A_i∈Gd≠0 = 1. B: The ancestor distribution (a_d) is plotted against the mutation rate (1 − q). The inset shows the ancestor distribution with respect to genotypes $a_d/\left(\begin{array}{c}\nu \hfill \\ d\hfill \end{array}\right)$ rather than genotype classes. The parameters were the same as in A.

The condition Eq. 7 is commonly known as the error threshold (maximum allowable 1 − q) or the information threshold (maximum allowable ν). This is because, for fitness landscapes with sharp peaks, the population composition at equilibrium exhibits a sharp transition at such a threshold (Fig. 1, 1 − q ≈ 0.045; Fig. 2). However, whether the system displays such a sharp transition (whether it has such a threshold) is actually irrelevant to the conclusion that the amount of information that can be maintained by evolution is limited [62]. To illustrate this point, let us consider the genotype-fitness map in which the fitness is defined as A_i∈G_d = s^−d where s is a constant and s > 1 (known as the multiplicative fitness landscape). In this fitness landscape, the system displays no sharp transitions and so has no error threshold (Fig. 3) [63]. Nevertheless, if the mutation rate is sufficiently high, evolution does not reflect any differences between genotypes (Fig. 3B, inset). Therefore, the system can lose information contained in G₀ because of erroneous replication. The absence of an error threshold only makes it difficult to say exactly at what point the information is lost.

Figure 3. No error threshold.

A: The equilibrium concentrations of genotype classes (z_d) are plotted against the mutation rate (1 − q). The thick line represents the fittest genotype (z₀), the thin lines, the mutant genotype classes (z_d>0). The inset shows the equilibrium concentrations of genotypes (x_i). In the inset, the thick solid line represents the fittest genotype; the dotted lines, the mutant genotypes. x_i was calculated as $z_i/\left(\begin{array}{c}\nu \hfill \\ i\hfill \end{array}\right)$. The parameters were as follows: ν = 50; A_i∈Gd = 1.05^−d. B: The ancestor distribution (a_d) is plotted against the mutation rate (1 − q). The inset shows the average Hamming distance of the ancestor distribution from the fittest genotype 〈d〉_ad = ∑_d a_dd as a function of the mutation rate. The parameters were the same as in A.

Next, we slightly modify Eq. 6 as follows. Let us suppose that G₀ is a sequence pattern embedded in a long sequence (hereafter referred to as the background sequence). If the background sequence contains a certain “correct” pattern (background pattern), G₀ increases the fitness of its carrier by a factor of σ (= A₀/A_m). If the background sequence contains wrong patterns, G₀ gives no effect. We assume that the mutant class G̃_m has the correct background pattern, and another mutant class (denoted by G̃_n) does not. In this new setup, Eq. 6 is modified into

$${\stackrel{.}{z}}_0=\sigma A_m{Q}_0{Q}_m{z}_0-\varphi z_0$$

$${\stackrel{.}{\tilde{z}}}_m=A_m{Q}_m{\tilde{z}}_m+\sigma A_m(1-Q_0)Q_m{z}_0-\varphi {\tilde{z}}_m$$ (8)

$${\stackrel{.}{\tilde{z}}}_n=A_n{\tilde{z}}_n+A_m(1-Q_m){\tilde{z}}_m+\sigma A_m(1-Q_m)z_0-\varphi {\tilde{z}}_n$$

where Q₀ = q^ν as before; Q_m = q^η where η denotes the length of the background sequence pattern; z_n denotes the concentration of G̃_n; and A_n, the fitness of G̃_n. Under what condition can G₀ survive through its own multiplication? The invasion criterion for z₀ is σA_mQ₀Q_m > ϕ. ϕ can be obtained from the steady state in which z₀ = 0, as follows. If z₀ = 0, Eq. 8 reduces to Eq. 6 except that the subscripts 0 and m are replaced by m and n, respectively. We now assume that A_mQ_m > A_n (i.e., evolution can maintain the background sequence pattern in G̃_m). Then, the steady state condition ${\stackrel{.}{\tilde{z}}}_m=0$ yields A_mQ_m − ϕ = 0. Therefore, the invasion criterion for z₀ is σA_mQ₀Q_m > A_mQ_m, which boils down to the same expression as Eq. 7. This result suggests a re-interpretation of Eq. 7 as follows. Let us suppose that some sequence pattern, if contained in a genotype, increases the fitness of its carrier by a factor of σ. For this sequence pattern to be maintained by evolution, the length of this pattern—i.e., the amount of information—must be smaller than ln σ/(1 − q) for the mutation rate 1 − q.

To illustrate why the limitation described by Eq. 7 arises, let us consider a model that does not show any such limitation [64]. In this model, we disregard the internal structure of genotypes and, instead, directly conceive a genotype space as follows. Genotypes are ordered by their fitness A_i such that A_i > A_i+1 where the subscripts denote genotypes (0 ≤ i ≤ n). A mutation happens with a probability 1 − Q per genome per replication. It damages the replicators’ genomes such that their fitness is decreased from A_i to A_i+1. We assume that mutations have no effect on the n-th genotype because this genotype is completely destroyed (back mutations are ignored). We then obtain the following equations:

$${\stackrel{.}{x}}_0=A_{0}Q{x}_0-\varphi x_0$$

$${\stackrel{.}{x}}_i=A_{i}Q{x}_i+A_{i-1}(1-Q)x_{i-1}-\varphi x_i$$ (9)

$${\stackrel{.}{x}}_n=A_n{x}_n+A_{n-1}(1-Q)x_{n-1}-\varphi x_n$$

where 0 < i < n and ϕ = ∑_j A_jx_j. It can be shown that the condition for the survival of the fittest genotype is A₀Q > A_n (apply invasion experiments consecutively). Thus, the fittest genotype always survives unless A_n > 0. A_n > 0 means that the completely destroyed genotype has a positive fitness (i.e., no lethal mutants exist). Since assuming A_n > 0 is unrealistic, there seems no limit on the amount of information that can be maintained by evolution (except under the unrealistic assumption) [64].

Eq. 9 is almost identical to Eq. 6 (they are identical if n = 1). The condition A₀Q > A_n from Eq. 9 corresponds to the condition σQ₀ > 1 from Eq. 6. Hence, the almost identical models have been used to draw the totally opposite conclusions. This paradox arises from the different interpretations of the parameters: A_n > 0 is considered unreasonable in Eq. 9, whereas σ = ∞ is considered unreasonable in Eq. 6. How does this difference arise? Two situations are possible in which σ = ∞ (σ = A₀/A_m): either A₀ = ∞ or A_m = 0. The former is obviously unnatural. The latter, however, is true if the sequence pattern in question (G₀) is essential for replication. In this case, there appears to be no limit on ν or q for the information to be maintained. This conclusion, however, no longer holds if we remove the unrealistic assumption (made through the definition of ϕ) that the total population size is always maintained at a positive value (e.g., consider the effect of spontaneous decay of replicators). Therefore, Eq. 7 is a necessary (but not sufficient) condition for the maintenance of information. It only describes the condition imposed by competition from mutants.

Eq. 9 has led to the odd conclusion that the survival of the fittest genotype depends on the fitness of the least fit mutant (i.e., A₀Q > A_n). This conclusion is derived from the assumption that the mutation rate per genome 1 − Q is identical for every genotype (except for the n-th genotype). This assumption, however, is unnatural. Let us suppose that the genomes of mutants are intact in some parts and broken in some other parts. Mutations in these broken parts should have no or reduced effects on the fitness. Thus, mutation rates should be effectively smaller for mutants than for the fittest genotype. This effect, however, is neglected in the present model because the internal structure of genotypes is disregarded. By contrast, it is incooporated in the model described by Eq. 6, where replication is (effectively) error-free for the mutant class (G̃_m). This is because mutations that convert genotypes within G̃_m cancel each other out. This canceling effect enables mutants to out-compete G₀ for high mutation rates, hence leading to the limitation described by Eq. 7. To sum up, the model of Eq. 9, though unrealistic, helps pinpoint why the limitation described by Eq. 7 arises: the mutant class has reduced effective mutation rates as compared with the fittest genotype.

The results described above can be restated with the quasi-species theory (Section 3.2). Let us suppose there are two classes of genotypes: ones carrying certain sequence patterns that increase the fitness and the others carrying no such patterns. We compare these two classes in terms of the average fitness of genotype neighborhoods. If the mutation rate is sufficiently high, the sizes of genotype neighborhoods are so large that the fitter genotypes make no difference to the average fitness of their respective genotype neighborhoods. In this case, evolution cannot “distinguish” between these two classes, hence the loss of information.

Using Eq. 7, we can obtain a rough idea about the length of sequence patterns that can be maintained by evolution in a system of self-replicating RNA molecules [39]. RNA-based polymerization has a high mutation rate, for example, 1 − q = 0.033 [22]. Assuming that some sequence pattern increases the replication rate by 10 fold as compared with random patterns, we obtain ν_max ≈ 70. This number seems too small to contain information about complex machinery of current living systems (e.g., the translation system). To increase ν_max, replicators must acquire additional machinery to increase the accuracy of replication (e.g., a protein polymerase with proof-reading mechanisms, which, however, requires the translation system). To implement such additional machinery, however, a greater amount of information than permitted by ν_max would be needed [39]—hence the catch-22 of prebiotic evolution (also known as Eigen’s paradox) [60, 65].

To sum up, there is a severe limit on the amount of information that can be maintained by evolution in simple RNA-like replicator systems. Moreover, selection pressure (σ) plays a minor role as compared with the mutation rate (1 − q) and the amount of information (ν) because σ appears as its logarithm in Eq. 7. This limitation poses an important question to the evolution of complexity in RNA-like replicator systems: How can an RNA-like replicator system increase the amount of information it contains by evolution? We will consider this question in Section 5.1.

4. Replicators with genotypes and phenotypes

4.1. RNA folding genotype-phenotype map

To illustrate the topic discussed in this section, let us return to Eq. 1. In Eq. 1 the population size is assumed to be infinitely large. Consequently, x_i > 0 for any genotype i at any time t > 0. That is, a population instantaneously “discovers” all possible genotypes, which is unrealistic. If this assumption is relaxed, there is no guarantee that a population can reach the fittest genotype (e.g., consider how a population can cross the valley in the fitness landscape described in Section 3.2). If the fitness landscape contains many local optima, evolution can stall at these optima, failing to discover the global optimum (see also Ref. [66]). Therefore, the structure of a genotype-phenotype-fitness map is crucial for the attainability of information by evolution.

What is the genotype-phenotype map of RNA-like replicators like? To consider this question, let us introduce the RNA-folding genotype-phenotype map. The secondary structures of RNA molecules can be computationally predicted from their sequences by free energy minimization (Fig. 4) [67]. This prediction can be viewed as a map from the genotype (sequence) of an RNA molecule to its phenotype (structure). The RNA-folding genotype-phenotype map has three advantages: the algorithm is efficient (O(ν³) in time and O(ν²) in storage); secondary structures capture the major component of folding energy in tertiary structures; secondary structures are important for biological functions.

Figure 4. An example of RNA secondary structures.

The RNA molecule depicted in this figure is the most abundant one at the end of the evolutionary simulation shown in Fig. 6 (time = 417650). The secondary structure was obtained with Vienna RNA Package [68]. The sequence is superimposed to the secondary structure.

Let us now consider the impact of the RNA folding genotype-phenotype map on the evolutionary dynamics of replicators. To this end, we consider the following model of an RNA replicator system [50, 69, 70]. The replication rates (i.e., fitness) of RNA molecules are defined as s^−d where s is a parameter (s > 1), and d is the distance between the secondary structures of RNA molecules and a predefined target structure (the distance is defined in Fig. 5). By defining such a target, evolution is here conceived as an optimization process.

Figure 5. Evolution in the RNA folding genotype-phenotype map.

The number of generations required for the system to discover a target structure is plotted against the length of RNA sequences (the error bars show SD). One generation is defined as the total number of replication events divided by the (target) total population size (1000). The graph also shows the average Hamming distance between initial sequences and the first molecules that achieved target structures. Each data point was obtained from 100 simulation runs. For each run, two RNA sequences were randomly generated (each base with an equal chance). One sequence was used to define the target structure; the other sequence, the initial population of RNA molecules. In each time step, every molecule replicated with a probability proportional to its fitness and was removed from the system with a probability proportional to ϕ defined in a way similar to the definition of ϕ in Eq. 1 (ϕ = (1/c₀) ∑_i f_i where f_i denotes the fitness of an RNA molecule i, c₀ the target population size (c₀ = 1000)). The fitness of molecules was defined as 1.5^−d where d is the structural distance between their secondary structures and the target structure. The structural distance was defined as the number of base pairs that must be opened and closed to transform one structure into another. The mutation rate was set such that (1 − q)ν = 0.5 where ν denotes sequence length, and 1 − q a mutation rate per base.

How well can a target be achieved in the above model? The dynamics of the model was simulated with a Monte Caro method (Fig. 5). Simulations showed that the system evolved target structures, on average, within a few thousand generations with the sequence length of 300 and the population size of 1000 (Fig. 5). With these figures, the number of genotypes “searched” by the system during evolution is bounded above by ~ 10⁶ (population size × number of generations). Thus, the fraction of the sequence space searched was as small as 10⁶/4³⁰⁰ ≈ 10⁻¹⁷⁴. Moreover, the required number of generations was proportional to the length of sequences (Fig. 5). This result is remarkable, given that the size of sequence space is an exponential function of the sequence length.

How can the system evolve target structures so well?. We here review only the essential points required to answer this question (see, e.g., Ref. [50, 69, 70], for more information).

(1) The RNA-folding genotype-phenotype map is a highly asymmetric many-to-one map. The number of all possible RNA sequences of a length ν is 4^ν. The corresponding number of all possible secondary structures is calculated as 1.4848ν^−3/21.8488^ν, which is an overestimate of the number of all structures that can actually be realized [71]. Therefore, as ν increases, structure/sequence ratio decreases rapidly. Exhaustive computation of secondary structures from all possible sequences consisting of Gs and Cs with ν = 30 gives the following numbers [72]:

number of all possible sequences	109
number of all realized structures	2 × 105
number of typical structures	2 × 104
sequences folding into typical structures	93%

(the typical structures are defined as follows: the number of sequences folding into any given typical structure is greater than the average number of sequences folding into one realized structure). The table shows that more than 90% of sequences fold into only 10% of realized structures. Therefore, the “coding” of structures by sequences is extremely redundant [50].

(2) The RNA-folding genotype-phenotype map has a feature called shape-space covering [71]. Namely, a small hypersphere located in an arbitrary position in a sequence space covers nearly an entire (typical) structure space. For example, the above exhaustive computation shows that sequences within any hypersphere of a radius 8 (Hamming distance) in the sequence space fold into, on average, 90% of the typical structures [73]. Moreover, the Hamming distance required for such high coverage of structures is almost proportional to ν (at least up to ν = 100) [73]. This result parallels the linear relationship seen in Fig. 5.

(3) The RNA-folding genotype-phenotype map has a feature called neutral networks [71]. Namely, sequences folding into an identical typical secondary structure form a network of genotypes connected by a unit mutation (i.e., substitution of one base or substitution of two bases that form a base pair in a structure; e.g., a G-C substitutes for an A-U). In addition, such neutral networks extensively percolate through a sequence space. For example, for more than 20% of randomly chosen sequences (ν = 100), the following holds: A sequence can be changed, with step-by-step unit mutations, into one differing from it in every position without ever changing its secondary structure [71]. Moreover, sequences that are “off” the neutral network by a unit mutation (i.e., one-step mutants) fold into a great variety of secondary structures [74]. Hence, a random walk in a sequence space results in “perpetual innovation along the neutral net(work)” [74].

These properties of the RNA folding genotype-phenotype map are extremely advantageous for evolution toward a target structure. For example, they indicate that RNA molecules having distinct secondary structures are interconvertible mostly with neutral mutations (i.e., two neutral networks “touch” each other in a sequence space). In fact, such interconvertibility has been experimentally demonstrated for actual ribozymes [75].

Now, let us closely examine the evolutionary dynamics of the above model. The typical dynamics consists of cycles of the following events (Fig. 6). A mutant appears whose secondary structure is closer to the target than those of a “wild-type” population (Fig. 6A, e.g., at the time indicated by the arrow). Such a mutant has increased fitness and so can out-compete the wild-type population. This succession is characterized by an abrupt drop in the average structural distance toward the target (Fig. 6A), a peak in the rate of sequence evolution (Fig. 6D, MSD/time), and a sudden decrease in the genetic heterogeneity of a population (Fig. 6B). Despite these abrupt changes, the succession leaves no clear mark on the long-term dynamics of the Hamming distance from the initial sequence (Fig. 6C).

Figure 6. Dynamics of RNA evolution toward a target structure.

A simulation was done with the model described in Fig. 5 (ν = 76; q = 0.999). The target structure was the same as shown in Fig. 4. A: The structural distance of a current population to the target structure (population mean and population minimum). B: The mean Hamming distance between all sequences in a population (i.e. genetic heterogeneity). C: The mean Hamming distance between a current population and the initial population (gray). The Hamming distance between the fittest sequence in a current population and the initial sequence (black). D: λ is the fraction of neutral substitutions in all possible single-base substitutions in a sequence; 〈λ〉 is its population mean. MSD/time is the mean square displacement of the consensus sequence per generation [76]. The “running ave.” is the running average of MSD/time.

After such succession, a population gradually spreads through a neutral network, whereby its genetic heterogeneity increases (Fig. 6B). During this process, the consensus sequence of a population constantly changes while the consensus secondary structure remains unchanged (i.e., neutral evolution) (Fig. 6AC).⁸ Moreover, robustness against mutations (i.e., the average neutrality 〈λ〉) increases as well (Fig. 6D) [51]. This increase indicates that a population moves toward an increasingly densely connected part of a neutral network (see Ref. [53], for a mathematical treatment).

After such neutral evolution, a new mutant appears whose secondary structure is closer to the target than those found in a (new) wild-type population (e.g., at time ≈ 0.65 × 10⁵). That is, a population discovers a “portal” to another neutral network that has an improved secondary structure [46]. Then, another round of the cycle begins. As the system undergoes these cycles, it discovers a secondary structure that is increasingly close to the target structure. During this process, 〈λ〉 decreases in a step-wise manner (Fig. 6D). That is, an increasing number of sequence positions become unable to mutate for sequences to fold into their structures. In other words, the system accumulates information.

To sum up, the evolutionary dynamics in this model is characterized by cycles of the two processes: nearly random “search” on a neutral network⁹ and adaptive transition to another neutral network [76].¹⁰ Through these cycles the system accumulates information.

Let us also comment on the evolution of mutational robustness described above. This evolution is neutral because no adaptive changes occur in phenotypes during this evolution [53]. Also, the increase of mutational robustness adds nothing to the fitness of individual genotypes. The term “neutral”, however, should not be interpreted as indicating that this evolution is due to genetic drift (i.e., random walk in a genotype space). In fact, the average population neutrality 〈λ〉 can be greater than expected when a population randomly (i.e., uniformly) spreads through a neutral network [53]. Therefore, this evolution is neither due to selective advantage of individual genotypes nor due to genetic drift—how could it be? This apparent paradox disappears if we apply the quasi-species theory (Section 3.2). Simply, the greater the neutrality of a genotype, the greater the fitness of the neighborhood around this genotype. Then, the evolution of mutational robustness is actually adaptive, for it increases the fitness of a genotype neighborhood (see Ref. [53], for the importance of population sizes and mutation rates). Therefore, the distinction between neutral and adaptive may not always be sensible or, at least, not trivial.

Finally, we consider how mutational robustness is actually achieved in an RNA molecule [52]. Fig. 4 depicts an RNA molecule that has evolved in one of the simulations. Its three stacks have three distinct sequence patterns, which help avoid the formation of wrong stacks. Namely, the upper stack largely consists of G-C pairs; the middle stack, A-U pairs; the bottom stack, alternating G-C and A-U pairs. Moreover, one of the internal loops consists largely of Cs, which also helps avoid the formation of wrong stacks. It is remarkable that evolution can generate such a “smart” genotype although there is no explicit selection pressure for it.

4.2. Phenotypic information threshold

In Section 3.3, we estimated the amount of information that can be maintained by evolution. To this end, we examined survival of the fittest genotype. However, if there is percolating neutrality in a genotype-phenotype map, the fittest genotypes are not unique. In this case, we must instead consider the fittest phenotype as follows [76].

Let us suppose there are two phenotypes, x_P and y_P. The replication rate of x_P is σ (> 1); that of y_P is normalized to 1. Genotypes are categorized by their phenotypes: x_G and y_G. We again ignore back mutations from y_G to x_G. By summing Eq. 1 separately for each phenotype [53, 80], we obtain the following equations [81]:

$$\stackrel{.}{x}=\sigma Qx+\sigma \mathrm{\Lambda }(1-Q)x-\varphi x$$

$$\stackrel{.}{y}=y+\sigma (1-\mathrm{\Lambda })(1-Q)x-\varphi y$$

where x denotes the concentration of x_P; y, that of y_P; Λ, the probability that a mutation occurring in a genotype in x_G is neutral (only base substitutions are considered); Q = q^ν and ϕ = σx+y as before (x+y is normalized to 1). For x_P to survive (through its own multiplication), its effective replication rate must be greater than that of y_P; thus, the condition is

$$\sigma [Q+\mathrm{\Lambda }(1-Q)]>1$$ (10)

To calculate Q + Λ(1 − Q), we make the simplest possible assumption: base substitutions are independent of each other (i.e., no epistasis) [81]. All possible single-base substitutions can be classified either as neutral or as deleterious. By the assumption, a mutant is neutral if and only if it has no deleterious substitution. The probability that replication introduces no deleterious substitution is Q + Λ(1 − Q) = [1 − (1 − q)(1 − λ)]^ν where λ is the fraction (probability) of neutral single-base substitutions. Because (1 − q)(1 − λ) corresponds to 1 − q in Eq. 7, we obtain the condition for the survival of x_P as follows:

$$\nu <\frac{\text{ln}\sigma }{(1-q)(1-\mathrm{\lambda })}$$ (11)

Three points are notable in Eq. 11. First, neutrality increases the maximum permissible sequence length of replicators (ν_max) by a factor of 1/(1 − λ). Likewise, it can also increase the maximum permissible mutation rate 1 − q_min. However, λ in the RNA folding genotype-phenotype map turns out to be a decreasing function of ν [81]. In addition, its values are not very high (between 0.2 and 0.5). Therefore, these increases are limited. Second, Eq. 11 closely approximates the survival condition of the fittest phenotype in a more complex model incorporating the RNA folding genotype-phenotype map [81]. This result is surprising because RNA folding involves many non-local interactions between bases [51, 82]. However, it can actually make intuitive sense as follows. The number of substitutions per sequence per replication ν(1 − q) cannot be very large despite neutrality (see above).¹¹ In such a case, epistasis cannot cause a large effect. Finally, although neutrality increases ν_max, it does not necessarily increase the amount of information that can be maintained by evolution. This is because the greater the neutrality is, the more randomness is permitted in sequence patterns coding a certain phenotype.

5. Replicators with interactions

5.1. Hypercycles

Evolution operates on genotype neighborhoods, that is, genotypes that are genetically close to each other (Section 3.2). It, therefore, does not permit the coexistence of genetically distant replicators, as far as replicator systems similar to Eq. 1 are concerned (also known as competitive exclusion). Moreover, there is a severe limit on the amount of information that can be maintained in such systems (Section 3.3). Taken together, this problem of information maintenance presents a severe obstacle to the evolution of complexity.

However, Eq. 1 makes a simplistic assumption that the fitness of replicators is completely determined by their respective genotypes. Instead, we here assume that the fitness is determined by interactions between replicators. This assumption makes possible the coexistence of genetically distant replicators (i.e., replicator “ecosystem”). Such coexistence is a potential solution to the problem described above. That is, we consider population-based maintenance of information (as opposed to individual-based maintenance of information) [83].

We consider the simplest kind of interaction, namely, the replication interaction. Replicators are assumed to catalyze replication of other replicators: R_i + R_j → R_i + 2R_j where R_i serves as a catalyst, and R_j as a template. These interactions form a network of replicators. We first consider the short-term dynamics of such a network by ignoring mutations in replicators. We can formulate the following equations describing the population dynamics of replicators (cf. Eq. 1) [40]:

$${\stackrel{.}{x}}_i={\displaystyle \sum _j{k}_{ij}{x}_j{x}_i-\varphi x_i}$$ (12)

where x_i denotes the concentration of the i-th replicator type (or “species”) R_i; k_ij, the catalytic activity of R_j replicating R_i; and ϕ keeps the total concentration constant (ϕ = ∑_i,j k_ijx_ix_j/∑_i x_i).¹² Eq. 12 is known as replicator equation [43]. The multiplication is hyperbolic in Eq. 12, whereas it is exponential in Eq. 1 (see Ref. [84–86], for sub-exponential multiplication).

We consider the simplest network consisting of two replicator species:

$${\stackrel{.}{x}}_1=(k_{11}{x}_1+k_{12}{x}_2)x_1-\varphi x_1$$

$${\stackrel{.}{x}}_2=(k_{21}{x}_1+k_{22}{x}_2)x_2-\varphi x_2$$ (13)

Under what condition can the two species coexist? Since x₁ + x₂ is constant (denoted by c₀), we can substitute x₂ = c₀ − x₁ in Eq. 13 [87, 88] and obtain

$${\stackrel{.}{x}}_1=c_0^{-1}{x}_1(c_0-x_1)[\alpha +\beta )x_1-c_0\beta ]$$ (14)

where α = k₁₁ − k₂₁ and β = k₂₂ − k₁₂, each representing the specificity of catalysis. For the coexistence, Eq. 14 must have a steady state x₁ = x̄₁ that satisfies 0 < x̄₁ < c₀ and is stable. These two conditions respectively correspond to

$$0<\frac{\beta }{\alpha +\beta }<1\text{ and }\alpha +\beta <0$$ (15)

From Eq. 15, we obtain k₂₁ > k₁₁ and k₁₂ > k₂₂. This result makes intuitive sense. If x₁ is in a majority, k₂₁ > k₁₁ causes x₂ to increase faster than x₁, hence a negative feedback to x₁. Likewise, k₁₂ > k₂₂ provides a negative feedback to x₂. Therefore, the two species must be mutualistically coupled to coexist stably [40].

The simplest network with mutual coupling is obtained by setting k₁₁ = k₂₂ = 0 and k₁₂, k₂₁ > 0. Likewise, we can conceive a network with n species: k_ij > 0 if i = j + 1 (1 ≤ j ≤ n − 1) or if i = 1 and j = n; k_ij = 0 if i and j take any other values. This type of a replicator network is called a hypercycle [40]. Numerical solutions of Eq. 12 (Fig. 7) indicate that hypercycles permit the stable coexistence of all member species (see Ref. [43] for a mathematical proof). Thus, a hypercycle can maintain the amount of information that is greater than permitted in one (quasi-)species. Therefore, it has been suggested that hypercycles are an essential intermediate stage of prebiotic evolution [40].

Figure 7. Dynamics of well-mixed hypercycle systems.

Numerical solutions of Eq. 12 are shown for hypercycles with various numbers of members. The coordinate is the concentration x_i of a hypercycle member (∑_i x_i is normalized to 1), and the abscissa is time. A: 3-member hypercycle. The dynamics has a stable steady state. B: 4-member hypercycle. The dynamics has an asymptotically stable steady state (the real part of the dominant eigenvalue is 0). The inset shows dynamics for longer duration. C: 5-member hypercycle. The dynamics displays oscillation. D: 9-member hypercycle. The amplitude of oscillations is greater than that in C. The parameters were as follows: k_ij = 1 if i = j + 1 (1 ≤ j ≤ n − 1) or if i = 1 and j = n; otherwise, k_ij = 0. x_i(0) = 0.1 for i ≠ n, and x_n(0) = 1 − ∑_i≠n x_i(0).

However, hypercycles involve a positive feedback loop (with delay), which is a source of instability. Their dynamics thus displays oscillation, whose amplitude enlarges as n increases. (Fig. 7CD). Therefore, if n is large, the total population size must be extremely large to prevent the system’s extinction, which can be caused by the stochastic loss of any member [89] (this issue is ignored in Eq. 12, which assumes an infinitely large population size).

Let us next consider competition between two 1-member hypercycles. In Eq. 14, we set k₁₂ = k₂₁ = 0 and k₁₁, k₂₂ > 0. In this case, the steady state that satisfies 0 < x̄₁ < c₀ is unstable (Eq. 15). In addition, the two steady states at x₁ = 0 and x₁ = c₀ are stable. Thus, the winner of the competition depends on the initial abundance of the two hypercycles. This result is expected because the two hypercycles, each behaving as a positive feedback loop, are negatively coupled through competition. This result indicates that once one hypercycle establishes itself, it excludes the establishment of the other hypercycles—that is, it prevents further evolution (but see Section 5.2). This situation is called once-for-ever selection [40] (also known as interference competition [90]).

The idea of hypercycles raises two questions [91]: How can a hypercycle originate?; How will it evolve? Let us first consider the latter. We can conceive two types of mutations that “improve” a hypercycle: those improving a mutant as a template; those improving a mutant as a catalyst. The former increases the mutant’s fitness; the latter does not. Thus, selection favors improved templates, but does not favor improved catalysts [91] (this issue is actually even worse as described in Section 5.3.2). Let us consider a mutation that improves a mutant as a template, but completely destroys it as a catalyst. Such a mutant (or a “parasite”) will out-compete the species (say R₁) from which it has originated. Then, R₂ is no more replicated and so goes extinct next. Eventually, the whole system goes extinct (Eq. 12 ignores this issue because it assumes a constant total population size). Therefore, hypercycles are evolutionary unstable (see also Section 5.2). Given this instability, it is difficult to conceive how hypercycles can originate through evolution.

5.2. Hypercycle and spatial self-organization

Eq. 1 and 12 make two implicit assumptions: all replicators instantaneously interact with each other; populations are continuous. These assumptions are made, not because they are natural, but because they allow us to use ODEs as a modeling framework. These assumptions, though merely ad hoc simplifications, significantly affect the dynamics of replicator systems, as will be seen soon. In this section, we consider an alternative modeling framework, namely, stochastic cellular automata (CA). CA models also make two implicit assumptions, which contrast with the two mentioned above: replicators interact only locally (i.e., only with those that are spatially close to them); populations are discrete [30, 92]. Using CA as a model, we reconsider the dynamics of hypercycles [93, 94].

A CA model of a hypercycle can be formulated as follows. Briefly, the model consists of a finite, 2-dimensional square lattice. A single lattice point either contains one individual replicator or is empty. Empty points are assumed to represent “resources” required for replication (e.g., substances and space). This assumption locally limits the population size of replicators. The dynamics of the model is driven by randomly selecting a lattice point and applying an algorithm that simulates reaction and diffusion in the vicinity of this point.¹³ We assume two kinds of reactions: replication ${\mathrm{X}}_i+{\mathrm{X}}_{i+1}+\mathrm{\varnothing }\stackrel{k_{i+1i}}{\to }{\mathrm{X}}_i+2{\mathrm{X}}_{i+1}$, and decay ${\mathrm{X}}_i\stackrel{d}{\to }\mathrm{\varnothing }$ where ∅ represents an empty lattice point (resources). Diffusion is treated as a second-order reaction. When it happens, it swaps the contents (including ∅) of two adjacent lattice points. Each reaction happens with a probability proportional to its rate constant determined by parameters. (See Ref. [59], for details; see Ref. [93], for the original work.)

The dynamics of the above model is distinct from that of the corresponding ODE model (Fig. 8). If the number of hypercycle members n exceeds 4, the spatiotemporal distributions of replicators self-organize into rotating spiral waves [93]. The spiral-wave formation requires other conditions; for example, all members must have similar parameters [94] (see below). The spiral-wave formation solves one problem of hypercycles: The system as a whole does not oscillate anymore (oscillations are spatially localized).

Figure 8. Spiral wave formation in hypercycles.

The upper panels show the dynamics of hypercycles obtained with Eq. 12. Colors represent different members of a hypercycle. The parameters were the same as in Fig. 7. For n < 5 (the number of members), the initial condition was also the same as in Fig. 7. For n ≥ 5, x_i(0) = 0.1 for i > 2, x₁(0) = 0.11 and x₂(0) = 0.11. The lower panels show snapshots of simulations with the CA model. Colors correspond to those in the upper panels. Reactions (including diffusion) were prohibited to occur across the lattice boundaries. The rate constants were the same as in Fig. 7. The diffusion rate was set to 1 (see Ref. [59]). The lattice size was 512 × 512 points (hereafter this is true unless otherwise stated).

The spiral-wave formation causes the following asymmetry: Only replicators forming the cores (centers) of spiral waves can leave descendants; the others forming the peripheries eventually go extinct. This asymmetry arises because replicators composing spiral waves propagate outward toward the boundaries of spiral waves as the dynamics proceeds (Fig. 9). This biased movement arises because interactions between individuals are spatially structured (of course, not because the movement of individuals itself is directional).

Figure 9. Core-periphery differentiation.

The spatiotemporal dynamics of rotating spiral waves is depicted by consecutive snapshots of a simulation (n = 9). At t = 0, the replicators in the core of one spiral wave were colored yellow; those around the core, blue. In this blue-yellow region, one member of the hypercycle was colored red. Replicators in the other regions were colored in gray-scale. The darkest replicators and the red replicators belong to the same hypercycle member. During the simulation, every replicator “inherited” the color of the replicator from which it is replicated. The parameters were the same as in Fig. 8.

This asymmetry between cores and peripheries makes hypercycles resistant against parasites [93]. Let us suppose that the system depicted in Fig. 9 is inoculated with parasites. If parasites are placed in the periphery of a spiral wave, they will be driven toward the boundary of the spiral wave. Being “pushed” into the boundary, parasites eventually go extinct, provided, for example, that the value of n is sufficiently large.¹⁴ If the parasites are placed in a core of a spiral wave, they drive this spiral wave to extinction. They, however, will still be in the peripheries of the other spiral waves. This resistance against parasites requires an important condition: populations must be discrete [89]. Discreteness allows the population size of parasites to become strictly zero. If, by contrast, populations are continuous (e.g., as in partial differential equations), the concentration of parasites never becomes zero (unless initially zero), not only globally, but also locally. In this case, hypercycles inevitably go extinct because every spiral-wave core contains a small amount of parasites.

A more striking consequence of the spiral-wave formation is that selection operates between spiral waves [94]. If there are multiple spiral waves each consisting of a distinct hypercycle, one that rotates fastest out-competes all the others, taking over the entire system [94, 97]. This rotation can be accelerated by increasing the reaction rates such as the replication rate (k_{i+1 i}) and the decay rate (d) [94]. Therefore, between-spiral-wave selection favors improved catalysts (i.e., increased k_{i +1 i}) and even replicators that decay quickly. This selection tendency is in stark contrast to that in the ODE model (Section 5.1). However, between-spiral-wave selection also poses a problem, in that the rotation decelerates as n increases.¹⁵ Thus, the selection tends toward decreasing the amount of information contained in hypercycles [94].

In addition, the spiral wave formation makes competition between hypercycles independent of their initial abundance (cf. Section 5.1). Because of this independence, a nascent hypercycle does not suffer from its small population size (provided it develops a spiral wave, e.g., by stochasticity). Thus, the spiral-wave formation circumvents the problem of once-for-ever selection, which is inherent in hyperbolic replicator systems (see Ref. [98], for an alternative treatment of this problem).

So far, we have considered the selection dynamics of hypercycles. Is the evolutionary dynamics of hypercycles stable as well? Unfortunately, the answer turns out to be “no” for two reasons [99]. First, there is a dilemma. If a hypercycle can form stable spiral waves, once spiral waves are established, new waves cannot easily be generated. By contrast, if a hypercycle cannot form stable spiral waves, spiral waves are continuously generated and destroyed. However, the lack of stable spiral waves makes the hypercycle vulnerable to parasites [89]. Second, the core-periphery asymmetry is a double-edged sword. Mutants appearing in the core of a spiral wave are likely to spread through the entire wave. Such frequent fixation of mutants can increase heterogeneity in the characters (such as k_{i+1 i} and d) of hypercycle members. This heterogeneity, however, is detrimental to the stability of spiral waves as described above (see Ref. [89, 100], on spatial heterogeneity). Moreover, the mutants that appear in the cores can also be parasites. In this case, they destroy spiral waves in whose core they appear. These two problems together render hypercycles evolutionarily unstable [99].

The last conclusion, however, should not blind us to the important insight obtained from hypercycles: Spatial self-organization can strongly impact the direction of selection.

5.3. Spatial self-organization and multilevel evolution

In this section, we describe how the insights obtained from hypercycles can be generalized and extended by considering a simpler replicator network.

5.3.1. Traveling waves

We consider a minimal replicator network consisting of one replicase and one parasite species (RP system, for short). We formulate an ODE model describing the RP system as follows:

$$\stackrel{.}{R}=k_R{R}^2\theta -dR$$

$$\stackrel{.}{P}=k_{P}RP\theta -dP$$ (16)

where R denotes the concentration of replicases; P, that of parasites; θ, the amount of resources required for replication; k_R and k_P, replication rate constants; d, a decay rate. For simplicity, we set θ = 1 − R − P (θ, defined in this way, corresponds to empty lattice points in the CA model described in Section 5.2).

If k_R < k_P (i.e., parasites serve as better templates than replicases do), parasites out-compete replicases and so drive the system to extinction. Does this conclusion also hold in a spatially extended system with discrete populations? To address this question, we consider a CA model describing the RP system. Such a model can be formulated in essentially the same way as explained in the previous section [96]. The CA model shows that the spatiotemporal distributions of replicators self-organize into traveling wave patterns (Fig. 10). The formation of traveling waves depends on two conditions: the diffusion rate is sufficiently low; and parasites are sufficiently strong (i.e., k_P is sufficiently greater than k_R). The traveling waves display the following dynamics. The “front” side of a traveling wave consists of replicases (Fig. 10, white pixels), and the “back” side of it consists of parasites (Fig. 10, black pixels). In the wave front, replicases propagate into an empty area. In the wave back, parasites out-compete replicases (because k_R < k_P ) and cause local extinction, leaving an empty area behind (this process can be called parasite propagation). Through such continuous propagation and extinction, waves travel and rotate. Traveling waves often collide with each other, in which case they get annihilated. This collision limits the lifetime of traveling waves (collision can also happen with the system’s boundary depending on the boundary condition). Importantly, traveling waves can generate new traveling waves. The generation of a new wave typically begins with a process by which replicases of an existing wave are isolated (i.e., escape”) from parasites of the same wave (Fig. 11). The isolated replicases can start a new local population. While this population expands, it is “infected” by nearby parasites and so becomes a new traveling wave [96]. Because this wave generation continually happens, the system as a whole can persist despite local extinction of replicases and frequent annihilation of traveling waves [99]. In more general terms, the dynamics of mesoscopic entities (waves) enables the system’s macroscopic stability despite its microscopic (i.e., local) instability.

Figure 10. Traveling-wave formation.

Snapshots depict the result of a CA model simulating the dynamics the RP system. The color coding is as follows: replicases are white; parasites, black; empty points, gray. The k_P /k_R ratio was chosen to be the maximum possible value for which the system can survive for a fixed value of k_R (viz., k_R = 0.39). The value of d is indicated in the figure. The diffusion rate was set to 0.01. The model also assumed rare mutations that transform a replicase into a parasite (10⁻⁴ per replication event). The CA was toroidal (i.e., no boundary). The lattice size was 512 × 512 points.

Figure 11. Generation of a new traveling wave.

Consecutive snapshots depict the spatiotemporal dynamics of the CA model describing the RP system. The colors and the parameters are the same as in Fig. 10. Time goes from left to right. Only a part of the CA field is shown (75 × 75 lattice points). The second panel from left shows that replicases are left behind a propagating layer of parasites. These isolated replicases then increase their population size; concomitantly, they are infected by nearby parasites.

At this point, a difference between the RP system and hypercycles is worth noting. For a hypercycle to be resistant against parasites, it must first self-organize into spiral waves before being infected by parasites. Moreover, spiral waves cannot generate new spiral waves (if they are stable). In the RP system, by contrast, replicases do not form any spatial patterns by themselves. Rather, their interactions with parasites generate traveling-wave patterns. Moreover, traveling waves can continually generate new traveling waves, and this continual generation makes the RP system macroscopically stable.

The stability of the RP system critically depends on the dynamics of spatial patterns. The system can survive the infection of strong parasites (the strength measured by k_P /k_R) if it contains many traveling waves of moderate sizes (Fig. 10). Whether this condition holds depends on various parameters in an intricate manner [96]. The important point, however, is simple: the very existence of this intricacy. The system’s macroscopic stability depends not only on the strength of parasites (k_P /k_R), but also on various other factors that influence the size and number of traveling waves. In the ODE model, by contrast, the system’s stability depends solely on a single parameter, k_P /k_R.

We next illustrate the importance of discrete populations. To this end, we compare the above CA model to a corresponding partial differential equation (PDE) model. The latter can be constructed by extending Eq. 16 as follows:

$${\partial }_{t}R=R^2\theta -dR+{\nabla }^{2}R$$

$${\partial }_{t}P=\widehat{k}RP\theta -dP+{\nabla }^{2}P$$ (17)

where R, P, and θ are the functions of t, x, y, where x and y are spatial coordinates; k̂ = k_P /k_R. As before, we assume that k̂ > 1. To determine whether replicases and parasites can coexist, we introduce the following variables [101]:

$$u=\frac{R}{R+P}$$

$$\upsilon =R+P$$

Using these variables, we can transform Eq. 17 into

$$\stackrel{.}{u}=(1-\widehat{k})u^2(1-u)\upsilon (1-\upsilon )+{\nabla }^{2}u+\frac{2}{\upsilon }\nabla u\cdot {\nabla }_{\upsilon }$$

$$\stackrel{.}{\upsilon }=u[u+\widehat{k}(1-u){\upsilon }^2(1-\upsilon )-d\upsilon +{\nabla }^2\upsilon$$ (18)

At the coordinates (x, y) where u is maximum, ∇u = 0⃗ and ∇²u < 0 hold. That is, diffusion always decreases the maximum value of u. This result makes intuitive sense since what diffusion does is to average things out. Thus, where u is maximum, u̇ < 0 must hold. Therefore, u → 0 must hold everywhere. When u becomes sufficiently small, υ̇ < 0 must also hold. Consequently, the whole system goes extinct. To sum up, discrete populations are necessary for the stability of the RP system (whether it is sufficient is described in Section 7.1; see Ref. [59] (Text S1) for what happens through a transition from discrete to continuous populations).

5.3.2. Complex formation

Eq. 16 describes replication as a second-order reaction, which amounts to the assumption that replication is an instantaneous process (the same holds for Eq. 12). The following consideration shows that relaxing this assumption gives a selective advantage to parasites [96]. A single molecule cannot serve both as a replicase and as a template at exactly the same moment. Thus, if replication is not instantaneous, catalyzing replication inhibits the own replication of replicases. Parasites, in contrast, do not waste any time catalyzing replication and so have a selective advantage over replicases. In Section 5.1, we stated that selection favors improved templates, but does not favor improved replicases. It is even worse—selection actually disfavors molecules serving as replicases.

Let us consider this advantage of parasites with a mathematical model. To incorporate non-instantaneous replication into Eq. 16, we consider complex formation between replicases and templates as follows [96, 102]:

$$2\mathrm{R}\underset{a(1-k_R)}{\overset{a{k}_R}{\rightleftharpoons }}{\mathrm{C}}_{\mathrm{R}}\stackrel{\kappa \theta }{\to }3\mathrm{R}$$

$$\mathrm{R}+\mathrm{P}\underset{a(1-k_P)}{\overset{a{k}_P}{\rightleftharpoons }}{\mathrm{C}}_{\mathrm{P}}\stackrel{\kappa \theta }{\to }\mathrm{R}+2\mathrm{P}$$ (19)

where R denotes a replicase; P, a parasite; C_R, a complex between 2 Rs; C_P, a complex between R and P; a, the overall speed of the association-dissociation reaction; k_R and k_P , the affinity of molecules toward replicases (0 ≤ k_R ≤ 1 and 0 ≤ k_P ≤ 1); κ, a replication rate constant; d, a decay rate; θ represents resources required for replication (θ = 1 − R − P − 2(C_R + C_P ) as before).

What is the effect of complex formation on the competition between replicases and parasites? In a well-mixed system, the outcome of the competition is determined by the relative concentrations of C_R and C_P because the multiplication rates of R and P are proportional to these quantities. For simplicity, we assume a ≫ κ (see Ref. [96], for the case where κ ≫ a). We consider the following two extreme situations:

(i) k_R ≈ 0 and k_P ≈ 0. In this case, the concentrations of C_R and C_P are negligible in comparison with those of R and P. By using the equilibrium assumption, the concentrations of complexes can be approximated as

$$C_R=k_R{R}_t^2$$

$$C_P=k_P{R}_t{P}_t$$

where R_t denotes the total concentration of R; P_t, that of P (R_t = R+2C_R+C_P and P_t = P + C_P ). Thus, the system reduces to that described by Eq. 16 as expected.

(ii) k_R ≈ 1 and k_P ≈ 1. In this case, the concentrations of complexes are non-negligible. To approximate C_R and C_P, we assume κ = 0 and C_R = C_P = 0 at t = 0 and then determine the equilibrium of the following reactions: 2R ⇌ C_R and R + P ⇌ C_P. We then obtain the following equations [103, Section 4.8]:

$$C_R=\frac{1}{2}\frac{R_t^2}{R_t+P_t}$$

$$C_P=\frac{R_t{P}_t}{R_t+P_t}$$

These equations show that parasites have two fold advantage in comparison with replicases.

If k_R and k_P take values between 0 and 1, the effect of complex formation, too, lies between the above two cases. That is, parasites have a (less than two fold) selective advantage over replicases. This advantage enables parasites to out-compete replicases even if they serve as worse templates than replicases do (i.e., k_R > k_P ) [96]. This result implies that mutations can easily convert replicases into parasites; mutations only have to destroy the catalytic ability of replicases. Therefore, the evolution of parasites is very likely [104].

5.3.3. Evolution at the level of traveling waves

So far, we have considered the dynamics of the RP system with the parameters kept constant. If the parameters are allowed to evolve, do they remain in the range for which the system survives? To answer this question, let us make a slight modification to Eq. 19. We assume that a parasite P switches between two conformations: the folded state (denoted by P_f ) and the template state (P_t). P_f cannot serve as a template and so cannot be replicated, whereas P_t can be replicated. For simplicity, we assume that the reaction P_t ⇌ P_f is so fast that it is always in equilibrium. Let K be the equilibrium constant of this reaction. Then, the concentration of P in the folded state and that in the template state can be expressed as P_f = l P and P_t = (1 − l)P, respectively, where l = K/(1 + K). Hence, we now have

$$\mathrm{R}+\mathrm{P}\underset{1-k_P}{\overset{(1-l)k_P}{\rightleftharpoons }}{\mathrm{C}}_{\mathrm{P}}\stackrel{\kappa \theta }{\to }\mathrm{R}+2\mathrm{P}$$ (20)

where a has been set to 1 for simplicity (cf. Eq. 19).

We consider the evolution of the parameters k_P and l (all the other parameters are fixed). In a well-mixed system, parasites are expected to evolve toward maximizing their own multiplication. That is, they evolve toward increasing k_P and decreasing l (i.e., increasing the affinity toward R and decreasing the ratio P_f/P, respectively). Because of this evolution, the system will eventually go extinct even if these parameters are initially set such that the system can survive.

Does this conclusion also hold in a spatially extended system with discrete populations? To address this question, we consider a CA model describing the above RP system. Such a model can be formulated in essentially the same way as explained in Section 5.2 [59]. In the CA model, each individual parasite is assigned the parameters k_P and l. Each parasite copies the values of these parameters from the parental individual from which it is replicated. When the values are copied, a mutation can slightly modify them with a certain probability (mutation rate) (see Fig. 12, for details). The model shows that the values of k_P and l evolve in a direction that is unexpected in the ODE model (Fig. 12, solid line). The evolutionary trajectory of k_P and l can be separated into two phases: the short-term evolution and the long-term evolution. During the short-term evolution, the trajectory goes toward a contour given by a certain functional relationship f(k_P, l) = 0. This contour indicates a constraint in the values of k_P and l parasites can take. During the long-term evolution, the trajectory goes along the contour in the direction in which l and k_P increase their values. This trend might appear puzzling because parasites do not gain any obvious selective advantage from being in the folded state.

Figure 12. Evolutionary trajectories of k_P and l.

The lines represent the trajectories of k_P and l (population averages) during evolution. The arrows indicate the values of k_P and l at the beginning of simulations. The black solid line is for the model of the spatially extended RP system. The numbers beside it indicate the time points at which the snapshots of the simulation shown in Fig. 14 were taken. The dotted line and the gray line are for the model of the compartmentalized RP system and for the modification thereof, respectively (these models are described in Section 7.2.2). The parameters were as follows: k_R = 0.6; κ = 1; d = 0.02. The value of k_P was slightly changed (i.e., mutated) with a probability 0.01 per replication. The size of this change was uniformly distributed in the range [−0.05, 0.05]. Likewise, l was mutated as well. The two types of mutations were mutually exclusive during one replication event. In the spatially extended RP system, D = 0.1 (diffusion). In the compartmentalized RP system, D = 1 and υ_T = 1000 (division volume). The boundaries of the system had no flux. The lattice size was 512 × 512 points.

The model does not explicitly assume any constraint in the values of k_P and l parasites can take. Thus, the observed constraint can be referred to as “emergent” trade-off.¹⁶ Increasing k_P and decreasing l accelerates the multiplication of parasites (see above). Given this fact, the slope of the contour f(k_P, l) = 0 indicates that the multiplication rates (or “fitness”) of parasites are maintained at a more or less constant level during the evolution. Thus, we obtain the first conclusion: In contrast to the well-mixed system, the spatially extended RP system is stable even if parasites can evolve their parameters (provided that mutation rates are not too high; see Ref. [59], for details).

How does the spatially extended RP system achieve the evolutionary stability? For simplicity, we consider a model in which only l is allowed to evolve. This model displays the following spatiotemporal dynamics. If each traveling wave is observed over time, parasites, which compose the back side of a traveling wave, evolve toward decreasing the value of l (i.e., the ratio P_f/P) (Fig. 13; the colors of the traveling waves change from cyan to yellow over time). That is, as far as individual waves are concerned, parasites evolve toward maximizing their multiplication, the same trend as expected in the well-mixed system. Nevertheless, at the level of the whole system, this maximization does not ensue; that is, the average value of l is maintained at a constant level. How is this possible?

Figure 13. Life history of traveling waves.

The figure shows consecutive snapshots of a simulation with the CA model with only l allowed to evolve (k_P was fixed at 1). The numbers in the snapshots identify individual traveling waves. The time was reset to zero when the first snapshot was taken. The parameters were the same as in Fig. 12, except that the mutation rate of l was increased to 0.19 per replication event and that the lattice size was 1536 × 1536 points.

As described before, the traveling waves can multiply by generating new traveling waves. Because diffusion is finite, the parasites of a newly generated wave are mostly descended from the parasites of the wave that has generated the new wave. That is, newly generated waves inherit their parasites from the parental waves. Moreover, variations exist in the character (l) of parasites both between and within traveling waves (Fig. 13). These variations are caused by finite diffusion and stochasticity, both of which reduce the effect of local competition between parasites. Traveling waves, therefore, display multiplication, inheritance, and variation, the three features required for evolution.

The generation of new waves is triggered by the stochastic isolation (i.e., “escape”) of replicases from parasites (Section 5.3.1). The probability of this isolation increases as the multiplication rate of parasites decreases (Fig. 13; newly generated waves, which are identifiable by their size, contain parasites having relatively great values of l). In this way, the character of parasites can modulate the frequencies at which waves generate new waves (i.e., the “fecundity” of waves). Each wave decreases its fecundity over time because its constituent parasites evolve toward maximizing their multiplication (i.e., waves undergo an aging-like process, so to speak). However, waves with higher fecundity can out-compete those with lower fecundity because collision between waves results in the annihilation of waves (we saw a similar dynamics for hypercycles as well; see Section 5.2). Therefore, evolution happening at the level of waves tends toward decreasing the multiplication rate of parasites. It thus counteracts the evolution of parasites happening within each wave (i.e., the maximization of the multiplication). Consequently, the multiplication rates of parasites are kept more or less constant at the level of the whole system. With the multiplication rate kept constant, the possible combinations of values in k_P and l are limited, hence the emergence of the trade-off between k_P and l.

The next question is why the values of k_P and l increase through the long-term evolution. It turns out that increases in these parameters coincide with an increase in the number of traveling waves in the system (Fig. 14). This correlation implies that increasing these parameters along the trade-off curve increases the fecundity of traveling waves. Let us consider whether this implication is true. The generation of a new wave is initiated by the replicases that are isolated from parasites. To generate a new wave successfully, these replicases must quickly increase their population to avoid being eradicated through re-infection by parasites. To what extent replicases can increase their population after being isolated from parasites? This question can be addressed with the ODE model describing the RP system (to this end, Eq. 16 are extended to incorporate Eq. 20 [59]). To simulate the situation where replicases have just been isolated from parasites, we set the model’s initial condition to R ≪ 1, P ≪ 1, and C_R = C_P = 0. We then assess how fast replicases can multiply in the face of competition from parasites. Since the system is largely empty (θ ≈ 1), the limiting process of multiplication is complex formation (i.e., $2\mathrm{R}\stackrel{k_{\mathrm{R}}}{\to }{\mathrm{C}}_{\mathrm{R}}\text{ and }\mathrm{R}+\mathrm{P}\stackrel{(1-l)k_P}{\to }{\mathrm{C}}_{\mathrm{P}}$) rather than replication (i.e., ${\mathrm{C}}_{\mathrm{R}}\stackrel{\kappa \theta }{\to }3\mathrm{R}\text{ and }{\mathrm{C}}_{\mathrm{P}}\stackrel{\kappa \theta }{\to }2\mathrm{P}+\mathrm{R}$). Thus, decreasing the complex formation rate of parasites (1 − l)k_P increases the multiplication rate of replicases by reducing competition. Indeed, the value of (1 − l)k_P decreases as the values of k_P and l increase along the trade-off curve. Thus, the implication described above is confirmed: The increase of k_P and l through the long-term evolution increases the fecundity of traveling waves. Therefore, this long-term evolutionary trend is caused by the evolution happening at the level between traveling waves [59].

Figure 14. Evolution of traveling waves.

Snapshots of the simulation taken at the three points along the evolutionary trajectory shown in Fig. 12 (black solid line).

5.3.4. Conclusion

In the RP system, traveling waves can generate new traveling waves. This ability of “multiplication” enables traveling waves to undergo evolution that further improves this ability (cf. hypercycles). The evolution of waves is manifested in the evolution of constituent parasites, through which, however, parasites themselves gain no immediate selective advantage (the instantaneous multiplication rates of parasites are maintained at a more or less constant level). Actually, this kind of phenomenon is familiar to all biologists: Selection acts on phenotypes; phenotypes are generated through the dynamics of microscopic entities that constitute organisms (such as DNA molecules); heritability resides in these microscopic entities. In short, evolution is a multilevel process. To sum up: the RP system, one of the simplest replicator systems, can display such multilevel evolution if spatial self-organization and discrete populations are taken into consideration.

6. Replicators with genotypes, phenotypes, and interactions

In Section 4, we considered how information can be acquired through evolution. There, we analyzed a model that incorporates a complex genotype-phenotype map of replicators. In that model, what phenotype provides the functionality of self-replication is predetermined, that is, what is informational is predefined. Such a model has an obvious limitation that one cannot investigate how novel (i.e., non-predefined) information can be generated through evolution. In Section 5, on the other hand, we considered whether information can be maintained by the coexistence of distinct replicator species (i.e., replicator ecosystems). There, we analyzed a model in which the structures of replicator networks are predefined, that is, replicator ecosystems are presupposed. Again, such a model has a limitation that one cannot investigate the origin and evolution of replicator ecosystems. These limitations can be transcended by combining the two models to complement each other. Namely, we consider a model in which

• the sequences of replicators determine the structures of replicators,

• the structures determine the interactions between replicators,

• the interactions produce a dynamic feedback onto the spatiotemporal distributions of the sequences of replicators,

that is, a model that incorporates a genotype-phenotype-interaction map of replicators [104]. Such a model allows us to consider the generation of novel information and the evolution of a complex replicator ecosystem simultaneously. Considering both of these issues at the same time is, as will be explained in this section, not mere complication, but a key to understanding the evolution of complexity.

A model seems to require two features to simulate the evolution of a complex replicator ecosystem. First, it should incorporate both a large degree of freedom and a complex biological structure in the genotypes and phenotypes of replicators. This feature can be implemented with the RNA-folding genotype-phenotype map (Section 4). Second, a model should also incorporate biological constraints that not every RNA molecule can catalyze replication and that not every RNA molecule can be recognized as a template by an RNA replicase. Clearly, a model cannot be chemically precise in this regard. This is not only because of the lack of knowledge about real RNA replicators (such replicators currently do not exist), but also because of the problem of “over-materialization” (as opposed to oversimplification). The over-materialization makes a model intractable, and it also renders the realism intended in assumptions spurious because the assumptions always contain insufficiency and inaccuracy. Rather, we shall consider a model that is artificial for the sake of simplicity, yet biologically relevant.

Let us consider the following model, which embodies the two features described above [104]. Each individual replicator has its sequence and secondary structure (determined by the RNA folding). As before, replication occurs in two steps: complex formation and replication reaction (Section 5.3.2). Complex formation occurs by binding between the 5’ dangling end of one replicator and the 3’ dangling end of another replicator.¹⁷ The binding affinity is determined by the number of complementary base-pair matches between the two dangling ends (thus, not every replicator can be recognized as a template). Replication reaction is assumed to be catalyzed by the replicators that have a catalytic secondary structure (thus, not every replicator can catalyze replication). The catalytic structure is arbitrary defined as that which consists of two hairpin loops that are joined by one multi-loop (see Ref. [104], for a graphical representation). defining a catalytic structure does not translate into defining what is informational (at least not completely) as will be seen below. Finally, complex formation and replication reaction are coupled as follows. A replicator can be replicated if its 3’ dangling end is binding to the 5’ dangling end of another replicator that has the catalytic structure. In other words, the 5’ dangling end and 3’ dangling end are the motif to recognize a template and the motif to be recognized by a replicase, respectively. In addition, replication reaction is assumed to produce a complementary sequence of a template and can introduce base substitutions (mutations) into a product. Finally, replicators also decay at a certain rate.

To summarize, the model has the following reaction scheme:

$$\mathrm{X}+\mathrm{Y}\underset{k_2}{\overset{k_1}{\rightleftharpoons }}{\mathrm{C}}_{{\mathrm{X}}^{5\prime ~3\prime }\mathrm{Y}}\text{ and }\underset{k_2^{\prime }}{\overset{k_1^{\prime }}{\rightleftharpoons }}{\mathrm{C}}_{{\mathrm{Y}}^{5\prime ~3\prime }\mathrm{X}}$$

$${\mathrm{C}}_{{\mathrm{X}}^{5\prime ~3\prime }\mathrm{Y}}+\theta \stackrel{\kappa }{\to }\mathrm{X}+\mathrm{Y}+{\mathrm{Y}}^{\mathrm{c}}$$

$$\mathrm{X}\stackrel{d}{\to }\theta$$

where X and Y denote replicators; C, a complex formed between X and Y (the subscripts indicate which dangling end binds to which dangling end); Y^c, a complementary copy of Y (but it can carry mutations); θ represents resources required for replication. The complex formation and dissociate rates k₁ and k₂ are, respectively, an increasing and decreasing functions of the number of complementary base pairs between the 5’ dangling end of X and the 3’ dangling end of Y (energetic differences between base pairs are taken into account). The replication rate κ is positive only if X has the catalytic structure (κ is independent of all other factors). Finally, the decay rate d is assumed to be identical for all replicators (see Ref. [104], for more details).

In the above model, evolution has neither a predefined goal nor a presupposed ecosystem to maintain (as in reality). Rather, evolution is modeled as a process of pattern formation at multiple levels of organization such as the genotypes, phenotypes, interactions, and spatiotemporal distributions of replicators. Although this is the most important feature of the model, it inevitably generates an observational issue too. The absence of a predefined goal translates into the lack of search images for the observers of the model. It also translates into the absence of predefined observables in the model. Consequently, the observation of the model becomes a highly nontrivial issue [29]. In the current case, pattern detection methods commonly used in bioinformatics turn out to be extremely useful, if not essential, to analyze the model.

Let us now consider how a simple replicator system can evolve into a complex replicator ecosystem with the model described above [104]. To start a simulation, the system is inoculated with a population of replicators having an identical sequence. The sequence is chosen such that replicators can replicate both their exact and complementary copies, so that the initial population can survive. (see Ref. [104], on how the sequence was chosen). As it proceeds, the simulation generates (as output) millions of sequences that have been present in the system through evolution. This large amount of data is not directly informative by itself (see the last paragraph). However, we can use phylogenetic trees to examine whether there is any pattern in these populations of sequences. Fig. 15 shows a phylogenetic tree constructed from sequences sampled from the system at some time step. The tree clearly indicates the existence of four distinct groups of sequences. Once these groups are recognized, we can use sequence logos [107] to analyze specific patterns of sequences in each sequence group (Fig. 16). The common secondary structure of each group can also be visualized with a simple statistical representation (Fig. 16, below the logos). These pattern analyses enable us to infer possible interactions between the four sequence groups as described below.¹⁸

Figure 15. Phylogenetic tree of replicators.

The tree was built from the genotypes (sequences) of 2000 replicators sampled from a simulation with the model explained in the main text (1 − q = 0.004). The leaves of the tree are color-coded based on the sequence composition in the dangling ends of the respective replicators. Each replicator, however, has up to four dangling ends (two ends in each of two complementary strands). Thus, one dangling end was chosen for each replicator as follows. If replicators are a replicase (i.e., if they have the catalytic structure), the 5’ dangling ends that recognize templates are chosen (i.e., the 5’ dangling end of the strands that actually fold into the catalytic structure). (Most replicases have the catalytic structure only in one strand.) The fraction of Cs and As in these dangling ends determine the colors of the corresponding leaves (the color scale is shown in the inset). If replicators are not a replicase, the 3’ dangling ends that have the most extreme sequence composition is chosen for each replicator. In this case, the colors are set to a function of the fraction of Gs and Us in these dangling ends (the color scale is shown in the inset). According to the above coloring scheme, each sequence class tends to appear as follows: C-catalysts tend to be cyan; A-catalysts, magenta; G-parasites, red; U-parasites, green. However, there are exceptions: The red leaves appearing in the clades of C-catalysts are not G-parasites, but are mutants (of C-catalysts) that have lost the catalytic structure. Thus, they belong to the C-catalyst quasi-species. Likewise, the green leaves in the clades of A-catalysts belong to the A-catalyst quasi-species. The tree was constructed with a maximum likelihood method implemented in PhyML [106]. However, owing to great divergence among sequences, the constructed tree does not necessarily depict evolutionary relationships between replicators.

Figure 16. Typical sequence and structure of each sequence class.

A sequence logo was obtained for each sequence class detected in the phylogenetic tree shown in Fig. 15. In sequence logos [107], the sizes of nucleotide characters (positively) correlate with the degree of conservation of the corresponding nucleotides in respective sequence positions. The sequence logos were generated with WebLogo [108]. The secondary structures beneath the sequence logos statistically represent the typical secondary structures in each sequence class (the structures are in the dot-bracket notation [109]). The colors of brackets indicate the frequencies of brackets in an alignment of secondary structures: the more red, the more frequent. A structure immediately beneath each sequence logo corresponds to the structure of the sequence represented by the respective logo. Below this structure is the structure of the complementary strand.

The replicator system has evolved four distinct groups of replicators (Fig. 15). Two of these groups have the catalytic structure and are distinguished by their distinct sequence patterns (Fig. 16). Based on the sequence composition of their 5’ dangling ends (the motif to recognize templates), they can be called C-catalysts or A-catalysts. C-catalysts and A-catalysts can each replicate themselves because their 3’ dangling ends are enriched with the respective complementary bases (i.e., G for C-catalysts, U for A-catalysts). The other two groups, which are non-catalytic, can be called G-parasites or U-parasites based on the sequence composition of their 3’ dangling ends (the motif recognized by replicases). G-parasites and U-parasites are replicated by C-catalysts and A-catalysts, respectively, as seen from the base composition of their dangling ends.

How do replicators achieve the interactions described above? For example, both of the two complementary sequences of U-parasites have a 3’ region enriched with Us and a 5’ region enriched with As (hereafter referred to as recognition regions) (Fig. 16, logo). Whichever of the complementary sequences is folded, this 3’ region always forms a part of the 3’ dangling end, so that it can serve as a motif to be recognized by A-catalysts (Fig. 16, structure). The 5’ region, in contrast, always forms a part of a hairpin loop. Thus, it is prevented from causing a complex formation that makes U-parasites unavailable as templates. These structural features are achieved by alternative stacking of three regions enriched with Gs and Cs (hereafter referred to as structural regions). Stacking between these GC-rich structural regions, which prevents stacking between the AU-rich recognition regions, relies on the fact that the bond between GC pairs is stronger than that between AU pairs (GC pairs have a greater energetic contribution than AU pairs).

G-parasites have basically the same strategy as that of U-parasites. In addition, G-parasites have evolved a way to solve the problem that they cannot rely on the energetic difference between AU pairs and GC pairs to prevent stacking between their GC-rich recognition regions.¹⁹ As a solution to this problem, G-parasites have evolved repeating CA (CpA) patterns in one of their recognition regions. Because of the intervening As, stacking between the recognition regions requires the formation of many bulges, which is, however, energetically disfavored. Therefore, stacking between the structural regions is favored to stacking between the recognition regions despite the fact both of these regions are enriched with Gs and Cs. (For brevity, we omit to describe C-catalysts and A-catalysts; see Ref. [104], for the description.)

Once we have recognized the existence of meaningful entities in the system, the four sequence groups described above, we can device a way to visualize the system’s dynamics.²⁰ The simplest way is to color-code every individual replicator according to the characteristics that can be used to distinguish which sequence group a replicator belongs to (or more precisely, which sequence group it is more closely associated with—for the sequence groups do not have definite boundaries). Such visualization enables the characterization of the spatiotemporal dynamics of the system as follows.

The four groups of replicators generate mutually invading, traveling wave patterns (Fig. 17). There are two types of waves: the waves whose front sides consist of C-catalysts (Fig. 17, cyan) and back sides consist of G-parasites (red) (henceforth, CG-waves); the waves whose front sides consist of A-catalysts (magenta) and back sides consist of U-parasites (green) (henceforth, AU-wave). The front of CG-waves can invade AU-waves because C-catalysts can replicate faster than A-catalysts. Conversely, the front of AU-waves can invade the back of CG-waves because A-catalysts are not affected by G-parasites. Because of this mutual invasion, both types of waves can persist indefinitely. Therefore, the four groups of replicators can coexist with each other (despite the fact that they all share identical resources θ).

Figure 17. Visualization of the model system.

The snapshot depicts mutually invading traveling wave patterns. Each replicator is colored in the same way as the leaves of the phylogenetic tree in Fig. 15. In this coloring scheme, each sequence class appears as follows: C-catalyst (cyan); A-catalyst (magenta); G-parasite (red); U-parasite (green). The arrows represent the direction of the propagation of wave fronts: the C-catalyst fronts (black); the A-catalyst fronts (white). The CA was toroidal. The lattice size was 512 × 512 points. (A movie is also available [104].)

How does the above replicator ecosystem evolve? The same visualization allows us to characterize the evolutionary dynamics of the system as follows (see also Ref. [104], for a movie, which is available from the publisher’s website). As an inevitable consequence of a birth-and-death process, the population is under selection for faster replication. Consequently, a highly optimized species of replicators evolve, which turn out to be C-catalysts (the specific sequences and structures of C-catalysts that evolve depends on the initial condition; see Ref. [104], for details). The evolution of C-catalysts entails the generation of a new ecological role (i.e., a niche) for a species that can parasitize C-catalysts. The generation of this new niche leads to the evolution of the second species, G-parasites, which arise through the speciation from C-catalysts (catalysts only have to destroy their 5’ dangling ends to be converted into parasites (Section 5.1)). Once the system evolves into the ecosystem consisting of C-catalysts and G-parasites, another new niche is generated, namely, a niche for a species that can escape from G-parasites (this escaping is distinct from that initiates wave generation (Section 5.3.1)). The ability to escape from G-parasites, that is, to not recognize G-parasites as templates, gives a selective advantage over C-catalysts in the presence of G-parasites. Consequently, A-catalysts evolve from C-catalysts through speciation. Finally, the evolution of A-catalysts, in turn, generates yet another niche for a species that can parasitize A-catalysts. This niche generation leads to the evolution of U-parasites.

To sum up: An existing, simpler ecosystem generates a novel niche. The generation of a novel niche leads to the evolution of a species that can occupy this niche. The evolution of a new species results in the development of a more complex ecosystem, which, in turn, generates another niche. In this way, a complex ecosystem can evolve through a chain reaction of niche generation and speciation. In addition, all these species with distinct niches are not predefined in the model. Rather, they evolved as patterns in the genotypes, phenotypes, interactions, spatiotemporal distributions, and ecological functions of replicators. Importantly, understanding of the system’s dynamics hinges on the concepts derived from the analyses of these patterns, which include, but are not limited to, the very concept of species. In this particular sense, the organization that has evolved in the system, be it genetic or ecological, are emergent [28].

This bootstrapping-like mechanism for the evolution of a complex ecosystem can be extended to the evolution of novel information, the other topic of this section. In fact, the evolution of novel information is inseparable from the evolution of a complex ecosystem in this model. The ecological functions of replicators depend on the genotypes of replicators (the functions are “coded” in the sequences). However, these functions (niche) are made possible by the ecosystem as already discussed. In other words, the ecosystem supplies a context in which the sequences gain functionality. Moreover, this ecosystem itself is a product of evolution. In short, evolution can generate information from within the system.

Taken together, the above bootstrapping-like mechanism can be restated as follows. An emergent ecosystem supplies a context in which a new species has a function (niche). The generation of a new niche leads to the evolution of sequences that can realize this function, that is, the evolution of novel information. The evolution of novel information, in turn, leads to the evolution of a more complex ecosystem. In short, the evolution of complexity is possible owing to the positive feedback loop between the evolution of information and that of organization.

Finally, let us consider the above results from the perspective of the previous section, where we described hypercycles. Hypercycles have been originally suggested as the simplest replicator network permitting the stable coexistence of multiple replicator species. This suggestion is based on the fact that such coexistence requires mutualistic coupling in well-mixed systems (Section 5.1). This requirement, however, is unnecessary in spatially extended systems with discrete populations, as indicated by the survival of the RP system (Section 5.3). The results described in this section further suggest that suppressive interactions such as parasitism and competition, rather than cooperation, underlie the evolutionary origin of complex replicator ecosystems (note that parasitism and competition decrease the diversity of well-mixed replicator systems as described in Section 5.1 and 3.2, respectively). (See Ref. [110], on a possible role of parasites for the evolution of new catalytic functions.)

7. Replicators with compartmentalization

In Section 5, we saw that spatial self-organization gives rise to selection operating at two levels of entities, namely, individual replicators and traveling waves (or spiral waves). This type of selection is generally known as multilevel selection. In the context of prebiotic evolution, a more obvious mechanism of multilevel selection is compartmentalization whereby replicators are enclosed in vesicle-like compartments that can grow, split, and decay. In compartmentalized replicator systems, selection operates not only between replicators but also between compartments.

Multilevel selection caused by compartmentalization is similar to that caused by self-organization. In both cases, selection operates on mesoscopic entities (waves or compartments), each consisting of a population of microscopic entities (replicators), which themselves are subject to selection. However, these two multilevel selection mechanisms critically differ in one respect. In compartmentalization, mesoscopic entities and the dynamics thereof are externally imposed upon replicators by boundaries such as vesicle membranes. In self-organization, by contrast, mesoscopic entities spontaneously arise from the dynamics of replicators. To indicate this difference, multilevel selection caused by compartmentalization may be called explicit multilevel selection, whereas that caused by spatial self-organization implicit multilevel selection.

In this section, we illustrate the effect of compartmentalization on the evolutionary dynamics of interacting replicators. Furthermore, we compare explicit multilevel selection with implicit multilevel selection to illustrate the general features of multilevel selection. We also contrast explicit and implicit multilevel selection in terms of the selection pressure they generate on respective mesoscopic entities.

7.1. Classical model of group selection

The classical model of group selection is the simplest example of explicit multilevel selection. In this model, a replicator system is partitioned into subsystems (i.e., groups) that are independent of each other within the generation time of replicators [111, 112]. How such partition is achieved is ignored in this model (this issue is revisited later). Each subsystem has n sites, each of which can contain at most one individual. Let us suppose there are two competing types of replicators, X and Y. X produces some sort of metabolite required for replication. The produced metabolite is available only within the subsystem where it is produced, and its amount is proportional to the population size of X in this subsystem (denoted by x_i where i denotes a subsystem). Based on these assumptions, the fitness of X in the i-th subsystem is set to f_xi = bx_i − c, and that of Y to f_yi= bx_i where c represents the cost of metabolite production, and b the benefit of the produced metabolite (this model is a slightly modified version of the model described in Ref. [112]). f_xi < f_yi holds in every subsystem. Therefore, the metabolite producer (X) is out-competed by the non-producer (Y ) if the distributions of replicators over subsystems (x_i and y_i) are uniform. The same conclusion, of course, holds if the system consists of only one subsystem.

If the distributions of replicators over subsystems are not uniform, can X out-compete Y? For X to be selected, the average fitness of X must be greater than that of Y [111]. That is,

$$\frac{\langle f_{x_i}{x}_i\rangle }{\langle x_i\rangle }>\frac{\langle f_{y_i}{y}_i\rangle }{\langle y_i\rangle }$$

where 〈〉 denotes the average over all subsystems. This inequality can be simplified into

$$b\left(\frac{\langle x_i^2\rangle }{\langle x_i\rangle }-\frac{\langle x_i{y}_i\rangle }{\langle y_i\rangle }\right)>c$$

or, equivalently, into

$$b\left(\frac{{\sigma }_x^2}{\langle x_i\rangle }-\frac{{\gamma }_{xy}}{\langle y_i\rangle }\right)>c$$ (21)

where ${\sigma }_x^2$ denotes the variance of x_i, and γ_xy the covariance between x_i and y_i. Eq. 21 encapsulates the essence of the group-selection theory [111].²¹ It is also analogous to Hamilton’s rule in the theory of kin selection [113] (see Ref. [114], for when this analogy breaks down).

As the simplest case, let us assume that the distributions of replicators over subsystems are random.²² If the number of individuals and that of subsystems are infinitely large, x_i and y_i follow the multinomial distribution. In this case, we have ${\sigma }_x^2=n{p}_x(1-p_x)$ and γ_xy = −np_xp_y where p_x = 〈x_i〉/n and p_y = 〈y_i〉/n. By substituting these equations, Eq. 21 is transformed into b > c [111, 112]. This result can be intuitively understood as follows. Because the distributions of replicators are random, the environments of individuals are statistically identical between X and Y (the “environment” of an individual here refers to the population size of X and Y in the subsystem where the individual is located excluding this individual). The only statistical difference between the factors affecting the fitness of X and Y is the effect an individual causes to itself.²³ Therefore, if the metabolite production benefits the producer itself (i.e., b − c > 0), the producer can out-compete the non-producer. To summarize, although the fitness of the metabolite producer is lower than that of the non-producer in every subsystem (i.e., f_xi < f_yi), the random distribution of individuals into many subsystems permits the evolution of the metabolite producer (see also Ref. [115]).

We next assume that X is a replicase that can replicate other individuals using them as a template (X can also serve as a template). The fitness of X is set to f_xi = k_x(x_i − 1) where −1 represents the fact that a replicase cannot replicate itself (replication requires at least two individuals). The fitness of Y is set to f_yi= k_yx_i (Y is a parasite). If we further assume that k_x = k_y (denoted by k), the current setup boils down to the previous setup with b = k and c = k. The equation b − c = 0 indicates the fact that an individual gains nothing from replicating other individuals (replication is here assumed to be instantaneous, but see Section 5.3.2). In this case, even if the populations of X and Y are randomly distributed over subsystems, X will not be selected (the condition b − c > 0 is not fulfilled). In fact, the average fitness of X is the same as that of Y. Thus, if k_x < k_y (i.e., if Y serves as a better template than X), Y out-competes X.

Let us contrast the above result with the result described in Section 5.3.1 (Eq. 17). There, we saw that if populations are treated as continuous quantities, parasites out-compete replicases even if diffusion is finite (parasites are assumed to serve as better templates than replicases). This result can be restated as follows: If local population sizes are infinitely large, parasites out-compete replicases even if the spatial distributions of replicators may be non-random (to treat populations as continuous quantities, the number of individuals must be infinitely large). To make a contrast, we restate the result described in the previous paragraph as follows: If the spatial distributions of replicators are random, parasites out-compete replicases even if local population sizes (i.e., n) are finite.

In Section 5.3, we saw that both discrete populations and spatial self-organization together allow the stable coexistence of replicases and parasites. To conceive a similar situation in the above group-selection model, we could assume the distributions of replicators are such that replicases tend to co-occur with their own kind (i.e., ${\sigma }_x^2>{\gamma }_{xy}$). Although this assumption itself is reasonable [116], there is a fundamental problem in this line of thought. Namely, ${\sigma }_x^2$ and γ_xy are parameters of the model and so independent of the population dynamics of replicators. However, how the spatial distributions of replicators are—which include the values of ${\sigma }_x^2$ and γ_xy—is exactly what the population dynamics of replicators is all about. In the next section, we will consider a model that is free of this problem.

7.2. Compartmentalization and multilevel evolution

7.2.1. Problem of parasites

Spatial self-organization is not the only way to circumvent the problem of parasites (Section 5.3): it can be circumvented by compartmentalization as well [59, 117–119]. By compartmentalization is meant that replicators are partitioned into compartments that can multiply and decay depending on replicators within each compartment (such compartments can be considered primordial cells or “protocells”).

The mechanism by which compartmentalization circumvents the problem of parasites is simple (at least theoretically). To examine this mechanism in detail, however, let us consider the model of the compartmentalized RP system as follows [59] (see Section 5.3.3, for the description of the RP system itself). The model contains the Cellular Potts model (CPM, for short) to simulate the dynamics of compartments, or more precisely, the dynamics of boundaries that demarcate compartments [120, 121]. The CPM is a two-scale, stochastic CA model. It defines compartments as sets of one or more lattice points that are in a state uniquely corresponding to every compartment (the definition of the states is arbitrary). The dynamics of the CPM is defined by an algorithm that stochastically updates the state of the CA. At the microscopic level, the algorithm tends to minimize the area of contact between different compartments, leading to the shrinkage of compartments. At the mesoscopic level, however, it tends to maintain the volume of each compartment (i.e., the number of lattice points constituting each compartment) at some target value (denoted by V ). The equilibrium between these two opposing tendencies determines the shape and size of compartments (see Ref. [121], for full details of the CPM).

The dynamics of replicators is simulated by a CA as described in Section 5. The size of this CA is set identical to the size of the CA that implements the CPM. The two CA are superimposed on each other [99], and their dynamics are coupled with the following assumptions [59]. First, both replicases and parasites cannot diffuse across compartment boundaries. Second, parasites that are in the folded state (see Eq. 20) can increase the target volume (V ) of the compartment in which they reside (e.g., by synthesizing the components of the compartment boundary). V also decays spontaneously at a constant rate (denoted by d_V ). Taken together, the dynamics of V can be expressed as V̇ = lP − d_V V, where l is the fraction of parasites in the folded state. Third, a compartment divides into two compartments when its volume (denoted by υ) exceeds a threshold (denoted by υ_T ). A compartment is divided along the line perpendicular to the main axis of the compartment (the axis is calculated by the principle component analysis). The internal replicators are distributed between the daughter compartments depending on which compartment the replicators reside at the moment of division. The target volume of the mother compartment is also distributed between the daughter compartments in proportion to their volumes right after the division.

As can be seen from the above description, V plays a crucial role for competition between compartments. A compartment with a greater value of V has a greater chance of growing (i.e., increasing υ) and so a greater chance of dividing. In addition to division, a compartment can also die either because its target volume decreases to zero (e.g., because it loses all internal parasites) or because it is squeezed by other compartments that have greater target volumes. Because the death of compartments can happen as a consequence of the model’s assumptions described above, it need not be explicitly implemented.

For simplicity, we first consider the evolution of k_P in the above model with all the other parameters kept constant (k_P is the affinity of parasites toward replicases; see Eq. 20). Simulations show that evolution tends toward increasing the average value of k_P in every compartment (Fig. 18, black lines). This tendency is expected because evolution within each compartment tends toward maximizing the multiplication of parasites. However, all compartments that exist at any given moment are descended from compartments in which the average value of k_P was small (Fig. 18, color lines). This result indicates that compartments with smaller values of k_P have a selective advantage over compartments with larger values of k_P.

Figure 18. Evolutionary dynamics within and between compartments.

Plotted is the value of k_P averaged over all parasites within each compartment as a function of time. The colored squares indicate the compartments that existed in the system at the designated time. The ancestral lineages of these compartments are indicated by colored lines. The other lineages are shown by black lines. (For visibility, the values of k_P are not plotted between the moments at which compartments died and those at which their last division happened.) The parameters were as follows: the value of l was fixed to 0.5; the diffusion rate was set to 1 so that a system within each compartment was relatively well-mixed; k_R = 0.6; κ = 1; d = 0.02; υ_T = 1000 (with this value of υ_T the total number of replicators within a compartment was on average about 200); the lattice size, 150 × 150 points.

Each compartment decreases its selective advantage over time because its internal parasites evolve toward increasing the value of k_P (compartments undergo an aging-like process; see also Section 5.3.3). Despite this clear trend, the evolutionary dynamics of parasites within each compartment is highly stochastic as indicated by its zigzag trajectory (Fig. 18). This high stochasticity arises from the fact that internal replicator systems of compartments have small population sizes [118]. This stochasticity generates variations between compartments, which are subject to selection at the between-compartment level. Since this selection favors compartments with small k_P values (see above), the resultant evolution tends toward decreasing the value of k_P. Thus, evolution at the between-compartment level counteracts evolution at the within-compartment level. Consequently, the average k_P value of the whole system is maintained at a moderate level; hence, the system can survive. This mechanisms of multilevel selection is basically identical to that involving traveling waves (Section 5.3) despite the obvious difference in mesoscopic entities (we come back to this point later). The mechanism of multilevel selection described above is also known as “stochastic corrector” mechanism [118].

7.2.2. Emergence of a novel evolutionary trend

Spatial self-organization gives rise to a novel evolutionary trend in the spatially extended RP system (Section 5.3.3). Can compartmentalization also cause such a novel evolutionary trend? To address this question, we now allow not only k_P but also l to evolve in the CA model described above. With both parameters allowed to evolve, the model displays dynamics similar to that displayed by the model of the spatially extended RP system (Fig. 12, dotted line). Namely, the evolutionary trajectory of k_P and l can again be separated into a short-term and a long-term evolution. During the short-term evolution, the trajectory goes toward a contour given by a function f(k_P, l) = 0, indicating that compartmentalization, too, leads to the emergence of trade-off between k_P and l. This trade-off arises from the balance between within-compartment evolution and between-compartment evolution described in the previous section. During the long-term evolution, parasites evolve toward increasing k_P and l. This trend, however, is neither novel nor unexpected since the model explicitly assumes that the folded state of parasites has a functionality to promote the compartment growth (cf. Section 5.3.3).

Whether this functionality is indeed the actual cause of the long-term evolutionary trend can be checked by removing this functionality. To this end, we modify the model by assuming that V is constant and greater than υ_T (V = 1.1υ_T) if a compartment contains at least one parasite, but V is zero otherwise.²⁴ Since V is fixed, all compartments are identical to each other in terms of their tendency to grow and divide; the only difference lies in their tendency to die. Compartment death results from a complete loss of parasites or replicases and so depends on the stability of the coexistence between replicases and parasites within each compartment. Consequently, between-compartment selection now operates exclusively on this stability.

Despite the removal of the functionality, a long-term evolutionary trend still emerges in the modified model (Fig. 12, gray line). Moreover, the direction of this trend depends on the mutation rates of k_P and l (i.e., the probabilities per replication of changing these parameters) as follows. If the mutation rates are sufficiently high, k_P and l decrease (Fig. 12); if they are sufficiently low, k_P and l increase (data not shown; see Ref. [59]). In the original model, by contrast, k_P and l increase independently of the mutation rates. Taken together, these results indicate that the evolutionary trend in the original model is caused, at least partly, by the functionality of parasites. Moreover, compartmentalization by itself causes long-term evolutionary trends independently of this functionality.

The longevity of compartments plays a key role for the emergence of long-term evolutionary trends (as expected since only this is subject to between-compartment selection). The longevity of compartments is a function of k_P and l (Fig. 19). The shape of this function depends on whether coexistence between replicases and parasites inside compartments is deterministically stable. The deterministically stable coexistence refers to the case where the replicator dynamics requires no interventions (such as stochastic corrector mechanism) to enable the coexistence. If the coexistence is deterministically stable, increasing the values of k_P and l increases the longevity (Fig. 19). Conversely, if the coexistence is deterministically unstable, decreasing the values of k_P and l increases the longevity.

Figure 19. The death rate of compartments as a function of the stability of internal replicator systems.

The death rate (i.e., the inverse of the longevity) of compartments is plotted as a function of l for various values of k_P (inset, magnified view). The abscissa Δl is defined as l − l_H. If Δl < 0, the coexistence of replicases and parasites is deterministically stable (the value of l_H was obtained from an ODE model). If Δl > 0, the coexistence is deterministically unstable. The abscissa is set to Δl in order to center the curves around l = l_H (l_H is an increasing function of k_P ).

The deterministic stability of the coexistence between replicases and parasites inside compartments depends on the mutation rates of k_P and l. Increasing these mutation rates increases the variability of parasites within each compartment and so accelerates within-compartment evolution. This acceleration reduces the relative impact of between-compartment evolution. Consequently, the balance between within- and between-compartment evolution is shifted toward destabilizing the coexistence. Let us suppose that such a shift renders the coexistence deterministically unstable. In this case, between-compartment evolution tends toward decreasing the values of k_P and l (Fig. 12, gray line), so that compartments increase their longevity (Fig. 19). Conversely, decreasing these mutation rates shifts this balance toward stabilizing the coexistence. If the coexistence is deterministically stable, between-compartment evolution tends toward increasing the values of k_P and l [59].

This dependency of the evolution on the mutation rates can also be explained by the quasi-species theory (Section 3.2). To this end, let us define the genotype of each compartment as the average values of k_P and l within each compartment. Reducing mutation rates decreases the size of a genotype neighborhood that affects evolution (Section 3.2). If mutation rates are sufficiently low, this size shrinks to a single genotype. In this case, compartments evolve toward the fittest genotype, the values of k_P and l that minimize the death rate of compartments (Fig. 19). By contrast, if the mutation rates are sufficiently high, this size is greater than a single genotype. In this case, compartments evolve toward a region of the genotype space that has the highest average fitness. This region corresponds to the values of k_P and l for which a change in these values leads to a small increase in the death rate of compartments (i.e., where the death-rate curve is flat (Fig. 19)).

The remaining question is how to relate the deterministic stability of the coexistence, the longevity of compartments, and the values of k_P and l. Briefly, since k_P and l determine reaction rates (Eq. 20), increasing these parameters turns out to strengthen the deterministic flows of the replicator dynamics [59]. The stronger the deterministic flows, the weaker the relative impact of stochasticity on the dynamics. Stochasticity only jeopardizes the coexistence if the coexistence is deterministically stable. If, however, the coexistence is deterministically unstable, stochasticity can actually promote the coexistence because it can disrupt the deterministic progression of the dynamics toward extinction.

To sum up: compartmentalization causes selection at the between-compartment level, which leads to evolutionary adaptation of compartments toward increased longevity. This adaptation is manifested in the long-term evolution of parasites.

Although the above mechanism of multilevel evolution is basically identical to that involving traveling waves (Section 5.3), there is a notable difference. Namely, traveling waves evolve toward increased fecundity, whereas compartments, by default, evolve toward increased longevity. This difference arises from a difference in the nature of mesoscopic entities. Traveling waves are a consequence of spatial self-organization. Thus, each wave is stable by itself and is annihilated only by external perturbations such as collision with other waves (such annihilation cannot be avoided in the current model, but see Section 6). The generation of new waves, however, rests on rare stochastic events and subsequent amplification thereof (viz., escape and multiplication of replicases). Thus, room for improvement by evolution lies in the success probability of these processes. Contrastingly, compartments are imposed on replicators by external boundaries, which also dictate compartment multiplication. Each compartment is unstable by itself because it is eventually destroyed by evolving parasites inside it. Thus, the stability of internal replicator systems is the Achilles heel of compartments. Accordingly, evolution tends toward stabilizing the internal replicator systems (unless this tendency is overwritten owing to coupling between compartment multiplication and internal replicator systems, e.g., via the functionality of parasites to promote compartment multiplication).

Finally, there is an interesting difference between compartments (i.e., protocells) and modern cells. This difference relates to the concept of genotypes as applied to these two kinds of “cells”. In modern cells, the genotype of a cell is an invariant character during the lifetime of the cell. In the compartments, however, internal replicators of a compartment, which can be viewed as the genome of the compartment, rapidly evolve over time comparable to the lifetime of the compartment (compare Fig. 18 and Fig. 19). Thus, compartments do not have stable genotypes as conceived in modern cells. The next section describes how such stable genotypes can arise through evolution (see also Ref. [122–124]).

8. Replicators with DNA-like function

At the core of all living systems lies the division of labor between the storage of genetic information (DNA) and the phenotypic implementation of genetic information (proteins). The RNA world hypothesis, however, posits that in primordial replicating systems, RNA molecules have both stored genetic information and provided chemical catalysis. This co-embodiment of a template and catalyst in one type of molecules is one of the fundamental characteristics that distinguish the RNA world from modern living systems (Section 2).

How can the division of labor between templates and catalysts evolve in RNA-like replicator systems? One way to approach this question is to ask whether RNA-like replicator systems can evolve replicators whose only function is to store information, that is, to serve as templates. Such replicators are functionally equivalent to DNA molecules in modern living systems.

The simplest possible model of a replicator system that includes DNA-like replicators can be conceived as follows [125]. The model assumes two types of molecules: RNA and DNA. They are identical to each other except in one respect: RNA molecules can serve both as replicases and as templates, whereas DNA molecules can only serve as templates (but a DNA template can store information about a replicase; see below). The two types of molecules allow for four different replication reactions, each specified by the molecular types of templates and products:

• RNA-dependent RNA polymerization (i.e., RNA replication),

• RNA-dependent DNA polymerization (i.e., reverse transcription),

• DNA-dependent RNA polymerization (i.e., transcription),

• DNA-dependent DNA polymerization (i.e., DNA replication).

A replicase is assumed to be either an RNA polymerase or a DNA polymerase (Rp or Dp, respectively, for short). The type of a polymerase, however, can be converted from one to the other as a result of rare mutations (see below). Replicases have a potential to discriminate between an RNA molecule and a DNA molecule as a template. They, however, cannot discriminate between different DNA molecules and between different RNA molecules as a template (such discrimination is omitted for simplicity, but see Section 6). Although a replicase is always an RNA molecule by definition, the information about a replicase can be stored both in an RNA molecule (RNA template), which itself also serves as a catalyst, and in a DNA molecule, which, however, can only serve as a template. To distinguish between the RNA form and the DNA form of a replicase, we use superscripts as follows: Rp^RNA, Rp^DNA, Dp^RNA, and Dp^DNA. When it is preferred not to distinguish between these forms, these superscripts are omitted.

Before describing details of the model, let us first consider the replicator system described above in general terms. The four replication reactions listed above allow for several replication cycles (Fig. 20). The simplest among these cycles consists of a single type of molecules, namely, RNA molecules that can replicate RNA molecules as templates (hereafter referred to as RdRp^RNA where Rd stands for “RNA dependent”) (Fig. 20A). This RNA-only system captures the essence of the RNA world: a single type of molecules embodies both catalyst and template. The replication cycle that most closely resembles that of modern cells consists of four types of molecules: DdRp^RNA, DdRp^DNA, DdDp^RNA, and DdDp^DNA (hereafter referred to as the transcription system). RNA molecules in this cycle serve only as catalysts, delegating the role of templates to DNA molecules (Fig. 20B). Thus, this cycle establishes the division of labor between templates and catalysts.

Figure 20. Schematic of possible replication cycles in a replicator system with or without DNA-like replicators.

RdRp denotes the RNA-dependent RNA polymerase (RNA replicase); DdRp, the DNA-dependent RNA polymerase (transcriptase); DdDp, the DNA-dependent DNA polymerase (DNA replicase); RdDp, the RNA-dependent DNA polymerase (reverse transcriptase); dual-Rp, the RNA polymerase that recognizes both RNA and DNA as templates; dual-Dp, the DNA polymerase that recognizes both RNA and DNA as templates. Superscripts indicate whether molecules are RNA or DNA.

The comparison between these two cycles reveals that DNA molecules bring an obvious disadvantage to a replication cycle: a reduction in the replication efficiency. The inclusion of DNA molecules into a replication cycle requires joint action of four distinct types of molecules (viz. Rp^RNA, Rp^RNA, Rp^DNA, and Dp^DNA) to complete a replication cycle (e.g., Fig. 20B). This increase of complexity entails a decrease in the concentration of each molecular type, assuming that the total concentration of replicators is constant. Consequently, the rate of multiplication is reduced in comparison with the RNA-only system.

The inclusion of DNA molecules, however, also brings a potential advantage; namely, it can reduce the harmfulness of parasites. In the RNA-only system, RNA molecules must serve not only as catalysts but also as templates in order to complete the replication cycle. A molecule, however, cannot serve as a template while replicating another molecule (since replication is not an instantaneous process; see also Section 5.3.2). Therefore, RNA molecules face the trade-off between serving as a catalyst and serving as a template. This trade-off brings an advantage to parasites because parasites spend all of their lifetime serving as templates (Section 5.1). In the transcription system, by contrast, RNA molecules do not serve as templates because they are produced by the transcription of DNA molecules. Therefore, RNA molecules are free of the aforementioned trade-off. This absence of the trade-off puts parasites at a relative disadvantage as compared with the case where the trade-off is present. In this sense, the transcription system is more resistant to parasites than the RNA-only system is (more on this point later).

The evolutionary dynamics of the replicator system defined above can be simulated by a CA model as follows. Replication is modeled as a two-step process as follows:

$$\mathrm{R}+\mathrm{T}\underset{1-k}{\overset{k}{\rightleftharpoons }}\mathrm{C}$$

$$\mathrm{C}+\theta \stackrel{\kappa }{\to }\mathrm{R}+\mathrm{T}+\mathrm{T}\prime$$

where R denotes a replicase, T a template, C a complex between R and T, T′ a newly produced copy of T, and θ represents resources required for replication. T′ is either RNA or DNA depending on whether the replicase R is Rp or Dp. Each replicase is assigned parameters R_rec and D_rec defining its template specificity (R_rec and D_rec stand for RNA recognition and DNA recognition, respectively). The template specificity determines the rate constant of complex formation as follows: k = R_rec if T is RNA; k = D_rec if T is DNA. R_rec and D_rec assume values between 0 and 1, ranging from the case of no complex formation to that of no complex dissociation, respectively. The rate constant of replication reaction κ is assumed to be identical regardless of the type of a replicase and a template that form a complex.

In addition to replication, the decay reaction can occur that converts molecules into the resources θ: $\mathrm{R}\stackrel{d}{\to }\theta \text{ and }T\stackrel{d}{\to }\theta$. The decay rate d is assumed to be identical to all molecules. The decay of complexes is treated as independent decay of each constituent molecule as follows: $\mathrm{C}\stackrel{2d}{\to }X+\theta$ where X is either R or T with an equal chance.

T′ inherits the characters of T from which it is produced. The inherited characters, however, can be modified by a mutation, which occurs with a certain probability during replication. There are four types of mutations, among which at most one can occur during replication: a change in the value of R_rec (its probability is μ_Rrec), a change in the value of D_rec (μ_Drec), conversion of the type of a replicase (μ_Rp→Dp and μ_Dp→Rp), and the conversion of a replicase into a parasite, which is defined as an RNA or DNA molecule that cannot serve as a catalyst but can serve as a template with increased affinity toward replicases (μ_P). These mutations can happen whether a replicase is in an RNA or DNA form.

The model described above can be formulated as a surface model in which replicators are attached to surfaces with limited diffusion or as a compartment model in which replicators are compartmentalized by vesicle-like boundaries (see Ref. [125], for details; see also Section 5.2 (surface) and 7.2 (compartment), for the brief descriptions of similar models). To formulate a compartment model one must specify how compartment dynamics and replicator dynamics are coupled together. For simplicity, we here assume that the target volume V of a compartment is proportional to the total number of RNA and DNA molecules within the compartment with a factor of proportionality f. In addition, replicators cannot diffuse across compartment boundaries.

For the sake of continuity with the previous section, we here review the results of the compartment model (those of the surface model will be briefly mentioned later). Before describing the main results, let us first describe the evolutionary dynamics of the RNA-only system with the evolution of DNA molecules disabled (i.e., with μ_Rp→Dp set to zero). The replicator system, after reaching an equilibrium, displays the following characteristics. The values of D_rec are uniformly distributed (a trivial consequence of the absence of DNA molecules). The values of R_rec show a unimordal distribution whose peak is at an intermediate value (Fig. 21A).

Figure 21. Two-dimensional frequency histograms of R_rec and D_rec.

The frequencies are shown separately for the population of Rp (left panel) and the population of Dp (right panel) (populations include both RNA and DNA molecules). The gray scale is indicated in the inset. The number of bins are 20 × 20 with an equally bin size. A: the RNA-only system (Fig. 20A). The data were obtained from a simulation where the evolution of DNA molecules was disabled (i.e., with no Dp initially present in the system and μ_Rp→Dp = 0). The parameters were as follows: κ = 1; d = 0.02; μ_Rrec = 0.01; μ_Drec = 0.01; μ_P = 10⁻⁵; μ_Dp→Rp = 0; υ_T = 500; f = 1.3 (V = fυ); the diffusion rate was 1; the boundaries had no flux, the lattice size was 512 × 512. The values of R_rec and D_rec were mutated in the same manner as described in Fig. 12. B: the transcription-like system (Fig. 20C). The data were obtained from a continuation of the simulation depicted in A with the evolution of DNA molecules enabled (i.e., with μ_Rp→Dp increased to 10⁻⁵).

The distribution of R_rec is maintained by the balance between two opposing evolutionary tendencies as follows. RNA molecules increase the chance of their own replication by increasing the time they spend serving as templates (i.e., by becoming more and more parasitic²⁵). Therefore, evolution within each compartment tends toward decreasing the value of R_rec (Fig. 22A). Conversely, the multiplication of compartments is accelerated if their internal RNA molecules increase the time they spend serving as catalysts. Therefore, evolution at the between-compartment level tends toward increasing the value of R_rec.

Figure 22. Evolutionary dynamics of well-mixed replicator systems with or without DNA molecules.

The model was modified such that interactions between molecules happen independently of the location of molecules (i.e., interactions are global). At the beginning of simulations, every population of RNA and DNA molecules was set in equal proportion. The initial values of R_rec and D_rec were as indicated in the figure (time = 0). The parameters were the same as in Fig. 21. A: the RNA-only system (Fig. 20A). This system contains no DNA molecules; hence, D_rec undergoes neutral evolution. R_rec decreases faster than D_rec, that is, faster than expected for neutral evolution. B: the transcription-like system (Fig. 20C). The curves are indistinguishable from that of D_rec in A, that is, from a curve expected for neutral evolution. C: the all-replicate-all system (i.e., the transcription-like system plus reverse transcription; Fig. 20D). R_rec and D_rec decrease faster than D_rec in A (i.e., faster than neutral evolution).

Do DNA molecules evolve in the RNA-only system? With DNA evolution enabled (i.e., with μ_Rp→Dp increased to 10⁻⁵), the model shows that the RNA-only system evolves (after a long transient) into a system that consists of DdDp and Rp recognizing both RNA and DNA as templates (Fig. 21B). This system is schematically depicted in Fig. 20C (hereafter referred to as the transcription-like system). The evolution of the RNA-only system into the transcription-like system requires sufficiently high mutation rates (μ_Rrec and μ_Drec) [125] (cf. Section 7.2.2).

To elucidate the evolution of DNA molecules (in the form of the transcription-like system), we consider the effect of DNA molecules on the tendency of within-compartment evolution. To this end, we compare the RNA-only system and the transcription system depicted in Fig. 20A and B, respectively (we will consider the transcription-like system later). In the RNA-only system, RNA molecules tend to evolve into parasites through within-compartment evolution (see above). In the transcription system, by contrast, RNA molecules do not serve as templates and so gain no selective advantage from increasing the time they spend serving as templates. Therefore, they do not tend to evolve into parasites through within-compartment evolution (nor do they evolve toward serving more and more as catalysts). This enhanced evolutionary stability of RNA molecules serving as catalysts gives a selective advantage to compartments containing the transcription system. This advantage, however, is offset by the disadvantage, namely, a reduction in the efficiency of multiplication (see above).

The relative significance of these advantage and disadvantage depends on the speed of within-compartment evolution. Accelerating within-compartment evolution exacerbates the evolutionary instability of catalysts in the RNA-only system. Thus, it increases the relative advantage of the transcription system. By contrast, the disadvantage of the transcription system is an inevitable consequence of increased complexity in the replication cycle. Hence, it is independent of within-compartment evolution. Therefore, if within-compartment evolution is sufficiently rapid, the advantage can most likely outweigh the disadvantage.

However, the above explanation is based on the consideration of the transcription system. Can the transcription-like system, the system that evolves in the model, also cause such evolutionary stabilization of catalysts? This question can be addressed by simulating the system that is completely devoid of between-compartment selection. Such a simulation shows that, in the transcription-like system, RNA molecules serving as catalysts are so stable that their evolutionary deterioration is indistinguishable from neutral evolution (Fig. 22B).

How does the transcription-like system, in which RNA molecules also serve as templates, cause the evolutionary stabilization of catalysts? The key to understanding the mechanisms of this stabilization is the direction of information flow. Both in the transcription system and in the transcription-like system (Fig. 20BC), information flows from DNA to RNA (transcription), but not from RNA to DNA (reverse transcription). If the latter flow is added to the transcription-like system (Fig. 20D), the evolutionary stabilization of catalysts is lost. The evolution of RNA molecules into parasites is much accelerated, with its speed approaching that of the RNA-only system (Fig. 22C). Therefore, the presence or absence of reverse transcription plays a critical role for the evolutionary stabilization of catalysts. The reason for this role is explained below.

In the transcription system, the relative frequencies of different RNA molecules are completely determined by those of the corresponding DNA molecules that “encode” them (replicases do not distinguish between different DNA molecules). Thus, even though RNA molecules increase the time they spend serving as templates, no difference is made to their relative frequencies.

In the transcription-like system, the relative frequencies of different RNA molecules are determined by the combination of two factors: the transcription of DNA molecules and the replication of RNA molecules. Because of RNA replication, RNA molecules can increase their relative frequencies by increasing the time they spend serving as templates—thus, parasitic RNA molecules have a selective advantage. This advantage, however, causes no difference to the relative frequencies of the DNA molecules because no reverse transcription happens. In addition, the transcription of DNA molecules tends to bring the relative frequencies of RNA molecules toward those of the corresponding DNA molecules. Thus, the population of DNA molecules serves as a buffer to the population of RNA molecules. In this way, DNA molecules can stabilize RNA molecules serving as catalysts.

If reverse transcription happens as well as transcription, they together form a positive feedback loop (as in the all-replicate-all replication cycle depicted in Fig. 20D). This feedback loop amplifies the aforementioned advantage of parasitic RNA molecules. Therefore, reverse transcription nullifies the stabilizing effect of DNA molecules.

Taken together, these arguments show that the evolutionary stabilization of catalysts requires both presence of transcription and absence of reverse transcription. That is, the flow of information must be from DNA (dedicated templates) to RNA (which serves as catalysts), but not vice versa. This unidirectionality in the flow of information indicates the division of labor between templates and catalysts.

Reverse transcription is nevertheless required for the emergence of DNA molecules. If a system contains no DNA molecules to begin with, it must perform reverse transcription to generate them. However, this fact, of course, does not contradict with the above conclusion since reverse transcription can happen during the transient phase of evolution before the establishment of the transcription-like system.²⁶ Although reverse transcription is necessary for the emergence of DNA molecules, it becomes inhibitory for their evolutionary maintenance.

To sum up: The inclusion of DNA molecules into a replication cycle allows the division of labor between dedicated templates (DNA) and molecules serving as catalysts (RNA). This division of labor evolutionarily stabilizes the molecules serving as catalysts and thereby brings a selective advantage to compartments. Because of this advantage, DNA-like replicators can evolve in a compartmentalized RNA-like replicator system. The ensuing stabilization of replicator systems within compartments can be considered a step toward the evolution of stable genotypes in protocells (see Section 7.2.2, for the discussion on genotypes of protocells).

The evolution of DNA and the resultant division of labor between templates and catalysts mark one of the major transitions in the way in which genetic information is transmitted between generations. It has been said that this transition, as well as the other major transitions, must be explained in terms of immediate selective advantage to individual replicators [65]. This restriction is beneficial in preventing explanation from going astray, given the difficulty of predicting non-immediate effects (especially without explicitly simulating evolutionary dynamics). However, as we saw above, DNA molecules can stabilize the genotypes of compartments, and this stabilization is advantageous only on a timescale longer than the lifetime of compartments, let alone that of replicators. Such an advantage may thus be called evolutionary advantage to distinguish it from immediate selective advantage. Such non-immediate, evolutionary advantage might play a significant role for the other major transitions in evolution as well.

Finally, let us briefly describe the surface model. DNA-like replicators also evolve in this model, and the underlying principle is again the division of labor between templates and catalysts [125]. The model, however, displays dynamics markedly different from that of the compartment model and thereby provides extra insights. In the surface model, the transcription system evolves (rather than the transcription-like system). Moreover, it coexists with the RNA-only system despite the fact that it multiplies more slowly than the latter (owing to its reduced replication efficiency; see above). This coexistence is mediated by parasites (arising from one-step mutations), which mainly target the RNA-only system because their population largely consists of RNA molecules (their role resembles that of G-parasites, which mediate the coexistence between C-catalysts and A-catalysts as described in Section 6). It, therefore, seems that the presence of parasites causes the evolution of the transcription system; however, this is only one side of the story. Even if the evolution of parasites is disabled (i.e., if no parasite is initially present in the system and μ_P = 0), the transcription system nonetheless evolves. Moreover, in this case, the RNA-only system undergoes speciation, whereby two sub-populations emerge: molecules with active replicase function (R_rec > 0 and D_rec ≈ 0) and molecules without such function (R_rec ≈ 0 and D_rec ≈ 0). The latter molecules serve only as templates and are thus effectively parasites. This speciation happens only if the transcription system is present. Therefore, not only can parasites cause the evolution of the transcription system, but also the transcription system can cause the evolution of parasites. Because of this bidirectionality in the cause-and-effect relationship, “which causes which” in the evolution is only historically decidable, but logically undetermined.

9. Summary and discussion

The results reviewed in this paper can be summarized as follows:

• The amount of information that can be maintained by evolution in one quasi-species is severely limited by erroneous replication—the problem of information maintenance, for short (Section 3.2 and 3.3).

• Neutrality in the genotype-phenotype map of replicators helps the acquisition of information through evolution (Section 4.1). This neutrality, however, does not help to solve the problem of information maintenance (Section 4.2).

• The population-based maintenance of information (i.e., coexistence of co-operatively coupled replicators) is severely hampered by the evolution of parasites—the problem of parasites, for short (Section 5.1).

• Spatial self-organization of replicators causes implicit multilevel selection, which can solve the problem of parasites (Section 5.2 and 5.3).²⁷ Moreover, it leads to multilevel evolution, whereby novel evolutionary trends emerge in replicators (Section 5.3.3).

• Spatial self-organization can not only render parasites harmless, but also make them contribute to the evolutionary generation of novel information (Section 6).

• Compartmentalization of replicators causes explicit multilevel selection, which can also solve the problem of parasites (Section 7.2.1). However, explicit multilevel selection can cause an evolutionary trend that is opposite to that caused by implicit multilevel selection (Section 7.2.2).

• With multilevel selection, the division of labor between templates and catalysts can emerge through the evolution of DNA-like replicators, which serve as dedicated information storage (Section 8).

As summarized above, we surveyed the evolutionary dynamics of RNA-like replicator systems from a view point of bioinformatics. In particular, we considered the maintenance, acquisition, generation, and storage of information in RNA-like replicator systems. Given these lines of research, a possible next step is to consider whether (and how) RNA-like replicator systems evolve the processing of information, for example, in the form of metabolic regulation. Let us explain why this question is worth addressing. To this end, we return to the question of the origin of life, that is, how non-life can become life. This question immediately raises another question that needs to be answered before addressing the first; namely, What is life? Although such a question might defy any definite answer, let us consider the following possibility [1, 131–133]: Life is an information-processing system or a symbol-processing system (but not vice versa; e.g., consider computers). For example, the translation of an mRNA involves processing of “symbols” whereby codons correspond to specific amino acids. There is no direct physicochemical affinity between codons and amino acids; the correspondence is mediated by tRNAs and aminoacyl-tRNA synthetases. Another example is found in the relationship between heat given to an organism and proteins produced in response to it. This relationship contrasts with that between heat given to a gas and a resultant increase of its volume. Life is, in fact, full of such examples—information processing is manifest in adaptability, excitability, regulation, signal transduction, taxes, immunity, and so forth. These processes, we think, are the hallmarks of living systems [122]. This answer to “What is life?” is worth considering, not because it is the correct answer to the question,²⁸ but because it renders the origin-of-life question interesting and concrete—the origin of an information-processing system. Thus, future research might be directed toward addressing how information processing can evolve in RNA-like replicator systems.

Highlights.

▶ We review computational studies on prebiotic evolution. ▶ We consider the maintenance, acquisition, generation, and storage of information in RNA-like replicator systems. ▶ We pose a future question for theoretical studies on the origin of life.