Evolution of microbial genomics: conceptual shifts over a quarter century

Abstract

Prokaryote genomics started in earnest in 1995, with the complete sequences of two small bacterial genomes, those of Haemophilus influenzae and Mycoplasma genitalium. During the next quarter century, the prokaryote genome database has been growing exponentially, with no saturation in sight. For most of these 25 years, genome sequencing remained limited to cultivable microbes. Together with next generation sequencing methods, advances of metagenomics and single cell genomics have lifted this limitation, providing for an increasingly unbiased characterization of the global prokaryote diversity. Advances in computational genomics followed the progress of genome sequencing, even if occasionally lagging behind. Several major new branches of bacteria and archaea were discovered including Asgard archaea, the apparent closest relatives of eukaryotes and expansive groups of bacteria and archaea with small genomes thought to be symbionts of other prokaryotes. Comparative analysis of numerous prokaryote genomes spanning a wide range of evolutionary distances changed the conceptual foundations of microbiology, supplanting the notion of species genomes with fixed gene sets with that of dynamic pangenomes and the notion of a single Tree of Life with a statistical tree-like trend among individual gene trees. Strides were also made towards a theory and quantitative laws of prokaryote genome evolution.

The birth of microbial genomics

In the fall of 1995, 25 years before the time of this writing, J. Craig Venter‟s research institute (then The Institute for Genome Research, TIGR) released the first two complete sequences of bacterial genomes, both from opportunisitc human pathogens, the 1.78 Mbp genome of Haemophilus influenzae [1] and the 0.57 Mbp genome of Mycoplasma genitalium [2]. Comparison of these two small bacterial genomes inspired an attempt to reconstruct the minimal genetic complement of life that was inferred to consist of about 250 genes [3]. The first sequenced genome of an archaeon, Methanococcus (currently, Methanocaldococcus) jannaschii, followed promptly [4], along with several additonal bacteria including the photosynthetic cyanobacterium Synechococcus sp [5]. Within 3–4 years, the exponential growth of the collection of complete bacterial and archaeal genomes has settled in [6] (Figure 1). The comprehensive comparative analysis of microbial genomes that started as soon as the first two genomes became available made it clear that 70–80% of the genes in each genome were highly conserved in evolution such that orthologs could be identified in distantly related bacteria and/or archaea [7]. Thus, the genomes of prokaryotes provided ample material for functional and evolutionary inferences conducive to experimental validation. The flourishing research field of microbial genomics was born. In this short review, we briefly address several achievements of microbial genomics that appear most impactful as well as the major challenges in this field.

Figure 1. Exponential growth of the prokaryote genome database.

Number of genome assemblies at different assembly levels (ftp://ftp.ncbi.nih.gov/genomes/ASSEMBLY_REPORTS/) in the GenBank section of the NCBI Assembly database. The data points correspond to the end of the respective year; the X-axis starts with 1995, the year when the first two complete genomes were published.

Complete genome: all chromosomes are fully assembled with gaps not exceeding 10 ambiguous bases.

Chromosome: all chromosomes are fully assembled, but possibly containing gaps or unlocalized scaffolds.

Scaffold: sequence contigs are connected across gaps, but not placed on the chromosomes.

Contig: only unconnected contigs are available.

Advances and pitfalls of computational genomics

Genome sequences are useless unless adequate capabilities exist for the annotation and comparative analysis of genomes. At the time the first prokaryote genomes came along in the mid 1990s, the computational biology community was at best partially prepared for the genomic revolution, but developments followed quickly. Computational methods for gene prediction are essential in genomics but, in the case of prokaryotes, are relatively straightforward because of the dense coverage of these genomes by largely non-overlapping protein-coding genes [8–11]. Arguably, the two key developments in computational microbial genomics were the approaches for the identification of orthologous genes and the utilization of genome context information for functional inferences. Although the concept of orthology – evolutionary relationship between genes derived from the same ancestral gene in the most recent common ancestor of the compared organisms – was introduced by Walter Fitch as early as 1970 [12], orthology became an important concept only with the appearance of complete genome sequences because orthologous relationships cannot be reliably identified on incomplete gene sets [7, 13, 14]. Notwithstanding all the limitations imposed by the complexity of biological processes, the orthology conjecture, which posits that orthologous genes are responsible for equivalent functions in the respective organisms, generally, appears to hold [15]. Therefore, clusters of orthologous genes (COGs) can serve not only as the units for the reconstruction of microbial genome evolution but also as the most adequate platform for functional annotation of the genomes [7, 16–20]. As pointed out above, 70–80% of the protein-coding genes in a typical prokaryote genome are conserved across long evolutionary distances, and for most of these, biological function,at least, in general terms, can be reliably assigned by automatic comparison to position-specific scoring matrices or hidden Markov models derived from multiple alignments of protein sequences in well-curated collections of COGs. Superimposing the patterns of gene presence-absence in COGs onto phylogenetic trees for the most highly conserved genes (see below), the history of gene gain and loss in microbial genomes can be reconstructed using either maximum parsimony or the more robust maximum likelihood methods [21–25]. Given the highly dynamic character of the prokaryote evolution, such reconstuctions are an essential approach in microbial evolutionary genomics.

A substantial fraction of genes in any prokaryote genome are organized into operons, arrays of cotranscribed genes that are typically involved in the same pathway or process [26–28]. The operonic architectures are only partially conserved among distantly related prokaryotes, such that comparison of operons from diverse organisms yields networks of (potentially) functionally linked genes [29, 30, 31]. Hence the “guilt by association” approach that allows systematic prediction of the functions of “hypothetical” microbial genes based on their consistent linkage with functionally characterized genes [32–35]. With the growth of the collection of genomes across a broad range of evolutionary distances, the guilt by association approach has evolved into a powerful strategy for the discovery of new functional systems, particularly, those involved in highly variable functions, such as biological conflicts and signal transduction. The recent discovery of numerous, enormously diverse antivirus defense systems that tend to form distinct islands in prokaryote genomes may be considered the prime case in point for these approaches [36–39].

The progress in computational genomics over 25 years has been momentous, but serious problems persist and are even exacerbated by the rapid accumulation of genome sequences, many of them incompletely assembled. The most general and, apparently, most damning outstanding problem is that the exponential growth of genomic databases necessitates a near complete automation of genome analysis procedures to replace the combination of automatic and manual, case by case analyses, which was the most efficient approach in the early days of genomics. This results in notorious error propagation both in the construction of COGs and in the downstream genome annotation [40–42]. Identification of orthology is straightforward for highly conserved, single-copy genes with conserved domain architectures but remains an incompletely resolved challenge for faster evolving gene families with complex histories that include domain rearrangements, lineage-specific amplification of paralogous genes and gene loss as well as multiple horizontal gene transfers (HGT). Devising and implementing robust and efficient, phylogeny-based algorithms for this task and creating reliable, regularly updated databases of orthologous gene clusters remains a key task for computational genomics.

Metagenomics and single cell genomics usher in a new revolution in microbiology

The completion of the first microbial genome sequences was brought about by the perfection of whole genome shotgun (WGS) technique which remained the principal genome sequencing method for about a decade and a half since then. However, over the next few years, WGS has been nearly completely supplanted by Next Generation Sequencing (NGS) which brought to the table unprecedented sequencing depths but also hard problems with sequence assembly [43–45]. Once these difficulties have been largely overcome thanks to new, highly efficient assembly algorithms, such as Spades, assembly of numerous nearly complete genomes from metagenomics and single cell genomics data has become realistic [45–48].

Given the estimates indicating that less than 0.1% of prokaryotes represented in most environments can grow in culture [49, 50], metagenomics ushered in a revolution in microbiology by allowing unbiased, genome level surveys of microbial diversity. The Tara Ocean project that broadly explored the marine prokaryote diversity is a prime example of such a global survey [51, 52], and the exhaustive analysis of the human gut microbiome is another strong case in point [53]. In parallel, the advances of sing-cell genomics provide for partial sequencing of thousands of prokaryotic genomes across diverse habitats, allowing taxonomic assignment for the majority of metagenomic reads [53]. Perhaps, even more importantly, metagenomics and single-cell genomics brought about the discovery of entire major groups of uncultivable bacteria and archaea that shed new light on major aspects of microbial physiology, ecology and evolution as discussed in the next section.

All these remarkable advances notwithstanding, metagenomics changed the very notion of what a microbial genome sequence is because, in virtually all cases, it is impractical to assemble a closed circular chromosome sequence from metagenomic contigs. Thus, at present, when a recently sequenced “genome” of a bacterium or an archaeon is reported, by default, this implies a collection of contigs that have been placed in the same bin based on statistical properties of the contig sequences, such as oligonucleotide frequencies. In the current genome sequence databases, the number of such conditionally “complete” genomes at the scaffold or contig level exceeds the number of literally complete, closed genomes by orders of magnitude (Figure 1). This state of genome sequencing puts an extra onus on computational approaches to genome analysis and annotation. In particular, numerous contaminations to prokaryotic genomes ensue, stimulating the development of dedicated computational decontamination methods [54]. Evidently, it is highly desirable that, for each new group of prokaryotes that is discovered through metagenomics and single cell genomics approaches, at least a few truly complete, closed genomes were sequenced, to alleviate the concerns of possible incompleteness and contamination, and to obtain an example of the full gene repertoire for the given group of organisms.

Discovery of novel major groups of bacteria and archaea: impact on our understanding of microbial physiology, ecology and evolution

At least two momentous discoveries were enabled by the advances of metagenomics. First, metagenomic sequences allowed the delineation of two expansive groups of bacteria and archaea with small genomes, between 0.5–1 Mb that are thought to be symbionts (parasites, commensals or mutualists) of other prokaryotes [55–58]. Because of their apparent dependence on the respective hosts and despite their ubiquity in various environmental habitats, these microbes have been missed in the pre-metagenomic era (or detected only in surveys of the 16S rRNA diversity and not characterized in any detail). The only prominent exception was Nanoarchaeon equitans which was originally discovered through the observation of tiny cocci attached to the cell surface of the crenarchaeon Ignicoccus hospitalis and subsequently sequenced using the DNA isolated from a co-culture with I. hospitalis [59]. With the advent of metagenomics, it became apparent that N. equitans belonged to a large group of archaea with small genomes that form a clade in most phylogenetic trees and are known as the DPANN superphylum (named after the 5 constituent major groups of archaea: Diapherotrites, Parvarchaeota, Aenigmarchaeota, Nanoarchaeota and Nanohaloarchaeota), which includes several phylum-level branches [56, 60–62]. Parallel to the discovery of DPANN, an expansive branch of bacteria with similar genome sizes has been discovered and became known as Candidate Phyla Radiation (CPR), or later, Patescibacteria [60, 63]. Genome analysis indicates that both DPANN archaea and Patescibacteria encode a minimum of metabolic enzymes and therefore have to depend on other microbes for most metabolites [55, 57]. However, for most of these bacteria and archaea, it remains unclear whether their tiny cells are actually attached to host cells or are simply members of complex microbial communities with an extreme dependence on other members [57].

The second major discovery of metagenomics is the Asgard superphylum that includes a broad variety of archaea (all named after Norse deities) that encode numerous Eukaryotic Signature Proteins (ESPs), that is, homologs of proteins involved in characteristic eukaryotic functional systems, such as endomembranes and cytoskeleton. In most phylogenies of universal genes, eukaryotes cluster with or even within Asgard archaea, suggesting that these are the closest archaeal relatives of eukaryotes [64–66]. The first cultivation of an Asgard archaeon has been reported after a substantial diversity of Asgards has already been revealed by metagenomics, demonstrating synthrophy with Delta-proteobacteria and methanogenic archaea, consistent with the syntrophic scenarios for the origin of eukaryotes [67–69].

Taken together, the discoveries of DPANN and Asgard archaea, and Patescibacteria highlight the potential of metagenomics to reveal entire new continents in the microbial world. It is equally obvious, however, that genome analysis can take microbiologists only so far, and biological follow-up is essential.

The conceptual shift in microbial evolutionary genomics: horizontal genomics, statistical tree of life and dynamic pangenomes

Apart from the characterization of the microbial diversity and discoveries of new major groups of bacteria and archaea, genomics (and later, metagenomics) have transformed the most fundamental concepts of the structure and dynamics of the microbial world. This conceptual shift was triggered by the observations, made shortly after the first several complete genomes of bacteria and archaea were sequenced, that phylogenies of different conserved genes had different topologies. The differences were found to be so extensive that they could not be explained away by methodological artifacts, leading to the conclusion on extensive HGT being a key factor in the evolution of prokaryotes [5, 70–73]. Hence the idea of giving up the concept of the Tree of Life (ToL) for a net of life, devoid of any vertical component, gained much ground, leading to vivid debates on the extent of “horizontal genomics” [74–77]. Nevertheless, a comprehensive comparison of the topologies of the phylogenetic trees for individual conserved genes demonstrates the existence of a statistically significant consensus tree-like trend in the evolution of prokaryotes, even though the gene flow is quantitatively dominated by HGT [78–80]. Thus, the ToL survived the genomic revolution, but in a transformed version, as a statistical trend within the forest of individual gene trees, rather than a definitive representation of genome evolution [81].

The second conceptual shift in microbiology is the emergence of the concept of the dynamic pangenome, that is, the entirety of the genes found in all representatatives of a prokaryotic species (notwithstanding the difficulty of defining the latter) [82–84]. The majority of the bacteria and archaea have open pangenomes, that is, sequencing of genomes of new isolates adds a set of new genes that were not detected in the previously available genomes from the same species, without obvious signs of saturation (Figure 2). This trend, certainly, does not imply that the pangenomes are infinite, but does indicate that most of them include orders of magnitude more genes than a typical individual genome [85]. Thus, in most bacteria and archaea, the relatively small, conserved core genome is associated with numerous accessory genes that comprise the bulk of the pangenome. However, the estimated size of pangenomes across the diversity of archaea and bacteria varies in within a broad range, with some having closed pangenomes that effectively saturate after a small number of isolates are sequenced (Figure 2).

Figure 2. Pangenomes of prokaryotes.

The plot (rarefaction curves; double logarithmic coordinates) show the increase in the total number of genes with the addition of new genomes for four clades of closely related bacteria. The points show the medians of the numbers of families of orthologous genes in 100 randomly sampled subsets of genomes within a clade. The clades represent four bacterial clusters from Alignable Tight Genome Clusters databse (ATGCs) [95]. ATGC001 is a cluster of 432 genomes from Escherichia, Salmonella, Enterobacter and other closely related families; ATGC052, 109 genomes of Staphylococcus aureus and S. argenteus; ATGC143, 103genomes of Campylobacter jejuni and C. coli; ATGC021, 73 genomes of Chlamydia trachomatis and C. muridarum.

Quantitative laws and theory of microbial genome evolution

Comparative analysis of prokaryote genomes has yielded several quantitative regularities that can be construed as “laws” of evolutionary genomics (Figure 3) [86]. The crucial corollary of the discovery of pangenomes is that the key evolutionary process in prokaryotes is not point mutation but rather gene replacement via HGT and gene loss. A plot of the gene commonality (that is, sharing of orthologs) in any set of prokaryote genomes shows a universal, skewed U-shape, regardless of the evolutionary distances between the compared genomes (Figure 3a) [87]. This universal curve consists of three components that correspond to the small core of nearly universal genes, the much larger ‘shell’ of modetately conserved genes, and the huge ‘cloud’ of rare and unique genes (also known as ‘orphans’) (Figure 3a). The proportions between the components dramatically differ on the phylogenetic depth of the group; at the domain level (Bacteria or Archaea), the core consists of 100–200 genes, the shell contains thousands of genes, whereas the cloud of rare genes reaches into hundreds of thousands and, at the current state of sampling, is effectively unbounded, in line with the openness of most prokaryote pangenomes. The formation of this distinct plot shape can be accounted for by a stochastic model of genome evolution with non-uniformly distributed genes replacement rates, that is, differential effect of selection on different genes [88].

Figure 3. Quantitative laws of prokaryote genome evolution.

The schematic plots show: A. The universal gene commonality distribution. Top: prokaryote domains level (across multiple phyla); bottom: ATGC level (across closely related species or genera) [95]. Dashed lines show the approximate contributions of the individual components (cloud, shell and core); the solid line shows the observed combined distribution.

B. Two classes of prokaryote genes with slow and near-instantaneous replacement rates. Dashed lines show the exponential decay of the fast- and slow-decaying components; the solid line shows the observed combined fraction of gene families that are shared at different evolutionary distances.

C. Differential scaling of functional classes of prokaryote genes with the total number of genes in a genome

Mathematical modeling of prokaryote genome evolution by gene replacement shows that, to fit the observed dynamics of gene commonality decay during evolution, it is necessary to introduce two classes of genes [89]. The first class, combining the core and shell of the universal commonality distribution, includes about 90% of the genes in each genome that are replaced relatively slowly, whereas the second class (the “cloud”) consists of the remaining 10% or so of the genes that are replaced virtually instantaneously, in comparison (Figure 3b). A notable inference from this model is that the prokaryote genome universe consists of billions of distinct genes. In an indepndent line of analysis, it has been shown that the rate of gene trunover in prokaryotic genomes is proprotional to the standing nucleotide variation in the population, which is compatible with the notion that replacement of accessory genes is a predominantly neutral process [90].

Different functional classes of prokaryote genes show distinct scaling exponents with the total number of genes [87, 91–93]. Genes involved in information processing (replication, transcription, translation) are characterized by sublineal scaling, metabolic enzymes and transporters scale close to linearity, whereas regulatory and signal transduction genes scale superlinearly (Figure 3c). These notable regularities in the evolution of the prokaryote genome content can be accounted for by a model that includes two distinct parameters, selection coefficient, which defines the gene loss rate, and genome plasticity that reflects gene gain [94].

The availability of multiple groups of closely related bacterial and archaeal genomes [95], combined with the realization that the evolution of prokaryotes occurs, primarily, via gene replacements, provided for the development of a basic population-genetic theory of prokaryote genome and pangenome evolution. The general theoretical model of genome evolution developed by Lynch holds that genome evolution is shaped by the power of selection (primarily, purifying selection) that itself depends on the product of the selection coefficient (s) and effective population size (Ne) [96, 97]. In organisms with large Ne, selection is efficient such that features with even small negative s values can be eliminated. In contrast, in organisms with small Ne, genetic drift is a major contribution to the evolutionary process, such that even moderately deleterious features are often fixed in the population. Prokaryotes typically live in large populations (Ne up to 10⁹), and so the theory predicts that, unlike the the genomes of multicellular organisms that accumulate large amounts of “junk” DNA, prokaryote should evolve under selection for streamlining because any piece of junk, even a small one, would be eliminated by purifying selection [98]. The dense packing of genes in prokaryote genomes appears to be compatible with this prediction. However, direct measurements of the strength of selection unexpectedly run afoul of the theoretical prediction. Selection in prokaryotes can be gauged by measuring the ratio of the non-synonymous to synonymous substitution rates (dN/dS) in multiple groups of closely related prokaryote genomes. Such measurements led to the unexpected conclusion on a highly significant negative correlation between dN/dS and the genome size in prokaryotes [99, 100] (Figure 4). In other words, in prokaryotes, the larger the genome, the stronger the protein-level selection. Stimulated by these findings, mathematical modeling of the evolution of prokaryote genomes by gene gain and loss has shown that the observed genome size distributions were best fit by models with positive, even if small, mean s values associated with gene gain [101, 102]. Thus, in the evolution of bacteria and archaea, the advantage of functional diversification conferred by the capture of new genes appears to often override the selection for genome streamlining, at least, up to a limit on the genome size.

Figure 4. Selection in the evolution of prokaryote genomes.

The schematic plots show:

A. The predicted and observed dependency of protein level selection, measured as dN/dS, on the total number of genes in a genome.

B. The utility hypothesis predicting that the gene gain rate decreases with the genome size because the relative impact on the fitness of a newly gained or lost gene is relatively smaller in larger genomes (diminishing return) whereas the intrinsic loss rate increases with the genome size because there are more genes to lose [101]. Vertical dashed lines around the equilibrium point indicate the range of the genome size expected to be observed in independently isolated genomes of the given species.

Along similar lines, capture of advantageous genes, in particular, those that ensure ability of microbes to explore new ecological niches, can lead to the expansion of pangenomes [103, 104]. Under this conceptual framework, organisms with large N_e tend to have larger pangenomes than those with small populations due to the strong positive selection driving fixation of acquired genes that confer even a slight fitness gain [105].

Conclusions

In the quarter century since its birth, microbial genomics evolved from a modest enterprise, where sequencing of each new genome was a feat in itself, to an expansive research field where discoveries stem from comparative analysis of hundreds or thousands of genomes. With the recent advances of metagenomics, the goal of completely charting the diversity of prokaryotes on earth, at least at coarse grain, might be within reach of the current generation of microbiology students. Above and beyond this striking quantitative progress, microbial genomics has transformed some of the fundamental concepts of evolutionary biology, replacing the notion of species genomes with fixed gene sets with that of dynamic pangenomes, and the single tree of life with the statistical tree-like trend in the forest of gene trees. Furthermore, comparative analysis of multiple genomes from many prokaryotic taxa provides for the discovery of quantitative laws of genome evolution and testing increasingly realistic theoretical models to explain the emergence of such laws. All these advances must not overshadow the hard challenges faced by microbial genomics, in particular, those associated with reliable automatic analysis of rapidly growing collections of genomes most of which in actuality are clusters of contigs rather than complete, closed genome sequences. Finally, we should never forget that genome analysis, however extensive, can only stimulate, augment and complement but by no means replace experimental microbiology.

• How can we radically improve automated genome annotation?

• How large is the diversity of bacteria and archaea on earth and how long will it take to chart it completely?

• What is the size of a typical prokaryote pangenome and what determines it?

• How common are host-parasite relationships between different prokaryotes?

• Are there domain-level or superphylum-level groups of prokaryotes remaining to be discovered?

Highlights.

• The database of prokaryote genomes has been growing exponentially for 25 years

• There is no saturation of prokaryote diversity currently in sight

• Most prokaryotes have dynamic, “open” pangenomes

• Metagenomics makes key contributions to the study of prokaryote diversity

• There is no single tree of life but a tree-like trend in evolution is discernible