Transparency in metabolic network reconstruction enables scalable biological discovery

Abstract

Reconstructing metabolic pathways has long been a focus of active research. Now, draft models can be generated from genomic annotation and used to simulate metabolic fluxes of mass and energy at the whole-cell scale. This approach has led to an explosion in the number of functional metabolic network models. However, more models have not led to expanded coverage of metabolic reactions known to occur in the biosphere. Thus, there exists opportunity to reconsider the process of reconstruction and model derivation to better support the less-scalable investigative processes of biocuration and experimentation. Realizing this opportunity to improve our knowledge of metabolism requires developing new tools that make reconstructions more useful by highlighting metabolic network knowledge limitations to guide future research.

Introduction

Mapping metabolic pathways has been a focus of significant scientific efforts dating from the emergence of biochemistry as a distinct scientific field in the late 19th century [1]. This endeavor remains an important effort for at least two compelling reasons. First, cataloguing and characterizing the full range of metabolic processes across species (which because of genomics are being discovered at an incredible pace) is a fundamentally important step towards a complete understanding of our ecological environment. Second, mapping metabolic pathways in organisms – many of which can be found with specialized properties shaped by their environment – facilitates metabolic engineering to advance nascent industrial biotechnology efforts ranging from augmenting/replacing petroleum-derived chemical precursors or fuels to biopharmaceutical production [2]. However, despite laudable efforts to enable high-throughput “genomic enzymology” [3], the traditional biochemical approaches of enzyme expression, purification, and characterization remain time-, capital-, and labor-intensive, and have not expanded in scale like our ability to identify and characterize life genomically. Characterizing new metabolic function is further hampered by the challenge of cultivating environmental isolates in laboratory conditions [4]. Fortunately, recent efforts to leverage genome functional annotation and established knowledge of biochemistry have enabled the computational assembly of “draft metabolic reconstructions” [5], which are parts lists of metabolic network components. In this context, a reconstruction is not just the information embodied in the stoichiometric matrix describing metabolic network structure, but also the associated metadata and annotation that entails an organism-specific knowledge base. Such a reconstruction can serve as the basis for making functional models amenable to mathematical simulation. Thus, a reconstruction is a bottom-up assembly of biochemical information, and a model can serve as a framework for integrating top-down information (for example, model constraints can be generated from statistically inferred gene regulatory networks [6]). Such computational approaches are significantly faster and less expensive than biochemical characterization [7]. They are also providing new resources facilitate cultivation of novel environmental isolates [8], and the scope of draft metabolic network coverage across the biome has increased much faster than wet lab characterization. If the distinction between reconstruction and model formulation can be strengthened and supported through software implementation, there is great opportunity for using both tasks to further advance rapid discovery of biological function.

The iterative process of manual curation of a draft metabolic network reconstruction to assemble a higher confidence compendium of organism-specific metabolism (a process termed “biocuration” [9,10]) remains time- and labor-intensive. Biocuration of metabolic reconstructions currently advances on a decadal time scale [11,12]. Thus, much research effort has focused instead on developing techniques for rapid development of models that are amenable to simulation [13,14]. Thousands of models have been derived from automatically assembled draft reconstructions [15], but most of these models consist of highly conserved portions of metabolism since they are propagated primarily via orthology. Though the number of models is large, they do not reflect the true diversity of cellular metabolic capabilities across different organisms [16]. Applying the rapid and scalable process of draft network reconstruction to support and accelerate the less-scalable processes of biocuration and in vitro or in vivo experimentation remains an unrealized opportunity. The path forward should focus on increased emphasis on transparently documenting the reconstruction process and developing tools to highlight, rather than obscure, knowledge limitations that ultimately cause limitations to model predictive accuracy.

More explicit annotation of metabolic network reconstruction and model derivation steps can help direct research efforts

The biocuration process of assembling biochemical knowledge from genomic annotation and published literature (i.e., assembling a reconstruction) involves identifying and resolving ambiguity inherent in information generated through ongoing experimental efforts to characterize biological systems. This bottom-up reconstruction process often introduces implicit hypotheses in the reconstruction. Such hypotheses, if made explicit, could be usefully exploited to prioritize experimental efforts. For example, reconstruction requires selection of a specific genome assembly, selection of a homology threshold for functional annotation, and interpretation of published literature regarding pathway information. Information about how these choices are made during the course of reconstruction can be very useful for informing subsequent research efforts, but such information is currently difficult to find because it is seldom included in published reconstructions. Furthermore, as has been highlighted in a recent review [17], most software currently available for assembling a reconstruction does not support the detailed level of annotation that would be needed for scalable hypothesis generation.

Testing implicit hypotheses arising from reconstruction assembly provides one opportunity for guiding experimental efforts. However, the very act of identifying ambiguous information in the literature should also be exploited to contribute to experimental efforts, independent of the choices a researcher makes in assembling a reconstruction. Preliminary steps to facilitate large-scale computational identification of biological uncertainty have been made, such as the development of the Evidence Ontology [18]. However, realizing the potential for using reconstruction assembly to highlight experimental opportunities will require a broader shift to emphasize the limits of our knowledge, rather than only the predictive power of a model that can be derived from a reconstruction. Computational reconstruction of metabolic networks provides two distinct opportunities for guiding experimental efforts even before a mathematically computable model is derived from the assembled knowledge: highlighting areas of uncertainty in the current knowledge of an organism, and introducing hypotheses of metabolic function as choices are made throughout biocuration efforts.

The subsequent process of deriving a mathematically computable model from a reconstruction provides additional opportunities for scalable hypothesis generation that could be exploited to inform experimental efforts. While stoichiometrically constrained models derived from reconstructions are “parameter-light” when compared to dynamic enzyme kinetic models, they are not really “parameter free” [19]. As modelers derive a model from an assembled reconstruction, they must make choices. And, like the ambiguities and choices that are made and should be highlighted in assembling a reconstruction, highlighting the choices made in deriving a model provides further opportunity for scalable hypothesis generation. Examples of choices that often arise in deriving a functional model include adding intracellular transport reactions, filling network gaps, or trimming network dead ends to improve network connectivity [20]. Researchers seeking to conduct Flux Balance Analysis (FBA) [21] or similar approaches must formulate an objective function, can include testable parameters such as ATP maintenance requirements, and can compare model predictions to designated reference phenotype observations. Each of these model-building and tuning activities presents opportunities to rapidly develop and prioritize new hypotheses of metabolic function – for example, if a model required the addition of an intercompartmental transport reaction to function, this mathematical necessity would suggest opportunity to improve genomic annotation or to discover a previously uncharacterized enzyme or function.

Metabolic network gaps may be filled algorithmically by optimizing for shortest path connections [22] or through other penalties, such as prioritizing reactions catalyzed by enzymes that have higher homology for functional genomic annotations [23]. Either approach generates a list of reactions that permit network flux computationally, but are only predicted to have the modeled biological function. Similarly, identifying network dead ends presents opportunities to shine a light on the understudied portions of “dark metabolism”. A close look at areas where the metabolic network remains unconnected can lead to surprising discoveries, such as the recent report of a riboneogenesis pathway in the central carbon pathway of yeast that could enable a flux bypass of the oxidative phase of the pentose phosphate pathway [24]. This function had not previously been documented despite the fact that glycolysis in yeast is perhaps the most extensively characterized pathway in the biosphere. It is thus extremely likely that many such surprises remain in even “well-studied” metabolic networks – and reconstruction biocuration and functional model development have great potential for facilitating such investigation.

To date, the process of selecting parameters such as gap-filling choice, biomass objective function definition, and constraint application during the model building process has been fairly implicit. Making the parameter selection process in model development more explicit provides a pathway to increase the speed of large-scale metabolic function discovery and characterization. Realizing the potential of reconstruction biocuration and model building for scalable hypothesis generation requires a traceable and reproducible method and software infrastructure for assembling a reconstruction, and for each step of deriving models from a reconstruction. Software should facilitate answering questions such as: What network gaps were filled, from what source, using what method? Why were those gaps filled? What dead ends were trimmed? How was the objective function formulated? What constraints were applied, and why? Is this set of constraints believed to be unique? How were reactions compartmentalized? What transport reactions were introduced? What changes were made for simulating different conditions?

Answering such a broad range of questions in a biocuration or subsequent model building effort is greatly facilitated by extensive annotation in a standardized data structure that can be used with a variety of software platforms. However, current standards for structured data formats that enable publishing and exchanging models (such as the Systems Biology Markup Language (SBML) [25] or the Minimal Information Required in the Annotation of Models (MIRIAM) standard [26] are designed for model exchange, rather than reconstruction exchange. It is likely that exchange and annotation of a reconstruction requires a database schema definition, rather than a markup language, which may be more suitable for exchange of functional models. This need is becoming more pressing as the scope of network reconstruction extends from metabolism to include Macromolecular Expression (ME) models [27]. One reconstruction schema that supports extensive annotation and provenance tracking has been defined as part of the Pathway Tools software [28], but this schema has not yet been adopted more widely or supported by other software implementations, nor has this schema yet been extended to support ME models or reconstructions. Similarly, there are also methods for deriving functional models from pathway databases such as those encoded by Pathway Tools [29], but these methods have not yet been standardized across the research community. Finding the necessary balance between standardization and the flexibility to enable research efforts and development of new methods remains an ongoing process of dialogue within the metabolic network research community.

Reconsidering model performance for scalable hypothesis generation

Model predictions of single gene deletion viability have been the key metric used for evaluating the performance of metabolic network models. However, over-emphasizing “improvements” to this metric carries risk of model implementation approaches that can hinder biological discovery. In contrast, careful consideration of differences between model prediction and in vivo observations provides another abundant opportunity for scalable hypothesis generation from metabolic network model building. It can be surprisingly difficult to generate a reference set of essential genes that can be used for evaluating model development. Even in genetic model organisms like Saccharomyces cerevesiae, recent work has highlighted that genes can be conditionally essential [30]; that the deletion of a single gene makes additional mutations more likely [31]; and that auxotrophic markers in laboratory strains meaningfully affect metabolic phenotype [32]. Thus, a model that has high predictive accuracy for one set of “essential” genes may in fact be over-fit, and have unexpectedly limited predictive ability of gene essentiality in non-reference environments. If recognized and highlighted, rather than obscured in an effort to improve apparent model prediction accuracy for each subsequent publication, such limitations can be usefully exploited for investigating which aspects of gene essentiality arise from stoichiometric constraints of the metabolic network itself, and which arise from other causes. Since “models based on pathway stoichiometry alone can only be used to ask questions that do not depend on more complex features” [19], model predictions that differ from observation may suggest biological functions arising from other factors.

Thus, models can be over-fit to improve performance by one metric at the expense of generality. There can be an inherent conflict between the goals of 1) assembling a reconstruction to reflect established knowledge (includes gaps, and highlights uncertainty) and 2) building a more focused model to be analyzed with a given performance metric. The same reconstruction can be used to derive models with high descriptive accuracy at the expense of predictive ability as well as models with better predictive ability but worse prediction of a reference observation. Use of condition-specific constraints makes the model more descriptive, but model developers must be conscious that applying such constraints move the scope of a model beyond stoichiometry, and can come at the expense of predictive power; such “models are not necessarily predictive but instead have a scoping nature by allowing us to assess what is metabolically feasible” [6].

Concluding remarks

The effort to computationally reconstruct biochemical knowledge to compile organism-specific reconstructions, and to derive computable models from these reconstructions, is a relatively young field of research with abundant opportunity for facilitating biological discovery of metabolic function. Judgment is required in assembling a reconstruction, and there should be careful consideration of the fact that judgment calls represent an implicit hypothesis. Making these hypotheses more explicit would help guide subsequent investigation. Bernhard Palsson and colleagues call for “an open discussion to define the minimal quality criteria for a genome scale reconstruction” [16] – an effort we fully support. We believe that such a beneficial “minimal quality criteria” should be guided by the goals of reproducibility and transparency, including those aspects that can help to guide discovery of novel gene functions. A structured format and software tools that facilitate transparent reconstruction, specific model derivation, and a variety of model test metrics would facilitate biological knowledge generation, enable new research approaches, and improve model utility. To assess the quality of a reconstruction, it is critical to have detailed and accurate information on how it was assembled, such as knowing which choices were made in generating a simulatable model, and what methods were used to inform such choices.

Highlights.

• Assembling a network reconstruction can reveal knowledge gaps

• Building a functional metabolic model enables testable prediction

• Recent work has found that most models contain the same reactions

• Reconstruction and functional model building should be explicitly separated