Computational Models in Systems Biology: Standards, Dissemination, and Best Practices

Abstract

Mathematical and computational models are a key technology in systems biology. Progress in the field depends on the replicability and reproducibility of their properties and behavior. For this, an essential requirement is a set of clear standards for model specification and dissemination. This review covers existing standards, and it highlights the most important areas where further work is required. This includes the specification of agent-based models, an increasingly common modeling approach.

Graphical Abstract

Introduction

Dynamic computational models represent a key enabling technology in systems biology that provides a powerful methodology for encoding the interactions of the different parts of a biological system as they generate its emerging dynamics. As with experimental approaches, model-based research must be reproducible and credible. For maximum impact and efficiency, models must be shareable and transparent, especially as the need for large-scale integrated models grows, such as for the development of medical digital twins [1]. This naturally raises the issue of best practices in model specification and dissemination. This brief critical review describes the current state of affairs, some successes, and the most pressing needs to be addressed.

One of the most prevalent modeling techniques has been systems of ordinary differential equations (ODEs). ODEs have a long tradition as a modeling technique for, e.g., biochemical reaction networks or gene regulatory networks, and have benefitted from sustained and largely successful attempts to develop best practices and standards for their specification. But other model types also have gained popularity, such as discrete models of various types, constraint-based models, and, in particular, agent-based models (ABMs). An additional specification challenge arises when considering multi-scale hybrid models, especially those that combine both mechanistic and data-driven models. Although such models are of increasing importance [2], they are out of the scope of this review.

Model credibility is important in systems biology, whether models are used to guide experiments, or drug development, or patient treatment optimization. This is particularly urgent as many data-driven models are black-box, and the sheer complexity of biological systems makes reverse-engineering from a relatively small set of observations difficult [3]. Replicability and reproducibility, directly connected to credibility, are generally considered challenges of both experimental and computational research [4, 5]. Even for models based on systems of differential equations, reproducibility is far from assured, as was shown in a study based on the BioModels database [6]. The situation for computational simulation models, such as ABMs, is considerably worse. While the problem of reproducibility can be somewhat alleviated by listing all the details of the ABM construction, the sheer complexity of describing these is a significant problem [6, 7]. Here, we focus primarily on the issue of standards for model specification.

Existing Model Specification Standards

Several ongoing standards development efforts are coordinated by the Computational Modeling in Biology Network (COMBINE) [8]. This includes the Systems Biology Markup Language (SBML) [9,10], the Systems Biology Graphical Notation (SBGN) [11–13], and others. Arguably, SBML, under development for two decades, represents the state of the art in this subject, even though it only covers models using systems of ordinary differential equations, with extensions that cover constraint-based models [14], logic models, and rule-based models [13]. It is commonly used in model databases, such as BioModels [15], which includes several thousand models. And it plays an important role in a larger ongoing effort to improve reproducibility of model-based results [16]. In 2013, a community effort resulted in an extension, SBML qual, of SBML to a wide class of time- and state-discrete models [17], that includes logical models, in particular Boolean networks and their generalizations, summarily referred to as qualitative models. It is used in model databases, such as the Cell Collective [18].

Rule-based computational models

Among the modeling techniques not covered by the standards discussed above, and not currently included in initiatives focused on model reproducibility, such as the recently established Center for Reproducible Biomedical Modeling, are rule-based models of multicellular and other systems which are becoming increasingly common. This includes, most prominently, agent-based models (ABMs). Only minimal mathematical expertise is needed to construct them, especially with the availability of low-entry barrier general purpose software platforms, such as Netlogo [19] for agent-based models. Even more complex platforms for virtual tissue models, such as CompuCell3D [20] or PhysiCell [21] are accessible to domain experts without extensive modeling experience. They make it easy to simulate stochastic processes in spatially heterogeneous environments, and require few general assumptions, such as systems being well-mixed in the case of many ODE models. The initial popularity of ABMs in the social sciences, ecology, and epidemiology expanded later to systems biology and biomedicine, including a wide range of applications, such as respiratory diseases [22], tumor growth [23], and liver fibrosis [24]. The cost of such ease of use and broad applicability, coupled with the absence of a mathematically rigorous formalism, is that standards and a formalism for model specification are exceedingly difficult to develop. The complexity of this issue is compounded by the increasing use of ABMs within multiscale models, combining, e.g., ODE-based intracellular models with ABM tissue models (see, e.g., [22]). Thus, there is an urgent need for model specification standards for this modeling technique to ensure reproducibility of results.

Although ABMs, sometimes referred to as individual-based models, are widely used, the state of the art in model specification leaves much to be desired [25]. In essence, ABMs are simulation models encoded in a variety of different computer languages. The most common method of specifying ABMs is text-based, whereas the actual code is not provided in the majority of cases [26, 27]. Even when both are available, it is far from straightforward to verify the concordance between the text-based description and the implementation into computer code, and this is rarely done at the manuscript reviewing stage [28]. Less than half of the publications reporting ABMs explicitly mention what programming language or platform was used, further limiting the ability to replicate results [27], let alone reproduce them. It is difficult to interpret conclusions from an ABM if emergent behavior differs significantly depending on which platform the model has been implemented on, as has been shown to happen using a model from ecology [29]. Lastly, a significant problem with describing models in natural language is that there could be many ways to implement the same procedure described in natural language [28, 30].

The main tool available for ABM model specification is the so-called Overview, Design Concepts, and Details (ODD) protocol [31–33], first introduced in the context of ecological modeling. It is a text-based description (in particular, not necessarily machine-readable) giving an overview of the model, entities, and processes driving the behavior of the individual entities in the model, in a prescribed format that can be effective in specifying model logic, when crafted carefully. However, the ODD protocol has been criticized as not being comprehensive or precise enough for making ABMs truly reproducible, and does not require platform-specific details for the replicability of models [34]. And, as mentioned above, it does not address the important issue of being able to map this description onto the actual simulation code. The development of automated linking between written descriptions in the ODD protocol and software implementing the model to reduce ambiguity has been suggested in the latest update of the ODD protocol, although the authors discuss several cautions in doing so [33].

An effort to specify ABMs in a graphical way is the usage of class diagrams in the Unified Modeling Language (UML), primarily used to describe object-oriented software [35]. Interestingly, in the first update of the ODD protocol for ABMs, it was suggested to not use UMLs to describe ABM structure, to make sure that ODDs are independent of how models are actually implemented [32]. More recently, there have been discussions about representing ABMs in SBML [36], although to our knowledge, there is no current standard way of doing so, limiting the ability of sharing, testing, and replicating ABMs.

Conclusions and future directions

Since early on, it has been recognized that computational models require specification standards to ensure replicability and reproducibility of model simulations. Much progress has been made for models based on systems of ordinary differential equations, owing both to their ubiquity and to their well-defined universal mathematical structure. The Systems Biology Markup Language, together with packages that extend the capability of SBML core, can be considered the gold standard of model description, despite some remaining challenges. Other relatively common model types, such as constraint-based models and logical models are also covered by extensions of SBML. The main outstanding challenge concerns ABMs, for which currently the only general standard available is the ODD protocol.

A concerted effort is needed to transfer the lessons learned from the development of existing standards to ABMs. The need for standards extends from text-based description to the computer code that represents the actual simulation model. Standards have to enable the unambiguous mapping of components in the text to matching components in the code. Ideally, tools should be developed to create machine-readable standards for ABMs. Another important step is the organization of model repositories for ABMs beyond platform-dependent repositories such as Netlogo, similar to BioModels for ODE and constraint-based models, and Cell Collective for discrete models. For this to be practical, standards for model description are also essential. We encourage the systems biology community to take on this challenge.

One possible starting point could be to use the modular structure for simulation models described in [37], which can provide a template for both the text description and the code structure. The key feature of this modular structure is the strict separation of model entities and the data describing the model state. All computational algorithms operate on a global model state that contains all data, all parameters, and all spatial specifications. Algorithms do not interact with each other directly and do not exchange data. Instead, they interact with each other indirectly by modifying overlapping data fields. This decomposition of the simulation into modular pieces, together with an ODD-like description, introduces a clear and discernible structure into the model specification.

Figure 1.

Work still needs to be done in order to make models truly reproducible, although much progress has been made for models based on ODEs. Standardization of ABMs is still in its infancy, limiting studies of the replicability and reproducibility of such models and, as a result, limiting their credibility and extendibility.