Genetic network structure and dynamics: identifying simple negative feedback loops

Abstract

A broad array of experimental techniques have been used to determine the interactions between genes that regulate key cellular processes such as differentiation, metabolism and the cell cycle. The experimental studies are often complemented by development of models of varying degrees of complexity. We consider the ‘inverse problem’: to determine the underlying interactions based solely on the observed dynamics. In earlier work, we considered a specific class of ordinary differential equations that are continuous analogues of a Boolean switching network. We developed techniques to analyse and classify the dynamics based on their logical structure. We also developed techniques to solve the inverse problem. In the current work, we extend these earlier methods to analyse a model equation for a genetic network proposed by Cummins and colleagues. For a simple negative feedback system in which there is a cyclic interaction diagram with an odd number of inhibitory links, if the data is sampled at a sufficiently fine time scale with sufficient accuracy that maxima and minima can be determined, the structure can be deduced by considering sequences of maxima and minima. Alternatively, one can use the sequence of logical states found by discretizing the dynamics based on the first derivative of the variables as a function of time. The most useful technique for determining the interactions involves assessing the dependence of the rate of change of each variable as a function of the other variables, taken one at a time.

1. Introduction

Multicellular organisms are a mosaic of different cell types. A given cell type arises at a time and location that are precisely controlled. These events, as well as the specific activation of genes and consequent synthesis of proteins are controlled by genetic networks that are in turn controlled by a variety of factors that are both internal and external to each cell in the body. Jacob and Monod discovered that in bacteria specialized proteins called transcription factors could control gene expression by binding to specific DNA sequences that regulate transcription of specific genes. They further proposed that the same sorts of mechanisms for activation or inhibition of gene expression in bacteria could be strung together to make genetic control circuits that in principle could lead to diverse cellular functions such as differentiation and the cell cycle [1].

We call the determination, using gene expression data, of the genetic networks controlling cellular functions the inverse problem. Diverse cellular functions have been considered including differentiation, proliferation, senescence, apoptosis, motility and metabolism. One typically wishes to determine key genes that code for transcription factors involved in the control circuitry and the manner in which multiple controllers of a single site may interact with one another.

One approach involves modern techniques that gather large amounts of data and seek to leverage these data to infer the structure of genetic networks on a large scale. Data can be collected using a number of techniques including RNA-seq to measure gene expression, ChIP-seq to measure transcription factor–DNA binding sites, and knock-out and knock-in experiments. Examples include determination of putative binding sites for transcription factors in the ENCODE project [2] and the determination of genetic networks based on massive data sets of gene expression in different cell types using bioinformatic techniques, as reviewed by Marku & Pancaldi [3]. These methods are being developed by bioengineers and data scientists.

An alternative approach involves small scale network inference being carried out in systems biology and mathematical biology, aiming for accurate models of specific delimited systems. This approach has a sharper focus on identifying key transcription factors and other regulatory molecules that are involved in the control of specific circuits in specific settings and explicitly developing models for the underlying circuits that can then be simulated and compared with experimental data. Representative settings in which a logical network approach has been proposed include bacteriophage lambda [4], sea urchin [5], yeast [6–9], Arabidopsis [10] and Drosophila [11–15]. Results are typically reported as an interaction diagram, often accompanied by simulations of mathematical models that capture key elements of the control circuits. Recent reviews describe modelling genetic and cellular control using logical models [16–18]. Software has been developed to analyse and simulate Boolean models of complex networks [19]. To avoid the synchronous updating inherent in Boolean models, transitions can be asynchronous such as occurs with stochastic updating, or based on the dynamics in appropriate differential equations.

This paper deals specifically with the logical network approach, which derives from the early insights of Jacob and Monod, and was placed in a more mathematical framework by Kauffman, Glass and Thomas in the 1970s [20–22]. There are four key ideas: (i) that expression of a transcription factor from a given gene will affect the expression of other genes in the network. These effects are typically represented by graphs, called interaction diagrams in which each vertex represents a gene. Edges from one vertex to a subset of the other vertices indicate an effect of the expression of one gene on its targets. Typically one assumes that a given gene will lead to either the activation or inhibition of proteins coded by its target. In principle, it is also possible that a given transcription factor could lead to either activation or inhibition of its targets depending on the presence or absence of other transcription factors. The interaction diagram, in certain cases with appropriate extensions that we will make clear, specifies the ‘logical structure’ of the network. (ii) Based on the interaction diagram, we construct a state transition diagram. In the state transition diagram, each vertex represents a region of state space. Directed edges between vertices represent allowed flows between adjacent regions of state space. A state transition diagram does not imply a specific dynamical system, but is meant to capture coarse-grained dynamics that can be found in a broad class of dynamical systems generally represented by nonlinear differential equations that have the underlying structure given by the interaction diagram. (iii) Analysis of the state transition diagram enables one to make predictions about the dynamics that will be observed in real or model networks that have the same interaction diagram. (iv) Just as one can make predictions concerning dynamics based on the interaction diagram from carrying out the steps (i), (ii) and (iii) in sequence, by analysing qualitative aspects of the dynamics we can obtain insight into the inverse problem and determine the interaction diagram. This last point is the focus of the current article.

To help fix ideas, we consider a classic example of a simple negative feedback loop proposed by Goodwin [23] and subsequently investigated by many others [24]. Assume three components $x$, $y$ and $z$, where $x$ activates the production of $y$, $y$ activates the production of $z$ and $z$ inhibits the production of $x$. Figure 1A shows the interaction diagram for the Goodwin oscillator, where the arrow represents activation and the blunt edge represents inhibition. The model is

Figure 1.

The Goodwin oscillator. (A) Network diagram showing influences between variables. (B) Time series data for equation (1.1) with $\alpha =2$ and $n=20$. (C) State transition diagram. As described in §2, this is also the state transition diagram for a 3 variable negative feedback system with truth table in table 1.

$${\displaystyle \begin{array}{rl}\frac{\text{d}x}{\text{d}t}& =\frac{\alpha }{1+z^n}-x,\\ \frac{\text{d}y}{\text{d}t}& =x-y,\\ \frac{\text{d}z}{\text{d}t}& =y-z,\end{array}}$$ (1.1)

where $\alpha$ and $n$ are parameters and we have assumed that each species undergoes exponential decay (we have modified parameters from the earlier papers to facilitate analysis). If we take $\alpha =2$, the fixed point is at $x=y=z=1$. There is a Hopf bifurcation for $n=16$. For $n>16$, there is a stable limit cycle oscillation, figure 1B. In early work, Tyson [24] considered the coarse grained flows between the octants formed by the three planes defined by $x=1$, $y=1$, $z=1$. When the fixed point is unstable, with these parameters for $n>16$, there was an attracting torus that passed through six of the octants. For each variable, if we assign it a discrete state 1 if it is larger than 1, and 0 if it is less than 1, then the flow goes through the sequence of octants

$$\begin{array}{lcr}& 000\to 100\to 110\to 111\to 011\to 001\to 000\to \dots & \end{array}$$ (1.2)

By applying the Brouwer fixed point theorem to the Poincaré map after one circuit around the sequence of octants, Tyson was able to prove the existence (but not uniqueness or stability) of a periodic orbit in this system. For this equation, if we start at initial conditions in either orthant 010 or 101, then we will always enter one of the orthants associated with the cycle. Therefore, we can generate the symbolic representation of the dynamics shown in figure 1C. The same state transition diagram will hold for a large number of different differential equations that have the same interaction diagram shown in figure 1A. If we had data, we could generate a coarse grained tessellation of state space and then determine the flows from one volume to the next. The criterion for the coarse graining would be that all trajectories are transverse to the boundaries between neighbouring regions. If the coarse graining generated a diagram with the same structure as in figure 1C, we claim that it would display the same underlying logical structure. In general, it might not be obvious what boundaries to use for coarse graining. An alternative is to use the derivative [25]: ‘The state of a variable is 1 if its first derivative is positive and 0 if the first derivative is negative. The state of a network of $N$ variables at any time is then an $N$ vector of ls and 0s giving the state of each variable.’ Using the derivative discretization, we also obtain the sequence in equation (1.2).

In §2, we present a brief summary of our approach to analyse complex switching networks [20,22,26,27]. In §3, we describe how this approach facilitates carrying out the inverse problem. This article has been motivated by talks and papers from Tomas Gedeon and colleagues presented at the workshop on ‘Computing combinatorial dynamics in high dimensional biological networks’ that was held in Leiden, The Netherlands in October 2023. An example presented in [28] provided a challenge for application and extension of our techniques.

2. Logical and continuous models of genetic networks

2.1. Basic results

We briefly review of our approach [27,29] and then discuss aspects that are especially relevant to the inverse problem. We propose a class of piecewise linear ordinary differential equations for complex genetic networks. The technique involves embedding a given logical structure as defined by a Boolean switching network into a continuous differential equation.

We designate Boolean variables by capital letters, and continuous variables by lower case letters. A Boolean switching network with $N$ elements is represented by

$${\displaystyle X_i(t+1)=F_i(X_{i_1}(t),X_{i_2}(t),\dots ,X_{i_{K_i}}(t)),i=1,\dots ,N,}$$ (2.1)

where ${\displaystyle F_i(X_{i_1}(t),X_{i_2}(t),\dots ,X_{i_{K_i}}(t))\in \{0,1\}}$ and $K_i$ is the number of inputs to $i$. In general, $F_i$ can be written as a sum of products of the Boolean variables $X_j$ and $\overline{X_j}$ where the overline designates negation [30].

The logical structure of equation (2.1) can be captured by a differential equation [20,31]. To a continuous variable $x_i(t)\ge 0$, we associate a discrete variable $X_i(t)$,

$${\displaystyle X_i(t)=0\text{if}0\le x_i(t)<{\theta }_i;\text{otherwise}{X}_i(t)=1\text{if}{x}_i(t)>{\theta }_i,}$$ (2.2)

where ${\theta }_i$ is a threshold for the $i$th gene. For any logical network, we define an analogous differential equation

$${\displaystyle \frac{\text{d}{x}_i}{\text{d}t}=-{\gamma }_i{x}_i+f_i(X_{i_1}(t),X_{i_2}(t),\dots ,X_{i_{K_i}}(t)),i=1,\dots ,N,}$$ (2.3)

where ${\displaystyle f_i/{\gamma }_i>{\theta }_i}$ if $F_i$ is 1 and ${\displaystyle 0\le f_i/{\gamma }_i<{\theta }_i}$ if $F_i$ is 0. If we express the Boolean function $F_i$ using a sum of products of $X_k$ and $\overline{X_k}$, then we can generate an analogous continuous function $f_i$ by substituting a monotonic increasing sigmoidal $S(x_k)$ for $X_k$ and a monotonic decreasing sigmoidal function $\overline{S(x_k)}$ for $\overline{X_k}$. We typically choose the Hill function [31]

$$\begin{array}{lcr}& S(x)=\frac{x^n}{x^n+{\theta }^n},\overline{S(x)}=\frac{{\theta }^n}{x^n+{\theta }^n},& \end{array}$$ (2.4)

where $n$ is a positive constant. In the limit $n\to \infty$ the sigmoidal functions are step functions and equation (2.3) is piecewise linear enabling direct analytic integration. When all decay constants are equal the trajectories in each orthant are directed towards a focal point in the orthant specified by the logical structure. Numerical results and analytical results find that the qualitative properties of equation (2.3) for a large range of values of $n$ are the same as in the limit $n\to \infty$, supporting the notion that the logical structure can be used to define an equivalence class of networks.

We assume that there is no self-input, i.e. the function $F_i$ does not depend on $X_i$ and $f_i$ does not depend on $x_i$. With this assumption there is a 1:1 correspondence between directed graphs on the $n$-dimensional hypercube ($n$-cube) and the Boolean switching network. Just as there are ${\displaystyle 2^{n\times 2^{n-1}}}$ ways to label the edges of an $n$-cube where each edge is oriented in a single orientation, there are ${\displaystyle 2^{n\times 2^{n-1}}}$ ways to choose a Boolean network with no self-input.

To illustrate the rules that follow, consider the truth table for a simple three-dimensional (3D) negative feedback network. If $A$ activates $B$, represented by $A\to B$, then $B(t+1)=A(t)$ and if $A$ inhibits $B$, represented by $A⊣B$, then $B(t+1)=\overline{A(t)}$. Consider the network in which $X\to Y\to Z⊣X$. The truth table for this network is shown in table 1.

Table 1.

Truth table for 3D negative feedback system.

$(XYZ{)}_t$	$(XYZ{)}_{t+1}$
(111)	(011)
(110)	(111)
(101)	(010)
(100)	(110)
(011)	(001)
(010)	(101)
(001)	(000)
(000)	(100)

The following construction generates the unique representation as a directed graph on an $n$-cube for a Boolean switching network with no self input. To help understand the construction, consider the truth table in table 1, and the associated state transition diagram in figure 1C.

Assume $\mathbf{X}(t+1)=F(\mathbf{X}(t))$ where ${\displaystyle X(t)}$ denotes the full state vector at time ${\displaystyle t}$. The (Hamming) distance between $\mathbf{X}(t+1)$ and $\mathbf{X}(t)$ is the number of loci in which they are different. On the $n$-cube from vertex $\mathbf{X}(t)$ we draw directed edges along the first edge of each shortest path from $\mathbf{X}(t)$ to $\mathbf{X}(t+1)$. If the distance between $\mathbf{X}(t)$ and $\mathbf{X}(t+1)$ is $d$, this construction will generate $d$ edges directed away from $\mathbf{X}(t)$ to $d$ neighbouring vertices. For example, for states 101 and 010 at time $t$ the Hamming distance to the state at $t+1$ is 3, leading to the three out arrows from 101 and 010 in figure 1C. For the remaining states, the Hamming distance is 1 leading to 1 out arrow from each of those states in figure 1C.

For a given network, the truth table has $2^n$ rows. A row gives the successor state for each of the states of the network. Columns in the truth table give the values of $X_i(t)$ and $X_i(t+1)$ for each of the states of the network. Since we assume no self-input, for each $i$, the distance from the column containing $X_i$ at time $t$ for each state of the network to its successor at time $(t+1)$ is $2^{n-1}$. Thus, in table 1, $n=3$ and summing over all the rows of the truth table, the total Hamming distance between each variable at times $t$ and $t+1$ is 4. It follows that the sum of the distances from the initial state to the successor state for a Boolean network with no self-input is $n{2}^{n-1}$, which is also the number of edges on the $n$-cube. For the example in table 1, the sum of the Hamming distances is $4\times 3=12$, which is the number of edges on the 3-cube.

This construction gives information about the flows in the differential equations with the same logical structure as a given network. The $n$-cube is the geometric dual of Euclidean state space: each vertex represents an orthant of state space and each edge represents the open hypersurface [32] between two orthants. A directed edge on the $n$-cube implies that there will be a transverse flow across the associated hypervolume in Euclidean space. As reviewed elsewhere [27,29], this discrete representation enables many results related to combinatorics and classification as well as prediction of dynamic properties of equation (2.3).

We say that two networks are structurally equivalent if their $n$-cube representations are superimposable under a symmetry of the $n$-cube. An in-vertex on the $n$-cube implies a stable steady state in the associated differential equation. We define a cyclic attractor as a cycle on the $n$-cube in which each vertex on the cycle is adjacent to $(n-2)$ edges that are not on the cycle, all of which are directed towards the vertex on the cycle. If there is a cyclic attractor in which three or more variables switch then there will always be a choice of functions in the continuous analogue that will guarantee the existence of a stable, unique limit cycle passing through the associated volumes of state space [32].

If for a given set of differential equations there is a cycle on the $n$-cube that is not a cyclic attractor (some vertices on the cycle have outgoing edges adjacent to the cycle) then it is possible to establish analytically whether or not there exists a periodic orbit following that cycle and if so, whether or not it is stable [26,33]. Necessary and sufficient conditions on parameter values to ensure this can be calculated exactly. In fact, stable periodic orbits can exist for a large class of such cycles on the $n$-cube, in the sense that there exist sets of differential equations with appropriate parameters in which such a periodic orbit occurs [34]. For example, any simple (non-self-intersecting) cycle on which there is at least one vertex with all edges adjacent to the cycle directed inwards can have a stable periodic orbit for appropriate equations. Thus, while cyclic attractors guarantee stable periodic orbits (if three or more variables switch), there are many other cyclic structures on the $n$-cube for which stable periodic orbits can also exist.

There are several shortcomings in the formulation of equation (2.3). There can be situations in which a product of a gene directly feeds back to activate or inhibit its own production [35]. Also, if a given gene activates or inhibits more than one other gene in the network, there is the possibility that the threshold for control would be different for the different interactions [28,36]. Both these situations could be dealt with by introducing additional variables, thereby increasing the dimension of the system [37,38]. This can preserve the hypercube geometry. Although each edge would be oriented on the higher dimension hypercube, the orientation on many of the edges would not be independent of each other. For situations in which there are multiple threshold levels for the influence of a gene on its downstream targets, it would be possible to have geometries in which some of the variables would have multiple logical values, rather than just two values. However, it is possible that genes that are inputs to multiple downstream targets have dynamics constrained to a range of values so that they can be considered either always above or always beneath some of the thresholds thereby reducing the complexity of the interaction network. As discussed in §3, this is the case for networks in [28].

We close this section with two general results. Hastings et al. proved existence of a periodic cycle in a simple negative feedback system with an unstable steady state [39]. Snoussi proved that a necessary condition for a stable oscillation in a differential equation is the presence of a negative feedback loop in the interaction diagram [40].

2.2. Simple negative feedback systems

Because of its biological relevance, and also relevance to an example in Cummins et al. [28], we illustrate these ideas with a discussion of simple negative feedback systems. Since early studies of Monod et al. [41], it has been recognized that negative feedback plays an important role in cellular metabolism [42]. In a simple negative feedback system, the interaction diagram is a loop consisting of $n$ elements in which each acts to either activate or inhibit production of the next element. In one example of a negative feedback system, the last product in a synthetic sequence acts to inhibit the production of the first product, while all others in the loop promote production of the next product. However, from a dynamical perspective all networks with an odd number of inhibitory links in a cyclic interaction diagram with $n$ variables have state transition diagrams that are superimposable under a suitable symmetry operation of the $n$-cube. Further, the state transition diagrams have cyclic attractors and can have stable limit cycles when the control functions are step functions [26,32].

One way to characterize and classify oscillations in the piecewise linear gene networks is to give the switching sequence, the sequence in which each gene crosses its threshold [43,44]. Further, on each crossing of each threshold, the slope will either be positive or negative. As presented in [31], consider the interaction diagram for the 3D negative feedback system considered in table 1, $X\to Y\to Z⊣X$.

We can represent the oscillation found for this network by a signed switching sequence in which the slope of threshold crossings of the continuous variables in time is given as a superscript:

$$\begin{array}{lcr}& X^{+}{Y}^{+}{Z}^{+}{X}^{-}{Y}^{-}{Z}^{-}{X}^{+}\dots & \end{array}$$ (2.5)

An alternative is to give the sequence in which the variables pass through maxima and minima, indicating a maximum by a + and a minimum by a $-$. These two ways of getting a switching sequence are clearly identical for a cyclic attractor with Heaviside control. To overcome limitations inherent in defining thresholds for systems with smooth switching functions, we propose two alternative, but equivalent, methods for deriving the signed switching sequence from data. One is based on the first derivative discretization described earlier. If a logical variable $X_j$ in the derivative discretization goes from 0 to 1, we generate the symbol $j^{+}$, and if it goes from 1 to 0, we generate the symbol $j^{-}$. Alternatively, if there is a maximum for variable $X_j$, we generate the symbol $j^{+}$, and if there is a minimum, we generate the symbol $j^{-}$. For cyclic attractors with step function control, the different ways of generating the signed switching sequences give the same result. For the Goodwin oscillator in figure 1, it is easy to confirm that one obtains the signed switching sequence in equation (2.5) for the different methods.

3. Solving the inverse problem from observed dynamics

3.1. A brief history

In developing mathematical models of genetic and biochemical systems, it is common to use equations of the form

$${\displaystyle \frac{\text{d}{x}_i}{\text{d}t}=-{\gamma }_i{x}_i+g_i(\mathbf{x}),i=1,\dots ,N,}$$ (3.1)

where ${\gamma }_i$ is a decay constant and $N$ is the number of components. The observation that RNA and proteins have half-lives justifies the exponential decay term. This also leads to equations being bounded if the functions $g_i$ are bounded. Depending on the particular example, the function $g_i$ can be linear, nonlinear or both depending on the component. Assumptions about the properties of the functions $g_i$ determine the procedures for developing techniques for the inverse problem. For example, an analysis of gene expression following a perturbation delivered at a steady state assumed that $g_i$ consisted solely of a sum of linear terms [45]. By contrast to the purely linear models, there is often an assumption of nonlinear control for one or more terms as in equations (1.1) and (2.3). The general problem of determination of functions governing the dynamics falls in the field of functional data analysis [46,47]. Assumption of the form of the underlying equation facilitates carrying out the inverse problem [7,44,48–55]. We briefly summarize several earlier results that share the common goal of determining the underlying logical structure (as opposed to detailed kinetic parameters).

First consider what it means for a gene $j$ to regulate another gene $i$. Gene $j$ regulates gene $i$ if $X_j$ plays a part in determining the output of $F_i$. There must be at least two system states $\mathbf{X}$ and ${\mathbf{X}}^{\prime }$ that (i) are identical in all elements except for the element corresponding to gene $j$ ($X_j\ne X_j^{\prime }$ and $X_{j^{\prime }}=X_{j^{\prime }}^{\prime }$ for all $j^{\prime }\ne j$) and (ii) for which $F_i(\mathbf{X})\ne F_i({\mathbf{X}}^{\prime })$. Equivalently, there must be two vertices of the $n$-cube corresponding to orthants where only gene $j$ is in a different state, and where the edge corresponding to gene $i$ is ingoing at one vertex and outgoing at the other vertex. If one knew the directions of all the edges on the $n$-cube for a particular network’s dynamics, one could systematically compute which genes regulate which. However, one need not know all the edges’ directions in order to have sufficient evidence for structure inference, or at least to compute plausible hypotheses on structure.

Perkins et al. [49] considered the inverse problem for equation (2.3) in high dimensions and with multiple inputs for each variable. To aid computations, they shifted the coordinates so that ${\theta }_i=0$, ${\gamma }_i=1$ and the regulation functions are composed of step functions: $f_i=F_i$ outputting a binary value ($-1$ and $+1$, rather than $0$ and $1$). They also assumed access to a continuously observed time series $\mathbf{x}(t)$ of the gene expression values. These assumptions lead to a simple inference rule: look for any moment $t$ in the time series where ${\displaystyle \text{d}{x}_i(t)/\text{d}t}$ (for some $i$) changes sign—or equivalently, where $x_i(t)$ (for some $i$) reaches a local minimum or maximum in time. At such a moment, at least one other gene must be changing Boolean state by crossing its threshold $\theta =0$, in order for $F_i(t)$ to change value. If precisely one other gene $j$ is changing Boolean state at time $t$, then we can deduce that gene $j$ regulates gene $i$. In essence, the dynamics of the system just before and just after (or at) time $t$ comprise one of those pairs of observations $\mathbf{X}$ and ${\mathbf{X}}^{\prime }$ that imply $n$-cube edge orientations that reveal regulation of gene $i$ by gene $j$. Although the scale of data required and the nature of the model are not suitable for real systems, the techniques were capable of carrying out the inverse problem for systems consisting of tens of thousands of genes. It was also possible to estimate the size of data sets needed as well as whether there were missing data.

These techniques were adapted to carry out the inverse problem for experimental data collected from an electronic circuit with five model genes that was built to simulate equation (2.3) for a connectivity that had been evolved to display ‘chaotic’ dynamics [50]. In this network, there were no a priori assumptions about decay rates or the thresholds or output values for the regulation functions. Voltages measured over time took the place of genes’ expression. They estimated high and low output values for each $F_i$, along with decay rates ${\gamma }_i$, by fitting exponential curves to segments of the time series where a variable was continuously falling or rising. Then, they simultaneously inferred both gene thresholds ${\theta }_i$ and the logic of the regulatory functions $F_i$ in the following way. For a given estimated set of thresholds ${\hat{\theta }}_i$, they divided the time series into different segments within the same orthant of state space—that is, to periods of time in which no variable changes Boolean state, corresponding to a single node of the $n$-cube. To estimate $F_i$, they assumed every variable could regulate every other. For each Boolean state of the network ($5$-cube vertex), they determined whether each variable was more often rising or falling in the time series $\mathbf{x}(t)$ (equivalent to determining a tentative orientation for edges between each vertex on the 5-cube and its neighbours). They then simulated that model starting from every observed time point until the next and measuring the quantitative agreement between the next observed value and the predicted value. They performed a local search for values ${\hat{\theta }}_i$ such that the overall prediction accuracy was maximized. They found that this procedure was sufficient to correctly infer the logic of the regulation functions, for vertices of the 5-cube that were observed in the data.

Since continuous observations in time are usually not possible in determination of genetic activity, Perkins and collaborators studied situations in which sampling was sparse [51,53]. They focused on the Boolean aspects of inference, essentially assuming that at each time point one could observe the Boolean state of each gene (if necessary by observing the real-valued expression values $\mathbf{x}(t)$ and knowing the thresholds ${\theta }_i$). They assumed that one could also observe whether each gene was heading towards a low or high state (which would be true, for instance, if one could observe the time-derivative of expression ${\displaystyle \text{d}\mathbf{x}(t)/\text{d}t}$, and one knew the decay rates ${\gamma }_i$ and possible output values of each regulation function $F_i$. In this case, the problem reduces to one of learning Boolean target functions $F_i$. To do this, they proposed to scan the input time series data and perform a combinatorial search for the/a minimum-size set of regulators for each gene that allows one to construct a truth table linking observed binary system states $\mathbf{X}(t)$ to the observed target function values $F_i(\mathbf{X}(t))$. They showed that this inference rule, while sometimes computationally expensive, could correctly reconstruct network structure, and they provided estimates of how many time series points would be needed to infer the entire structure—based on additional assumptions regarding the sampling rate or, more precisely, how many variables typically change Boolean state between observations.

3.2. Inverse problem for a six-dimensional oscillator

The logical structure of a genetic network, as partially represented in the interaction diagram, has been used by many different groups to help understand the function of genetic networks. A key question involves predicting possible sequences of activation or inhibition of genes in the network. As discussed earlier, such transitions must follow edges in the state transition diagram. In general, one does not need to assume specific differential equations to predict those transitions and some groups have specifically avoided equations and considered what happens under other methods for updating such as stochastic updating [8]. However, as emphasized by Cummins et al. [28,56], representations of the interaction diagram may not be sufficient to understand a given network since parameters such as production rates and thresholds will determine dynamics. For example, even though a given gene may activate another one, if the production is such that it is always above its threshold, then its target would always be activated. Also, for systems in which a gene has multiple targets, the thresholds for different targets can be different. In this case the sequence of gene activation may depend on parameter values as well as on the underlying interaction diagram. If a given genetic network is modelled by a differential equation with parameters, then in order to understand the range of dynamics that can be exhibited by the network, it is necessary to understand how the state transition diagram changes as parameters change.

The datasets that we use in the current paper were used earlier by Cummins et al. [28] in the development of the Inherent Dynamics Pipeline (InDyPi) for determination of genetic network structure based on the dynamics. The InDyPi combines three earlier techniques to first identify important nodes, second identify important interactions and third test combinations of nodes and edges as possible explanatory network models of the provided data. Cummins et al. only used the second and third steps. The second step employs the local edge machine, an earlier technique developed by McGoff et al. [54] that predicts the best single regulator for each gene by optimizing Hill function model parameters using a Bayesian approach. The output is a probability distribution over all potential input edges for each node and thus $p$-values can be used for rank-ordering inputs to each gene. The third step of InDyPi analyses network models using Dynamic Signatures Generated by Regulatory Networks (DSGRN) with associated software [36,56]. It systematizes and classifies the changes in topology of the state transition diagrams as parameters and network structure change, and carries out a pattern matching analysis by checking the order of maxima and minima of experimental data compared with the order found with different network topologies and parameter values [57,58]. In this fashion, it rejects models that do not agree with the data. The final list of top-ranked interactions identified by InDyPi is determined by the prevalence of interactions in networks that match the order of extrema of the data. Since experiments can be done under different conditions, or in the same organism after knockout of a given gene, the techniques may be helpful in determining genetic networks in experimental settings.

As illustrated by an in silico model that was used to develop concepts that were used to study genetic networks in yeast [28], challenges arise from the combinatorial explosion of possible behaviours under parameter changes. That paper illustrated dynamics of a single six-dimensional system under three different sets of parameters. We chose one set of parameters given in equation (A 1) (dataset 270) for detailed analyses. We attempted to determine the network using methods complementary to those used in the InDyPi technique. The architecture of the network is shown in figure 2A and the time series data for dataset ‘270’ are plotted in figure 3A. The data are given for 31 time points, equally spaced at 0.5 time units over a period of 15 time units.

Figure 2.

Two networks used to demonstrate inference techniques. (A) A network from Cummins et al. [28]. (B) A five-gene simplification of that network retaining the effective regulatory influence of each gene in the ‘270’ time series.

Figure 3.

Demonstration of inference algorithms on the ‘270’ time series data from Cummins et al. [28]. (A) Time series data for inference, annotated with maximum and minimum time points for each gene. (B) Inference based on prediction of estimated regulation function. Within each box are plotted estimated target gene regulatory function values ${\hat{f}}_i$ on the y-axis versus candidate regulator gene expression values $x_j$ on the x-axis. The best Hill function fit is shown in black, with the correlation coefficient between the two indicated above. If the y-values appear to be a function of the x-values, then regulation is suggested.

As described above, in earlier work we developed techniques to do the inverse problem for equations of the form of equation (3.1) for situations in which the control was via switching functions [49,50]. The key idea was using the dynamics to try to determine the control function. In the current context, we adopt those methods. Assuming the differential equation model ${\displaystyle \text{d}{x}_i/\text{d}t=f_i(\mathbf{x})-x_i}$, we estimated ${\displaystyle f_i\approx \text{d}{x}_i/\text{d}t+x_i}$, where $x_i$ is the observed expression and the derivative is estimated by averaging forward and backward differences. We attempted to explain each gene’s $f_i$ estimates as either an increasing Hill function of another gene: $f_i\approx a+b{x}_j^n/(x_j^n+{\theta }^n)$ or a decreasing Hill function: $f_i\approx a+b{\theta }^n/(x_j^n+{\theta }^n)$. For each target gene $i$ and candidate regulator $j$, the parameters $a$, $b$, $n$ and $\theta$ were fitted using Python’s scipy.optimize.minimize function to minimize the mean square error between the two sides of the equation. We plot the best fits in figure 3B. For instance, gene D is not a good candidate regulator for gene A, because for very similar values of $x_D$ we can observe divergent high and low values for ${\hat{f}}_A$. However, gene F would be a good explanation of gene A, because of the apparent smooth functional relationship between $x_F$ and ${\hat{f}}_A$, with correlation near one. If we select the best single regulator for each gene, then we find the relationships A $⊣$ D $⊣$ C $\to$ B $⊣$ F $\to$ A and F$\to$E. This suggests that $E$ is not an input to any of the other genes. The remaining five genes form a negative feedback network that can support oscillatory behaviour. Note, however, that figure 3B also suggests a fairly strong negative regulation of gene D by gene E, though not as strong as D’s regulation by A. This leads to another possible five gene reduced network in which gene A is omitted. We consider this possibility further below.

We next considered the first derivative discretization and the sequence of maxima and minima. The original data in [28] were not on a sufficiently fine time scale to determine the precise sequence of maxima and minima (figure 3A). For instance, both C and B achieve local maximum and enter a period of negative derivative simultaneously at time $t=1$, making ambiguous which might influence gene F, the next gene to switch. Therefore, we have recomputed the time series on a finer time scale and applied the first derivative discretization to obtain the logical oscillation

$$\begin{array}{lcr}& 000100\to 000101\to 100101\to 100111\to 100011\to 101011\to 111010\to 111000\to 011000\to 011100\to 010100\to 000100.& \end{array}$$ (3.2)

From this, we derive the signed switching sequence for one cycle:

$$\begin{array}{lcr}& F^{+}{A}^{+}{E}^{+}{D}^{-}{C}^{+}{B}^{+}{F}^{-}{E}^{-}{A}^{-}{D}^{+}{C}^{-}{B}^{-}.& \end{array}$$ (3.3)

Because the order of A and E are reversed during their two occurrences, this sequence is not consistent with a simple negative feedback system. Using the techniques in [44], we could generate a logical network that would generate exactly this sequence. However, when we carried out the rather tedious process of determining the network interactions that would generate this sequence, it appeared quite different from the actual network interactions given in [28].

There is another possibility that is suggested by the preceding analysis and also from an examination of the time series and the governing equations. If we eliminate either A or E from the sequence then we obtain two subsequences, which are both consistent with a simple negative feedback system. If we eliminate A from the sequence, we obtain $F^{+}{E}^{+}{D}^{-}{C}^{+}{B}^{+}{F}^{-}{E}^{-}{D}^{+}{C}^{-}{B}^{-}$, consistent with the interaction diagram $F\to E⊣D⊣C\to B⊣F$. If we eliminate E from the sequence, we obtain $F^{+}{A}^{+}{D}^{-}{C}^{+}{B}^{+}{F}^{-}{A}^{-}{D}^{+}{C}^{-}{B}^{-}$, consistent with the interaction diagram $F\to A⊣D⊣C\to B⊣F$. Since we know the equations, we carried out simulations in which we eliminated $A$ or $E$ from the original six-dimensional system. Removing $A$ led to a steady state, but removing $E$ led to a five-dimensional system for which the dynamics of the variables in the reduced system were very similar to the dynamics of those variables in the original six-dimensional system. We now consider that reduced five-dimensional system.

3.3. The embedded five-dimensional network

The above analysis identified a negative feedback network comprised of five genes that we claim is driving the dynamics in the ‘270’ dataset. Since elements $C$, $D$ and $E$ have multiple inputs, this finding was not expected. However, by examining the equations for $C$, $D$ and $E$, we found that only one of the inputs has a significant effect in driving the dynamics of that element, and that $E$ had minimal effect in driving the dynamics of the other variables. To confirm this observation, we consider a reduced set of equations in which we assume fixed values (at either the high value or low value of the Hill function depending on the observed dynamics), and considered the inverse problem for the reduced system given in equation (A 2) in appendix A. The interaction diagram and the dynamics are shown in figures 2B and 4A, respectively. In addition to simplifying the dynamics compared with the six-dimensional example above, we have sampled time more finely in this data series, every 0.075 time units.

Figure 4.

Demonstration of inference algorithms on the five-dimensional simplified oscillator. (A) Time series data for inference evaluated at 201 time points, annotated with the maximum and minimum for each gene. (B) Inference based on prediction of estimated regulation function.

We first tried to coarse grain the state space and determine the transition diagram based on the observed transitions. However, we do not know the correct thresholds based on the dynamics. Although in other examples, choosing the midpoint of the range of values for each variable would give the expected logical structure for the negative feedback, this did not work in the current example. Based on [39], we expect that a coarse graining based on the coordinates of the fixed point would be appropriate. For equation (A 2), the fixed point is at $A\approx 0.78,B\approx 0.005,C\approx 0.87,D\approx 1.38,F\approx 0.68$. Using these coordinates, we generate the sequence $10001\to 10101\to 11101\to 11100\to 01100\to 01110\to 01010\to 00010\to 00011\to 10011\to 10001\to \dots$. This is consistent with the interaction diagram in figure 2B.

As described above, the sequence of maxima and minima can also be used to determine the interaction diagram for negative feedback networks if the Hill functions are sufficiently steep. From the sequence at the bottom of figure 4A, we find $C^{-}{B}^{-}{F}^{+}{A}^{+}{D}^{-}{C}^{+}{B}^{+}{F}^{-}{A}^{-}{D}^{+}$, from which we can derive the interaction diagram in figure 2B. As with the data for the full six-dimensional system, the quantitative derivative fitting approach shown in figure 4B identifies the network accurately.

In summary, we have identified the simple negative feedback loop

$$F\to A⊣D⊣C\to B⊣F$$

as the driver of the dynamics for the ‘270’ time series. There are two different possible paths to find this structure. Most direct is the plotting of the first derivative for each variable as a function of each of the other variables in figure 3B. If we take the putative input with the best fit for each variable, then we find the underlying five-dimensional network. The second method considered the sequence of maxima and minima as measured on a sufficiently fine time scale to resolve the ordering. By hypothesizing that the underlying controlling network was a simple negative feedback loop with five nodes, we identified two possible structures, one of which we identified as the underlying driver of the oscillation.

3.4. Inference from two alternative time series for the six-dimensional system

To further test the network inference procedures above, we turned to the two additional time series that Cummins et al. [28] generated for the six-gene network in figure 2A. They used the same network architecture but different parameters, so that for multi-input genes, the dominant regulator could be different, leading, in effect, to a different network structure. Their data for the two additional time series are shown in figures 5A and 6A.

Figure 5.

Time series and inference analysis for the ‘779’ dataset of Cummins et al. [28]. For explanation of panels, see figure 3 caption.

Figure 6.

Time series and inference analysis for the ‘1945’ dataset of Cummins et al. [28]. For explanation of panels, see figure 3 caption.

For the ‘779’ time series of figure 5A, we can analyse the strongest input to each node using the fits of the first derivatives in figure 5B. We once again find the simple feedback network $F\to A⊣D⊣C\to B⊣F$; E is not the principal controller of any of the other nodes in the network. The sequence of maxima and minima of gene expression is $E^{-}$$F^{-}$$A^{-}$$D^{+}$$C^{-}$$B^{-}$$F^{+}$$E^{+}$$A^{+}$$D^{-}$$C^{+}$$B^{+}$$E^{-}$. Since the sequence of F and E is not preserved, this sequence is not consistent with a simple negative feedback with all six nodes. However, as before, if we consider possible structures for a simple negative feedback system with five nodes, we once again generate the $F\to A⊣D⊣C\to B⊣F$ as well as the alternative $E\to A⊣D⊣C\to B⊣E$. Notice that since E and F are highly correlated with each other, the second putative control mechanism would be expected to be consistent with the data. Simulations using parameter values for the ‘779’ time series and appropriate constants for non-essential Hill functions confirm that the simple negative feedback loop $F\to A⊣D⊣C\to B⊣F$ is sufficient to generate dynamics similar to those observed in the full equations (figure 7A).

Figure 7.

Simulations of simplified models of ‘779’ and ‘1945’ networks. (A) Simulation of ‘779’ network after removing regulation of gene $C$ by $A$ and $E$ and regulation of $D$ by $E$, leaving the core regulatory loop of $F\to A⊣D⊣C\to B⊣F$. (B) Simulation of the ‘1945’ network after removing regulation of gene $C$ by $A$ and $D$ and regulation of $E$ by $B$, leaving the core regulatory loop of $F\to E\to C\to B⊣F$.

In the ‘1945’ time series shown in figure 6A, the sequence of maxima and minima is not easily interpreted because of several instances at which multiple genes achieve an extremum simultaneously. However we can once again analyse the strongest input to each node using the fits of the first derivatives in figure 6B. From this analysis, we extract the four-dimensional simple negative feedback loop: $F\to E\to C\to B⊣F$. In addition, we also find that $F\to A⊣D.$ However, $D$ is a dead end and does not exert strong effective input on any other variables. As in the earlier examples, we carried out simulations using parameter values for the ‘1945’ time series and appropriate constants for non-essential Hill functions to confirm that the simple negative feedback loop $F\to E\to C\to B⊣F$ is sufficient to generate dynamics similar to those in the full equations (figure 7B). Although the period of the oscillation will depend on parameters, the observation that the period of oscillation of the ‘1945’ time series is shorter than the period of oscillation of the other parameter choices is consistent with the observation that the period of oscillation in simple negative feedback loops increases as the number of elements in the loop increases [44].

4. Discussion

Determination of genetic network structures regulating cellular function is a key scientific question that is facilitated by rapidly developing technologies. We have focused on a limited class of questions related to gene network determination: determining the network based solely on the dynamics. We have proposed several different techniques: (i) coarse-graining state space and determining the flows between regions; (ii) generating a sequence of logical states based on whether the first derivative is positive or negative; (iii) generating a sequence of the times of maxima and minima for each element; (iv) determining the dependence of the first derivative of a given variable on its own value as well as on the values of other variables in the system considered one at a time.

In earlier work, we showed that for arbitrarily complex networks with Heaviside step function control, equation (2.3), the first technique in which state space is coarse grained could be used to determine the interaction diagram and showed that the amount of data needed depended on the details of the network construction [49]. In the current work, we found that even with high quality data, this technique had problems in examining systems with non-step function control since the necessary values for coarse graining were not known and were not readily determined. However, for a negative feedback system, it was shown previously [39] and confirmed here that the coordinates of the steady state provide suitable values for determining orthants of the coarse grained flows. However, when the coordinate values for the orthants are not known, it was possible to gain information about the interaction diagram from consideration of the symbolic sequences that are determined from the derivatives or the sequence of maxima and minima, methods (ii) and (iii) above. Finally, for equations of the form in equation (3.1), when there is one effective input for each gene and the ${\gamma }_i=1$, we can successfully determine the underlying network directly by plotting ${\displaystyle \text{d}{x}_i/\text{d}t+x_i}$ as a function of each of the other variables in the system. If the ${\gamma }_i$ were allowed to vary, it would be possible to estimate them using the techniques in [50] so we could still carry out the inverse problem. Further work is needed to determine if the direct estimation of $f_i$ will be possible if it is a function of two or more variables.

These methods depend on having sufficiently high accuracy at sufficiently finely spaced time points. In realistic settings, the data will necessarily be noisy and the sampling will be more sparse. This would degrade our knowledge about the derivative as well as the sequence of maxima and minima and would require further work to delineate the limitations of our approaches.

Of the techniques we applied, the most useful involved the determination of the first derivative as a function of the other variables considered one at a time; see panel (B) in each of figures 3–6. This method enabled us to find simple negative feedback loops consisting of five nodes for the ‘270’ and ‘779’ datasets and four nodes for the ‘1945’ dataset. Although we confirmed that the proposed simple negative feedback networks were sufficient to generate oscillations in qualitative agreement with oscillations in the full equations, it is possible that another reduced network in which some other interactions between nodes in the full equations were eliminated would also give agreement with the observed dynamics. This is especially so since the network in figure 2B contains redundant parallel pathways that lead to activation or inhibition of one node by another (e.g. node A can activate node C directly, or through consecutive inhibitory interactions $A⊣D⊣C$). In accord with the earlier analysis in [28], we found that node D was not necessary in the underlying network generating the ‘1945’ data, but was likely involved in the networks underlying the other two datasets. However, nodes B and F were implicated in our proposed simple negative feedback network for the observed dynamics for datasets ‘270’ and ‘779’ whereas node E was not necessary. This was not expected based on fig. 5 in [28].

A central focus of our group has been on the mathematical question of prediction of dynamics in equations based solely on the underlying logical structure by coarse graining of phase space and analysing the flows in the coarse grained phase space. This approach is related to current approaches in nonlinear dynamics that reduce flows in nonlinear differential equations to a graph; the resulting combinatorial structure of the flow is then amenable to further analysis [59,60]. The equations proposed to model genetic networks are providing a fertile ground for advancing mathematical questions relating to combinatorial aspects of flows in differential equations [56].

For many applications, we do not need to know the detailed equations for biological networks. Qualitative information in the form of interaction network diagrams is a usual goal of research. For such networks, issues related to robustness of the dynamics can sometimes emerge. Thus, if in a logical network given by equation (2.1) there is an attracting path, then the dynamics in a real-world realization of this system, for example in yeast, will attract to this path [6,9,14]. Therefore, we do not need to develop differential equations since the transitions are constrained by the logic and any updating scheme including stochastic updating gives the same sequence. Logical models help identify robust network architectures that are essential to maintain stable dynamics in controlled biological systems [7,12,13,61].

Appendix A.

We have analysed an in silico gene network that was presented by Cummins et al. [28] in their analysis of methods to determine gene network architecture. We have focused on a particular parameter set: column 1 of table 2 and fig. 3B in [28] for the ‘270’ time series. We give eqn (4) from [28] with parameters for the ‘270’ time series inserted in the equations (not keeping all the decimals):

$${\displaystyle \begin{array}{c}\frac{\text{d}A}{\text{d}t}=-A+0.19+1.45\frac{F^5}{F^5+{0.728}^5}\\ \frac{\text{d}B}{\text{d}t}=-B+0.001+1.95\frac{C^5}{C^5+{2.98}^5}\\ \frac{\text{d}C}{\text{d}t}=-C+(0.101+0.258\frac{A^5}{A^5+{0.24}^5}+1.65+0.39\frac{E^5}{E^5+{0.009}^5})\times (0.062+1.33\frac{{1.08}^5}{D^5+{1.08}^5})\\ \frac{\text{d}D}{\text{d}t}=-D+(0.054+0.89\frac{{0.686}^5}{A^5+{0.686}^5})\times (0.22+3.66\frac{{0.539}^5}{E^5+{0.539}^5})\\ \frac{\text{d}E}{\text{d}t}=-E+(0.001+0.713\frac{{1.51}^5}{B^5+{1.51}^5})\times (0.018+0.82\frac{F^5}{F^5+{1.305}^5})\\ \frac{\text{d}F}{\text{d}t}=-F+0.107+1.20\frac{{0.005}^5}{B^5+{0.005}^5}.\end{array}}$$ (A 1)

As described in the text, for several of the Hill function controls, the controlling variable is oscillating at predominantly higher or lower values than the Hill function thresholds and does not act as an effective input to its target. In this case, we assume the appropriate asymptotic value depending on whether its value tends to be higher or lower than the threshold for the Hill function. For example, variable E tends to be higher than the threshold for the production of C, so we assume the asymptotic value (0.39) of the controlling Hill function. However, since its oscillation tends to be lower than the threshold for the inhibition of D we assume a constant value of the Hill function term (3.66) where $E=0$. In this fashion, oscillations in E that are present in the full equations do not play a role in modulating the activities of C or D. We have generated a reduced five-dimensional set of equations to highlight the underlying logical structure of this network. To facilitate comparison with equation (A 1), we do not simplify the equations:

$${\displaystyle \begin{array}{c}\frac{\text{d}A}{\text{d}t}=-A+0.19+1.45\frac{F^5}{F^5+{0.728}^5}\\ \frac{\text{d}B}{\text{d}t}=-B+0.001+1.95\frac{C^5}{C^5+{2.98}^5}\\ \frac{\text{d}C}{\text{d}t}=-C+(0.101+0.258+1.65+0.39)\times (0.062+1.33\frac{{1.08}^5}{D^5+{1.08}^5})\\ \frac{\text{d}D}{\text{d}t}=-D+(0.054+0.89\frac{{0.686}^5}{A^5+{0.686}^5})\times (0.22+3.66)\\ \frac{\text{d}F}{\text{d}t}=-F+0.107+1.20\frac{{0.005}^5}{B^5+{0.005}^5}.\end{array}}$$ (A 2)

Ethics

This work did not require ethical approval from a human subject or animal welfare committee.

Data accessibility

Program code and data are available in the electronic supplementary material [62].

Declaration of AI use

We have used AI-assisted technologies in creating this article.

Authors’ contributions

T.J.P.: conceptualization, investigation, software, writing—original draft, writing—review and editing; R.E.: conceptualization, investigation, writing—review and editing; L.G.: conceptualization, investigation, software, writing—original draft, writing—review and editing.

All authors gave final approval for publication and agreed to be held accountable for the work performed therein.

Conflict of interest declaration

We declare we have no competing interests.

Funding

This work was supported in part by Discovery Grants RGPIN-2019-06604 and RGPIN-2017-04042 from the Natural Sciences and Engineering Research Council of Canada (NSERC) to T.J.P. and R.E., respectively, and by Project Grant PJT-180458 from the Canadian Institutes of Health Research (CIHR) to T.J.P.