Bifurcation and Singularity Analysis of a Molecular Network for the Induction of Long-Term Memory

Abstract

Withdrawal reflexes of the mollusk Aplysia exhibit sensitization, a simple form of long-term memory (LTM). Sensitization is due, in part, to long-term facilitation (LTF) of sensorimotor neuron synapses. LTF is induced by the modulatory actions of serotonin (5-HT). Pettigrew et al. developed a computational model of the nonlinear intracellular signaling and gene network that underlies the induction of 5-HT-induced LTF. The model simulated empirical observations that repeated applications of 5-HT induce persistent activation of protein kinase A (PKA) and that this persistent activation requires a suprathreshold exposure of 5-HT. This study extends the analysis of the Pettigrew model by applying bifurcation analysis, singularity theory, and numerical simulation. Using singularity theory, classification diagrams of parameter space were constructed, identifying regions with qualitatively different steady-state behaviors. The graphical representation of these regions illustrates the robustness of these regions to changes in model parameters. Because persistent protein kinase A (PKA) activity correlates with Aplysia LTM, the analysis focuses on a positive feedback loop in the model that tends to maintain PKA activity. In this loop, PKA phosphorylates a transcription factor (TF-1), thereby increasing the expression of an ubiquitin hydrolase (Ap-Uch). Ap-Uch then acts to increase PKA activity, closing the loop. This positive feedback loop manifests multiple, coexisting steady states, or multiplicity, which provides a mechanism for a bistable switch in PKA activity. After the removal of 5-HT, the PKA activity either returns to its basal level (reversible switch) or remains at a high level (irreversible switch). Such an irreversible switch might be a mechanism that contributes to the persistence of LTM. The classification diagrams also identify parameters and processes that might be manipulated, perhaps pharmacologically, to enhance the induction of memory. Rational drug design, to affect complex processes such as memory formation, can benefit from this type of analysis.

INTRODUCTION

Biochemical and gene networks are complex systems with multiple nonlinear interactions among signaling molecules and genes. This complexity often makes understanding and predicting network behaviors nonintuitive (1–3). One of the key goals of systems biology is to develop tools that may lead to a better understanding of nonlinear behaviors of biochemical and gene networks at both the molecular and systems levels (4).

A gene and protein network can be described by a mathematical model consisting of ordinary differential equations (ODEs). Bifurcation analysis is a mathematical technique that enables determination of the stability of a system with respect to a parameter (5,6). Bifurcation diagrams describe the dependence of a state variable on a continuous change in a chosen system parameter, termed a bifurcation parameter. A bifurcation is said to take place when there is a change in the number or the stability of solutions of a system. For example, steady-state solutions for the values of the dependent variables may appear, disappear, change stability, or multiple steady-state solutions may coexist. The coexistence of multiple steady-state solutions at a particular value of a bifurcation parameter is termed multiplicity (or multistability, if the solutions are stable to small perturbations). Multiplicity may occur with supralinear and positive feedback interactions among components of a system of coupled ODEs, and oscillatory dynamics may be sustained if a negative feedback loop of interactions is present. Singularity theory (for review see (7,8)) provides a systematic framework to determine how many topologically distinct bifurcation diagrams exist in a nonlinear dynamic system, and to partition the multidimensional parameter space of the model into regions in which different types of bifurcation diagrams exist. This information can be used to classify control parameters, which play a crucial role in determining system dynamics by governing transitions between qualitatively different bifurcation diagrams.

This study applies bifurcation and singularity analysis to a relatively complex signal transduction and gene network that underlies the induction of long-term memory (LTM) to examine model dynamics and determine control parameters. Sensorimotor neuron synapses of the mollusk Aplysia have been utilized extensively as a model system for the study of the cellular and molecular processes underlying learning and memory (9–13). These synapses exhibit both short- and long-term facilitation after exposure to 5-HT. Long-term facilitation (LTF) requires both activation of protein kinase A (PKA) and transcription. Molecular processes that underlie LTF have been studied in detail (for review see (13)). LTF is a correlate of long-term sensitization (LTS) of Aplysia defensive withdrawal reflexes, a form of long-term memory (LTM) (13–16). Pettigrew et al. (17) developed a mathematical model of biochemical processes that underlie the induction of LTF. The model simulates experimental observations of the time courses of activity of protein kinases essential for LTF, specifically PKA and the MAP kinase isoform termed extracellular-regulated kinase (ERK), after temporally spaced or massed exposures to 5-HT. The model also simulates induction of a gene product (Aplysia ubiquitin hydrolase, or Ap-Uch) that is essential for LTF.

This study addresses the following questions regarding the nonlinear features of the Pettigrew model:

1. Is a multiplicity of steady states possible?

2. What key control parameters determine the dynamics of the system?

3. What distinct steady-state system behaviors exist?

4. What parameter values might be predicted to enhance the induction of LTF?

5. Which system behaviors are robust?

6. Might bifurcation and singularity analysis be helpful to design drugs to improve the induction of memory?

In analyzing the dynamics of the model, this study focuses on the role of a positive feedback loop involving reciprocal induction of PKA activity and Ap-uch gene expression. The results indicate that the positive feedback loop can generate a robust and irreversible switch between a basal steady state of low PKA activity and gene expression, and a stimulated steady state of high PKA activity and gene expression. A prolonged steady state of high gene expression might contribute to the strengthening of synapses between neurons, and therefore to the induction and maintenance of LTM. Singularity theory was used to construct classification diagrams of the parameter space. These diagrams help identify molecular processes that might be manipulated, perhaps pharmacologically, to enhance the induction and maintenance of LTM. These methods of systematic analysis will also be helpful to understand a broad class of complex biological phenomena amenable to modeling, such as development and cellular differentiation (18–23), or the cell cycle (3).

METHODS

The model

For this analysis, the model of Pettigrew et al. (17) was used (see Table 1). The Pettigrew model (17) was developed to study biophysical mechanisms that may be implicated in learning and memory. Analysis of this model shows that it does not exhibit multiplicity of solutions (the coexistence of multiple steady-state solutions) with the parameter values previously published in Pettigrew et al. (17). A goal of this study is to illustrate a methodology for finding various types of bifurcation diagrams and their location in the parameter space. By altering four parameters, i.e., k_ApSyn, k_ApSynBasal, K_pka, and K_erk (new values given in Table 1), the model exhibits multiplicity and, therefore, a more diverse set of bifurcation diagrams. With these four new parameter values, the model continues to respond to stimuli as in Pettigrew et al. (17).

TABLE 1.

Long-term memory (LTM) model (17); differential equations of the LTM model and standard parameter values

Differential equations of the LTM model and standard parameter values
Differential equations for modeling PKA activity after exposure to 5-HT and hydrolysis of PKA R-subunits by Ap-Uch:
(1)
(2)
(3)
(4)
where
Differential equations for modeling REG and pREG dynamics:
(5)
(6)
(7)
Differential equations for modeling the Raf/MEK/ERK pathway:
(8)
(9)
(10)
where
(11)
(12)
where
Differential equations for Ap-Uch synthesis:
(13)
(14)
(15)
Model parameters and their values:
Vsyn = 0.002 min−1, kfpka = 105.0 min−1, kAp-Uch = 0.007 min−1, Ap-Uch basal = 0.10, kdpka = 0.00048 min−1, τ = 250 min, kbaspka = 12 min−1, Kreg = 0.00064, ktranslation = 4 min−1, vrphos = 1 min−1, Krphos = 1.5, vdreg = 0.16 min−1, Kdreg = 0.0015, kdsm = 0.02 min−1, vmREG = 2 × 10−5 min−1, vdmreg = 0.00225 min−1, Kdmreg = 0.01, kdmreg = 3 × 10−5 min−1, kf,Raf = 0.00345 min−1, kf,MAPKK = 0.7 min−1, kf,ERK = 0.44 min−1, kb,Raf = 0.001 min−1, kb,MAPKK = 0.12 min−1, kb,ERK = 0.12 min−1, [Raftot] = 0.5, [MAPKKtot] = 0.5, [ERKtot] = 0.5, KMK = 0.08, ERKbasal = 0.015, kphos1 = 0.01 min−1, kdephos1 = 1.5 min−1, PPhos = 0.1, kphos2 = 0.005 min−1, kdephos2 = 0.5 min−1, kApSyn = 0.02 min−1, kApSynBasal = 0.0009 min−1, Kpka = 0.2, Kerk = 0.004, kdeg = 0.01 min−1.

The key biochemical cascades represented by the model are illustrated in Fig. 1. Two transcription factors, TF-1 and TF-2, regulate expression of Ap-uch. When neither TF is phosphorylated, TF-2 represses Ap-uch and TF-1 has no effect. After exposure to 5-HT, ERK phosphorylates TF-2, relieving repression by TF-2, and PKA phosphorylates TF-1, allowing TF-1 to induce Ap-uch. (For further description of the model development and its validation, see (17).) Pettigrew et al. (17) assumed specific molecular identities for the two transcription factors. TF-1 was assumed to be the cAMP response element binding protein (CREB1), and TF-2 was assumed to be CREB2. However, recent results (24) suggest that induction of Ap-uch by 5-HT may not be mediated directly by CREB1 and CREB2, but rather by as yet unidentified transcription factors. Nevertheless, Ap-uch induction is regulated via the cAMP signaling pathway (25), and [cAMP] elevation activates both PKA and ERK in neurons (26,27). Therefore, it remains plausible that PKA and ERK induce Ap-uch by phosphorylating transcription factors, as the model assumes (Fig. 1).

FIGURE 1.

Schematic diagram depicting the model. Serotonin (5-HT) elevates cAMP, which binds to the PKA holoenzyme and initiates dissociation of free catalytic (C) and regulatory (R) subunits. The C subunits phosphorylate the transcription factor TF-1. TF-1 then activates Ap-uch gene expression, resulting in Ap-Uch protein synthesis. By increasing the availability of free ubiquitin, Ap-Uch enhances degradation of R subunits to sustain PKA activation, closing a PKA/TF-1/Ap-Uch positive feedback loop. 5-HT also induces translation and phosphorylation of a hypothetical protein termed REG. REG slows the reassociation of C and R subunits of PKA, sustaining intermediate-term (1–3 h) PKA activation. 5-HT also activates ERK (via Raf and MAPKK), and ERK then phosphorylates TF-2, relieving TF-2's repression of Ap-uch transcription. A basal phosphatase activity dephosphorylates TF-1 and TF-2.

The model consists of a set of 15 nonlinear differential equations (Table 1): 12 ODEs (Eqs. 1 and 5–15) and three time-delay differential equations (DDEs) (Eqs. 2–4). The dynamics of the model can be described by the time course of the activity of the catalytic subunit of PKA (heretofore referred to as PKA activity). The model represents three biochemical pathways essential for the induction of LTF. The first pathway is a positive feedback loop in which PKA phosphorylates the transcription factor TF-1, which in turn enhances the expression of the gene coding for Ap-Uch. Ap-Uch promotes degradation of the regulatory subunit of PKA, releasing a free, active catalytic subunit. The resulting increase in PKA activity closes the positive feedback loop. The second pathway is the cascade in which 5-HT activates ERK, leading to phosphorylation of TF-2, and reinforcing induction of Ap-uch by TF-1. In the third pathway, translation of a hypothetical protein, denoted REG, is induced by 5-HT. REG prolongs PKA activity for ∼1–3 h by preventing free catalytic subunit from reassociating with the regulatory subunit. This intermediate-term prolongation of PKA activity contrasts with the long-term activation of PKA by Ap-Uch, which lasts for >20 h.

In the model, long-term PKA activity is assumed to correlate with LTM. This assumption appears plausible for the following reasons. Inhibition of PKA after 5-HT exposure blocks LTF, which is a correlate of LTM (28). Also, Ap-Uch mediates long-term PKA activation (28), and inhibition of Ap-Uch function or expression blocks LTF (25).

Methods of predicting nonlinear features of the model

The 15 differential equations (Eqs. 1–15, Table 1) can be represented as a vector equation,

(16)

with initial conditions: .

In Eq. 16, is a 15-dimensional vector of nonlinear functions; x = ([cAMP], [R], [C], [RC], [REG], [pREG], [mRNA_REG], [Raf], [MAPKK], [MAPKK^pp], [ERK], [ERK^pp], P_pka, P_erk, [Ap-Uch]) is the 15-dimensional vector of system variables; λ is the bifurcation parameter; τ denotes the time delay in Eqs. 2–4; and is the 38-dimensional vector of system parameters other than λ (i.e., the kinetic rate constants listed in Table 1). The full parameter space of the model, , is defined to include all real values for all parameters including λ. However, a restricted parameter space is also defined, which does not include λ, because a principle goal of bifurcation analysis is to subdivide this restricted parameter space into regions that exhibit qualitatively different dynamics when λ is varied. In practice, biochemical constraints (e.g., plausible rate constants and concentrations) constrain the portion of parameter space that is physiologically relevant. The concentration of 5-HT, [5-HT], is selected as the bifurcation parameter λ because [5-HT] is an important extracellular stimulus commonly varied in experiments, including experiments to determine a threshold for persistent PKA activation.

To use bifurcation and singularity theory to analyze the nonlinear properties of this dynamic system, steady states are first obtained by setting dx/dt = 0 in Eq. 16, yielding 15 nonlinear algebraic equations

(17)

Equation 17 does not contain the time delay τ (in Eq. 16). Although τ has a significant effect on the transient dynamics of the system and might affect the stability of some steady states, it has no effect on the existence of a steady-state solution, or on the existence and location of limit points and singular points in the steady-state bifurcation diagrams. Therefore, setting the time delay τ to zero does not alter the steady-state bifurcation and singularity analysis described in this article. Hence, we simplify the steady-state bifurcation analysis by eliminating τ. We subsequently verify the predictions of the bifurcation analysis by simulation of the full model including the time delay.

Bifurcation diagrams, singularity theory, and classification diagrams

Two bifurcation diagrams are defined to be similar if the number, order, and orientation of the steady-state solutions x change in an identical way as the bifurcation parameter λ is varied. Singularity theory (8,29,30) was used to determine the regions in the restricted parameter space (not including λ) that contain similar bifurcation diagrams, by tracing the region boundaries. Golubitsky and Schaeffer (8) showed that the boundaries consist of one of three types of hypersurfaces: a hysteresis surface (HS), an isola surface, or a double limit set (DLS). When the values of parameters are biochemically constrained to a certain range (e.g., nonnegative concentrations), a fourth type of hypersurface, a boundary limit set (BLS), subdivides the parameter space into allowed and disallowed regions. The hypersurfaces are defined within the full parameter space including λ (Eqs. 19–22, Appendix). However, their projections into the restricted parameter space are used to subdivide the restricted space into regions with qualitatively different bifurcation diagrams. DLS surfaces only exist with bifurcation diagrams displaying five or more steady-state solutions for specific values of the bifurcation parameter. The coexistence of more than three steady states was not observed in this study, and thus, DLS surfaces do not appear to exist in the parameter space of the present model.

Steady-state singular points or singularities are points in the full parameter space at which the Jacobian of the function f in Eq. 17 (see Appendix) has a zero eigenvalue (7,8). Singular points are defined to be of codimension n, if n−1 parameters, in addition to the bifurcation parameter, are specified to locate the point (see Appendix). A limit point, or saddle-node bifurcation point, is a point on a bifurcation diagram where the number of steady-state solutions changes by two as the bifurcation parameter λ is varied. A limit point is a singular point of codimension-1 because only the bifurcation parameter λ needs to be specified to determine the point (Appendix, Eq. 18). HS, isola, and DLS hypersurfaces are also comprised of singular points. These singular points are of codimension-2, because, to determine the point, two parameters (λ and an additional parameter P₁) are specified (Appendix, Eqs. 19, Eqs. 21, and Eqs. 22). Column 3 of Table 2 illustrates the types of bifurcation diagrams that exist at critical singular points on the various boundaries (HS, isola surface, or BLS) between the regions of the restricted parameter space. Columns 2 and 4 of Table 2 display the bifurcation diagrams obtained by universal unfolding of these hypersurfaces, with universal unfolding defined as perturbation away from the surface by a small change of a parameter other than λ (8,31). Unfolding HS leads to two possible scenarios, either a monotonically dependent bifurcation diagram with no limit point, or a hysteretic loop type bifurcation diagram (bistability) with two limit points that coalesce on the critical hysteresis surface (HS). Unfolding an isola surface causes an isolated branch of solutions to appear (see Table 2, Isola1 and Isola2). Table 2 illustrates two types of isolated branches, or isolae, i.e., elliptic and hyperbolic. Unfolding the BLS causes the limit point of the bifurcation diagram at the feasibility boundary of the bifurcation parameter λ to move either into or away from the feasibility range. All these unfolding scenarios are observed with the present model, as discussed below (Results, Fig. 3).

TABLE 2.

Schematics of universal unfolding of the bifurcation diagrams of steady-state codimension-2 singular points (7,8,31)

Steady-state codimension-2 singularities applied to LTM model	Bifurcation diagram of singular point perturbed in one direction	Bifurcation diagram at the singular point	Bifurcation diagram of singular point perturbed in opposite direction
Hysteresis surface(HS)
Isola 1(hyperbolic)
Isola 2(elliptic)
Boundary limit set(BLS)

x is the vector of system variables, and λ is the bifurcation parameter ([5-HT] in the LTM model). (Column 1) Types of hypersurfaces in LTM model. (Column 3) Bifurcation diagrams at the codimension-2 singular points. (Columns 2 and 4) The corresponding unfolded bifurcation diagrams resulting from a small perturbation of any parameter P₁ other than λ. The direction of the perturbation (increase versus decrease of P₁) is opposite in columns 2 and 4. Depicted are steady states that are stable (solid lines) and unstable (dashed lines) as well as bifurcation points (solid circles). The boundary limit set for the LTM model is at λ([5-HT]) = 0.

FIGURE 3.

Typical bifurcation diagrams of PKA activity versus [5-HT] at the values of k_ApSyn within the seven different intervals marked 1–7 of Fig. 2 B. Insets show expanded view of negative [5-HT] axis for 3, 4, and 6. Critical points are marked by squares. Arrows depict direction of trajectory mapped onto the bifurcation diagram.

In this study, singularity theory was applied to identify the qualitatively different types of steady-state bifurcation diagrams (x vs. [5-HT]) exhibited by the model of Fig. 1 (Table 1). Using two-parameter planes, curves were constructed that represent the intersections of the planes with the hypersurfaces HS, isola, and BLS. These curves are loci of codimension-2 points, and are boundaries between regions that contain qualitatively distinct steady-state bifurcation diagrams. Such a partition of a two-parameter plane into regions of distinct bifurcation diagrams is termed a classification diagram. For example, a classification diagram of the plane (k_Ap-Uch, k_ApSyn) provides the types of bifurcation diagrams that may be obtained by varying the parameters k_Ap-Uch and k_ApSyn (with all other parameters fixed).

Computational methods

An outline of the general procedure for constructing a classification diagram is as follows (see Appendix for further details). A steady-state bifurcation diagram is constructed to identify codimension-1 limit points (e.g., Fig. 2 A). These limit points are used as initial points to construct a curve, or locus, of limit points. This locus constitutes a codimension-1 diagram (e.g., Fig. 2 B). An arc-length continuation scheme (32,33) is used to construct the curve (locus of limit points) by linear extrapolation of two points on the curve to determine a third point by the Newton-Raphson iteration scheme for solving nonlinear algebraic equations (34). The codimension-1 locus contains critical points that are of type codimension-2. The codimension-2 points are then used as initial points for constructing a diagram of loci of codimension-2 singularities, termed a classification diagram (e.g., Fig. 4). To construct this diagram, the arc-length continuation scheme is applied again.

FIGURE 2.

Construction of codimension-1 diagram. (A) Bifurcation diagram of PKA activity versus [5-HT]. Limit points are marked by cross points. For this and subsequent figures, parameter values are as in Table 1 unless noted otherwise. (B) Codimension-1 diagram in the ([5-HT], k_ApSyn) plane illustrating the limit set (LS), loci of limit points (black curve). The two cross points on LS correspond to the two limit points in A with k_ApSyn = 0.02 (value in Table 1) (green dash-dot line). Six codimension-2 singular points are denoted by red dots marked a–f and labeled HS1, HS2, Isola1, Isola2, BLS1, and BSL2. These points divide the k_ApSyn axis into seven intervals marked 1–7 (in blue). Dotted lines show values of k_ApSyn where codimension-2 singular points are. The inset expands the region of Isola1 and BSL1 and shows interval 4 on the k_ApSyn axis.

FIGURE 4.

Classification diagram constructed by the loci of six codimension-2 singular points. The (k_ApSyn, k_Ap-Uch) plane is divided into seven regions, marked 1–7, by curves that are continuations of the six codimension-2 singular points in Fig. 2 B (blue, hysteresis surface (HS); black, boundary limit set (BLS); red, Isola). Each region (1–7) with k_Ap-Uch = 0.007 contains a qualitatively different type of bifurcation diagram of PKA activity versus [5-HT] corresponding to the diagrams in Fig. 3, 1–7, respectively. Region 4 is very small, and an inset (above main figure) is required for its visualization.

To perform a comprehensive bifurcation and singularity analysis and locate all of the steady-state solutions, it is necessary to investigate both positive and negative ranges of the bifurcation parameter, [5-HT]. After completing the overall bifurcation and singularity analysis, the singularity analysis can be simplified and focus on the physiologically relevant range [5-HT] ≥ 0, as is illustrated in Figs. 5–8. The analysis, carried out on both positive and negative values of the bifurcation parameter before simplification to physiological values, is necessary to locate all steady-state solutions.

FIGURE 5.

Bifurcation diagrams of PKA activity versus [5-HT] for nonnegative [5-HT]. (A) Unique steady-state branch of low PKA activity for k_ApSyn = 0.0022. (B) Bistability with a reversible switch for k_ApSyn = 0.015. (C) Bistability with an irreversible switch and three steady states at [5-HT] = 0 for k_ApSyn = 0.03. (D) Unique steady-state branch of high PKA activity for k_ApSyn = 0.1.

FIGURE 8.

The size and location of the irreversible-switch domain, shown in Figs. 6 c and 7 c, can be altered by parameter values. (A) Effects of doubling k_phos1 or k_phos2 on the boundaries of the domain, BLS1 and BLS2. Depicted are k_phos1 = 0.01 and k_phos2 = 0.005 (default values) (solid curves); k_phos1 doubled to 0.02 (dotted curves); k_phos2 doubled to 0.01 (dashed curves). (B) Effects of doubling k_ApSyn or k_Ap-Uch. Depicted are default values: k_ApSyn = 0.03 and k_Ap-Uch = 0.007 (solid curves); k_ApSyn doubled to 0.06 (dotted curves); and k_Ap-Uch doubled to 0.014 (dashed curves).

FIGURE 6.

Simplified codimension-2 classification diagram for nonnegative [5-HT] in the (k_ApSyn, k_Ap-Uch) plane. The plane is divided into four regions (a–d) by curves that are continuations of three codimension-2 singular points; HS1, BLS1, and BLS2 (Fig. 2 B). The solid circles (at k_Ap-Uch = 0.007) in each region correspond to the bifurcation diagrams of PKA activity versus [5-HT] shown in Fig. 9 A.

FIGURE 7.

Simplified codimension-2 classification diagram for nonnegative [5-HT] in the (k_phos1, k_phos2) plane. The parameter plane is divided into four regions, marked a–d, by curves that are continuations of three codimension-2 singular points; HS1, BLS1, and BLS2 (Fig. 2 B).

Computations were performed on a PC with a Pentium 4 processor (2 GHz). Constructing codimension-2 diagrams required several minutes (<10 min to generate the diagram of Fig. 2 B) and under 1 h to generate the classification diagram of Fig. 4. The method should scale linearly for higher-order models.

RESULTS

Construction of the limit set identifies seven distinct bifurcation diagrams

To begin to analyze the steady-state features of the model, a bifurcation diagram was constructed (Fig. 2 A) to provide initial codimension-1 points. The bifurcation diagram exhibits multiplicity with three steady states of PKA activity within a range of [5-HT] (0.064 < [5-HT] < 0.105), The lower and upper branches are stable and the middle branch is unstable. Beyond this range of [5-HT], the diagram exhibits a single stable branch, low PKA activity for [5-HT] < 0.064, and high PKA activity for [5-HT] > 0.105.

To further examine the steady-state solutions of the model, the PKA/Ap-Uch positive feedback loop (see Methods) was analyzed in greater detail. The synthesis of Ap-Uch plays a key role in regulating the strength of the positive feedback loop, because Ap-Uch is induced by PKA activation and subsequently hydrolyzes the R subunits of PKA to support long-term PKA activation. Using the two limit points (marked by crosses) in Fig. 2 A as initial codimension-1 points, a codimension-1 diagram was constructed in the parameter plane of the Ap-Uch synthesis rate constant (k_ApSyn) versus the bifurcation parameter [5-HT] (Fig. 2 B). To construct this diagram, the locus of limit points, also called the limit set (LS), was traced using an arc-length continuation scheme (see Methods). The two crosses marked on the LS in Fig. 2 B correspond to the two limit points in Fig. 2 A with k_ApSyn = 0.02 (standard value in Table 1). The LS in the (k_ApSyn,[5-HT]) plane is a closed loop.

Six codimension-2 singular points were found on LS, denoted in Fig. 2 B as HS1, HS2, Isola1, Isola2, BLS1, and BLS2. These singular points divide the range of k_ApSyn into seven intervals labeled 1–7 in Fig. 2 B. Each interval represents a distinct region of parameter space that is described by a different bifurcation diagram (see Fig. 3).

Bifurcation diagrams in each of these seven intervals were characterized. In interval 1 of Fig. 2 B where k_ApSyn is less than the critical value of HS1, PKA activity is a monotonic function of [5-HT] (bifurcation diagram in panel 1 of Fig. 3), because no limit point exists in this interval. PKA activity is at a very low level at [5-HT] = 0. In interval 2 in which k_ApSyn is greater than HS1, multiplicity appears. The bifurcation diagram exactly at the HS1 point is schematized in Table 2 (HS). After crossing HS1, the bifurcation diagram exhibits two limit points, a hysteresis loop, and an interval of [5-HT] with three steady states of PKA activity. A diagram from interval 2 is shown in panel 2 of Fig. 3, in which a hysteresis loop is illustrated by the transitions between the two branches of stable steady-state solutions, which occur upon reaching the two limit points.

Upon increasing k_ApSyn to intervals 3 and 4, another hysteresis surface point, HS2, is crossed. Two new limit points, and a narrow range of multiplicity between them, appear at negative [5-HT] (panels 3 and 4 of Fig. 3, insets). The bifurcation diagrams in panels 3 and 4 of Fig. 3, with four coexisting limit points, are of the mushroom type (8). In interval 4 (see Fig. 2 B, inset and the segment of the LS between BLS2 and HS1), a line with fixed k_ApSyn intersects the LS curve at four points, only one of which is at positive [5-HT]. In the corresponding bifurcation diagram (panel 4 of Fig. 3), only one limit point is at positive [5-HT]. By further increasing k_ApSyn in Fig. 2 B, HS1 and HS2 merge at the point (d: Isola1). Here, an isola hypersurface intersects the (k_ApSyn,[5-HT]) plane. The bifurcation diagram at Isola1 is illustrated in Table 2 (Isola1, hyperbolic), in which the two branches of the mushroom merge at the value of [5-HT] at Isola1. The number of limit points decreases from four to two. Upon crossing Isola1, k_ApSyn goes to intervals 5 and 6. An isola of steady-state solutions appears (panels 5 and 6 of Fig. 3). For those values of [5-HT] at which three steady states of PKA activity coexist, the lower and upper states are stable whereas the middle state is unstable (5–8). However, with an isola, the two stable steady-state branches are not connected to each other. Therefore, the system cannot pass from one branch to the other through a hysteresis mechanism or loop. However, isolae demonstrate a mechanism of irreversible, nonhysteretic transition between steady states. For example, suppose in either panel 5 or 6 of Fig. 3 that the system is initialized in the lower steady state. Then, upon increasing [5-HT], the lower steady state will disappear (right boundary of isola) and the system will transit to the upper steady state. The transition is irreversible in that if [5-HT] is then decreased to zero, the upper steady state remains stable, and PKA activity remains high. However, a brief but large dynamic perturbation of system variables or parameters might cause a state transition in either direction by forcing the system out of the basin of attraction of either steady state.

With a further increase of k_ApSyn, the isola shrinks to a point at Isola2 (Fig. 2 B, corresponding bifurcation diagram in Table 2 (Isola2, elliptic)). By crossing Isola2 to interval 7, the isola disappears, leaving only a unique steady state of high PKA activity (panel 7 of Fig. 3).

In addition to the effects of the HS and isola points (Fig. 2 B) on the model behavior, the effects of the boundary limit set (BLS) points also need to be considered to characterize the complete set of steady-state behaviors of the model. The BLS is the hypersurface in the full parameter space of limit points located at zero [5-HT] (i.e., at the physically meaningful boundary of [5-HT]). Fig. 2 B shows two codimension-2 singular points on the LS loop, denoted BLS1 and BLS2, at which the BLS hypersurface intersects the (k_ApSyn,[5-HT]) plane. At BLS1, one limit point in the bifurcation diagram is at zero [5-HT] (see BLS in Table 2). BLS1 divides the k_Apsyn interval containing a mushroom-type bifurcation diagram into two subdomains, intervals 3 and 4 in Fig. 2 B. Typical diagrams in these intervals are illustrated in panels 3 and 4 of Fig. 3, respectively. Upon unfolding BLS1 (i.e., perturbing k_ApSyn slightly above or below BLS1), either one steady-state solution (panel 3) or three solutions (panel 4) appear at [5-HT] = 0. At BLS2, the right limit point of the isola is at zero [5-HT], while the other limit point lies in negative [5-HT] (not shown). In Fig. 2 B, BLS2 divides the isola interval into two subdomains, intervals 5 and 6. Typical bifurcation diagrams are shown in panels 5 and 6 of Fig. 3, respectively. In Fig. 2 B, the curve that connects BLS2 and HS1 represents a threshold of 5-HT stimulation. If [5-HT] remains to the right of this curve, then the lower steady state of PKA activity no longer exists (the system is to the right of the lower limit points in Fig. 2 A or panels 2–5 of Fig. 3). The system will transit to the only stable state, of high PKA activity.

It is evident from the above that the LS in Fig. 2 B plays a key role in bifurcation analysis. The LS enables rapid determination of all qualitatively different bifurcation diagrams of PKA activity versus [5-HT] that can be obtained at different values of a system parameter (k_ApSyn). Furthermore, the LS allows for the identification of codimension-2 singular points (HS, isola, BLS points in Fig. 2 B). These points, and the associated hypersurfaces, are needed to characterize the effect of changing two or more parameters on the dynamics of the system. Specifically, loci of these codimension-2 points in the plane of any two parameters P₁ and P₂ (in the parameter space P) can be constructed. A classification diagram of these loci subdivides the plane into regions, each of which manifests a qualitatively different type of (x,λ) bifurcation diagram.

The loci of six codimension-2 singular points partition a key classification diagram into seven regions

In addition to k_ApSyn, a second parameter that determines the strength of the PKA/Ap-Uch positive feedback loop is k_Ap-Uch. The rate constant k_Ap-Uch describes degradation of the R subunit of PKA by Ap-Uch, enhancing the catalytic (C) activity of PKA (Fig. 1, Eqs. 2–4 in Table 1). Therefore, a classification diagram in the (k_ApSyn,k_Ap-Uch) plane, illustrating qualitative changes in dynamics as these parameters are varied, is likely to provide significant insight into regulation of the model's behavior. By continuation (varying k_Ap-Uch, see Methods) of the six codimension-2 singular points (in Fig. 2 B) in the (k_ApSyn,k_Ap-Uch) plane, this classification diagram of Fig. 4 was constructed. At any point on a codimension-2 curve, the bifurcation diagram of PKA activity versus [5-HT] is qualitatively like that of the corresponding codimension-2 singular point in Table 2. For example, at any point on the Isola1 curve, the bifurcation diagram will show an isola pinching off from another branch of stable solutions (Table 2). At any point within a region between curves in Fig. 4, the bifurcation diagram is qualitatively like the corresponding diagram in Fig. 3. For example, throughout region 2 of Fig. 4, the bifurcation diagram is like that of panel 2 of Fig. 3, with two limit points and a hysteresis interval of [5-HT] between the limit points. Three steady states will be present within this [5-HT] interval, and only one steady state will be present for all other values of [5-HT]. The specific diagrams of Fig. 3 lie along the line k_Ap-Uch = 0.007 min⁻¹ in Fig. 4.

Only a subset of bifurcation diagrams is present for physiologically relevant 5-HT concentrations

By using the loci of codimension-1 singular points (LS in Fig. 2 B), different bifurcation diagrams (Fig. 3) can be derived and insights can be obtained into the ways in which these diagrams transit between each other via codimension-2 singular points (Table 2). Loci of these codimension-2 points partition classification diagrams (Fig. 4) into regions with distinct bifurcation diagrams. Analogous diagrams exist for any two parameters other than [5-HT]). These results were derived allowing [5-HT] to vary over negative, zero, and positive values. Physiologically, only nonnegative [5-HT] is relevant. Can the portions of bifurcation diagrams at negative [5-HT] be neglected and a simplified analysis derived that focuses on physiological significance?

To address this question, the seven bifurcation diagrams of Fig. 3 were further inspected. By neglecting the portions of diagrams with negative [5-HT], the diagrams can be reclassified into four types. In panel 1 of Fig. 3, the PKA activity exhibits a unique, monotonically increasing function for [5-HT] , but because the PKA/Ap-Uch positive feedback is weak (low k_ApSyn), PKA activity remains low. In panels 2 and 3 of Fig. 3, PKA activity manifests three steady-state values within a positive range of [5-HT]. In panels 4 and 5 of Fig. 3, PKA activity manifests three steady states within a range that includes [5-HT] = 0. In panels 6 and 7 of Fig. 3, a unique steady state again exists, but because the PKA/Ap-Uch positive feedback is strong (high k_ApSyn), PKA activity is high even at zero [5-HT]. These bifurcation diagrams with [5-HT] are plotted in Fig. 5. Fig. 5, A–D, corresponds to panels 1–7 of Fig. 3 as follows: Fig. 5 A to panel 1; Fig. 5 B to panels 2 and 3; Fig. 5 C to panels 4 and 5; and Fig. 5 D to panels 6 and 7.

There are three codimension-2 singular points with either positive or zero [5-HT] in Fig. 2 B: HS1, BLS1, and BLS2. These three points divide the range of k_ApSyn into four intervals, a, b, c, and d. The intervals a–d correspond to the intervals 1–7 of Fig. 2 B as follows: a to 1; b to 2 and 3; c to 4 and 5; and d to 6 and 7. The corresponding bifurcation diagram in interval a, below HS1, is shown in Fig. 5 A. Interval b, bounded by HS1 and BLS1, has the bifurcation diagram shown in Fig. 5 B. In interval b, a line with fixed k_ApSyn will intersect the LS in Fig. 2 B at two points for [5-HT] 0. These points correspond to the two limit points in the bifurcation diagram shown in Fig. 5 B (LP1 and LP2). In interval c of k_ApSyn, bounded by BLS1 and BLS2, the bifurcation diagram corresponds to Fig. 5 C. A line with fixed k_ApSyn in Fig. 2 B intersects the LS at only one point for [5-HT] 0, and the corresponding bifurcation diagram in Fig. 5 C also shows one limit point. In interval d, with k_ApSyn above BLS2, the bifurcation diagram corresponds to Fig. 5 D and has no limit point for [5-HT] 0.

The bifurcation diagram in Fig. 5 B illustrates a hysteretic loop between two limit points. Such a structure is termed a reversible switch or toggle switch, because alterations in [5-HT] can carry the system to the left or right of both limit points, inducing transitions of PKA activity between the upper and lower steady states. This multiplicity is termed bistability since the upper and lower solutions are stable whereas the middle solution is unstable. The limit point with larger [5-HT], LP1, defines a threshold of [5-HT] required to induce high PKA activity. Physiologically, such a threshold might act as a filter, by which a weak 5-HT stimulus can be regarded as noise and neglected. A saturation of PKA activity was also observed in this bifurcation diagram for [5-HT] > ∼0.16. Saturation might help to protect neurons against excessive gene induction and energy expenditure in response to frequent stimuli.

Although Fig. 5 loses part of the detailed information about how these bifurcation diagrams originate (Fig. 3) and transit between each other, it is derived from the overall bifurcation and singularity theory, and focuses on the physiologically relevant concentrations of 5-HT. A classification diagram similar to Fig. 4, but restricted to [5-HT] 0, can be constructed (Fig. 6) by continuing the codimension-2 singular points of Fig. 2 B that have nonnegative [5-HT] (HS1, BLS1, and BLS2) in the k_ApSyn-k_Ap-Uch plane. Regions a–d bounded by the loci of these singular points have the bifurcation diagrams illustrated in Fig. 5, A–D, respectively.

Bifurcation analysis reveals a robust irreversible switch

The type of bifurcation diagram is largely determined by the strength of the positive feedback loop in which PKA activity and Ap-uch expression reciprocally enhance each other (Fig. 1). Whereas Fig. 5 B displays a bifurcation diagram that is analogous to a reversible switch (k_ApSyn = 0.015), increasing the strength of the positive feedback generates a bifurcation diagram analogous to an irreversible switch (Fig. 5 C, k_ApSyn increased to 0.03). For Fig. 5 C, the system switches from the off-state (low PKA activity) to the on-state of high activity upon crossing the limit point LP where the off-state ceases to exist. However, because only one limit point exists for nonnegative [5-HT], the system remains at high PKA activity if [5-HT] is then reduced to zero. In the classification diagram of the (k_ApSyn, k_Ap-Uch) plane (Fig. 6), the reversible and irreversible switches lie within regions b and c, respectively. At first glance, it might appear that the irreversible switch originates from the broken hysteretic loop of Fig. 5 B. However, the analysis indicates that the irreversible switch originates from an isola bifurcation (Figs. 3 and 4), since interval 5 of k_ApSyn in Fig. 2 B, where the isola exists, is much larger than interval 4, with the broken hysteretic loop. If the positive feedback is very strong, the bifurcation diagram of PKA activity versus [5-HT] again manifests a unique steady state, with high PKA activity even at zero [5-HT] (Fig. 5 D). This would be analogous to an unphysiological state of constant kinase overactivity.

In the classification diagram of Fig. 6, region c, which corresponds to the irreversible-switch bifurcation diagram, is relatively large. This observation suggests that, generically, the irreversible switch will be preserved for small parameter variations. However, the (k_ApSyn,k_Ap-Uch) plane is only one of the possible slices through the restricted parameter space. Does the irreversible-switch region retain substantial size if the parameter space is examined using different parameters? To address this question, the (k_phos1,k_phos2) parameter plane was used to construct an analogous classification diagram (Fig. 7) (k_phos1 is the rate constant with which PKA phosphorylates TF-1; k_phos2 is the rate constant with which ERK phosphorylates TF-2). Although Figs. 6 and 7 appear somewhat different, their essential characteristics are similar. In both figures, one hysteretic surface (HS1) and two boundary limit sets (BLS1 and BLS2) divide the plane into four regions that correspond to the topologically distinct bifurcation diagrams in Fig. 5, A–D. As in Fig. 6, the region corresponding to the irreversible switch (region c in Fig. 7) remains relatively large. The other regions a, b, and d also remain of substantial size. This result strengthens the conclusion that the irreversible switch, as well as the dynamics represented in the other regions, are robust to small parameter variations.

To further assess the robustness of the irreversible switch, the effects of some large parameter changes on region c, Figs. 6 and 7, were examined. Fig. 8 A illustrates the consequences of doubling either k_phos1 or k_phos2. It is evident that the size of region c decreases somewhat upon doubling k_phos1 or k_phos2. The shift of the BLS2 boundary to the left is more significant than that of BLS1. However, the region is still substantially large. In particular, when k_Ap-Uch is high, a large range of k_ApSyn generates an irreversible switch. Fig. 8 B exhibits the consequences of doubling either k_Ap-Uch or k_ApSyn. Increasing k_ApSyn does not significantly alter the size of the irreversible switch region bounded by BLS1 and BLS2. Increasing k_Ap-Uch can considerably enlarge the irreversible switch domain. These results illustrate that the irreversible switch exists in a substantial region of parameter space, and is robust to large parameter changes. This robustness suggests that the irreversible switch mechanism might be observed to function in vivo.

Simulations verify steady-state bifurcation analysis

The above bifurcation analyses do not consider the time delay τ (Eq. 16) in the original model of Fig. 1. The presence of a time delay cannot eliminate any of the steady states in Figs. 2–8. However, a delay can affect the dynamic behavior as the system evolves toward steady states, and may alter their stability. Therefore, it is important to complement the steady-state analyses with simulations in which the system is initially far from steady state.

Four distinct steady-state bifurcation diagrams are plotted in Fig. 9. As k_ApSyn is increased, a monotonic steady state of low PKA activity (k_ApSyn = 0.0022) changes to a hysteretic loop with two limit points and three steady states of PKA activity between these points (k_ApSyn = 0.021). At higher k_ApSyn (0.03), an irreversible switch is obtained, with only one limit point at positive [5-HT] and three steady states at [5-HT] = 0. At yet higher k_ApSyn (0.1), only one steady state exists, with high PKA activity. These four bifurcation curves correspond to the four regions of Fig. 6 and are marked in Fig. 6 by the dots on the line k_Ap-Uch = 0.007 min⁻¹.

FIGURE 9.

Dynamic simulations of full LTM model. (A) Typical examples of four qualitatively different steady-state bifurcation diagrams of PKA activity versus [5-HT] (≥0) are plotted together. These diagrams correspond to the black dots in Fig. 6. (B–E) Time courses of four simulations with 100 h of 10 μM 5-HT stimulation beginning at t = 0. Parameter values correspond to the four bifurcation diagrams in A.

Fig. 9, B–E, illustrate simulations of the model (Table 1) with the time delay (τ = 250 min). By applying a long (100 h) stimulus, these simulations verify the stability of the steady states in Fig. 9 A. With k_ApSyn = 0.0022, 100 h of 10 μM 5-HT stimulation elevates PKA activity to a plateau (Fig. 9 B). The activity returns to its baseline rapidly after the stimulus ends, and this baseline corresponds to the lowest steady state in Fig. 9 A with [5-HT] = 0. With k_ApSyn = 0.021 (the reversible switch bifurcation diagram in Fig. 9 A), similar dynamics to Fig. 9 B are observed (Fig. 9 C). However, after the same 5-HT stimulation, PKA activity slowly returns to the steady-state value (in Fig. 9 A with [5-HT] = 0, k_ApSyn = 0.021). With k_ApSyn = 0.03 (the irreversible switch bifurcation diagram in Fig. 9 A), a prolonged stimulus of 10 μM 5-HT induces a transition to a high, stable level of PKA activity. This state remains stable when [5-HT] returns to 0 μM at t = 100 h, and PKA activity is permanently locked in a high state (Fig. 9 D). Finally, with very strong positive feedback (k_ApSyn = 0.1) (Fig. 9 E), PKA activity converges to the highest stable state of Fig. 9 A even with [5-HT] fixed at 0 μM.

The model (Table 1) contains three delay differential equations (DDEs) (Eqs. 2–4). To locate the steady-state solutions, limit points, and singular points, the model in vector form (Eq. 16) was reduced to the algebraic system (Eq. 17) without time delay. Time delays do not alter the existence of a steady state, or the locations of codimension-1 and -2 steady-state singular points (35). The focus of this article is to use steady-state singularity analysis to compute the singular points and construct classification diagrams of parameter space to determine the regions in which different steady-state bifurcation diagrams exist. This analysis is invariant to the presence of a time delay.

Analyses of DDEs and ODEs do differ in terms of the procedure to determine the stability of a steady state. Time delays may destabilize a steady state under certain conditions. In this study, we did not systematically investigate how the time delay affects the stability of steady states through analysis of the eigenvalues of linearized DDEs. Instead, we verified the stability of several key steady-state solutions on the different bifurcation diagrams in Fig. 9 A by simulation of the model using different values of τ. For example, we investigated the stability of the upper steady-state branch of the irreversible switch at [5-HT] = 0 (with k_ApSyn = 0.03) in Fig. 9 A. A delay equal to or smaller than that used by Pettigrew et al. (17) (τ ≤ 250 min) has a negligible impact on the transients as the system converges to the stable steady state. Increasing the delay 20-fold to 5000 min causes the system to converge to the steady state by way of damped oscillations with small amplitude. With yet a longer time delay (10,000 min), the amplitude of the damped oscillation increases, but the stability is still not altered. Similar results were found for the lower steady-state branch of the irreversible switch (with k_ApSyn = 0.03) in Fig. 9 A. The middle steady-state branch is unstable, therefore it cannot be reached with transient simulations. We also examined the stability of steady-state solutions of the three other bifurcation diagrams in Fig. 9 A and found that the stability is not altered by time delays as large as 5000 min. Therefore, we conclude that physiologically reasonable time delays do not alter the stability of the steady states in the parameter space that we investigated.

By applying pulsed stimulation protocols, which are often implemented experimentally (27,28), simulations were performed in the parameter regimes of the reversible switch (k_ApSyn = 0.021) and irreversible switch (k_ApSyn = 0.03) (Fig. 10). After five pulses of 5-HT (10 μM for 5 min, with an interstimulus interval of 20 min), PKA activity is strongly activated at ∼24 h in both cases (Fig. 10 A). However, PKA activity gradually drops to its baseline in the reversible switch case after removing 5-HT, whereas PKA activity remains at high level in the irreversible switch case (Fig. 10 B). After three pulses of 5-HT, PKA activity is only slightly and briefly elevated in both cases (not shown). Thus, three pulses of 5-HT are subthreshold, whereas five pulses cross the threshold for long-term PKA activation. Empirically, five pulses of 5-HT, but not three, induce long-term PKA activation (36), as well as LTF (10,37). In vivo, four or five spaced shocks to Aplysia induce long-term sensitization (LTS) of withdrawal reflexes, whereas three shocks do not (11). These observations suggest the threshold stimulus duration for long-term PKA activation (five pulses) corresponds to a threshold for the formation of LTS, a simple form of LTM.

FIGURE 10.

Time courses of PKA activity. (A) Twenty-five hours during and after five pulses of 5-HT (10 μM for 5 min, interstimulus interval of 20 min). The time courses correspond, respectively, to the values of k_ApSyn for the reversible (0.021) and irreversible (0.03) switches in Fig. 9 A. (B) Same as A, except continued for 1000 h.

For the value of k_ApSyn (0.021) corresponding to the reversible switch, the simulation protocol of Fig. 10 was repeated, varying the amplitude of the 5-HT stimulus. These simulations illustrated a dynamic threshold for stimulus amplitude at ∼7.5 μM [5-HT]. This stimulus is the minimum amplitude to induce high PKA activity by using the five-pulse stimulation protocol. However, the steady-state threshold for PKA activity in Fig. 9 A is just above a 5-HT stimulus of ∼0.106 μM. This steady-state threshold is the minimal stimulus amplitude to induce high PKA activity by continuous 5-HT application or very long stimulus duration. A simulation with [5-HT] held at 0.15 μM, slightly above the steady-state threshold, showed that to induce high PKA activity, ∼4500 min of stimulation was required.

DISCUSSION

Analyses with parameters unconstrained, and with parameters restricted to physiological ranges, are important for characterizing the model

The classification diagram of parameter space in Fig. 6 is a simplified version of Fig. 4 that considers only the three codimension-2 singular points for [5-HT] ≥ 0 in Fig. 2 B. It might appear that if analysis focuses on physiologically relevant parameter values, a subset of the singular points may suffice to classify parameter space. However, for a complete analysis it is necessary to investigate both the positive and negative ranges of the bifurcation parameter [5-HT]. This analysis is based on locating the limit points of the model, some of which are at negative [5-HT] (Fig. 2 B), to characterize the possible bifurcation diagrams. The limit points are used to locate the codimension-2 singular points (Fig. 2 B), which mark the boundaries between various types of bifurcation diagrams (Fig. 3). In our analysis, most of these singular points are at negative [5-HT] (Fig. 2 B). Only after identifying these points, and the intervals between them, can we classify the different types of bifurcation diagrams and construct specific examples (Fig. 3). If only positive values of the bifurcation parameter are examined, we risk missing a boundary between regions in the full classification diagram (Fig. 4), as well as incorrectly identifying the transitions between them (see Table 2).

For example, examining negative [5-HT] allows us to locate the BLS2 limit point. BLS2 represents the boundary between regions c and d in the classification diagram (Fig. 6) corresponding to the bifurcation diagrams of the irreversible switch type (Fig. 5 C) and monotonic type (Fig. 5 D), respectively. The irreversible switch is a potential mechanism for the induction or maintenance of LTM. In addition, without the analysis with negative [5-HT], it is not possible to establish that the bifurcation diagrams containing an irreversible switch (Fig. 5 C, corresponding to both intervals 4 and 5 of k_ApSyn in Fig. 2 B) mainly originate from a broken isola bifurcation, and not from a broken hysteretic loop. The origin of the irreversible switch can be observed from Figs. 2 B and 4, in which interval or region 5 (corresponding to the isola bifurcation diagram in Fig. 3, panel 5) is large, whereas interval or region 4 (corresponding to the hysteretic bifurcation diagram in Fig. 3, panel 4) is small (see Fig. 2 B, inset).

Bifurcation analysis and simulation complement each other

Simulation is essential to study the temporal evolution of a model, whereas bifurcation analysis provides a steady-state view. These two methods of analysis complement each other, as illustrated by comparing the simulations of Figs. 9, B–E, and 10 with the bifurcation diagrams of Fig. 9 A. Simulations with the full model (Table 1) including the time delay τ are consistent with steady-state features of bifurcation diagrams derived without τ. This comparison emphasizes the necessity of performing both steady-state analysis and simulations to understand the complex interactions within a biochemical network. Bifurcation analysis systematically characterizes the parameter space to determine regions that generate qualitatively different steady-state or oscillatory solutions. For complex models, this analysis is necessary, because it is computationally impractical to characterize the dynamics by a very large number of random or systematic simulations throughout the multidimensional parameter space. In contrast, simulations can generate time courses of system variables that provide detailed information on how the variables respond to stimuli and interact with each other. Simulations also provide insights into the methods that may be used to reach steady states (e.g., by determining the minimum stimulus amplitude and duration required to reach a specific steady state).

Irreversible switch dynamics may contribute to the formation of LTM

Irreversible switches have been suggested to be important for cellular differentiation and development (18–23); entry into mitosis in the cell cycle (3); ligand-induced autocrine growth factor signaling (38); Sonic Hedgehog signaling (39); and long-term memory formation (40–44). The common feature is conversion of a transient stimulus into an irreversible response through the activation of a network of intrinsically reversible signaling molecules and gene products.

Several types of irreversible switches have been proposed to contribute to the maintenance of LTM. Lisman and Fallon (41) and Roberson and Sweatt (44,45) suggested several possible switches that might sustain long-term changes in the strength of synapses. Lisman and co-workers (43,46,47) proposed CaMKII activity may be sustained indefinitely by positive feedback in which active, phosphorylated CAMKII multimers further phosphorylate their subunits, and this CAMKII activity might maintain long-lasting synaptic strengthening or potentiation (LTP) and therefore LTM. Iyengar and co-workers (40,48) showed that a feedback loop involving protein kinase activities and second messengers (including protein kinase C, Raf, MAP kinase kinase, and MAP kinase) can produce a bistable molecular switch at a synapse, which when activated may induce LTP. Kandel and co-workers proposed a positive feedback mechanism for protein oligomerization (49–51) that may be important for sustaining persistent modification of synaptic structure, allowing for LTM. The irreversible switch exhibited by the present model is similarly based on a positive feedback loop, with reciprocal induction of PKA activation and Ap-uch expression (see Methods). Such a switch (Figs. 5 C and 6) could maintain long-lasting PKA activation, transcription factor (TF) phosphorylation, and gene induction in response to relatively brief stimuli, and could therefore be important for consolidation of LTM, which is known to require gene expression.

Switches based on positive feedback loops, such as the irreversible switch (Fig. 5 C), are commonly described by deterministic ODEs or DDEs, which use chemical concentrations as continuous variables. This representation implicitly assumes that the number of molecules in the system is sufficiently large to ensure that deterministic, mass-action kinetics can describe the dynamics. However, if the copy numbers of some key kinases, or other molecular species, in the feedback loop are relatively low (e.g., tens), then stochastic noise and fluctuations in molecular copy numbers may induce spontaneous transitions between states (23,52). Therefore, molecular noise may be a key factor that affects the robustness of the bistable switch when the copy numbers of molecules are low.

Whether the PKA/Ap-Uch positive feedback loop does, in fact, play an important role in the formation of LTM remains to be determined experimentally. If this loop exhibits switchlike behavior, then a relatively brief activation of PKA by an agonist that elevates cAMP levels, such as forskolin or 5-HT, should induce a subsequent, long-lasting (many hours or days) phase of PKA activation and Ap-uch expression. Indeed, in Aplysia neuronal ganglia, application of five 5-min pulses of 5-HT induces PKA activation measured 24 h later (36). Although this result is compatible with long-lasting positive feedback, the time course of PKA activation needs to be examined at additional time points. Crucially, concomitant long-lasting induction of Ap-uch would need to occur to support the hypothesis that brief stimuli can induce irreversible switchlike activation of the positive feedback. Furthermore, a molecular pathway from PKA activation to Ap-uch gene induction should be identified. As discussed above (see Methods), Ap-uch does not appear to be induced via phosphorylation of the CREB1 transcriptional activator. However, PKA activation could lead to phosphorylation of an unspecified TF that induces Ap-uch. A major goal of this article, not dependent on these model details, is to point out that positive feedback loops may be important for sustaining long-lasting kinase activation and gene expression to induce, or maintain, LTM.

We note that in vivo, any biochemical switch associated with the induction of LTM is not likely to remain irreversible. It is likely that processes of molecular turnover, not represented in the present model, would tend to return kinase activities and synaptic strengths to average levels. For example, several current theories of long-term synaptic plasticity envision homeostatic mechanisms that rescale synaptic weights over time (53,54).

Classification diagrams can help predict effects of drugs or other environmental stimuli

Classification diagrams of parameter space (e.g., Figs. 4 and 6–8) provide boundaries within the parameter space that differentiate among multiple bifurcation diagrams and multiple stimulus-response outcomes (e.g., hysteresis or irreversible switches). Classification diagrams help to determine which dynamic features are robust to variation of parameter values. Only those bifurcation diagrams that occupy a region of significant size in multiple classification diagrams (e.g., Figs. 6 c and 7 c, and perturbations of this region in Fig. 8) are likely to describe the physiological dynamics.

Codimension-1 bifurcation diagrams analogous to Fig. 2 B can be used to determine parameters that, if varied by pharmacological or environmental intervention, would have a major qualitative effect on the dynamics of the system. For example, Fig. 2 B illustrates that a compound affecting the rate of Ap-uch transcription or protein synthesis (the rate constant k_ApSyn) would significantly alter the dynamics (e.g., by changing hysteretic behavior to monotonic or irreversible-switch behavior). Diagrams similar to Fig. 2 B can be combined with considerations of which parameters could be altered relatively selectively by chemical or other stimuli. In this way, potential targets could be identified for drug design and therapy, as well as for toxins or other environmental stimuli.

Classification diagrams analogous to Figs. 4 and 6 provide a visual map to determine how the system will respond if two parameters are varied simultaneously by pharmacological or environmental stimuli, or if one parameter is varied in systems characterized by differing values of a second parameter. Therefore, classification diagrams may provide a visual means of determining the effect of two drugs applied simultaneously, and possibly illustrate a synergetic effect, which would not be observed if the drugs were applied individually or sequentially. For example, Fig. 6 illustrates that compounds or drugs might enhance the induction of LTM by altering either k_ApSyn or k_Ap-Uch or both. Increasing the strength of positive feedback by increasing k_ApSyn and/or k_Ap-Uch favors more prolonged and greater activation of PKA. A monotonic relationship of PKA activity versus [5-HT] (region a) is converted sequentially to a hysteretic reversible switch (region b), then to an irreversible switch (region c). In vivo, compounds that increase Ap-Uch synthesis (k_ApSyn) and/or increase the effectiveness with which Ap-Uch promotes PKA activation (k_Ap-Uch) would plausibly promote persistent PKA activation, TF phosphorylation, and synaptic strengthening. Near boundary curves of classification diagrams, the system is particularly sensitive to parameter changes, because small changes can qualitatively alter system behavior.

Pettigrew et al. (17) proposed (their Fig. 6 A) that a compound able to reduce the dephosphorylation rates of MAPKK (k_{b, MAPKK}) and ERK (k_{b, ERK}), might enhance 5-HT-induced Ap-uch expression and PKA activation. The effect of reducing both these rate constants was therefore examined. Bifurcation analysis of the unaltered system (k_{b, MAPKK} = k_{b, ERK} = 0.12 min⁻¹) and the altered system (k_{b, MAPKK} = k_{b, ERK} = 0.06 min⁻¹) (other parameters as in Table 1) show that the qualitative nature of the bifurcation diagram (a hysteretic loop/reversible switch) is not changed by decreasing k_{b, MAPKK} and k_{b, ERK}, although the threshold for PKA activation by 5-HT is considerably reduced (not shown). An irreversible-switch type of bifurcation diagram cannot be generated by further decreasing k_b,MAPKK and k_b,ERK (not shown), because the strength of the positive feedback loop (which is governed by k_ApSyn and k_Ap-Uch) remains insufficient. These simulations suggest alteration of kinetic parameters outside the positive feedback loop can decrease the threshold for 5-HT necessary for significant PKA activation, but is less likely to enhance the duration of PKA activation, TF phosphorylation, and gene induction. The analysis predicts that alteration of kinetic parameters within the positive feedback loop might be more likely to enhance both the intensity and duration of gene induction, and the magnitude of LTM.

The proposal that alteration of MAPKK/ERK kinetic parameters might enhance PKA activation was based originally on sensitivity analysis (17). For each parameter in the model, a small change was made, and the resulting change in the magnitude of PKA activation was determined. PKA activation was found to be particularly sensitive to k_b,MAPKK and k_b,ERK. However, a caveat of sensitivity analysis is that small parameter changes will not change the qualitative bifurcation diagram when a state is not located near the boundary curves of a classification diagram. Larger parameter changes can qualitatively alter the bifurcation diagram. Such changes might simulate a mutation or a pharmacological intervention. Such alteration often corresponds to a nonlinear change in dynamic stimulus responses that could not be predicted by scaling up the effect of a small parameter change. Classification diagrams of parameter space (Figs. 4 and 6) and codimension-1 bifurcation diagrams (Fig. 2 B) may be used to identify parameters which, when varied, may move system behavior into a regime described by a different type of bifurcation diagram.

However, sensitivity analysis cannot be discounted. It is rapid and easily performed. A complex parameter space generally has too many dimensions to be explored thoroughly by bifurcation analysis. Therefore, sensitivity analysis is needed to determine a class of control parameters, defined as those that strongly affect the stimulus response of a system. Bifurcation analysis can then focus, at least initially, on characterizing how the dynamics are altered by larger changes in the subset of control parameters, which are plausible targets for drugs or other biochemical stimuli (17).

There appears to be significant conservation of biochemical mechanisms underlying synaptic strengthening and memory formation between organisms as diverse as Aplysia and vertebrates, including mammals. In mammals, as in Aplysia, the formation of many examples of LTM correlates with changes in synaptic strength (55,56). These changes, like Aplysia LTF, are associated with MAP kinase and PKA activation and gene induction (57–60), and with phosphorylation and activation of the CREB family of transcription factors (61–63). Activation of CREB appears critical for the formation of at least some forms of vertebrate LTM (64–68). Because of this conservation of mechanisms, it is likely that the methods of analyses used here will help us to understand the dynamics of memory formation in other organisms, and the ways in which memory formation might be altered by drugs or other environmental interventions.

APPENDIX

Further details of the equations and methods for calculation of steady-state singular points (8,31,69) are given here. The codimension-1 singularity termed a limit point is defined by the set of conditions

(18)

with f as in Eq. 17 and with 〈.,.〉 denoting the standard inner product, i.e., 〈u, u〉 = u^T · u. J is the Jacobian matrix of f, with the elements . The value u is the eigenvector of J corresponding to zero eigenvalue. The proceeding set of 15 × 2 + 1 algebraic equations determines the 15 × 2 variables (x,u) and the bifurcation parameter λ. Because these equations only determine one parameter, λ, the limit point is codimension-1. By varying continuously an additional parameter P₁, the loci of the limit points in the (P₁,λ)-plane can be obtained. This curve, or the corresponding hypersurface of singular points in the full parameter space, is usually termed a limit set (LS) (the intersection of the hypersurface with the (P₁,λ) plane yields the curve). The LS curve can be effectively traced out by an arc-length continuation scheme (32,33). It has been shown (see Results) that the LS enables rapid determination of all the qualitatively different bifurcation diagrams that can be attained for different values of the parameter P₁.

Codimension-2 singular points on HS, isola, or BLS hypersurfaces (see their bifurcation diagrams in Fig. 3 and Table 2) are defined as follows. On an HS,

(19)

where J* is the conjugate transpose of the Jacobian matrix J. The values u and v are the eigenvector and adjoint eigenvector of J, and the second differential vector of the vector function f is

(20)

The set of Eqs. 19 includes 3 × 15 + 2 equations that determine the 3 × 15 variables (x,u,v), the bifurcation parameter λ, and an additional parameter P₁. Because both λ and P₁ are determined, the point is codimension-2.

A point on an isola surface (elliptic or hyperbolic) (8,31) satisfies the conditions

(21)

These 2 × 15 + 2 equations determine 2 × 15 variables (x,v), the bifurcation parameter λ, and an additional parameter P₁. By continuously varying another parameter P₂ and using arc-length continuation, the curve of isola points in the (P₂,P₁)-plane can be obtained.

A boundary limit set (BLS) is a hypersurface on which a limit point in the bifurcation diagram (x vs. λ) occurs at the physiological boundary of the bifurcation parameter. In our model, the physiological boundary occurs when the concentration of 5-HT reaches 0. A point on the BLS therefore satisfies the conditions

(22)

These 2 × 15 + 2 equations determine the 2 × 15 variables (x,u), the bifurcation parameter λ, and an additional parameter P₁. By varying continuously another parameter P₂ the BLS curve can be constructed in the (P₂,P₁) plane using an arc-length continuation scheme. In our model, additional BLS surfaces could be defined for each biochemical rate constant, because they are constrained to be nonnegative.

To solve the above equations, the differential equations (Eq. 16) are first numerically integrated to obtain an approximate steady state. The steady state is more precisely computed by the Newton-Raphson method (34). Arc-length continuation is then used to trace out the bifurcation diagram in the (x,λ) plane and obtain the codimension-1 limit points (the turning points of the bifurcation diagram). Arc-length continuation is used again to trace out a curve of limit points in the (λ,P₁) plane. Some of these points may be identified as codimension-2 points that solve Eqs. 19, 21, or 22 for the fixed value of a second parameter P₂. If so, arc-length continuation is used yet again, varying P₁ and P₂, and tracing out curves of the codimension-2 points in the (P₂,P₁) plane. As discussed above, these curves are intersections of the HS, isola, or BLS surfaces with the (P₂,P₁) plane, and make up the classification diagram that divides the (P₂,P₁) plane into regions with qualitatively different x versus λ-bifurcation diagrams.