Associating changes in output behavior with changes in parameter values in spiking and bursting neuron models

Abstract

Several recent studies have demonstrated that neuronal models allow multiple parameter value solutions for a given output. In the face of this variability of parameter values, what can be learned about neural function through parameter value differences? Here, in two different models, we examine this question by attempting to reconstruct the source of model output changes based on simple statistical analyses of parameter distributions generated by automated searches. We conclude that changes to parameter values or their associated distributions do not reliably reflect the specific mechanisms responsible for a given change in output.

1. Introduction

Mathematical models of neuronal activity provide an excellent platform for rigorous examination of hypotheses of neuronal function and as such have been a hallmark of neuroscience for decades. However, as these models have increased in complexity, it has also become apparent that the values of the parameters controlling model behavior can differ and yet still result in functionally identical model behavior. This variability of parameter values has been shown in simple and complex single compartment neuron models (Foster et al. 1993; Goldman et al. 2001) and at the small network level (Prinz et al. 2004). However, modeling studies often rely on the relationship between parameter value differences and output differences as a means to identify neuronal mechanisms related to a particular function (Hill et al. 2001; Purvis and Butera 2005). Thus, the apparent non-uniqueness of parameter values could call this practice into question.

Parameter value non-uniqueness is likely the result of under-specification of the model’s parameters; that is, in many modern neuronal models, the number of model parameters exceeds the number of model outputs of interest (i.e. the models behavior) that could be used to specify those parameters. As more complex models, with hundreds or even thousands of parameters are developed, the phenomenon will become more severe. Left unaddressed, an increasing level of parameter value non-uniqueness in neuronal models could potentially threaten to undermine the effectiveness of neuronal modeling itself.

In contrast, it could be argued that experimental study assumes variability from the outset. Even with the most stringent experimental controls, no researcher expects every functionally equivalent neuron examined to be exactly the same. Instead, experimentalists expect measurements of neural properties to be merely similar under controlled conditions. Thus, rather than an expectation of uniqueness, experimentalists expect measured values to be constrained in a manner that can be captured by statistical methods with experimental results being reported as averages, standard deviations, correlation coefficients etc. This begs the question: Can parameter value non-uniqueness in neuronal models be handled in a similar manner?

In this study, we explore the idea that neuronal models can be characterized in a practical manner similar to real neurons and neural circuits; namely, by statistical examination and profiling of populations of functionally equivalent parameter sets. Specifically, we seek to examine the premise that changes in underlying mechanisms responsible for changes in model behavior will be reflected in a statistical examination of populations of neuron models. Based on the work presented here, we conclude that parameter value non-uniqueness can be characterized by statistical means with based on a reasonable number of automated searches, but that statistically linking output shifts to the exact parameters responsible for those shifts is not reliable. Instead, the results appear to indicate that statistically determined parameter shifts are better taken as general indicators of the underlying mechanism responsible rather than literal sources of model behavior shifts.

2. Methods

2.1. Mathematical models

To ensure that a more general conclusion could be drawn, our studies used two, notably different neuronal models: a single-compartment model of a pre-Bötzinger complex (pBc) bursting neuron (Butera et al. 1999), and a multi-compartment model of a cat spinal cord motoneuron (Kuo et al. 2006; Shapiro and Lee 2007). The pBc bursting neuron model is based on the Hodgkin-Huxley formalism, and contains a fast-inactivating sodium conductance, a slow-inactivating persistent sodium conductance, a non-inactivating delayed rectifier potassium conductance, and a passive leak conductance. Our pBc bursting neuron model differs from the initial published model in that it does not contain the synaptic current I_tonic-e, or the applied current I_app, and that the state variable for sodium channel inactivation was separate from the state variable of potassium activation. (The base parameter values of this model are given in Table A.1.) This model was coded with both the Simulink modeling package (Mathworks, Inc), and with DIESEL (Simatra Technologies) using a variable-step Dormand-Prince integrator with a minimum step size of 10⁻⁶ s, a maximum step size of 0.1 s, and a relative tolerance of 10⁻⁴. For every simulation of the model, the model output was given 10 s. to settle, after which 100 s of simulation data were recorded.

The spinal cord motoneuron model used for these studies was a simplified version of that presented in Shaprio 2007, as it contained only three compartments corresponding to the dendrite, soma, and initial axon segment (IS) of the neuron. The salient features of this model are four-state sodium channels derived from the Kuo and Bean twelve-state Markov Model (Kuo and Bean 1994), Hodgkin-Huxley style delayed-rectified potassium channels, and multiple calcium channels and transporters. (The base parameter values of this model are given in Table A.2.) This model was coded in object-Pascal and DIESEL (Simatra Technologies). Evolution of the model’s state variables was calculated using a variable-step, 1^st order solver. Without external stimulation, the model is at rest, and the initial conditions for each simulation were adjusted to reflect steady-state values for each state variable.

2.2. Terminology

In our experience, characterizing and describing properties of populations of parameter sets can be daunting without some specialized nomenclature (Mitchell and Lee 2007). Thus, we use the term “behavior” to denote the set of model output quantifications derived from one or more stimulation protocols. We use the term “homolog” to describe a single set of parameter values that result in model behavior that meets a stated goal. Thus a homolog is an individual member of a population of parameter sets all result in the same model behavior. Within this population we will denote the homolog that was used to define the goal behavior for that population as the “base homolog.” Finally, as we will be creating new populations by shifting parameter values in the base homolog, we will denote these new bases as “shifted base homologs”

2.3. Model behavior measurement

As stated above, for each model, we examined the output of the model for several standardized protocols (i.e. the model’s behavior) at a base parameter value set (i.e. the base homolog for that model). The model’s behavior was used to define a target behavior that was used as the goal for parameter searches. The protocols and measures that constitute each model behavior metrics are described below.

The pre-Bötzinger complex (pBc) bursting neuron model generates intrinsic rhythmic bursting activity. For each simulation, we recorded 100 s of simulation data, and then measured six outputs of the system based on this intrinsic activity: period (measured from the first spike of a burst to the first spike of the next burst), burst duration, number of spikes per burst, mean spike height (measured from 0 mV), slow-wave trough (the minimum membrane potential between bursts), and slow-wave peak (the maximum of the minimum inter-spike membrane potentials during a burst). These protocols are shown in Figure 1. The base homolog produced rhythmic bursting with a period of 4.1 s, a burst duration of 0.9 s, 43 spikes per burst, slow-wave trough and peak of −53.6 mV and −42.2 mV, respectively, and mean spike height of −3.7 mV.

Figure 1.

Stimulation protocol and typical output for pre-Bötzinger complex bursting neuron model.

The spinal cord motoneuron (MN) model is intrinsically quiescent, producing activity only in response to external stimulation. We quantified the behavior of the model based on six different stimulation protocols: sag, spike, spike threshold, rheobase, ramp frequency-current (F-I), and current-voltage (IV). From these protocols, fifteen outputs were used as the target of the guided parameter search: first spike level, F₀, I₀, SP gain, input conductance, G_{Fast Min}, fast onset, rheobase, sag ratio, after-hyperpolarization (AHP) duration, AHP magnitude, spike height, late after-depolarization (ADP) time to peak, spike width, and voltage threshold. These protocols and typical outputs are shown in Figure 2.

Figure 2.

Stimulation protocols and typical outputs for spinal cord motoneuron model. A: Sag protocol. B: Spike protocol. C: Rheobase protocol. D: Spike train elicited by F-I protocol. E: F-I response of F-I protocol. F: I-V protocol.

In the sag protocol, a −4 nA current step is applied to the soma, and the sag ratio (ratio of final to peak voltage deflection) is measured. In the spike protocol, a 30 nA, 0.3 ms current pulse is applied to the soma in order to elicit a single spike. From the spike protocol we take the following output measurements: spike height (measured from the pre-pulse resting potential), spike width (measured at 10 mV above the resting potential), after-hyperpolarization (AHP) duration (measured from when the membrane potential crosses below the resting potential to when the membrane potential regains the resting potential), AHP magnitude (measured from the resting potential), and late after depolarization (ADP) time to peak (measured from spike initiation). In the spike threshold protocol, successive 50 ms current pulses are administered are applied to determine the threshold current for spiking to within 1 nA. From the spike threshold protocol, voltage threshold is measured (measured as the maximum membrane potential at which the neuron does not fire). The rheobase protocol calculates the current necessary to elicit a single spike 50% of the time within 50 ms of current application. From the rheobase protocol we measured the threshold current of firing (rheobase). In the ramp F-I protocol, we apply a 6 nA/s current ramp with 0.5 nA of sum-of-sinusoids pseudo-noise with a base frequency of 0.02 Hz added to the neuron (Marmarellis & Marmarellis). From this protocol, the following measurements were taken: first spike level (magnitude of first spike in spike train), steady-state primary-range (SP) gain (the slope of the F-I relationship, excluding the first five spikes and last spike), F₀ (instantaneous spike frequency of the first inter-spike interval), I₀ (value of ramp current when neuron begin rhythmic firing). In the I-V protocol, the model is voltage clamped with an 8 mV/s ramp function added to a 0.15 mV, 125 Hz sine wave. From this protocol, we measure the fast onset (point of zero slope in the fast I-V curve), input conductance (G_in), and minimum conductance of the fast I-V function (G_{Fast Min}, measured at the point of maximum negative slope on the fast I-V curve).

2.4. Parameter Searches

To generate homologs for a model behavior target, we applied a random perturbation to the base homolog parameter values, creating a set of perturbed parameters. In the motoneuron model, the perturbation for each parameter was drawn from a uniform random distribution over the range +/− 5% of the base parameter value. In the pBc bursting neuron model, the perturbation for most parameters was drawn from a uniform random distribution of +/− 1% of the base parameter values. The pBc bursting neuron model used a smaller perturbation because it was much more sensitive to parameter perturbation than the motoneuron model; perturbations of 5% would almost always drive the model to a state where the search algorithm (described below) failed to converge on an acceptable solution.

The perturbed parameters were then used as the starting point for a modified Powell conjugate search algorithm that attempted to drive the model behavior to the target behavior (Powell 1964 - See Discussion). A detailed description of this method is presented in Mitchell and Lee (2007). Briefly, each parameter is adjusted in turn (the order in which parameters were adjusted was randomized for each search), to minimize the residual sum squared error of the model’s behavior compared to the target behavior. As each parameter may be minimized several times, subsequent to such repetitions a conjugate parameter vector is generated. This vector points in the aggregated direction of all the parameter shifts since any prior conjugate vector and is used for a single search with the goal of accelerating convergence. In general, steps along these conjugate vectors were not overly large. Thus, acceleration was achieved but not dramatically so. Searches were terminated based on convergence criteria specific to each model and described below.

For the pBc bursting neuron model, a simple sum-of-squares error was calculated for the target output:

$$\xi =\underset{i=1}{\overset{6}{\Sigma }}{\left(\frac{x_i-x_{it}}{x_{it}}\right)}^2$$ Equation 1

where x_i is the i^th output, and x_it is the target value for that behavior metric. We allowed two iterations of the search algorithm (a total of 42 individual parameter minimizations and two minimizations along the Powell conjugate vector) for the model to converge. Of course, not all converged searches produced model behavior sufficiently close to the target behavior. Therefore, we accepted the convergence only when all of the outputs were within 10% of the target behavior measures (i.e. max error <10%)

For the motoneuron model, we weighted each output measure based on the known inaccuracies of experimental techniques (Gustaffson and Pinter 1984; Kernell 1965a; 1972; 1965b; c; Zengel et al. 1985):

$$\xi =\underset{i=1}{\overset{15}{\Sigma }}\frac{1}{W_i}{\left(\frac{x_i-x_{it}}{x_{it}}\right)}^2$$ Equation 2

where W_i is the weighting factor for the i^th output. The weighting factors are provided in the appendix. We allowed 500 iterations for the outputs of the model to converge on the target behavior. We accepted convergence of the outputs when the final weighted error was less than the number of target outputs (15). Thus the sum of all errors was approximately equal to or less than one standard experimental deviation.

For each model, we repeated the perturbation and search process for approximately one week in order to produce enough distinct parameter points to allow statistical analysis. For the pBc bursting neuron model, 102 distinct points were produced, and for the motoneuron model, 110 points were produced.

2.5. Statistical Methods

Since the goal of this study was to assess the connection between shifted and “found” parameter values, the overall variability for each parameter needed to be determined. Each parameter (p) was first normalized by the median value of that parameter (p_med) across a population of homologs:

$$p^{\prime }=(p-p_{med})∕p_{med}$$ Equation 3

where p’ is the normalized value of the parameter. Next outliers (defined as greater than 5 standard deviations out from the mean) were eliminated. Note that the entire homolog was eliminated if even one parameter was determined to be an outlier. Eliminating the outliers left us with n = 98 homologs for the pBc bursting neuron model and n = 105 homologs for the motoneuron model.

All statistical analysis was performed using the Matlab statistics toolbox. For each parameter distribution, we first performed a Lilliefors test at the p = 0.05 significance level to determine if the data were normally distributed. Because a majority of the parameter distributions were non-normal in character, we relied on nonparametric methods for comparison of the different distributions. The base and shifted homolog populations were compared with a Kolmogorov-Smirnov test. As we were performing multiple comparison tests across the same set of data, we used a Bonferroni correction to the 0.05 significance level to determine significance. As an additional metric, we also compared the distributions using a Wilcoxon rank-sum (Mann-Whitney U) test for different medians, using the same Bonferroni-corrected level of significance.

3. Results

The central hypothesis examined in this study is that neuronal models can be characterized by the distributions of the parameter values that produce equivalent behavior. These distributions are presumed to represent the inherent constraints on parameter values within neuronal models and as such can be treated in a manner similar to experimental measures where significance is assessed by statistical comparison of distributions of measured values. To test this hypothesis we must examine the parameter value non-uniqueness in our test models, and then analyze the resultant distributions. We can then compare how the distributions change as the targeted behavior of the model changes.

To examine parameter value non-uniqueness, we first located sets of parameters that produce equivalent behavior (i.e. homologs). Starting with a previously determined base homolog that produced the target behavior of the model, we applied a random perturbation to every parameter value; a subsequent automated search was then used to adjust the parameter values so that the model once again produced the target behavior (See Methods). The perturbation and tuning process was then repeated until a large number of homologs were generated. We refer to this population of homologs as the base population.

3.1 Overall Effect of Searching on Parameter Values

Before examining specific parameter value changes we made a more general examination of the effect of searching on the variability of parameter values across the base population of homologs. Interestingly, with very few exceptions, parameter variability increases or remains largely unchanged rather than decreasing as would be expected if the search converged the parameters towards “correct” values (Figure 3 - panels A and B). Additionally, there is no correlation between parameter-space distance from the base homolog (i.e. the search-induced movement in the parameter values) and behavior-space distance from the base behavior (i.e. the final error; See Figure 10 – panels C and D).

Figure 3.

Effect of Searching on Parameter Distributions. A: Motoneuron model parameters sorted in order of range of values in base population. Parameter value ranges post-search (open squares) are larger than the initial perturbation used to start the searches (solid circles). B: Same as A but for pBc bursting neuron model. C: Relationship between final overall behavior error and overall parameter distance from median for Motoneuron model. D: Same as C but for pBc bursting neuron model.

Figure 10.

Probability density functions for statistically different parameter distributions between the base and gCa1.2 shifted searches for spinal cord motoneuron model. Note that the distribution of g_Ca1.2 itself was not different between the base and the shifted search.

To provide greater insight into these search-induced changes in parameter space, we used Principle Component Analysis (PCA) to examine the parameter-space dimensionality. The cumulative distribution of variance across dimensions is shown in Figure 4 for the pBc bursting neuron model and the motoneuron model. As there is no universally accepted method for extracting dimensionality from PCA we used two simple methods. The first is the lowest dimension that contains at least 95% of the variance. This method is more sensitive to the tail of the variance at higher dimensions. The second method is essentially the “area over the variance curve” that is more sensitive to the variance assigned to the lower dimensions:

$$d=1+\underset{i=1}{\overset{n}{\Sigma }}2(1-V_i)$$ Equation 4

where d is the assessed dimensionality, n is the total possible number of dimensions, and V_i is the variance explained at the i^th dimension. As a reference, the PCA results for random data with the same number of samples is provided (dotted lines). As expected, the PCA results for the initial perturbation of the parameters closely match those of random data for both the pBc bursting neuron model (Figure 4 panel A) and the motoneuron model (Figure 4 panel C). However, the post-search variance in parameter values is reduced with both dimensionality assessment metrics yielding essentially the same reduction (3 versus 3.3 for the pBc bursting neuron model and 5 versus 5.5 for the motoneuron model).

Figure 4.

Effect of Searching on Base Population Parameter-Space Dimensionality. A: Pre-search variance explained as a function of PCA dimension for pBc bursting neuron model. Pre-search variance (solid curve) is nearly identical to random data with same sample size (heavy dashed curve) as assessed by 95% (or more) of variance explained (light vertical dashed line) and “d-score” (see text). B: Post-search variance explained as a function of PCA dimension for pBc bursting neuron model. Markings same as A. Searching reduced dimensionality. C: Same as A but for motoneuron model. D: same as B but for motoneuron model.

Taking the increase in overall variance (Figure 3) and the reduction in dimensionality (Figure 4) due to searching together, it would appear that the specific search process used (i.e. unconstrained gradient descent) tends to cause a preferential or skewed divergence in parameter values to achieve the search goals.

3.2. Distribution of Individual Parameter Values

In the pBc bursting neuron model, the parameter search process generated 102 distinct parameter sets that produced the base model behavior. The difference between these parameter values can produce noticeably different steady-state activation, inactivation, and voltage-dependent time constant curves, even though the metrics underlying model behavior are equivalent (Figure 5). Likewise, in the spinal cord motoneuron model, the parameter search process generated 118 distinct parameter sets that produced the canonical output. As was the case with the pBc bursting neuron model, the values of these parameters are remarkably different even though the output of the model is equivalent (Figure 6, Table 1).

Figure 5.

Example of parameter value non-uniqueness in pre-Bötzinger complex bursting neuron model. Black and Grey traces indicate different homologs (sets #40 and #116 in the base distribution). Both homologs produce the same assessed behavior but with differing parameter values. A, B: Bursting activity produced by the two homologs. C: Table of maximal conductances, reversal potentials, and membrane capacitance. D-F: Steady-state activation (squares) and inactivation (circles) curves. G-I: Voltage-dependent time constant curves.

Figure 6.

Examples of behavior similarity from different homologs in spinal cord motoneuron model. Model behavior for four homologs from the base distribution are shown. A: Response to sag protocol B: Response to spike protocol C: Response to rheobase protocol. D: Frequency response to F-I protocol. E: Spike level response to F-I protocol F: Response to I-V protocol.

Table 1.

Selection of parameter values for outputs shown in Figure 3. Maximal conductances are given in nS

	Base	Point 8	Point 86	Point 89
Dendrite Kdr gmax	602.50	601.19	717.46	588.81
IS Kdr gmax	1000.00	892.32	1038.11	1040.67
Dendrite Na 1.6 gmax	4500.00	4343.83	4692.44	4326.92
IS Na 1.6 gmax	6000.00	5037.19	4712.13	5168.94
Soma Na 1.2 gmax	6000.00	6307.08	6173.50	5410.39
Dendrite gleak	292.00	282.57	290.62	286.42
Soma gleak	223.08	218.79	216.75	227.04
IS gleak	3.85	3.87	4.49	4.10
Dendrite C (pF)	1250.00	1281.90	1208.81	1202.38
Soma C (pF)	113.10	108.51	98.40	116.04
IS C (pF)	18.85	18.39	18.19	15.46

Subsequent analysis will require us to characterize the distribution of parameter values for each model. This characterization encompasses the structure of the distributions, differences in scale, and the position of the base parameter values within their respective distributions. It is worth noting here that these characteristics of the distributions are qualitatively similar between the two models. These characteristics are discussed below.

To identify appropriate statistical tests, we first analyzed the structure of the parameter distributions. In the pBc bursting neuron model, only 10 of the 21 parameter distributions studied were determined to be normal or log-normal based on a Lilliefors test at the p = 0.05 significance level (Figure 7). In the spinal motoneuron model, the same test revealed that only 21 of the 57 parameter distributions were normal or log normal (Figure 8). Although the lack of normality across the parameter distributions makes statistical analysis of the parameter distributions somewhat more difficult, this finding is not surprising, given that the distributions likely represent very complex shapes in their respective 21- and 57-dimensional spaces. Based on the lack of normality across all parameters we must use non-parametric statistical methods to analyze the relationships between parameters.

Figure 7.

Profile of the parameter value distributions in pre-Bötzinger complex bursting neuron model. All parameters are normalized to their median value across the distribution and arranged in decreasing order of their coefficient of variance. Each black point indicates the location of a particular parameter value. Vertical grey bar indicates the mean value of the parameter across the set. Horizontal grey bar shows +/− one standard deviation. Asterisks indicate location of the base parameter value in the normalized parameter space.

Figure 8.

Profile of the parameter value distributions in spinal cord motoneuron model. All parameters are normalized to their median value in the set and arranged in decreasing order of their coefficient of variance. Each black point indicates the location of a particular parameter value. Vertical grey bar indicates the mean value of the parameter across the set. Horizontal grey bar shows +/− one standard deviation. Asterisks indicate location of the parameter values.

To accommodate differences in units and magnitude, parameter values must be scaled. Given the wide variety of parameter types and the presence of normal and non-normal distributions, scaling parameters by their median value is a reasonable approach that can be applied in all cases. Thus scaled, it can be seen that some parameters are more constrained than others are. Interestingly, the ratio of the most constrained to the least constrained parameter (as measured by the coefficient of variation) was roughly the same for the pBc bursting neuron model and the motoneuron model (8.7 and 8.6, respectively). Finally, the parameter values comprising the base homolog for each model can be seen to be reasonably close to the median values (i.e. within 1/2 standard deviation). However, there is nothing “special” about base homologs, excepting that they are the only homologs to produce the targeted behavior with truly zero error.

3.3. Parameter distributions change with model output

To investigate how these distributions change as model output changes, we altered a single parameter value in the base homolog, creating a shifted base homolog and its corresponding shifted behavior. A new shifted distribution of parameter values was then generated by perturbing the parameter values and targeting the automated search to the shifted behavior. Statistical comparison of the shifted and base distributions can then identify which parameters are significantly different. Analysis of these changes in parameter value distributions should indicate whether the non-uniqueness phenomenon ultimately masks our ability to discern underlying mechanisms based solely upon parameter value differences.

In the pBc bursting neuron model, we examined the effect of shifting the following parameters in the base homolog to generate shifted base homologs: E_leak (the leak reversal potential), g_leak (the leak conductance), g_P (the maximal conductance of the persistent sodium current), and θ_mP (the half-maximal activation voltage of the persistent sodium current). Parameter values shifts were 2x the standard deviation of the base distribution for that parameter (Table 2). For each shifted base homolog a shifted behavior target was produced and used to generate a subsequent shifted population each with its own shifted distributions. These four sets of searches respectively produced populations with 77, 82, 84, and 77 homologs.

Table 2.

Parameter shifts and behavior targets for shifted searches in pBc bursting neuron model

	Baseline	Eleak shifted	gleak shifted	θmP shifted	gP shifted
Amount shifted		3.0 mV	0.5 nS	−3.0 mV	0.7 nS
Period (s)	4.1	1.8	6.3	2.0	3.0
Burst Duration (s)	0.9	0.82	0.72	0.94	1.0
Spikes/Burst	43	19	41	31	48
Slow-wave Trough (mV)	−53.6	−48.8	−53.6	−52.0	−53.5
Slow-wave Peak (mV)	−42.2	−44.5	−42.1	−43.9	−41.9
Spike Height (mV)	−3.7	−6.1	−2.9	−3.1	−3.8

The shifted distributions were qualitatively similar to the canonical distribution. Because many of these distributions were non-normal, Kolmogorov-Smirnov tests were used to determine if any of the shifted parameter distributions were statistically different from their canonical counterparts (Sachs 1982) at the p = 0.0024 significance level (.05/21, Bonferroni correction). A Wilcoxon rank-sum test at the same significance level was also used to evaluate differences between distributions (Sachs 1982); the results were consistent with the Kolmogorov-Smirnov test.

The statistically different parameters for each shifted population are shown in Figure 9. In the E_leak shifted search, the distribution of E_leak was not significantly different from the canonical distribution, but the distributions of the parameters g_K, g_P, τ_hP, and τ_mK were all different from the canonical. The θ_mP shifted search showed differences in the distributions for θ_mP, as well as g_P, τ_hP and σ_mK. The g_leak search showed differences in the distributions for g_leak and θ_hP. Finally, the only parameter significantly different between the two distributions in the g_P shifted search was g_P itself.

Figure 9.

Probability density functions for parameters with statistically different parameter value distributions between the base and shifted searches for pBc bursting neuron model. Top Row: statistically different parameters for Eleak shifted search. Second Row: Statistically different parameters for θ_mP shifted search. Third Row: Statistically different parameters for g_leak shifted search. Bottom Row: Statistically different parameters for g_P shifted search.

In the case of the g_P shifted search, both the canonical and shifted distributions of g_P were normal, so a standard t-test was used to further evaluate these distributions. The canonical and shifted distributions of g_P were significantly different at the p = 2.8×10⁻⁶ significance level, and the 95% confidence interval in the difference of the means is 0.2 to 0.6 nS, lower than the 0.7 nS shift that was applied to the canonical parameters to obtain the shifted output.

Results were qualitatively similar for the spinal cord motoneuron model where shifted distributions based on shifting g_Ca1.2 and g_Na1.2 were assessed. Shifting g_Ca1.2, the maximal conductance of the Ca 1.2 calcium channel increased the value of this conductance from 430 to 491 nS and caused an increase in first spike level and AHP magnitude and a decrease in F₀, I₀, SP gain, fast onset, input conductance, rheobase, AHP duration, late ADP time to peak, and spike width. As with the pBc bursting neuron model, searches for the shifted behaviors were performed and produced 118 homologs that met the convergence criteria of the shifted behavior target. Between the base and shifted distributions, nine parameters were found to be significantly different, as determined by a Kolmogorov-Smirnov test at the p = 8.8×10⁻⁶ significance level (.05/57, Bonferroni correction). Wilcoxon rank-sum tests produced consistent results. The base and shifted distributions of these nine parameters are shown in Figure 10. Additionally, we performed a similar shifted search for the parameter g_Na1.2 producing 117 shifted homologs. In this case four parameters were found to be different (Figure 11).

Figure 11.

Probability density functions for statistically different parameter distributions between the base and g_Na1.2 shifted searches for spinal cord motoneuron model.

For the g_Ca1.2 shifted search, the results were similar to the E_leak search in the pBc bursting neuron model: the “mechanism” identified by the search process involves significant shifts in multiple parameters and no shift in the targeted parameter. Although, g_Ca1.2 was not shown to be significantly different between the two distributions, five of the nine significantly different distributions are for calcium-related parameters, such as g_Ca1.3 and g_K(Ca). It would appear the changes exhibited in g_Ca1.3, g_K(Ca), and other parameters, as a group, are able to mimic the changes made in the output by shifting g_Ca1.2 alone.

For the g_Na1.2 shifted search, the results were similar to the g_leak and θ_mP shifted searches in the pBc bursting neuron model: that is, the mechanism identified by the search process included the target parameter and other parameters.

3.4 Behavior Space Dimensionality

Finally, we examine the dimensionality on the output side of the models by applying PCA to the variability across the homologs for the base population (Figure 12). This variability is necessarily limited to the Error tolerance of the search. Nonetheless, its dimensionality can still be examined. As before, we assess dimensionality with the 95% explained metric and the d-score metric. For both models, dimensionality is notably reduced from the random data reference. For the pBc bursting neuron model the reduction was 1 and 2 respectively for the two methods (Figure 12 panel A). For the motoneuron model the reduction was 3 for both methods (Figure 12 panel C).

Figure 12.

Dimensionality of Base Population Behavior-Space. A: Post-search variance explained as a function of PCA dimension for pBc bursting neuron model. Post-search variance (solid curve) is notably less than random data with same sample size (heavy dashed curve) as assessed by 95% (or more) of variance explained (light vertical dashed line) and “d-score” (see text). B: Effect of expanding Behavior Space with intermediate behavior measures for pBc bursting neuron model. Markings same as A. Extra behavior measures add about ¼ of a dimension on average. C: Same as A but for motoneuron model. D: same as B but for motoneuron model.

For comparison, we also greatly expanded the number of measures of behavior for each model by assessing intermediate points. Thus, these new measures could capture higher order variability. (For example, instead of measuring spike width at one point, it was measured at three different points on the spike in the motoneuron model.) These intermediates were not added to the search criteria however, and so were completely free to vary. (Note that we did not need to re-search and develop a new population, rather we simply assessed the additional behaviors on the final homologs of the base population.) Overall the additional behavior measures increased the behavior-space dimensionality of the models by about a 1 per 4 added ratio. The pBc bursting neuron model increased its dimensionality by only 4/2 (95%/d-score) despite the inclusion of 12 intermediates (Figure 12 panel B). Likewise the motoneuron model dimensionality increased by 10/6.1 despite the addition of 28 intermediates. It would appear then that despite being intermediates that would be expected to be “redundant,” the added measures do increase the dimensionality albeit at a fractional rate.

4. Discussion

In two different neuronal models, we have demonstrated that parameter value non-uniqueness can be characterized through basic statistical methods based on a reasonable number of automated searches, and that those statistical methods can be used to identify parameters generally related, but not necessarily identical to the known source of changes in model behavior. Based on these findings we conclude that a priori parameter tuning to achieve desired model behavior cannot be used as the basis for determining exact mechanistic differences/effects. However, it can be effectively and efficiently used to qualitatively indicate mechanistic differences. That is, careful, systematic parameter turning can be used to provide insight into the mechanisms behind observed differences in neuronal behavior, but it cannot be used, in and of itself, as the basis for conclusions regarding precise mechanistic effects.

At issue is how literally modeling results, based on parameter value differences, are to be taken. The data presented here suggest that parameter-based modeling conclusions are best viewed as indicators of where key differences might lie rather than precise predictions of biological mechanisms. In this light, the approach taken here can be seen as a means to strengthen and quantify those conclusions. Additionally, these results suggest that non-uniqueness is a reality in modern models that can and should be actively managed. While it is a straightforward process to analytically or numerically examine the redundancy of parameters within a model and link highly redundant parameters thereby reducing overall parameter dimensionality, this practice can introduce unmanaged bias into subsequent results. We suggest a more structured approach wherein parameters are linked by strong conceptual hypotheses that can be explicitly tested (Shapiro and Lee, 2007).

4.1 Statistical analysis of model parameter values

A secondary, but nonetheless important finding of this work is that parameter value non-uniqueness can be functionally and efficiently quantified using traditional statistical approaches. Numerous other studies have already demonstrated the existence of parameter value non-uniqueness at both the single neuron and the network level (Foster et al. 1993; Goldman et al. 2001; Prinz et al. 2004). In these previous studies, the non-uniqueness phenomenon was demonstrated only for the maximal conductances. Here, we have shown that parameter value non-uniqueness extends to all model parameters, including for example, half-maximal activation voltages and time constants. The physiological properties related to these parameters are known to be the target of neuromodulators, so this aspect of the non-uniqueness phenomenon is likely to be present in biological systems as well (Harris-Warrick et al. 1995; Kiehn and Harris-Warrick 1992; Kloppenburg et al. 1999).

It is true that this statistical approach does not mathematically determine the precise shape of the n-dimensional manifold that comprises the complete solution space for a given behavior target that a truly exhaustive search would provide, but it does provide some assessment of that manifold with a “resolution” that is a direct function of the number of homologs identified. Thus, with as few as a hundred homologs some idea of the extent of parameter distribution can be obtained and used accordingly. Additionally, more advanced statistical techniques (cross-correlation analysis, factor analysis, component analysis etc. see Mitchell and Lee, 2007) can be applied to provide further insight into model mechanisms.

4.2 “One-off” manual hand tuning

It remains a common practice in neuronal modeling to hand tune parameter values to adjust model behavior to desired targets. This practice has always had the potential to introduce bias. However, this work indicates that this approach is probably acceptable for determination of a base homolog (as was the case here) as the hand-tuned result was a reasonable approximation of the median values of the parameter distributions. Nonetheless, the results of parameter shifting strongly warn against using manual turning as a means to examine mechanistic differences between differing model behavior.

4.3 Effect of search algorithms

An important consideration not examined here is how the search methods themselves affect the resulting distribution of non-unique parameters. The results presented here utilized a local gradient-based search method to find homologs. Previous studies of the non-uniqueness phenomenon have used systematic multi-dimensional searches, random bounded searches, or genetic algorithms to produce sets of non-unique parameters (Achard 2006; Goldman et al. 2001; Keren et al. 2005; Prinz et al. 2004). All of these methods produce results. Thus, the issues are 1) does the method introduce bias, and 2) is it computationally efficient. In our experience, gradient-based methods tend to be more efficient and introduce less parameter value noise than random walk-based methods. However, they do rely on the model being well-behaved (i.e. outputs always measureable and not too severely nonlinear). Further investigation of search methods with regards to their effect on the distribution of parameter values should provide insight into the appropriateness of different methods.

Appendix

A.1 Pre-Bötzinger Complex Bursting Neuron Model

The pBc bursting neuron model is a single compartment model based on the Hodgkin-Huxley formalism. The evolution of the compartment’s membrane potential, V_m, is given by:

$$C\frac{d{V}_m}{dt}=-I_{NaP}-I_{Na}-I_K-I_{leak}$$ Equation 5

where C is the whole cell capacitance. The evolution of the state variables, x, are given by:

$$\frac{dx}{dt}=\frac{x_{\infty }\left(V_m\right)-x}{{\tau }_x\left(V_m\right)}$$ Equation 6

$$x_{\infty }=\frac{1}{1+\exp [(V_m-{\theta }_x)∕{\sigma }_x]}$$ Equation 7

$${\tau }_x=\frac{{\stackrel{‒}{\tau }}_x}{\cosh [(V_m-{\theta }_x)∕\left(2{\sigma }_x\right)]}$$ Equation 8

The voltage-gated ionic currents I_Na, I_{K, and} I_P are given as follows:

$$I_{Na}={\stackrel{‒}{g}}_{Na}\cdot m_{\infty Na}^3\left(V_m\right)\cdot h_{Na}\cdot (V_m-E_{Na})$$ Equation 9

$$I_P={\stackrel{‒}{g}}_P\cdot m_{\infty P}\left(V_m\right)\cdot h_P\cdot (V_m-E_{Na})$$ Equation 10

$$I_K={\stackrel{‒}{g}}_K\cdot m_K\cdot (V_m-E_K)$$ Equation 11

where h_Na, h_P, and m_K are state variables whose evolution is given by equations 2-4. m_Na and m_P are instantaneous functions of V_m given by the steady-state equation 3. The passive current I_leak is given by:

$$I_{leak}=g_{leak}(V_m-E_{leak})$$ Equation 12

The base values of the parameters are given in Table A.1. The output of the model with the base parameter values were given in the methods.

A.2 Spinal Cord Motoneuron Model

The scmn model is modified version of the model presented by Kuo et al. (2006). Here, we only note differences between that model and the one used in this work. The scmn model used here has three compartments: dendrite, soma, and initial segment. The dendrite consisted of a leak channel, 2 voltage-gated calcium channels (Ca_v1.2,Ca_v1.3), a Na⁺ channel (Na_v1.6), an h channel, a Ca²⁺ activated K⁺ channel (K(Ca)), a delayed rectifier K⁺ channel (Kdr), and a pump (Na⁺-Ca²⁺ and Ca²⁺ ATPase) and buffer system that controlled intracellular calcium. The initial segment consisted of a leak channel, Na_v1.6, and Kdr. The soma contained a leak channel, Kdr, and Na_v1.2.

In this work, instead of using a Na⁺ channel with 12 transition states (Taddese and Bean 2002), we used only 4 states. We assumed that most of the intermediate transition states occurred much more quickly than the final transitions. This simplification was done for computational reasons. Thus, the reduced Na⁺ channel model has a pair of half-activation and half-inactivation potentials.

For the Kdr channel, the time constant was modeled as a voltage-dependent Boltzmann equation instead of a constant, with

$${\tau }_m={\tau }_{Min}+{\tau }_{Max}\cdot \left(\frac{b}{f}\right)$$ Equation 13

$$b=\exp \left(\frac{V_m-b{V}_h}{b_s}\right)$$ Equation 14

$$f={(1+b)}^2$$ Equation 15

The h channel is modeled with a single m-gate with constant half-activation potential, time constant, and voltage sensitivity. h channel Selectivity is the percentage of Na ions out of total ions that flow through the h channel.

All other equations are identical to Kuo et al. (2006) except that parameter values may vary. The base values of the parameters are given in Table A.2, and the output of the model, along with the weighting factors used for Equation 2, are given in Table A.3.

Table A.1.

Base parameter values for pre-Bötzinger complex bursting neuron

Parameter	Base Value
C	21 pF
Na - gmax	28 nS
P - gmax	2.8 nS
K - gmax	11.2 nS
gleak	2.8 nS
ENa	50 mV
EK	−85 mV
Eleak	−65 mV
σmNa	−5 mV
θmNa	−34 mV
σhNa	4 mV
θhNa	−29 mV
τhNa	10 ms
σmP	−5 mV
θmP	−24 mV
σhP	6 mV
θhP	−48 mV
τhP	10 s
σmK	−4mV
θmK	−29 mV
τmK	10 ms

Table A.2.

Base parameter values for spinal cord motoneuron model

Location	Parameter	Base Value
Global	Ca Buffer Backward Coefficient	5
Global	Ca Buffer Forward Coefficient	20000
Global	Ca L-type Channel - σm	6 mV
Global	Ca L-type Channel - τm	30 ms
Global	Ca L-type Channel - θm	−20 mV
Global	Cav1.3 Channel - σm	6 mV
Global	Cav1.3 Channel - τm	24 ms
Global	Cav1.3 Channel - θm	−41 mV
Global	K(Ca) Channel – mb	15
Global	K(Ca) Channel – md	100000
Global	Kdr Channel – σm	20 mV
Global	Kdr Channel – θm	−25 mV
Global	Kdr Channel – tmax	11.9 ms
Global	Kdr Channel – tmin	1.4 ms
Global	Kdr Channel – ts	5.5 mV
Global	Kdr Channel – θt	−39 mV
Global	Na-Ca Pump – τ	10 ms
Global	Na-K H Channel - σm	−5.3 mV
Global	Na-K H Channel – τm	50 ms
Global	Na-K H Channel - θm	−75 mV
Global	Na-K H Channel - selectivity	0.65
Global	Na 1.1 Channel – h1ss	5
Global	Na 1.1 Channel – τh1	0.2 ms
Global	Na 1.1 Channel – h2ss	−5.26
Global	Na 1.1 Channel – τh2	0.2 ms
Global	Na 1.1 Channel – θm1	−16.5 mV
Global	Na 1.1 Channel – θm2	−103 mV
Global	Na 1.1 Channel - τm	0.005 ms
Global	Na 1.1 Channel – mVs	12 mV
Global	Na 1.6 Channel – h1ss	7
Global	Na 1.6 Channel - τh1	0.05 s
Global	Na 1.6 Channel - Sh2	−4.5
Global	Na 1.6 Channel - τh2	0.05 ms
Global	Na 1.6 Channel – θm1	−23 mV
Global	Na 1.6 Channel – θm2	−93 mV
Global	Na 1.6 Channel – τm	0.005 ms
Global	Na 1.6 Channel - mVs	12 mV
Dendrite	Capacitance	1250 pF
Dendrite	gleak	0.292 μS
Dendrite	Ca L-Type Channel - gmax	0.43 μS
Dendrite	Ca 1.3 Channel - gmax	0.015 μS
Dendrite	Na-K H Channel - gmax	0.25 μS
Dendrite	K(Ca) Channel - gmax	0.5 μS
Dendrite	Kdr Channel - gmax	0.6025 μS
Dendrite	Na 1.6 Channel - gmax	4.5 μS
Dendrite	Ca-ATP Pump - Imax	1 nA
Dendrite	Ca-ATP Pump – [Ca2+]	0.0002 mol/m3
Dendrite	Na-Ca Pump - Imax	7.25 nA
Dendrite	Na-Ca Pump – [Ca2+]	0.03 mol/m3
Initial Segment	Capacitance	18.85 pF
Initial Segment	gleak	0.00385 μS
Initial Segment	Kdr Channel - Gmax	1 μS
Initial Segment	Nav1.6 Channel - gmax	5 μS
Initial Segment/Soma	G axial	4 μS
Soma/Dendrite	G axial	1.792 μS
Soma	Capacitance	113.1 pF
Soma	gleak	0.22308 μS
Soma	Kdr Channel - Gmax	4 μS
Soma	Nav1.1 - Gmax	6 μS

Table A.3.

Base behavior and weighting factors for spinal cord motoneuron model

Protocol	Output Name	Base Value	Weight
F-I	First Spike Level	24.5 mV	0.1
F-I	Fo	10.2 Hz	0.5
F-I	Io	3.0 nA	0.25
F-I	SPGain	9.8 Hz/nA	0.125
IV	Fast Onset	−51.4 mV	0.5
IV	GFast Min	−1.8 μS	0.05
IV	Gn	0.5 μS	0.05
Rheobase	Rheobase	2.9 nA	0.5
Sag	Sag Ratio	0.1	0.015
Spike	AHP Duration	221.84 ms	10
Spike	AHP Magnitude	3.56253 mV	0.15
Spike	Spike Height	84.6467 mV	0.5
Spike	Late ADP Time to Peak	247.415 ms	10
Spike	Spike Width	3.57556 ms	0.05
Spike Threshold	Voltage Threshold	−48.371 mV	0.5