Inference of Network Connectivity from Temporally Binned Spike Trains

Abstract

Background

Processing neural activity to reconstruct network connectivity is a central focus of neuroscience, yet the spatiotemporal requisites of biological nervous systems are challenging for current neuronal sensing modalities. Consequently, methods that leverage limited data to successfully infer synaptic connections, predict activity at single unit resolution, and decipher their effect on whole systems, can uncover critical information about neural processing. Despite the emergence of powerful methods for inferring connectivity, network reconstruction based on temporally subsampled data remains insufficiently unexplored.

New Method

We infer synaptic weights by processing firing rates within variable time bins for a heterogeneous feed-forward network of excitatory, inhibitory, and unconnected units. We assess classification and optimize model parameters for postsynaptic spike train reconstruction. We test our method on a physiological network of leaky integrate-and-fire neurons displaying bursting patterns and assess prediction of postsynaptic activity from microelectrode array data.

Results

Results reveal parameters for improved prediction and performance and suggest that lower resolution data and limited access to neurons can be preferred.

Comparison with Existing Method(s)

Recent computational methods demonstrate highly improved reconstruction of connectivity from networks of parallel spike trains by considering spike lag, time-varying firing rates, and other underlying dynamics. However, these methods insufficiently explore temporal subsampling representative of novel data types.

Conclusions

We provide a framework for reverse engineering neural networks from data with limited temporal quality, describing optimal parameters for each bin size, which can be further improved using non-linear methods and applied to more complicated readouts and connectivity distributions in multiple brain circuits.

1. INTRODUCTION

There remain fundamental difficulties in decoding brain activity and relating it to function across multiple spatiotemporal and sensitivity scales (1–3). Nervous systems operate as composite biological networks critical to diverse functions including motor tasks, sensory perception, sensation, and memory, carrying intricacies at both the cellular and molecular levels across expansive interconnected units and circuits that are hard to detect in full (4–6). Consequently, the interpretation of experimental neural data that are inherently limited in scope relies heavily on innovative analysis and automated learning methods to extract and predict network features relevant to function (7,8). Such methods have been used to decipher neural circuitry directly from data and predict brain function in clinical context, unlocking new possibilities for treatment of neurological diseases (9,10). Extracting meaningful information depends on data quality, scope, and type, with steady progress made towards upgrading sensing and hardware capabilities to improve the selection and scale of neurobiological data available to in silico studies. High density neural probes with increased spatial coverage provide chronic electrophysiological readouts of direct spiking activity from hundreds and thousands of units (11–14) but these devices remain invasive and provide information from limited regions in the brain (15,16). Other readout types with reduced temporal detail, such as optical imaging of slow calcium fluctuations, can still offer large scale single unit activity critical to understanding neural circuitry (17–19) and can be deconvolved computationally to uncover underlying action potential information with reasonable precision (20–22). Noninvasive approaches present less tractable options for deconvolution both temporally and spatially (23–25). Whole-brain neuroimaging of blood flow was found to partially correlate with electrophysiology (26) and neurotransmitter systems (27,28), but emerging sensors are now providing direct volumetric recordings of calcium levels (29,30), neurotransmitter dynamics (31–33), and electromagnetic fields (34–39). The temporal resolution for these advanced techniques is limited depending on the phenomenon studied and the physical hardware of the modality in use, highlighting the need for methods to reconstruct network connections from subsampled data that supplement existing approaches.

Currently established techniques for inferring connectivity from direct spiking measurements rely on a spectrum of approaches including cross-correlation (CC) (40–42), generalized linear models (GLMs) (43,44), Granger causality (45,46), transfer entropy (TE) (47,48) and others (49–51) to leverage spike history, information latency, and other dynamics underlying synaptic connectivity. Advanced adaptations to these methods parse connectivity information more effectively from data of limited quality by combining CC and GLMs to optimize classification for networks with varying pre- and post-synaptic firing rates (52). Others explore the specific problem of inferring connectivity from subsampled, low-resolution data by correlating signals on various temporal scales to produce functional connections (53) and expanding on transfer entropy to capture neuronal bursting activity (54). While significantly advanced, these efforts do not explore optimal parameters for inferring connectivity from temporally binned conditions. Supervised learning of feed-forward perceptron networks are specifically suited for making predictions related to synaptic strength and information storage with missing information and silent synapses (51,55). By matching learning rates with different network states, it is possible to make accurate spike predictions and to quantify information storage and learning capabilities. In this work, we develop a learning model based on adaptations from the perceptron, typically used to determine presynaptic connectivity from binary events, to construct a basis of implementation of more considerable models. We use a summation of binary presynaptic spike train data within time bins of varying sizes as input for supervised learning and demonstrate the performance of the algorithm in learning unknown presynaptic weights and predicting unseen postsynaptic spikes. We then expand on these findings by quantifying the effect of temporal resolution on performance at multiple neuronal firing rates, representative of active and stimulated neuron states (56–58), to provide a general guideline for optimal learning rates that can be used under each of these conditions. We then test the physiological relevance of the algorithm by implementing a three-layer leaky-integrate-and-fire (LIF) network model displaying bursting activity and evaluate performance with different degrees of access to presynaptic spike data. Moreover, we utilize in vitro multielectrode array (MEA) recordings to test the algorithm as a learning mechanism for real data and analyze its ability to improve spike prediction for a network without a priori knowledge of neural connections. This work lends to neuroscience a useful technique for reconstructing biological neural networks from data which represents discrete neural firing rates rather than precise, single-spike events.

2. METHODS

2.1. Feedforward network initialization

We constructed a network model containing a heterogeneous presynaptic cell population (n = 200) with simulated spike trains at firing rates ranging between 5 and 80 Hz. Half of the cell population was unconnected and half consisted of excitatory and inhibitory cells (4:1 ratio) connected to a postsynaptic cell, with weights $w_i\in \left(08\right)$ and (−8 0) (mV), $i\in \{1..n\}$ distributed uniformly (Fig. 1a). Spike trains were generated over 5000 trials of 1000 ms duration for each presynaptic cell, with each spike representing a single-millisecond event (Fig. 1b). Postsynaptic spike event $y$ was determined by a simple integrate-and-fire mechanism:

$$V={\sum }_{i=1}^n{w}_i·x_i$$ (1)

$$y=\left\{\begin{array}{ll}1,& V>\alpha \\ 0,& \mathit{otherwise}\end{array}\right.$$ (2)

Fig. 1 -. Performance of adapted perceptron on varying presynaptic bin sizes for an example network.

(a) Representation of a neural network, with a distribution of inhibitory and excitatory presynaptic cells connected to a single postsynaptic neuron of interest, and neurons with minimal connection to the postsynaptic cell. (b) 1000 ms of binary spiking data for the presynaptic cells. Highlighted by the black box is a smaller subset of 10 neurons over 400 ms duration. (c) Depiction of the summation of the presynaptic spikes highlighted in panel b into lower resolution time bins. From top to bottom are the discrete 1ms spikes, followed by 5 ms, 20 ms, 100 ms and 200 ms bins. Gray boxes depict respective time bin sizes representing the temporal detail fed to the perceptron. (d-h) Comparisons of the inferred presynaptic weights with true weights. Each panel depicts the inferred weights provided by the learning rate with the lowest RMSE for each bin size, in order of ascending sizes. (i) The root-mean-square error of this representative network was calculated for the inferred weights against the true weights and plotted for each combination of predetermined bin size and learning rate.

Where $V$ is the postsynaptic membrane potential, $\alpha =20\mathrm{m}\mathrm{V}$ is the postsynaptic firing threshold, $x_i$ and $w_i$ are the spike input and weight for neuron $i$, respectively, and $n$ is total number of presynaptic neurons.

2.2. Firing rate perceptron adaptation

We utilized a modified perceptron algorithm for supervised inference of presynaptic connectivity based on firing rate data. A randomized weight matrix was initialized prior to the first iteration. Weights were updated every iteration according to the product of learning rate $\eta$ and the sum $\left({\mathrm{\Delta }}_b\right)$ of each discrete error $\delta =y-y_l$ within time bin $b$:

$${\mathrm{\Delta }}_b=\sum _{k=l}^m{y}_k-y_{l,k}$$ (3)

$$w_i(t+1)=w_i\left(t\right)+\eta *{\mathrm{\Delta }}_b·x_i$$ (4)

Where $l$ and $m$ are the first and last discrete timepoints in bin $b$, respectively, and $y$ and $y_l$ are the true and predicted postsynaptic responses, respectively. The algorithm was trained over the first 2500 trials for a total of 100 iterations for each dataset or condition. Bin sizes of 1 ms, 5 ms, 20 ms, 100 ms and 200 ms were investigated. For each bin size, seven learning rates were independently investigated (1·10⁻², 5·10⁻³, 1·10⁻³, 5·10⁻⁴, 1·10⁻⁴, 5·10⁻⁵, 1·10⁻⁵), for a total of 35 Resolution-Rate Combinations (RRC). Temporal resolution of data was defined as summation of ground truth single spike events for a given bin size (Fig. 1c) and fed into the modified perceptron as input data by randomly redistributing the calculated sum of spikes to discrete millisecond events within the same bin size.

2.3. Inference of synaptic weights

The performance $P$ of the adapted perceptron model corresponding to the prediction quality of postsynaptic firing rates, was quantified according to equations (5)–(7):

$$R_b=\sum _{k=l}^m{s}_k,{\hat{R}}_b=\sum _{k=l}^m{\hat{s}}_k$$ (5)-(6)

$$P=1-\frac{1}{c}\sum _{a=1}^c\frac{\left|R_a-{\hat{R}}_a\right|}{m-l}$$ (7)

Where, $R_b$ and ${\hat{R}}_b$ represent the sum of true and predicted discrete postsynaptic spike events $s_k$ and ${\hat{s}}_k$, respectively, within bin $b$ and averaged over the total number of bins $c$ in a training set. Here, $s$ differs from $y$ as it represents testing data rather than training data. We used root-mean-squared error (RMSE) to determine the ability of the adapted perceptron to reconstruct presynaptic weight matrices on the final learned weight matrices to quantify the inferred connectivity provided by the model for each RRC (Fig. 1d–i).

2.4. Spike predictions

We evaluated the accuracy of single-spike predictions by generating postsynaptic responses from 5000 newly generated 1000 ms trials of presynaptic spike trains. Predicted postsynaptic spikes were calculated using inferred weights and compared to the postsynaptic spike generated using true weights. We determined true positive rate (TPR) for sensitivity and true negative rate (TNR) for specificity, as described in equations (8)–(9), where $S$ represents the subset of events in which a postsynaptic spike occurs $\left(s_p=1\right)$ and $\mathrm{N}$ represents the subset of events in which there is not a postsynaptic spike:

$$TPR=1-\frac{1}{\left|S\right|}\sum _{k\in S}{s}_k-{\hat{s}}_k,TNR=1-\frac{1}{\left|N\right|}\sum _{k\in N}{\hat{s}}_k-s_k$$ (8)-(9)

2.5. Leaky integrate-and-fire network

A LIF network model was implemented to evaluate performance with physiologically relevant bursting spike patterns. Equation (10) describes the membrane potential ($V_m$) for a given neuron at time $t+\Delta t$ as a function of resting potential $\left(V_e=-75\mathrm{m}\mathrm{V}\right)$, membrane current $\left(I_m\right)$, membrane resistance $\left(R_m=10\mathrm{M}\mathrm{\Omega }\right)$, and membrane capacitance $\left(C_m=100\mathrm{p}\mathrm{F}\right)$, where time constant ${\tau }_m=C_m\times R_m=1\mathrm{m}\mathrm{s}$:

$$V_m(t+\Delta t)=V_m(t)+\Delta t·\frac{-\left(V_m(t)-V_e\right)+I_m·R_m}{{\tau }_m}$$ (10)

Equation (11) defines membrane current as a weighted sum of presynaptic spikes normalized by the postsynaptic current $\left(I_{\mathit{post}}=1\mathrm{n}\mathrm{A}\right)$.

$$I_m=I_{post}\sum _i{w}_i·spik{e}_i$$ (11)

For times $t$ where the membrane voltage was greater than the threshold voltage $\left(V_{th}=-65\mathrm{m}\mathrm{V}\right)$, an action potential occurred, and membrane voltage was set to 15 mV. For subsequent $t+\Delta t$, neurons hyperpolarized to the reset potential $V_{\mathit{reset}}=-80\mathrm{m}\mathrm{V}$.

A LIF-based three-layer network was built with a first layer (L1) consisting of spike trains generated using equations (1)–(2) at temporal step of 0.2 ms. Resting L1 firing rate was set to 5 Hz, with five 150 Hz bursts, lasting up to 12 ms, injected asynchronously into each second of data. 200 unique L1 neurons were fed into the LIF model to produce voltage traces for each layer 2 (L2) neuron. This process was repeated for 200 L2 neurons acting as presynaptic inputs to layer 3 (L3) to generate leaky postsynaptic data. Layer 2 data was temporally compressed to 1 ms resolution and methods from Section 2.1 were repeated to produce simple integrate-and-fire L3 trains. By bounding learned weights between (−8 8) (mV), four variations of L3 scenarios were tested; unbounded integrate-and-fire (UIF), unbounded LIF (ULIF), bounded integrate-and-fire (BIF) and bounded LIF (BLIF). Each network was tested for different fractions of neurons recorded within the network to test performance in common experimental scenarios with limited accessibility (20%, 50%, 80%, 85%, 90% and 95%). In all cases, the same L1 data were used to produce and learn on 5000 s of L3 data.

2.6. Microelectrode array recordings

Harvard Biosciences transparent MEAs (60MEA100/10iR-ITO-gr) were selected for experimental recordings and covered with 50 μL drops of 0.1 mg/mL polyethyleneimine (408727–100ML, Sigma-Aldrich) with 4 μg/mL laminin (23017–015, Thermo Fisher Scientific). Rat cortical neurons (Gibco, 1e6 cells/mL, lot number: 2214638, catalog number: A1084001) were cultured as described previously (59). Cultured cell volumes of 50 μL (1·10⁶ cells/mL) were plated on the active sites of sterilized MEAs in Thermo Fisher plating media (Neurobasal Plus + 1x GlutaMAX^™ + 10% fetal bovine serum) and maintained in growth media without serum four hours after plating. Subsequent media changes (aspirate 480 μL and add 640 μL) occurred every 2–3 days. MEAs signals indicated cellular activity starting at DIV 14 and were recorded between DIV 18 and DIV 22. The recording was performed using an MZ60 MEA interface at ~6104 Hz with 32 channels, streaming through a PZ5 neurodigitizer amplifier to an RZ5P base processor (Tucker-Davis Technologies, Alachua, FL). Recordings for each dataset were collected in 900 sec intervals with a 60 sec stimulation epoch at the start of each recording.

2.7. Experimental data analysis

Raw data files were loaded into MATLAB R2021a using a custom TDT software development kit and were subsequently postprocessed to represent 1 ms resolution binary data to maintain experimental parameters equivalent to in silico spike train data. Correlations were determined for all MEA channel pairs with all electrodes iteratively designated as the single ‘postsynaptic’ cell, such that a subsequent 30×30 matrix with values between [−1 1] represented the correlation of all presynaptic to postsynaptic connections. A baseline for spike prediction was established by defining correlations greater than or equal to 10⁻⁴ as excitatory connections. Excitatory weights were uniformly randomized between (0 8) (mV) and normalized by the strongest correlation. Unlearned weights for inhibitory correlations, in which maximum value did not meet the threshold, were randomized between (−8 0) (mV) and normalized to the strongest inhibitory (minimum) coefficient. A second set of weights were initialized using a time-shifting approach to align channel activity. Here, correlation for each pre- and postsynaptic pair was found for intervals of [−5 5] ms presynaptic lag with the maximum correlation determining the time shift. Excitatory designations were given to correlations of at least 10⁻⁴ and respective weights were initialized as described above. For correlations below the threshold, minimum coefficients were used to determine lag and initialize normalized weights between (−8 0). The iterative learning was repeated for this time-shifted data. Learned weights from in vitro recordings were unbounded.

Performance was quantified by testing binary spike prediction on the same recording dataset. True positive rate and true negative rate were calculated for all 30 ‘postsynaptic’ electrodes using zero-lag and time-aligned learned weights after 100 iterations, as well as for their respective unlearned weights. To better analyze binary classification given uneven positive and negative event distribution, Matthews Correlation Coefficient (MCC), as defined in equation (12), was also found. Here, true negative ($\mathrm{T}\mathrm{N}$), true positive ($\mathrm{T}\mathrm{P}$), false negative ($\mathrm{F}\mathrm{N}$), and false positive ($\mathrm{F}\mathrm{N}$) represent the dimensionless metrics within a confusion matrix.

$$MCC=\frac{TN*TP-FN*FP}{\sqrt{(TP+FP)(TP+FN)(TN+FP)(TN+FN)}}$$ (12)

2.8. Computational cost analysis

Run times for both IF and LIF networks were analyzed during 100 iterations with a single learning rate $\left(\eta =1·{10}^{-4}\right)$ on all five bin sizes. The total computational times for an IF network with 2,500 seconds of data (intervals of 1,000 ms) over 100 iterations were 1.24·10³ s, 1.36·10³ s, 1.17·10³ s, 1.16·10³ s, and 1.26 ·10³ s, in order of ascending bin size. The mean elapsed time for each interval was 12.42 ± 0.33 s, 13.62 ± 0.15 s, 11.68 ± 0.19 s, 11.62 ± 0.47 s, and 12.62 ± 0.76 s, respectively. The total elapsed times for the LIF network, in order of ascending bin size, were 1.25·10³ s, 1.44·10³ s, 1.22·10³ s, 1.16·10³ s, and 1.20 ·10³ s, with respective iteration means of 12.55 ± 0.39 s, 14.43 ± 1.04 s, 12.16 ± 0.33 s, 11.61 ± 0.30 s, and 12.03 ± 0.16 s. Timing studies were conducted in Matlab R2021a using a 3.8 GHz Intel Core i7–9800X Processor with 64 GB RAM. Parallelization was not implemented for this study.

3. RESULTS

3.1. Baseline weight inference with increasing presynaptic bin size

The ability of the adapted perceptron to reconstruct synaptic weights with training sets of varying levels of presynaptic temporal detail is demonstrated in Fig. 1 and Fig. 2. An example network (Fig. 1) was reconstructed based on spike data with increasing temporal bin sizes (Fig. 1d–h, bin sizes: 1, 5, 20, 100 and 200 ms, respectively). We quantified reconstruction accuracy by calculating the minimal RMSE of inferred weights for different bin sizes and learning rates tested (Fig. 1i).

Fig. 2 – Learning-rate-dependent iterative performance for temporally constrained presynaptic inputs.

(a-e) Iterative performance of the adapted perceptron. Each panel depicts the performance of the learning rates used in this study for a particular bin size, ordered by bin sizes of 1 ms, 5 ms, 20 ms, 100 ms and 200 ms, respectively. (f) Collective RMSE for n = 10 datasets. The average RMSE of these datasets is represented by the darker shade. Mean firing rate was 20 spikes/sec.

Fig. 2 shows iterative prediction accuracy for multiple networks (n = 10, 1–100 iterations). The average RMSE over all datasets and RRCs reached minimal values of 6.93·10⁻³ ± 3.02·10⁻⁴, 0.28 ± 0.02, 0.59 ± 0.03, 1.30 ± 0.04, 1.92 ± 0.09, in order of ascending bin size (Fig. 2f) with respective Pearson correlations of r = 1.00, 1.00, 0.98, 0.92 and 0.85, each ($p\sim 0$), demonstrating that the ability to reconstruct presynaptic weights diminishes with decreased temporal resolution. For small bin sizes (b ≤ 20 ms, Fig. 2a–c) learning converged to maximal performance with fewer iterations at high learning rates. For large bin sizes (b ≥ 100 ms, Fig. 2d–e) high learning rates failed to converge weight predictions. However, we found that with smaller learning rates, performance was improved: RMSE at $\eta =1·{10}^{-2}$ was 3.69·10² ± 81.1 (r = 0.39, $p\sim 0$) and 8.17·10² ± 27.0 (r = 0.71, $p\sim 0$) for bin sizes of 100 and 200 ms, and 4.44 ± 0.63 (r = 0.39, $p\sim 0$) and 8.90 ± 1.15 (r = 0.59, $p\sim 0$) at $\eta =1·{10}^{-3}$, respectively. Performances for smaller learning rates and large bin sizes were not asymptotic at maximum number of iterations tested (100), indicating improved optimum can be reached with higher number of iterations at the expense of algorithm runtime.

3.2. Firing rate dependence of weight inference

To find conditions for improved connectivity reconstruction, we turned to comparing firing rate dependent performance for low and high temporal resolution data encompassing both low and high firing rates corresponding to baseline and stimulated activity, respectively (Fig. 3). For the lowest temporal bin size tested (200 ms) with mean presynaptic firing rates of 5, 10, 40, and 80 Hz, (Fig. 3a–d), we found minimal average RMSE values of 5.50 ± 0.20 (r = 0.24, $p\sim 0$), 4.50 ± 0.16 (r = 0.54, $p\sim 0$), 0.64 ± 0.05 (r = 0.98, $p\sim 0$), and 0.38 ± 0.02 (r = 0.99, $p\sim 0$), respectively. Lower firing rates (5 Hz and 10 Hz, Fig. 3a–b) exhibited reduced performance instability compared with higher firing rates (40 Hz and 80 Hz, Fig. 3c–d), but low quantity of information for small number of spikes predictably prevented proper convergence of learned weights to true weights (Fig. 3e–f). For higher firing rates, we found greater performance variability and instability, but optimal learning rates utilized the availability of data properly, displaying convergence between learned and true weights (Fig. 3g–h). The mean RMSE for the 200 ms, 5 Hz RRC ranged between 5.50 and 22.8 (Fig. 3i). Comparatively, the respective range for the 80 Hz case ranged between 0.38 and 39.8 (Fig. 3l). Based on these and all other RRCs tested (Fig. 3i–l), we obtained a generalized map that defines the optimal learning rate for a given temporal bin size and presynaptic firing rate (Fig. 3m).

Fig. 3 – Effect of Presynaptic Firing Rate on Performance of Adapted Perceptron.

(a-d) Iterative trends on spike redistribution performance for varying presynaptic firing rates during rest (<10 Hz) and stimulated (> 10 Hz) states. Each panel represents the performance of learning rates as a mean and standard deviation (n = 10) of datasets under 200 ms temporal resolution condition. (e-h) Comparisons of learned weights against true weights for 200 ms time bins. Each panel represents a sample dataset, selected from the learning rate that provides the lowest average RMSE, as shown in panels i-l. (i-l) Collective RMSE for multiple networks (n = 10) for differing presynaptic firing rates, bin sizes, and learning rates, with the average represented by the darker shade. (m) A single map was generated to provide optimal learning rates for each RRC.

3.3. Single-spikes Predictions

Based on our recipe, we quantified the ability to properly predict activity at the single-spike level (Fig. 4). Figure 4a introduces subsets of predicted spike trains based on weights learned from networks firing at 20 Hz at multiple temporal bin sizes. Predictably, TPR indicated high sensitivity of the algorithm to single-spike predictions for high temporal resolution data that deteriorated as resolution decreased. Mean TPR values were 99.85 ± 0.02%, 95.52 ± 0.21%, 90.65 ± 0.40%, 78.64 ± 0.95% and 70.76% ± 1.36%, in order of ascending bin size (Fig. 4b). Specificity of the algorithm persisted as temporal resolution decreased, with mean TNR values of >99.99 ± 3.40·10⁻⁴ %, 99.91 ± 4.73·10⁻³ %, 99.80 ± 0.01%, 99.56 ± 0.01% and 99.38 ± 0.03% (Fig. 4c). For other presynaptic firing rates (Fig. 4d), networks with mean spike rate of 40 Hz provided the highest average prediction of true positives in a 200 ms environment, with 13.98 ± 1.85%, 34.81 ± 1.73%, 90.86 ± 0.87% and 80.62 ± 3.68% mean TPR for 5, 10, 40 and 80 Hz, respectively (Fig. 4e). The algorithm remains highly specific under these test conditions, as demonstrated by the respective false positive rates of 98.79 ± 0.16%, 99.35 ± 0.05%, 99.61 ± 0.05% and 99.92 ± 0.02% (Fig. 4f).

Fig. 4 – Postsynaptic Spike Prediction from Learned Presynaptic Weights.

(a) 10 seconds of spike prediction is shown for a network firing at 20 Hz. From top to bottom are the true postsynaptic spikes, followed by the predicted spikes for 1 ms, 5 ms, 20 ms, 100 ms and 200 ms time bins, respectively. Grey spikes indicate correct predictions, while red and cyan represent failed predictions and false positives, respectively. (b) Average (n = 10) percentage of correct predictions for each 20 Hz time bin. (c) Performance against false positives for each 20 Hz time bin, represented as the average (n = 10) percentage of correctly predicted non-spike events. (d) Representation of 200 ms resolution spike prediction for presynaptic firing rate, with each subsequent two rows demonstrating true spikes and predicted spikes, respectively, for the firing rate. (e) Average (n = 10) percentage of correct predictions for differing presynaptic firing rates and 200 ms time bins. (f) Performance against false positives for differing presynaptic firing rates and 200 ms time bins, represented as the average (n = 10) percentage of correctly predicted nonspike events.

3.4. Physiologically relevant Model

Experimentally recorded neuronal activity displays bursting spiking patterns at lognormal firing distributions, and usually provides access to only a fraction of a given network circuit (4). To test performance in physiologically relevant scenarios we implemented a network of LIF neuronal models displaying bursting spiking patterns and tested the ability to reconstruct connectivity with variable degrees of accessibility to presynaptic data and temporal resolution (Fig. 5). Stimulation applied through the first layer (L1) manifested in biologically relevant firing bursts at frequencies ranging between 5.83 and 26.96 Hz ($\mu =16.98$, $\sigma =3.57$) in L2 neurons (Fig. 5a). Further, we tested performance with learned weights bounded to a physiological range of [−8 8] (mV). Performance in unbounded integrate-and-fire (UIF), bounded integrate-and-fire (BIF), unbounded leaky integrate-and-fire (ULIF) and bounded leaky integrate-and-fire (BLIF) configurations was analyzed by determining the RMSE for inferring weights of L2 neurons presynaptic to L3. RMSE values for varying degrees of temporal resolution and accessibility to L2 activity data were determined for the respective subset of L2 neurons analyzed (Fig. 5b–e). For 1 ms temporal resolution UIF, optimal learning rates of 5·10⁻⁴, 5·10⁻³, and 5·10⁻⁵ achieved minimum RMSE values of 2.72, 1.09 and 3.31 for 20%, 50% and 100% of the network accessible to the algorithm, respectively. When introducing very low temporal resolution (200 ms bin), respective performance values predictably degraded by 13.6 – 62.5% to RMSE values of 5.24, 2.96 and 4.34 with optimal learning rates of 5·10⁻⁵, 1·10⁻³ and 5·10⁻⁴. In ULIF cases representing a multilayer LIF network with unbounded synaptic weights (Fig. 5d), RMSE degraded with larger time bins for 20% and 100% of spike data accessible to the algorithm (from 4.04 to 4.89 and 3.84 to 4.02, for 1 ms and 200 ms, respectively) but improved for the 50% case (from 2.91 to 2.40). In addition, 200 ms bins also showed improvement from UIF to ULIF (7.3 – 23.1%). This suggests that low temporal resolution data may be favored for predicting connectivity in LIF models due to jittery presynaptic input contributing to spike generation compared with naive memoryless feedforward networks (60). Moreover, having access to only a limited fraction of network data can result in higher performance in some cases and with the number of iterations and configuration tested.

Fig. 5 – Reconstructing connectivity from firing rates of bursting network subpopulations.

(a) Demonstration of layer 2 (L2) bursting behavior resulting from superimposed layer 1 (L1) spike trains and corresponding layer 3 (L3) voltage trace. L2 neurons shown have strong excitatory and inhibitory connections to L3 neuron (excitatory - left; inhibitory - right). (b-e) RMSE results for UIF, BIF, ULIF and BLIF under varying L2 accessibility conditions. (f) A map is provided for determining optimal learning rates for given accessibility and bin size under BLIF conditions.

Bounding synaptic weights to physiological values showed a predictable decrease in performance for all cases with small time bins and varying degrees of performance change for large bins compared with unbounded cases (Fig. 5c and 5e). For BLIF (Fig. 5e), 1 ms bins yielded RMSE values of 4.23, 2.98 and 3.70 for 20%, 50% and 100% of the network accessible to the algorithm while increasing bin size to 200 ms had varying effect on performance with a large reduction for 20% (28.2%, RMSE = 5.89), moderate improvement for 50% (17.1%, RMSE = 2.54) and minimal change for 100% (2.76%, RMSE = 3.81). Overall, the ability of our method to reconstruct connectivity of bursting LIF networks is less predictable compared with simple IF networks and heavily depends on a combination of temporal resolution and the network coverage, demonstrating the necessity for choosing optimal learning parameters to infer weights from a proper subset of presynaptic inputs (Fig. 5f). For small bin sizes, relatively low learning rates are predictably sufficient for optimal weight inference (Fig. 5f, left side of map). This corresponds to the ability of the algorithm to make small corrections to weight values during each iteration and steadily improve inference before converging to optimal values, even for limited access to network activity. Further, we demonstrate that higher learning rates are more suitable for medium bin sizes when access to the network activity data is sufficiently high (Fig. 5f, upper middle portion of map). In contrast, for very large bin sizes (e.g. 200 ms bins) the algorithm performs better with lower rates (Fig. 5f, lower right size of map) correlating with higher learning stability (Fig. 4c–d). It is expected that longer training across more iterations or with larger datasets will more consistently increase performance with lower learning rates, assuming a fraction of neurons are observed above a threshold, as demonstrated previously (61).

3.5. Experimental data spike prediction

Cortical rat neurons cultured over a 32-channel microelectrode array provided the basis for the in vitro (n = 30) spike prediction study (Fig. 6a). Prediction was assessed using standard binary classifier performance metrics, primarily true positive rate (TPR), true negative rate (TNR), and Matthews correlation coefficient (MCC) (Fig. 6b and 6c). For each ‘postsynaptic’ channel and bin size pair, the learning rate which achieved the highest TPR was used for analysis. TPR, FPR and MCC values of 0 or unbounded were excluded. With random weights normalized to correlation between channels, no postsynaptic activity (TP = 0; FP = 0) was predicted in all experimental cases, which can likely be attributed to the randomization of weights prior to normalizing. For learned weights, spike prediction improved markedly in both non-shifted cases and time-aligned studies. TPR for 1 ms bins increased to a mean of 2.58% (SE = 1.26%, n = 18) without a shift and 4.28% (SE = 1.30%, n = 20) when aligned. Respective mean FPR values were 4.67·10⁻²% (SE = 1.30·10⁻²%, n = 18) and 5.70·10⁻²% (SE = 1.77·10⁻²%, n = 20). Net performance increase is indicated by MCC values, with respective coefficients of 8.44·10⁻² (SE = 1.90·10⁻², n = 18) and 1.24·10⁻¹ (SE = 1.86·10⁻², n = 20). Classification was best for 200 ms bins, with unaligned mean TPR and FPR values of 4.57% (SE = 1.21%, n = 19) and 3.87·10⁻²% (SE = 1.70·10⁻¹%, n = 19) and an associated MCC value of 1.16·10⁻¹ (SE = 1.97·10⁻², n = 19). When time-aligned, these respective values were 6.83% (SE = 1.52%, n = 23), 2.97·10⁻¹% (SE = 1.36·10⁻¹%, n = 23), and 1.56·10⁻¹ (SE = 2.02·10⁻², n = 23). This data demonstrates the efficacy of this blurred extraction approach in biological neural networks.

Fig. 6 – Predicting in vitro spike activity from microelectrode array recordings.

(a) Filtered MEA recording from representative channels, with 1 ms binary representations and associated ‘binning’ within 20 ms bins (b) True positive and true negative rates of prediction for spike trains with variable time-alignment conditions and binning parameters (c) Matthews correlation coefficient demonstrating improved binary classification over unlearned conditions.

4. DISCUSSION

This study delineates a recipe for inferring neural connectivity from temporally subsampled synaptic data and provides a framework for adoption in more involved experiments. By adapting a single-layer perceptron to varying orders of low temporal resolution, we demonstrate empirically optimized reconstruction of feedforward presynaptic connections with parameterized model conditions. Furthermore, we show that properly inferred connections can predict network activity at the single-spike level with relatively high precision, even with limited access to the network. The results of this study approach the under-addressed problem of reverse engineering neuronal network activity from the physically limited temporal dynamics of various sensing modalities.

Our protocol prescribes optimal parameters to decode connectivity of networks receiving low sampling rate signals with no assumptions made to spiking patterns within a given time bin. We test this generalized method on networks displaying physiological spike bursting patterns and find optimized learning rates and bin sizes for reconstructing presynaptic weights, with results comparable to existing methods. For example, we test prediction of synaptic connections using standard receiver operating curve (ROC) analysis and extrapolate true positive rates for a constant false positive rate of 0.01 across all bin sizes. For a bin size of 20 ms, where performance is degraded in transfer entropy methods (54), our model achieves a maximal TPR of 0.36 for the BLIF network. This performance persists with larger bin sizes as well, whereby 200 ms bins achieve a maximum TPR of 0.38. For these same cases, the optimal pairs of values (FPR, TPR) are (0.31, 0.78) and (0.10, 0.68), respectively, which closely resemble connectivity for discrete data in previous studies (52). When a fraction (20%) of the network is observed, the area under the curve (AUC) for excitatory connections remains significant, with 200 ms bins providing a maximum AUC of 1.00, a metric which outperforms previous studies (61) but at the cost of lower performance for classifying inhibitory connections. Future adaptations can be tailored to train over specialized spiking patterns (62), including spikes phase locked to sensory stimulus (63), network oscillations, and log-normal spiking behavior generally seen in the brain (4) which is expected to also improve classification of inhibitory connections. By redistributing single spikes within a given temporal bin during learning iterations, our perceptron variant can optimize performance preferentially to neural activity of a specific persuasion and can potentially achieve more accurate connectivity maps and single-spike predictions with biological data as input. Assumptions on spiking behavior can be integrated with approaches shown to successfully reconstruct undersampled networks with only sparse datasets observable from subpopulations of the network (61), and can in turn allow for processing of larger networks and datasets commonly reconstructed using generalized linear models that overcome computationally exhaustive data processing (52,64). Other relevant postprocessing tools used to deconvolve calcium readouts, EEG signals and similar temporally limited readouts into time-resolved action potentials (19,20,61,65) can also improve the ability to decipher connectivity by serving as precursors for upgrading the input data prior to learning of weights. Additionally, single-neuron calcium spikes and similar biophysical events have specialized nonlinear features that are leveraged by deconvolution methods to decode underlying voltage spikes by supervised learning of large amounts of data (20,22). Future variations of the method presented here will mimic specific signal properties relevant to biophysical processes and train over less naïve presynaptic input and signal shapes, as well as use leaky integrate and fire neurons more suitable for recreating tonic firing and bursting network activity (66,67).

Comprehensive reconstruction of mammalian cortical and subcortical microcircuitry uses ultrastructural, morphological, functional and computational analyses to reveal diverse connectivity that can reenact brain states in silico (68). Here, we presented a proof-of-concept study for naive, linear feedforward networks with uniform distribution of presynaptic connectivity and tested this approach on microelectrode array recordings of an in vitro network of primary rat cortical neurons. Processing of diverse biological datasets to explain realistic networks will rely on analogous modifications to more comprehensive connectivity approaches, such as widely used generalized linear models (GLM) (52,69), and applying these adaptations to recurrent neuronal circuits with interconnected hubs and heterogeneous weight distributions comprising both unidirectional and reciprocal connections (68,70,71). Furthermore, recordings occurring at time scales involving both short-term and long-term plastic processes related to learning and memory will require dynamic assignment of weights. In order to account for these and other nonlinear functional and topological network traits, dynamically responsive approaches where learning rates are not held constant during iterative training are expected to allow for improved performance (72). Our training sets correspond to neurobiological context, with millisecond timescales, all-or-nothing membrane potential fluctuations frequencies, and typical synaptic strength, but can be used more broadly for reverse engineering functional biological networks limited by temporally constrained readouts.

Highlights.

• Supervised learning can infer synaptic connectivity from data with low temporal resolution

• Inferred connectivity can predict postsynaptic spike trains

• Stimulation of presynaptic cells improves reconstruction of networks

• Classification using physiological data is comparable to existing methods

Availability of data and materials

Custom Matlab scripts used in this study are available here: https://github.com/avareberg/Blurry_Spike_Trains.git