Stability of motor cortex network states during learning-associated neural reorganizations

Abstract

A substantial reorganization of neural activity and neuron-to-movement relationship in motor cortical circuits accompanies the emergence of reproducible movement patterns during motor learning. Little is known about how this tempest of neural activity restructuring impacts the stability of network states in recurrent cortical circuits. To investigate this issue, we reanalyzed data in which we recorded for 14 days via population calcium imaging the activity of the same neural populations of pyramidal neurons in layer 2/3 and layer 5 of forelimb motor and premotor cortex in mice during the daily learning of a lever-press task. We found that motor cortex network states remained stable with respect to the critical network state during the extensive reorganization of both neural population activity and its relation to lever movement throughout learning. Specifically, layer 2/3 cortical circuits unceasingly displayed robust evidence for operating at the critical network state, a regime that maximizes information capacity and transmission and provides a balance between network robustness and flexibility. In contrast, layer 5 circuits operated away from the critical network state for all 14 days of recording and learning. In conclusion, this result indicates that the wide-ranging malleability of synapses, neurons, and neural connectivity during learning operates within the constraint of a stable and layer-specific network state regarding dynamic criticality, and suggests that different cortical layers operate under distinct constraints because of their specialized goals.

NEW & NOTEWORTHY The neural activity reorganizes throughout motor learning, but how this reorganization impacts the stability of network states is unclear. We used two-photon calcium imaging to investigate how the network states in layer 2/3 and layer 5 of forelimb motor and premotor cortex are modulated by motor learning. We show that motor cortex network states are layer-specific and constant regarding criticality during neural activity reorganization, and suggests that layer-specific constraints could be motivated by different functions.

INTRODUCTION

Motor learning manifests itself as the emergence of a reproducible movement pattern (Fig. 1A), which involves the concurrent reorganization of synaptic connectivity, neural activity, and neuron-to-movement transformation in recurrent neural networks (Fig. 1B) (Chen et al. 2015; Costa et al. 2004; Makino et al. 2017; Masamizu et al. 2014; Peters et al. 2014, 2017a). The restructuring of complex neural networks during learning poses fundamental questions as to the stability of the network state (Liu and Barabási 2016).

Fig. 1.

Does cortical population activity deviate from the critical network state while neural circuits undergo massive reorganizations during motor learning? A: traces of lever position in multiple trials from one representative mouse. Gray, individual trials; black, average of all trials; red, onset of the movement; left: one session from naive stage; middle: one session from middle learning stage; right: one session from expert stage. B: schematic of a recurrent neural circuit consisting of neurons (gray), connections (black), neural representation of movement (letters: P for position, v̄ for velocity, v for speed) and network state (background color). Question marks mean the network states are unknown. C: neuronal avalanches (gray) are contiguous bouts of spikes (black raster) across the population of neurons. The spike count within an avalanche determines the avalanche size. D: the shape of the avalanche size distribution reflects the level of spatiotemporal correlation within the network. A power law avalanche size distribution is a characteristic feature of the critical network state. Question mark means whether the network state is critical state is unclear. PDF, probability density function; S, avalanche size.

The motor cortex has emerged as a central locus where changes in neural circuit take place during motor learning (Peters et al. 2017b; Sanes and Donoghue 2000). For example, learning of motor skills can increase the representation of corresponding body parts in the motor cortex (Nudo et al. 1996), individual motor cortex neurons can change their activity during motor learning (Costa et al. 2004; Li et al. 2001; Masamizu et al. 2014; Rokni et al. 2007). These changes generate a neural activity pattern that relates to the learned movement (Peters et al. 2014). At the synaptic level, motor learning induces LTP-like plasticity in the motor cortex (Rioult-Pedotti et al. 2000), synaptic reorganization among functionally related layer 5 neurons (Biane et al. 2019), and turnover of dendritic spines and inhibitory synapses in a cell-type-specific manner (Chen et al. 2015; Fu et al. 2012; Xu et al. 2009; Yang et al. 2009).

Whether this massive synaptic and neuronal reorganization during learning takes place within the constraint of stable cortical circuit dynamics is unknown. Several pieces of evidence point toward reorganization while maintaining a given network state. First, learning is accompanied by spine formation, followed by a period of spine elimination, which returns the total number of spines to baseline (Harms et al. 2008; Xu et al. 2009; Yang et al. 2009). Second, learning-induced synaptic potentiation may be balanced by synaptic depression rather than generally potentiating local connections (Cohen and Castro-Alamancos 2005). Third, learning-induced changes in inhibition may be controlled by the cell-type specific activation of neurons with low and high excitatory-to-inhibitory synaptic-density ratios (Donato et al. 2013). Together, these observations suggest network-state invariance during learning as a plausible possibility.

Previously described states of cortical circuits include oscillatory (Hoseini et al. 2017; Hoseini and Wessel 2016; Speed et al. 2019), asynchronous (Renart et al. 2010), chaotic (van Vreeswijk and Sompolinsky 1998), or critical (Beggs and Plenz 2003) network states. Of these, the critical network state (Fig. 1, C and D) plays a prominent role in the study of cortical circuit dynamics, as the critical network state is optimized for signal processing (Clawson et al. 2017; Gautam et al. 2015; Haldeman and Beggs 2005; Tomen et al. 2019), may provide neural circuits with an optimal balance between robustness and flexibility (Muñoz 2018), and can represent a homeostatic set point (Ma et al. 2019). Qualitatively, the critical network state resides at the boundary between strongly and weakly coordinated population activity corresponding to phases of order (strongly coupled) and disorder, respectively. Mathematically, the critical network state is characterized by scale-free activity covariance, i.e., the distance dependence of the covariance follows a power law (Beggs and Timme 2012). This brief survey of network states then bares the significant question: Does population activity in motor cortex reorganize during motor learning within the constraint of operating near the critical network state?

To address this question, we reanalyzed data from previous experiments in which we monitored via population calcium imaging the activity of hundreds of pyramidal neurons in L2/3 and L5 of forelimb M1 and M2 of head-fixed mice during motor learning (Makino et al. 2017; Peters et al. 2014, 2017a). Water-restricted mice were trained daily on a lever-press task for ∼14 days, while we recorded the activity of the same neuronal populations from ∼200 neurons per mouse. We analyzed the population activity with respect to the reorganization of neural activity, its relation to lever movement, and the network state. We found that motor cortex network states are layer-specific and remain constant with respect to criticality during massive neural activity reorganization.

MATERIALS AND METHODS

We reanalyzed data from three previously published data sets (Makino et al. 2017; Peters et al. 2014, 2017a). All procedures were in accordance with protocols approved by the UCSD Institutional Animal Care and Use Committee and guidelines of the National Institute of Health. All data are available upon reasonable request.

Experimental Design

Behavior.

Water-restricted mice were trained daily for 14 days to perform a lever-press task, while two-photon imaging from motor cortex was applied simultaneously (Makino et al. 2017; Peters et al. 2014, 2017a).

The lever consisted of a handle glued to a piezoelectric flexible force transducer (LCL-113G, Omega Engineering). Voltage from the force transducer, which was linearly proportional to the lever displacement, was continuously monitored at 10 kHz using a data-acquisition device (LabJack) and software (LabVIEW, National Instruments).

A 6-kHz tone marked a period during which lever press was rewarded with water (∼8 µl per trial) paired with a 500-ms, 12-kHz tone, followed by an intertrial interval (variable duration, see below). Lever-press was defined as crossing of two thresholds (∼1.5 mm and ∼3 mm below the resting position) within 200 ms. The 3-mm threshold defined the displacement required, and the 1.5-mm threshold ensured that the mouse did not hold the lever near the lower threshold. Failure to press during the cue period triggered a loud white noise and an intertrial interval. Lever presses during intertrial intervals were neither rewarded nor punished. The intertrial interval was increased over the first three sessions from 2–4 to 8–12 s. Each session lasted 20–30 min and 100–200 trials until terminated when mice stopped performing or consumed 1 ml of water. Experiments lasted for 11 to 14 sessions.

Voltage from the piezoelectric lever during each session was parsed into movement and quiescence epochs as previously described (Peters et al. 2014). Briefly, movement was first identified by velocity threshold. Movement epochs were then refined by combining nearby epochs, eliminating small epochs and refining the start and end times of movement epochs according to when the lever position, respectively left or entered a baseline defined by adjacent quiescent epochs. Visual inspection confirmed accurate demarcation of behavior.

Imaging.

For all imaging data sets considered (Makino et al. 2017; Peters et al. 2014, 2017a), we conducted imaging in awake animals with a commercial two-photon microscope (B-scope, Thorlabs) running Scanimage using a ×16 0.8 NA objective (NIKON) with excitation at 925-nm (Ti-Sa laser, Newport). We acquired images with Scanimage 4.1 (Vidrio Technologies) at a rate of ∼28 Hz, covering 512 × 512 square pixels. Within the same animal, we imaged the activity of the same neural population over the course of 2 wk, while mice simultaneously learned and performed the lever-press task.

We transfected and imaged L2/3 neurons in the caudal forelimb area of the motor cortex (M1) with the Ca indicator GCaMP5G and identified excitatory and inhibitory neurons as described before (Peters et al. 2014). Within a field of view covering 472 × 508 µm, a total of 202±18 [mean±standard error of the mean (SE); n = 7 animals] neurons were imaged in each animal, with 20.9±6% being inhibitory.

For the imaging of L5 neuron activity in M1, we used the Cre-FLEX system to selectively express the calcium indicator GCaMP6f in corticospinal cells in the motor cortex as described before (Peters et al. 2017a). In brief, we imaged the apical dendritic trunks of GCaMP6f-expressing dendrites passing through L2/3. The locations of the dendrites were stable across days and the same dendrites could be reliably identified each day. GCaMP6f fluorescence within these dendrites was observed as bright discrete points in a very low-noise background, allowing for automated region-of-interest creation. We performed simultaneous imaging of dendrites and their parent somata by rapid 3D scanning and found that dendritic and somatic signals were nearly completely correlated. Thus, calcium signals in dendritic trunks of these neurons faithfully report somatic spiking activity. Also, imaging of sibling dendrites that connect with the same parent somata showed that the activities of sibling dendrites are very highly correlated with each other. We were able to use this information to detect “presumed siblings” based on activity correlations and combine them by averaging the activity of those presumed siblings to avoid counting the same neurons multiple times. Within a field of view covering ∼340 μm × 340 μm, we thus inferred the activity from 194±68 corticospinal neurons per mouse (mean ± SD; n = 8 animals).

For the imaging of L2/3 and L5 neuron activity in M2, we used CaMK2-tTA::tetO-GCaMP6s double transgenic mice as described before (Makino et al. 2017). Within a field of view covering 472 × 508 µm, imaging was performed ∼250 µm deep from the dura for L2/3 and 500 µm for L5. M2 L5 neurons were identified by cortical depth only, so they are a mixed population not defined by their projections. A total of three mice were imaged for L2/3 only, two mice for L5 only, and five mice for both (alternating between L2/3 and L5 each day).

Signal Processing and Statistical Analysis

Processing fluorescence time series.

For each ROI, we generated 1) background-subtracted and smoothed fluorescence time series (ΔF/F₀) and 2) calcium events, as described before (Peters et al. 2014). In brief, calcium events were defined as sharp rises of ΔF/F₀ above a “threshold” value. A calcium event trace was then constructed, which was zero except for frames with detected events. Each event was assigned an amplitude equal to the difference between the peak ΔF/F₀ and the “baseline” ΔF/F₀ for that event. Inferred spike probability (ISP) was computed from the background-subtracted and smoothed fluorescence time series (ΔF/F₀) using a fast, nonnegative deconvolution method (Vogelstein et al. 2010). All calcium signals were deconvolved to extract underlying spiking activity and thus the effect of different kinetics of sensors has been minimized. The inferred spike rate was computed using the spike triggered mixture algorithm (Theis et al. 2016). Briefly, the ΔF/F₀ traces were first upsampled to 100 Hz. For each time point, the spike number during the 10-ms bin was inferred using “an extension of generalized linear model” trained on simultaneous recordings of spikes and calcium traces. For all the analysis below on either single neuron activity or neural population activities, calcium events, inferred spike probability and inferred spike rate yielded qualitatively similar results.

Evaluating coordinated inferred neural activity.

From the inferred spike rates (ISR) of the recorded pyramidal neurons, we obtained the zero-lag pairwise Pearson cross-correlation coefficients of the inferred spike probability using the MATLAB corrcoef routine. We clustered the resulting matrix of cross-correlation coefficients in session 1, using hierarchical clustering with maximum or complete-linkage clustering (MATLAB dendrogram routine). We retained the neuronal order in subsequent sessions. To quantify the change of Pearson cross-correlation coefficients across sessions, we computed the correlations of the Pearson cross-correlation coefficients across adjacent sessions.

Relating inferred neural activity to lever position.

To investigate a potential relation between neural activity and lever position within a session (∼30 min), we concatenated all movement episodes and the corresponding inferred spike probability (ISP) for each recorded neuron, resulting in concurrent long time series of duration T; one for the lever position and one for the inferred spike probability for each recorded neuron. Here, we assumed zero lag between the lever position and inferred spike probability (but see The Relation Between Neural Activity and Lever Position Reorganizes in RESULTS). We spatially binned the total range of continuous lever position (typical range was below 2 mm) into discrete position segments of 0.04-mm bin size. For a given neuron, we computed the average inferred spike probability for each position segment and determined the “peak averaged inferred spike probability” for that neuron and session.

To evaluate the statistical significance of the peak averaged inferred spike probability, we shifted the concatenated lever position time series (of duration T) by an amount δt (<T) relative to the inferred spike probability. To ensure continuity of the concatenated lever position time series, we inserted the clipped end back at the beginning, thus accomplishing a circular shuffling. From the thus shifted concatenated lever position time series, we computed the average inferred spike probability for each position segment. We repeated this procedure 1,000 times, each time drawing the value for δt from a uniform distribution in the range from 0 to T. Finally, we accepted the peak averaged inferred spike probability for the original time series as significant, when the peak averaged inferred spike probability for the original time series was larger than the peak averaged inferred spike probability for at least 950 of the shuffled cases.

For a neuron with a significant peak averaged inferred spike probability in two adjacent sessions, we determined the “lever position of peak averaged inferred spike probability” in each session. We classified a neuron as “consistent,” when the difference in the lever position of peak averaged inferred spike probability for adjacent sessions was 0.2 mm or less. For all pairs of adjacent sessions, we computed the fraction of consistent neurons. In addition, we computed the fraction of consistent neurons for pairs of sessions of different intervals.

Relating inferred neural activity to three aspects of lever movement.

In addition, we evaluated the relation between the neuron’s recorded ΔF/F₀ and three related aspects of lever movement, namely lever position, velocity (time derivative of lever position), and speed (absolute value of lever velocity). For a given session, we concatenated all movement episodes and the corresponding ΔF/F₀ for each recorded neuron, as described above. From the maximum value of the cross-correlation between the lever position and the population activity (sum of ΔF/F₀ for all recorded neurons), we estimated a session-unique lag between the lever position and the single-neuron ΔF/F₀ time series. We then calculated the cross-correlation coefficients between a neuron’s ΔF/F₀ and the three aspects of lever movement, i.e., lever position, velocity, and speed. To evaluate the significance of the thus calculated cross-correlation coefficients, we compared against corresponding cross-correlation coefficients calculated from a neuron’s ΔF/F₀ and the three aspects of lever movement from shuffled movements (95% confident intervals). Specifically, while keeping the sequence of concatenated neural ΔF/F₀ recordings, we shuffled the movement chunks and then concatenated them. We repeated that procedure for 1,000 times. Each time, we also reversed the time series within each movement chunk once. From the thus-described evaluation of significance, we calculated the fraction of neurons with activity significantly correlated with at least one aspect of lever movement within a session.

Following this classification of the relation between neural activity and aspects of lever movement for each session, we assigned a numerical value (−2 to +2) to each neuron labeling the type of relation: lever speed (−2), lever velocity (−1), no relation (0), lever position (1), and more than one aspect of lever movement (2). While these choices of numerical labels are arbitrary, we checked that the subsequent analysis is robust with respect to the particular choice being made. This labeling procedure resulted in a vector of length equal the number of neurons and of five values ranging from −2 to 2. We refer to the cross-correlation coefficient from pairs of such vectors from adjacent sessions as the “Correlation of representations in adjacent sessions.” To evaluate the statistical significance of these cross-correlation coefficients, we repeated the analysis on shuffled lever movement epochs as described above.

Neuronal avalanche analysis.

The recording resolution of two-photon calcium imaging was around 28 Hz. We inferred the spike probability (rate) by different signal processing methods, including nonnegative deconvolution (Vogelstein et al. 2010) and spike-triggered mixture algorithm (Theis et al. 2016). The nonnegative deconvolution methods returned spike probability with roughly 28 Hz. We upsampled the signal to 100 Hz in the spike triggered mixture algorithm. We did neural avalanche analysis based on both 28-Hz and 100-Hz data. Both signal preprocessing procedures returns us similar results on network state study.

For each recorded neuron, we thresholded the ISP (ISR) at 3 SDs (SD determined from the inferred spike probabilities of the entire population in each session), and thus binarized the ISP of each neuron into a binary activity time series. We also tested threshold at 2.8 SDs and 3.2 SDs. The evaluations of network state (see Motor Cortex Network States Remain Stable with Respect to Criticality During Neural Activity Reorganization in RESULTS) were robust for threshold values within this range. To obtain the network activity, we summed the binarized ISP values across neurons in a time bin. Based on the network activity, we defined a “neuronal avalanche” by introducing a threshold. We tested the threshold from 25th percentile to 50th percentile of network activity. Since the results of network state evaluations were robust for network thresholds in this range, we chose the 35th percentile value of network activity as the network threshold for all subsequent analysis.

Neuronal avalanche distributions are naturally disturbed at the lower end from systematic recording errors (noise) and at the upper end from finite size effects. Such disturbances bias attempts of power law evaluations. We therefore followed documented methods (Karimipanah et al. 2017a; Shew et al. 2015; Virkar and Clauset 2014) of truncated power law fitting to the observed neuronal avalanche distributions. Using maximum likelihood estimation methods (Clauset et al. 2009), we fitted a truncated power law $f(S)=\frac{S^{-\tau }}{{\displaystyle {\sum }_{S_{\min }}^{S_{\text{max}}}{S}^{-\tau }}}$ to the avalanche size distribution of N_av avalanches using the following iterative procedure (Karimipanah et al. 2017a). 1) The maximum avalanche size S_max was taken as the largest observed avalanche size S_k. 2) The exponent τ was estimated for all values of the minimum avalanche size S_min ranging from 1 to S_k/20 and the corresponding Kolmogorov-Smirnov (KS) values were obtained. 3) The minimum avalanche size S_min and the corresponding exponent τ yielding the smallest KS value were chosen. 4) When $\text{KS}<\frac{1}{\sqrt{N_{\text{av}}}}$ the exponent estimation was completed. Otherwise, the procedure 2) to 4) was repeated with the maximum avalanche size S_max reduced by 1 until the condition $\text{KS}<\frac{1}{\sqrt{N_{\text{av}}}}$ was satisfied. In some cases, KS was stable (δKS<0.001, δKS is the absolute change of KS) when we changed maximum avalanche size; we then forced to break the iteration. We applied the same fitting procedure to the avalanche duration distributions.

To evaluate whether a power law was a plausible fit of an avalanche distribution, we performed hypothesis testing. We simulated 1,000 artificial power law distributions (surrogate distributions) with the same exponent, number of avalanches, minimum avalanche size, and maximum avalanche size, as estimated from the experimental avalanche distribution. Specifically, using the inverse method, the surrogate distributions were generated according to $S=S_{\text{min}}{\left(1–r\right)}^{-\frac{1}{\left(\text{τ}-1\right)}}$ where r was a random number drawn from a uniform distribution between 0 and 1. Thereafter, the distribution was upper-truncated by setting a cutoff at the higher boundary value in the empirical data S_max. This procedure worked well for generating avalanches of S_min larger than 1. For S_min=1, we used an alternative acceptance-rejection method (Clauset et al. 2009).

The deviation between the simulated surrogate distributions and a perfect power law was quantified with the KS statistics as KS_surrogated. The P-value was calculated as the fraction of the surrogate distributions with KS values larger than the KS value of the corresponding experimental avalanche distribution. KS_surrogated here is also influenced by the number of avalanches. When the number of avalanches is small, KS_surrogated tends to be large. That’s a natural flaw of small number of avalanches. We took the significance level to be 0.05, i.e., for P<0.05 the power law hypothesis was rejected, whereas for P≥0.05 the power law hypothesis was not rejected.

The uncertainty of the exponent estimation was computed using bootstrap method. After estimating the exponent from the experimental avalanche distribution, we resampled actual avalanches (with replacement) 1,000 times and then fitted the resampled data to a power law and estimated the exponent. The standard deviation of reestimated exponents provided an estimate of the uncertainty in the exponent estimation from the experimental avalanche distribution.

To test whether average avalanche size scales with duration according to <S>∼D^β, we estimated the fitted exponent β_fit from the experimental data using linear regression in logarithmic axes. We then compared the fitted exponent β_fit to the predicted β_pred, where the predicted exponent ${\text{β}}_{\text{pred}}=\frac{\text{α}–1}{\text{τ}–1}$ was obtained from the estimated size and duration exponents (see Motor Cortex Network States Remain Stable with Respect to Criticality During Neural Activity Reorganization).

To evaluate the deviation of avalanche size distributions from power laws across sessions, we used the previously developed statistical measure κ (Shew et al. 2009, 2011). In brief, we fitted a power law and the corresponding exponent τ to an empirical avalanche size distribution and then generated a surrogate (theoretical) avalanche size distribution within the range from S_min to S_max. We quantified the two distributions in terms of the corresponding cumulative probability distributions G(S) and G^Theoretical(S), for empirical and theoretical distributions, respectively. The cumulative probability distribution describes the probability of observing an avalanche with size less than S. By definition, κ equals 1 plus the average of differences between F^Theoretical(S) and F(S), i.e., $\text{κ}=1+\frac{1}{N_{\text{max}}-N_{\text{min}}+1}\sum _{S=N_{\text{min}}}^{N_{\text{max}}}\left(G^{\text{Theoretical}}\left(S\right)-G\left(S\right)\right)$. In other words, κ≈1 indicates a good match between the empirical distribution F(S) and the theoretical power law distribution F^Theoretical(S).

Friedman et al. (2012) demonstrated that the size versus duration exponent, β, can be used to accomplish avalanche shape collapse, where avalanche profiles of different sizes are revealed to be copies of each other at different scales. When the system is operating near the critical state, avalanches of all durations will reflect the same scaled mean shape. The average number of neurons firing at time t within an avalanche of duration D can be described as S(t,D)∝D^γ F(t/D), where F is a universal function determined by t/D. Given $S\left(D\right)={\int }_0^{D}s\left(t,D\right)\text{d}t$, the relationship between γ and β is γ=β−1. We utilized the Neural Complexity and Criticality (NCC) toolbox in MATLAB (Marshall et al. 2016) to perform shape collapse on our data. To be more specific, a scaling exponent β was assigned to compute γ, and then $\langle \frac{s\left(t,D\right)}{D^{\text{γ}}}\rangle$ was calculated as F(t/D). Here, $\langle \rangle$ denotes the average over avalanches with the same duration. A collection of F(t/D) were extracted for various avalanche durations as F. The average normalized variance of F(t/D) over D is the error for shape collapse under this scaling exponent, as Var(F)/(max(F)−min(F))². Via this method, a range of scaling exponents (β) are tested, and the exponent that produces the best shape collapse (smallest variance-based error) is selected as the scaling exponent. We used this method to calculate variance-based error to measure how closely scaled avalanche shapes fit the mean.

Neural avalanche analysis validation test on model.

To validate of the neural avalanche analysis method, we used a probabilistic integrate and fire (PIF) neural model (Karimipanah et al. 2017b; Larremore et al. 2011) with N=2,000 neurons. The strength of the connection from neuron j to i is quantified in terms of the transition probability P_ij, which is the probability that a spike in neuron j causes a spike in neuron i in the next simulation time step. For a network of N neurons and an average connectivity K (we chose K=3*N/100), each neuron is connected to other neurons with probability K/N For each connection among neurons, a P_ij is assigned by drawing a random number from uniform distribution in the interval [0 2/K]. With a sufficiently large N, this yields a network with a uniformly distributed transition probability P_ij with average value equal to 1/K, and a matrix P_ij with maximum absolute eigenvalue of 1. At each time step, the states of all neurons are updated synchronously according to the following probability:

$$P_i\left(t+1\right)=P\left(i_{t+1}|J\left(t\right)\right)=1-\left(1-{\eta }_i\left(t\right)\right){{\displaystyle \prod }}_{\left\{j\in J\left(t\right)\right\}}\left(1-P_{ij}\right)$$

where P(i_t+1|J(t)) denotes the probability of neuron i spikes at time t+1 given that the set of neurons j∈J(t) spike at the previous time-step t, η_i(t) is the probability of neuron i spikes only due to external input, and J(t) denotes the set of all neurons that spike at time t. It should be noted that this equation is valid only when the spikes of all neurons in j∈J(t) can be assumed as independent with a good approximation. In other words, it entails a locally treelike propagation of activity, which has been shown to be a good approximation for a wide range of connectivity (Larremore et al. 2011). At the large size limit (N≫1) and assuming that P_ij inversely scales with N (P_ij∼1/K), the above equation can be approximated as follows:

$$X_i\left(t+1\right)=\theta \left[\left(1-{\eta }_i\left(t\right)\right){{\displaystyle \sum }}_j{P}_{ij}{X}_j\left(t\right)+{\eta }_i\left(t\right)-{\xi }_i\left(t\right)\right]$$

where the binary state X_i(t) of neuron i denotes whether the model neuron spikes (X_i(t)=1) or does not spike (X_i(t)=0) at time t. Here, ξ_i(t) is a random number in [0 1] drawn from a uniform distribution, and θ is the step function. In addition to the update rule, a refractory period of two time steps was implemented. The external input η_i(t) was chosen to be smaller than the transition probability P_ij, which is small for large networks, P_ij∼1/K. The maximum eigenvalue λ of the transition probability matrix P_ij describes the network state at the infinite-size limit: λ≈1denotes the critical regime.

We applied the KS avalanche analysis method we used on the empirical avalanche distribution onto this PIF modeling spikes. To study the influence of convolution-sampling-deconvolution procedure on avalanche distributions, we set each time step as 2 ms. To mimic the time course of the experimentally observed fluorescent signal, we convolved the simulated spike trains. To imitate the low temporal resolution of imaging, we then sampled the convolved version of the simulated spike trains at the lower temporal resolution. Next, we applied nonnegative deconvolution (Vogelstein et al. 2010) to the thus sampled convolved simulated spike train and thus computed the inferred spike probabilities. We utilized the same avalanche analysis procedure on the thus obtained inferred spike probabilities from simulated spike trains. This convolution-sampling-deconvolution returns us avalanche size/duration distributions with bias from true power law in the head and tail part. The test on simulated data validated the KS power law fitting method on avalanche distributions with lower and upper limits (see results).

Branching ratio.

We followed a published method to compute the branching ratio (Wilting and Priesemann 2018). For a branching process, if the branching parameter is a fixed value, then $\langle A_{t+1}|A_t\rangle =m{A}_t+h$, where A_t is full activity at time t, $\langle \rangle$ denotes the conditional expectation, m is the branching ratio, and h is the mean rate of external drive. Subsampled activity a_t is proportional to A_t on average $\langle a_t|A_t\rangle =l{A}_t+g$ with constants l and g. Subsampling process leads to a bias $m\left(l^2\frac{Var\left[A_t\right]}{Var\left[a_t\right]}-1\right)$ on the estimator. Instead of using time t and t+1, the improved method focuses on times t and t+k with different time lags k = 1…, k_maximum. Under full sampling, one expects r_k = m^k, where r_k is a collection of linear regression slopes. Under subsampling $r_k=l^2\frac{Var\left[A_t\right]}{Var\left[a_t\right]}{m}^k=b{m}^k$, where b is a constant. Generating m^k with a collection of k, we can obtain an exponential curve, and then compute the branching ratio m from this curve.

RESULTS

To understand how the change of neural activity during learning impacts the network state (Fig. 1), we sought to quantify the neural activity reorganization while mice improved on a quantifiable behavior (Fig. 2). In these previously published experiments (see methods), we trained water-restricted mice daily for 14 days to perform a cued lever-press task. Mice received a water reward when, following an auditory cue, they pushed a lever across a threshold. Lever movements became increasingly stereotyped across training sessions (Peters et al. 2014). The motor cortex is necessary for this task (Makino et al. 2017; Peters et al. 2014). We therefore monitored via chronic two-photon population calcium imaging the neural activity in M1 and M2 for both L2/3 and L5 of the same populations of hundreds of neurons over the course of two weeks while head-fixed mice performed the described lever push task (Makino et al. 2017; Peters et al. 2014, 2017a). From the background-subtracted and smoothed fluorescence time series (ΔF/F₀) for each ROI, we inferred neural signal using several algorithms (methods).

Fig. 2.

Imaging and quantifying the neural population activity in motor cortex while mice improve on a quantifiable behavior. A: experimental paradigm with lever-press task and chronic imaging of neural activity. B: GCaMP5G expression in layer 2/3 neurons of M1. C: fluorescent signal (ΔF/F₀) for one representative neuron, the corresponding calcium events (CaEvent), inferred spike probability (ISP), and inferred spike rates (ISR). ITI, intertrial interval.

Neural Activity Reorganizes During Motor Learning

The time-averaged inferred spike rate of individual recorded neurons within a given session changed across the 14 recordings sessions [Fig. 3A, Supplemental Fig. S1 (https://figshare.com/s/9245c5019517cf3912ec)]. The observed changes included neurons which were silent at the early sessions but then became active with relative high rates in later sessions and vice versa. Specifically, 33% of the 5,899 neurons increased their ISR by more than 10%, whereas 25% of the neurons decreased their ISR by more than 10% from early to late session. Relevant for the question of network stability addressed in this study, the overall population activity remained largely stable for the four populations of neurons. Specifically, the average ratio of population firing rate of late to early session over all samples was 1.07 ± 0.012.

Fig. 3.

Neural activity reorganizes during motor learning. A: the time-average inferred spike rates (color coded, see vertical scale bar) are shown for each recorded neuron across 14 learning sessions. The ranking of the inferred spike rates (ISR) in session 1 determines the order of neurons for all sessions. Representative data are shown for M1 L2/3, M1 L5, M2 L2/3, and M2 L5. Qualitatively similar reorganizations are observed for all mice tested [see Supplemental Fig. S1 (https://figshare.com/s/9245c5019517cf3912ec)]. B: pyramidal neurons (M1 L2/3) were clustered by their functional correlation (Pearson correlation), only 50 neurons with top correlations were shown here for visualization. C: cross correlation between the cross correlations of adjacent sessions [see violin plots in Supplemental Fig. S2 (https://figshare.com/s/9245c5019517cf3912ec)]. Errors bars are standard errors. D: significance test (t test) on correlations between correlations of adjacent sessions for naive stage (first day), middle stage (days 5 to 7) and expert stage (last 3 days). *P < 0.05; **P < 0.01; ***P < 0.001. Error bars are SE [see ANOVA test in Supplemental Fig. S2 (https://figshare.com/s/9245c5019517cf3912ec)].

The rearrangement of neuronal activity implies a concurrent rearrangement of coordinated activity. To evaluate such coordination, we computed the Pearson cross-correlation of inferred spike rate for all pairs of neurons for each session, clustered the correlation matrix on the first session, and kept the order of neurons for all subsequent sessions (Fig. 3B). Finally, we quantified changes of coordination across sessions by computing the correlation of the correlation matrix from adjacent sessions (Fig. 3C). Changes in coordination were most pronounced from session 1 to 2, and the changes were not significant for subsequent sessions [Fig. 3, C and D, Supplemental Fig. S2 (https://figshare.com/s/9245c5019517cf3912ec)]. Specifically, according to two-samples t test, the correlations of correlations for adjacent days were significantly smaller (P < 0.05) for the naive stage (sessions 1 and 2) compared with middle (sessions 5–7) and late (sessions 11–14) stages.

The Relation Between Neural Activity and Lever Position Reorganizes

For any given session, a subset of the recorded neurons displayed a largely repeated activity time course across the many epochs of stereotyped lever movement within that session. For each session, we quantified the relation between neural activity and lever position as follows (Driscoll et al. 2017). For a given neuron, we computed the average inferred spike probability versus lever position and determined the “peak position” and its statistical significance of averaged inferred spike probability for that neuron and session (Fig. 4A). We noticed that the peak position for a given neuron varied across sessions. To quantify this variability, we computed for a given neuron the “difference of peak position” for adjacent sessions (Fig. 4A). Finally, we classified a neuron as consistent when the difference in the peak position for two sessions was 0.2 mm or less (Fig. 4B). Armed with this quantification framework, we computed the fraction of neurons with consistent peak position for pairs of sessions of different intervals to evaluated changes in peak position across different time scales. During the first seven sessions of motor learning, the fraction of neurons with consistent peak position decreased with increasing time interval between sessions; this trend persisted, but slowed, for expert mice (second week) (Fig. 4C). To evaluate the statistical significance of the fraction of consistent neurons, we kept the “lever position of peak averaged inferred spike probability” in each session, but reassigned them to any of the recorded neurons at random. We then classified a neuron as consistent, as described above. In this shuffled condition, the fractions of consistent neurons were substantially lower, indicating that the analysis depends on each neuron’s activity relative to the lever position.

Fig. 4.

The relation between neural activity and lever position reorganizes. A: the “peak movement position” for a single neuron is the lever position where the across-trial average inferred spike probability peaks and is significantly above noise. Each curve represents the normalized trial-averaged ISP of neuron. Two adjacent sessions are shown. The “difference of peak position” is marked by the dashed lines. B, top: histogram of the ”difference of peak” from session 1 to session 2. Bottom: histogram of the difference of peak from session 13 to session 14. We classified neurons for which difference of peak is smaller than a certain threshold as “neurons with consistent peak position” (within the fuchsia dashed box). C: fraction of neurons with consistent peak position for sessions with different intervals of first week (orange) and second week (cyan). Transparent curves: fractions with cells shuffled. Error bars are SE.

Motivated by the observed relation between motor cortex single-neuron activity and lever position (Fig. 4), we asked whether a relation between single-neuron activity and other aspects of lever movement existed as well. To address this question, we selected three aspect of lever movement, namely lever position, velocity (derivative of lever position), and speed (absolute value of lever velocity) (Fig. 5A). For each neuron, we calculated the cross-correlation coefficients between single-neuron activity and the three lever movement aspects during lever movements (Fig. 5A). For a given recording session, neural activity of a fraction of the recorded neurons were significantly correlated with either lever position, velocity, or speed, or with two or three of the lever movement aspects [Fig. 5B; M1 L2/3; session 2; Supplemental Fig. S3 (https://figshare.com/s/9245c5019517cf3912ec)]. Monitoring the same group of neurons for 14 days revealed a drift in the correlation of the neural activity with the lever movement aspects across sessions [Fig. 5C; same mouse as Fig. 5B, Supplemental Fig. S4 (https://figshare.com/s/9245c5019517cf3912ec)]. Importantly, the correlation between neural activity and lever movement aspect in one session was significantly more similar with the correlation in the previous session than chance (Fig. 5D). This similarity of correlations in adjacent sessions increased from the naive to the expert mice [Fig. 5E, Supplemental Fig. S3 (https://figshare.com/s/9245c5019517cf3912ec)].

Fig. 5.

The relations between single-neuron activity and aspects of lever movement drift across sessions. A: lever position, velocity, and speed traces in multiple trials from a representative mouse. Gray, individual trials; black, average of all trials; blue, one example trace for lever position; green, the corresponding example trace for lever velocity; purple, the corresponding example trace for lever speed; left: one session from naive stage; right: one session from expert stage. Bottom: example trace of single neuron activity (ΔF/F₀) above for one trial. B: the significant correlations (Corr) between ΔF/F₀ and the aspects of lever movement indicate a relation between single neuron activity and lever position, velocity, speed, or multiple lever movement aspects for a fraction of the recorded neurons (session 2 from one example animal; M1 L2/3). Open circles indicate neurons with no significant correlation of their activity and the lever movement. C: for each session and neuron, the significant correlation of the neuron activity with aspects of lever movement are indicated by the corresponding color as defined in B. White indicates no significant correlation. Neurons are sorted in session 1 and this order remains fixed for the display of all other sessions, thus revealing the drift in relations across sessions. Neurons are from the same imaging field as B. D: correlation of representation across adjacent sessions versus sessions for all n = 30 imaging fields. Representations were assigned numerical values to compute correlations (see methods). Red curves, experimental; pink curves, shuffled. Error bars are SE. E: according to two-sample t test, the adjacent correlation of neural representation was significantly smaller (P < 0.05) for the naive stage (sessions 1 and 2) compared with middle (sessions 5–7) and expert (sessions 11–14) stages [see ANOVA test in Supplemental Fig. S3 (https://figshare.com/s/9245c5019517cf3912ec)]. *P < 0.05; **P < 0.01.

In summary, our new analysis confirms and extends previous findings that the activity of all four populations of neurons (L2/3 and L5 in M1 and M2) reorganizes during motor learning.

Motor Cortex Network States Remain Stable with Respect to Criticality During Neural Activity Reorganization

The observed massive reorganization of neural activity (Fig. 3) and its relation to lever movement (Figs. 4 and 5) raised the question as to the stability of the motor cortex network state during motor learning and cortical reorganization. Among the many possible states of cortical circuits (Beggs and Plenz 2003; Harris and Thiele 2011; Holcman and Tsodyks 2006), evidence for the critical network state in cerebral cortex has proliferated in recent years (Muñoz 2018; Tomen et al. 2019; Wilting and Priesemann 2019). Qualitatively, the critical network state resides at the boundary between weakly and strongly coordinated population activity corresponding to concurrent phases of disorder and order, respectively. Quantitative tests of the critical network state include scale-freeness of neuronal avalanches and the scaling relationships between the measured exponents (Friedman et al. 2012; Marshall et al. 2016). Within this conceptual framework, neuronal avalanches are bouts of elevated network activity, revealing correlations across neurons and time (Orlandi et al. 2013). Here we investigated whether such fingerprints for the critical network state existed in the population activity of L2/3 and L5 of M1 and M2 during motor learning.

For each recorded neuron, we thresholded the ISP at 3 SDs (SD determined from the inferred spike probabilities of the entire population in each session) and thus binarized the ISP of each neuron into a binary activity time series (Fig. 6A). From the sum of the binarized ISP values across neurons, we obtained the “network activity” in a time bin. Based on the network activity, we defined a neuronal avalanche by introducing a threshold at the 35th percentile of network activity (Hartley et al. 2014; Johnson et al. 2019; Karimipanah et al. 2017a; Larremore et al. 2014; Poil et al. 2008, 2012). An avalanche starts when the network activity crosses the threshold from below and ends when the network activity crosses the threshold from above. We quantified each neuronal avalanche by its size S, i.e., the integrated network activity between threshold crossings, and its duration D, i.e., the time between threshold crossings.

Fig. 6.

Motor cortex network states remain stable with respect to criticality during neural activity reorganization in L2/3. A: methods on neuron avalanche analysis. Top: inferred spike probability (ISP) for one representative neuron. Spike probability was inferred from calcium fluorescent signals using a spike inference algorithm. Spike probability was then thresholded (dashed red line) to a level of 3 SDs above 0, and converted to 1 (active) or 0 (inactive). Middle: raster plot of activity constructed using the thresholded spike probability. Each row represents single neuron, and each mark represents the inferred spiking activity of that neuron, i.e., the thresholded spike probability with value 1 (active). Bottom: “Network activity” (black) is the sum of all spiking neurons. A threshold (dashed red line) at median network activity defines the start and end of a “neuronal avalanche” as the time points of crossing this threshold. The avalanche size (S, yellow) is the integrated network activity for the avalanche duration (D), i.e., the time between threshold crossings. B and C: probability density functions (PDF, purple dots) of avalanche sizes and durations for M1 L2/3 in session 1 (B) and M1 L2/3 in session 14 (from same animal) (C) followed power laws with exponent τ for size and α for duration distribution. Purple dots are distributions based on empirical data, only filled dots were fitted to power laws (black lines). We shuffled spikes for each neuron separately and got the distribution for shuffled spikes (dashed gray line) for comparison. D and E: exponents for avalanche size (τ) and duration (α) distributions of all subjects and sessions for M1 L2/3 (D) and M2 L2/3 (E). The gray scale of the symbols decreases from dark to light with increasing session number. F and G: The network state quantified by κ (size distribution only). When κ is close to 1, the distribution is close to power law, otherwise, it’s not. Here the κ for M1 L2/3 (F) and M2 L2/3 (G) are close to 1. For clarity of visualization of the many overlapping points for each session, we jittered the points laterally.

Avalanches for L2/3 population activity of M1 and M2 were diverse in spatiotemporal scale. Specifically, both avalanche size and duration distributions were consistent with power laws, P(S)∼S^−τ and P(D)∼D^−α (Fig. 6B). The closeness of the avalanche distributions to power laws was evaluated using rigorous statistical criteria (methods). We validated the power law fitting method by using simulated data from a PIF model [methods, Supplemental Fig. S5 (https://figshare.com/s/9245c5019517cf3912ec)]. The choice of single neuron signal threshold and network activity threshold within a certain range returned robust results [methods, Supplemental Fig. S6 (https://figshare.com/s/9245c5019517cf3912ec)]. Importantly, recorded population activities resulted in robust power-law avalanche distributions across all 14 sessions [Fig. 6C: same animal as Fig. 6B, Supplemental Fig. S7 (https://figshare.com/s/9245c5019517cf3912ec)]. Interestingly, the exponent values covered a broad range (Fig. 6, D and E). Such range of exponent values and the apparent linear relationship between size and duration exponents coincides with earlier experimental observations (Beggs and Plenz 2003; Fontenele et al. 2019; Hahn et al. 2010; Johnson et al. 2019; Priesemann et al. 2013; Shew et al. 2015). For completeness, we note that exponent values did not change with the progression of recording sessions and did not vary with the number of recorded neurons [Supplemental Fig. S8 (https://figshare.com/s/9245c5019517cf3912ec)].

To evaluate the potential deviation of avalanche size distributions from power laws across sessions, we used the previously developed statistical measure κ (Shew et al. 2009, 2011). In brief, we fitted a power law and the corresponding exponent τ to an empirical avalanche size distribution and then generated a surrogate (theoretical) avalanche size distribution. The statistical construct κ measures the difference between the two distributions (methods). By its definition, κ≈1 indicates a good match between the empirical distribution and the theoretical power law distribution. For all mice tested, L2/3 avalanche size distributions followed power laws across all 14 sessions as indicated by stable κ values near 1 (Fig. 6, F and G).

Power laws provide necessary but not sufficient evidence for critical dynamics (Stumpf and Porter 2012). Additional tests are needed to determine whether criticality underlies the experimentally observed power laws. Two such tests arise from a particular relationship between the size and duration of avalanches, which is predicted to occur at criticality (Friedman et al. 2012; Scarpetta et al. 2018; Sethna et al. 2001). First, the average avalanche size scales with duration according to $\langle S\rangle \sim D^{{\beta }_{\text{fit}}}$. Second, the exponent β is not independent, but rather depends on the exponents τ and α according to ${\beta }_{pred}=\frac{\alpha –1}{\tau –1}$. Our experiments confirmed both these predictions from the scaling relation for all sessions. Average avalanche size scaled with duration according to a power law $\langle S\rangle \sim D^{{\beta }_{fit}}$ (Fig. 7A) and the observed values of τ and α provided a prediction exponent ${\beta }_{pred}=\frac{\alpha –1}{\tau –1}$ of the fitted exponent β_pred for all 14 sessions (Fig. 7, B and C). Most of the differences between β_pred and β_fit are smaller than 0.2 (within the shaded area). As tested by a RNN model in (Ma et al. 2019), a difference of less than 0.2 is a useful criterion for the critical network state. The exponent β is itself meaningful; it quantifies how quickly an avalanche “fans out.” For instance, if β is precisely 1.0, avalanche size increases linearly with duration. Emerging data suggest that a wide variety of species and preparations generate scaling exponent β values of ∼1.2 (Fontenele et al. 2019). Our recorded L2/3 population activity revealed a stable β for all 14 sessions which was close to 1.2 (Fig. 7, D and E).

Fig. 7.

Neuronal avalanche scaling relation and shape collapse analyses support the notion that criticality underlies the observed power laws in L2/3. A: the linear relationship on logarithmic axes revealed a power law relationship <S>∼D^β between average avalanche size (S) and duration (D) as predicted by criticality theory. Scaling exponent predicted from critical theory (${\beta }_{\text{pred}}=\frac{\alpha -1}{\tau -1}$, solid line) was close to the exponent fitted with experimental data (purple dots) (M1 L2/3 session 14; same animal as Fig. 6). B and C: the predicted scaling exponents and fitted scaling exponents for M1 L2/3 (7 animals) (B) and M2 L2/3 (8 animals) (C). Gray scale changes from dark to light with increasing session number. Yellow shades indicate the criterium for consistency with criticality, i.e., an exponent difference of less than 0.2 (see Ma et al. 2019).Consistency with this criticality criterium is seen in 91/96 samples in M1 L2/3 and in 60/75 samples in M2 L2/3. D and E: fitted scaling exponents (β_fit) on each single session for M1 L2/3 (D) and M2 L2/3 (E). Colors denote different subjects. F: in experimental data, scaled avalanches across durations (gray, avalanches with duration from 3 to 10 frames) show little error around the polynomial fit (red). We show one example from M1 L2/3 session 2 of a single animal. G: shuffled data (gray) have no characteristic shape and are characterized by larger error around the fit (red). Shuffled data did not accomplish shape collapse. H and I: the variance-based errors for M1 L2/3 (H) and M2 L2/3 (I) are below 0.01 (black curves, averages; colored squares, samples). In contrast, the variance-based errors for shuffled data (gray line and squares) are above 0.01. Error bars are standard errors. The two curves are significantly different on each single session (P < 0.05).

A related test for critical dynamics arises from the “shape collapse,” i.e., the scaled avalanche profiles follow the same shape for all durations (Friedman et al. 2012; Marshall et al. 2016; Sethna et al. 2001). The L2/3 avalanche temporal profiles increased and decreased in size in a largely stereotypical manner. Avalanches exhibited shape collapse over an expansive set of durations. In other words, avalanches all had a similar “hump shape” no matter how long they lasted (Fig. 7F) and this was significantly different for shuffled data (Fig. 7G). In more quantitative terms, scaled avalanches across a range of durations showed little error (variance) around the polynomial fit (Fig. 7, H and I). Most of the variance-based errors of L2/3 are below 0.01, while errors of the shuffled data are 0.022 ± 0.006 (mean ± SD). The t test of all experimental and shuffled pairs suggested significant difference (P < 0.05). These tests establish that L2/3 networks in M1 and M2 consistently operate near criticality throughout learning.

In contrast to L2/3, L5 population activity in M1 and M2 displayed relatively fewer large avalanches; avalanche distributions were typically small scale and not consistent with power laws [Fig. 8A, Supplemental Fig. S9 (https://figshare.com/s/9245c5019517cf3912ec)]. When forcing a power law fit to the avalanche size and duration distributions (methods), the deviations of the avalanche distributions from power laws remained stable across all 14 sessions (Fig. 8B). As expected, the resulting exponent values for τ and α failed to provide a prediction for the scaling exponent β (Fig. 8, A and B). For all mice tested, L5 avalanche size distributions largely deviated from power laws across most of the 14 sessions as indicated by κ values below 1 (Fig. 8, C and D). Avalanche profiles failed to map onto one shape (Fig. 8E). In more quantitative terms, scaled avalanches across a range of durations showed relatively larger error around the polynomial fit (Fig. 8, F and G). The errors of L5 networks were significantly larger than those of L2/3 for both M1 (P = 2.5434e⁻¹⁵) and M2 (P = 2.0207e⁻¹²) and were not significantly different from shuffled data (P > 0.05 for 21 experimental and shuffled session pairs out of 28 sessions). These results indicate that, as opposed to L2/3, L5 networks in both M1 and M2 operate away from criticality throughout learning.

Fig. 8.

Motor cortex network states remain stable with respect to criticality during neural activity reorganization in L5. A: probability density functions (PDF) for avalanche sizes (S) and durations (D) for M2 L5 session 1 do not follow power laws, and the scaling relates fails. B: similar as A, for M1 L5 session 11 (same animal). C: κ values for M1 L5 (size distribution) were far away from 1. For clarity of visualization of the many overlapping points for each session, we jittered the points laterally. D: similar as C, for M2 L5. E: in experimental data for M1 L5, scaled avalanches across durations (gray) do not accomplish shape collapse (duration range 3 to 10). F and G: the variance-based error (black curve and colored squares) in M1 L5 (F) and M2 L5 (G) are not significantly different (t test, P > 0.05 for 21 pairs out of 28) from shuffled data (gray line and squares).

Complementary to the avalanche analysis, we evaluated network state by employing the “branching ratio” (Wilting and Priesemann 2018). The branching ratio is the expected number of neurons activated by one neuron in the previous time step (see methods). Importantly, the branching ratio approach does not employ the avalanches construct. A network operating at the critical point will have a branching ratio near 1.0. For L2/3 neurons, branching ratio values were near 1, whereas branching ratio values for L5 neurons were significantly lower [Supplemental Fig. S10 (https://figshare.com/s/9245c5019517cf3912ec)].

DISCUSSION

Our results indicate that cortical L2/3 circuits operate near the critical network state during learning, while L5 circuits operate away from criticality throughout learning. Specifically, the observations of power laws, scaling relations, shape collapse, and branching ratio together provide strong evidence that the inspected L2/3 cortical microcircuits of mouse M1 and M2 remain stable near the critical network state throughout motor learning, whereas L5 population activity continue to operate away from the critical network state.

Dynamic Cortical Representations During Motor Learning

How motor neuronal activity correlates with movement is an essential and open question in neuroscience (Moran and Schwartz 1999; Mussa-Ivaldi 1988; Schwartz 1994). Previous investigations of this question have largely focused on electrical recordings of neural activity, such as single unit activity (SUA) (Paninski et al. 2004), local field potential (LFP) (Mehring et al. 2003), electrocorticogram (ECoG) (Pistohl et al. 2008), and intracranial EEG (iEEG) (Hammer et al. 2016). Two-photon calcium imaging 1) yields signals from a large number of neurons simultaneously and 2) monitors spiking activity from the same group of neurons during different stages of motor learning across several days. So it serves as a useful complementary tool for the investigation of motor learning (Huber et al. 2012; Komiyama et al. 2010; Masamizu et al. 2014; Peters et al. 2014, 2017a). With two-photon calcium imaging recording, our study is focused on both single neuron and neural population dynamics.

We investigated the heterogeneity of neurons. Heterogeneity in neural system (Chelaru and Dragoi 2008; Golomb and Rinzel 1993; Mejias and Longtin 2012; White et al. 1998), especially the heterogeneity of motor cortex neural activity and its relation to movement, has long been discussed (Churchland and Shenoy 2007; Rokni et al. 2007). Here, we combined motor learning with heterogeneity study.

The present work confirms the restructuring of neural activity during motor learning (Makino et al. 2017; Peters et al. 2014, 2017a) (Fig. 3) and demonstrates the concurrent reorganization of the relation between neural activity and movement (Figs. 3 and 4).

Neural circuits are known to undergo significant remodeling over multiple timescales, even in the absence of behavioral training (Chambers and Rumpel 2017; Clopath et al. 2017), possibly leading to a massive, but dynamically balanced, remodeling of excitatory and inhibitory synaptic networks on the time scale of weeks (Attardo et al. 2015; Villa et al. 2016). Such substantial synaptic restructuring may imply that a given behavior can be realized by multiple configurations of synaptic strengths and that such wandering among synaptic configurations with equivalent behaviors, but different neural representations, may be advantageous for learning (Rokni et al. 2007). The theme of dynamic circuit remodeling is not limited to the encoding of motor outputs (Carmena et al. 2005; Flint et al. 2016; Masamizu et al. 2014; Peters et al. 2014), but also resurfaces in the encoding of sensory stimuli during learning (Huber et al. 2012) or sensory deprivation (Margolis et al. 2012) and the dynamic neural coding of place in hippocampus (Ziv et al. 2013).

The observation of the fluid representation of different aspects of the lever movement during learning raises an important question (Fig. 5). Does the flexible neural code for position and velocity underlying lever movement control determine which new movements are easier to learn than others (Sing et al. 2009)? Neural elements of motor control often encode information about limb position and velocity (Ashe and Georgopoulos 1994; Paninski et al. 2004). Here, we quantified the dynamic nature of this neural representation of position and velocity/speed across recording sessions (Fig. 5). Such representational flexibility, mediated by mixed task selectivity (Yang et al. 2019), is likely to be crucial for the learning of new movements (Bathellier et al. 2013; Rokni et al. 2007). Much future experimental work will be needed to test such hypothesized relation between the fluidity of representation and the ease of learning.

Stability of Network State Near Criticality

The present work provides a missing link between the reorganization of neural circuits during learning (Fig. 3–5) and the stability of the network state (Chambers and Rumpel 2017; Rokni et al. 2007), by focusing on the network state with respect to criticality (Fig. 6–8).

What is the critical network state (Fig. 1C) and why is it relevant? Network dynamics is an emergent property of neuronal interaction. This dynamic property determines the computational regime of a network. In general, cortical circuits are often assumed to 1) be “balanced,” such that runaway gain does not drive the network toward saturation or silence, 2) encode and transmit information across a wide range of spatial and temporal scales, 3) have a broad dynamic range, and 4) be capable of processing complex information. Such circuit properties arise out of a specific nonequilibrium regime of population dynamics often referred to as “criticality.” This critical regime of network dynamics has been proposed as a possible set point for certain cortical circuits (Bertschinger and Natschläger 2004; Karimipanah et al. 2017a; Priesemann et al. 2014; Shew et al. 2015). Qualitatively, the critical network state arises at the boundary between strongly and weakly coordinated population activity. These correspond to phases of order (strongly coupled) and disorder, respectively. A long-standing hypothesis at the interface of physics and neuroscience is that neural circuits in cerebral cortex operate near the dynamical critical network state (Herz and Hopfield 1995; Stassinopoulos and Bak 1995), thereby maximizing aspects of sensory information processing (Shew and Plenz 2013; Tomen et al. 2019). This criticality hypothesis (Beggs 2008) is partially supported by evidence for steady-state critical dynamics observed in the neural system (Beggs and Plenz 2003; Friedman et al. 2012; Petermann et al. 2009).

However, the stability of the critical network state with respect to “perturbations” and intrinsic changes has only begun to be explored. Of such perturbations, the level of arousal, sustained attention, or tonic alertness on the network state has garnered much interest. Literatures show stability of network state across many conditions. Recordings from cerebral cortex and hippocampus of behaving rats indicate that signatures of network criticality are stable across waking, slow-wave sleep, and rapid-eye-movement sleep but collapse during anesthesia (Ribeiro et al. 2010). Dense intracranial depth recordings in humans reveal avalanche distributions closely following a power law for each vigilance state, including slow-wave sleep, wakefulness, and rapid eye movement sleep (Priesemann et al. 2013). Other studies focus on the influence of levels of consciousness onto network states. Long-range temporal correlations, which naturally emerge in the vicinity of a critical network state, decline during sustained wakefulness in the human brain (Meisel et al. 2017a) and in rat brains (Meisel et al. 2017b). Focused cognitive task in humans induces subcritical dynamics (Fagerholm et al. 2015). States of consciousness impact the network state (Lee et al. 2019). Specifically, the brain network states deviate from criticality during anesthesia (Bellay et al. 2015; Fagerholm et al. 2018; Fekete et al. 2018; Ribeiro et al. 2010; Tagliazucchi et al. 2016). In the awake condition, the network states vary with eye-open versus eye-closed conditions (Hahn et al. 2017). In addition, task modulations of network state at short time scale is also a hot topic. Cognitive tasks impact the neural activity, but not the network state, in nonhuman primates (Yu et al. 2017). Sensory stimuli are known to modulate neural activity, yet the network state with respect to criticality remains largely unchanged (Arviv et al. 2015; Karimipanah et al. 2017a). Such stability of network state at short time scale can be mediated by sensory adaptation (Shew et al. 2015). At the time scale of days, when cortical input is perturbed by visual deprivation, an established homeostatic challenge, cortical network states deviate from criticality and subsequently return to the critical network state within 48 h, thus establishing that criticality is a homeostatic set point of complex activity in visual cortical circuits (Ma et al. 2019). At the longer time scale of development, distributions of neuronal avalanches and long-range temporal correlations remain stable during the first year of life in human infant (Jannesari et al. 2020).

In the present work, we have advanced the study of the stability of the network state near criticality toward the field of learning (Del Papa et al. 2017). Learning is qualitatively different in its specificity of reorganization, compared with the above-discussed external perturbations (e.g., anesthesia), which are global in nature. Such large-scale disturbances render a global response in network state plausible, as both perturbation and response can happen at the same global spatial scale. In contrast, learning a new motor task implies specific and spatially localized changes in the underlying neural activity (Figs. 1–5), mediated by changes in the synaptic circuitry (Biane et al. 2019; Chen et al. 2015; Fu et al. 2012; Rioult-Pedotti et al. 2000; Xu et al. 2009; Yang et al. 2009). Whether the massive synaptic and neuronal reorganization during learning materialize within the constraint of stable cortical circuit dynamics with respect to criticality was until now unknown. Earlier observations at the microscopic level indicated the possibility of some form of network-state invariance during learning, as spine formation may be balanced by spine eliminations (Harms et al. 2008; Xu et al. 2009; Yang et al. 2009), synaptic potentiation may be balanced by synaptic depression (Cohen and Castro-Alamancos 2005), and changes in inhibition may be balanced by changes in E-to-I connectivity (Donato et al. 2013). Here, through the concurrent 14-day observation of neural reorganization (Figs. 2–5) and the network state (Figs. 6–8) during learning, we showed that motor cortex network states remain stable with respect to criticality. In other words, L2/3 neural circuits of motor cortex hover around or near the critical network state during the neural reorganization. Thus, as shown previously in visual cortex (Ma et al. 2019), the critical network state may be an attracting point of cortical circuits rather than a state at which it is always perfectly poised. Preliminary evidence for layer-specific network states has been discussed before (Gireesh and Plenz 2008), but those studies were limited to local field potential recordings from somatosensory cortex in urethane-anesthetized rats and brain slices. Such layer specificity of network state could be motivated by differences in function. From this functional perspective, self-organized criticality may provide a unifying explanation for the large variability of neural activity (Karimipanah et al. 2017a, 2017b; Linkenkaer-Hansen 2003). This raises the question whether operating away from criticality decreases the variability of neuronal activity and thus may benefit the L5 system which serves as descending output layer (Anderson et al. 2010; Brecht et al. 2013; Cramer et al. 2020; Weiler et al. 2008).

The discovery of the dynamical criticality constraint during learning raises important new questions. First, what biophysical mechanisms self-organize the network state in L2/3 near criticality while the neural circuit reorganizes during learning? A number of synaptic plasticity mechanisms can mediate self-organization toward the critical network state (Levina et al. 2007, 2009). More complex model investigations have been extended toward simple sequence learning tasks (Del Papa et al. 2017) and to network state homeostasis (Ma et al. 2019; Zierenberg et al. 2018). Significantly more future work, experimental and computational, will be needed to demonstrate the interplay of biophysical mechanisms mediating the self-organization of the network state near criticality during learning and the concurrent neural circuit reorganization. Second, does network criticality optimize learning? Neuronal variability is essential for learning (Bathellier et al. 2013; Rokni et al. 2007; van Beers et al. 2004). On the other hand, robustness in biological networks is essential for reliable performance (Aldana et al. 2007; Barkai and Leibler 1997). How then do biological systems balance the need for both variability and robustness (Chambers and Rumpel 2017)? This question is particularly pertinent for motor learning and neural circuits (Golub et al. 2018; Peters et al. 2017b). In this context, it is tantalizing to hypothesize that the critical network state may provide L2/3 circuits with a balance between robustness and flexibility that is optimized for learning (Muñoz 2018). Many future experiments will be needed to test this far-reaching learning-at-criticality hypothesis.

GRANTS

This research was supported by a Whitehall Foundation grant (no. 20121221; R. Wessel), a National Science Foundation Collaborative Research in Computational Neuroscience grant (no. 1308159; R. Wessel), National Institutes of Health (NIH) grant (R01-S091010A; T. Komiyama), and NIH McKnight Scholar Award (T. Komiyama).

DISCLOSURES

No conflicts of interest, financial or otherwise, are declared by the authors.

AUTHOR CONTRIBUTIONS

Z.M., T.K., and R.W. conceived and designed research; H.L. performed experiments; Z.M. analyzed data; Z.M., H.L., T.K., and R.W. interpreted results of experiments; Z.M. prepared figures; Z.M. drafted manuscript; Z.M., H.L., T.K., and R.W. edited and revised manuscript; Z.M., H.L., T.K., and R.W. approved final version of manuscript.