Rapid Integration of Artificial Sensory Feedback during Operant Conditioning of Motor Cortex Neurons

Summary

Neuronal motor commands, whether generating real or neuroprosthetic movements, are shaped by ongoing sensory feedback from the displacement being produced. Here we asked if cortical stimulation could provide artificial feedback during operant conditioning of cortical neurons. Simultaneous two-photon imaging and real-time optogenetic stimulation were used to train mice to activate a single neuron in motor cortex (M1), while continuous feedback of its activity level was provided by proportionally stimulating somatosensory cortex. This artificial signal was necessary to rapidly learn to increase the conditioned activity, detect correct performance, and maintain the learned behavior. Population imaging in M1 revealed that learning-related activity changes are observed in the conditioned cell only, which highlights the functional potential of individual neurons in the neocortex. Our findings demonstrate the capacity of animals to use an artificially induced cortical channel in a behaviorally relevant way and reveal the remarkable speed and specificity at which this can occur.

Highlights

• •All-optical brain-machine-brain interface for neuroprosthetic control
• •Mice rapidly learn to activate single neurons under optogenetically evoked feedback
• •Population imaging reveals that learning is restricted to the conditioned neuron
• •Novel “in cerebro” learning paradigm for neural circuit dissection

Prsa et al. demonstrate that cortical stimulation can provide informative feedback about the current state of a voluntarily modulated neuron. Rapid learning of this artificial paradigm holds promise for its use as a research tool and in neuroprosthetic applications.

Introduction

Brain-machine interfaces (BMIs) impose arbitrary associations between neural activity patterns and prosthetic actions. A neuroprosthetic skill is considered successfully acquired when this association is learned and the prosthesis can be controlled in a goal-directed manner. The underlying principle is that neural activity, reinforced via operant conditioning, can be volitionally generated. Operant conditioning of cortical neurons has been successfully achieved in monkeys (Engelhard et al., 2013, Fetz, 1969, Fetz and Baker, 1973, Hwang et al., 2013, Moritz et al., 2008, Sadtler et al., 2014), rodents (Arduin et al., 2013, Clancy et al., 2014, Gage et al., 2005, Hira et al., 2014, Koralek et al., 2012), and humans (Cerf et al., 2010) by translating their activity into auditory or visual feedback signals and reinforcing desired patterns with reward. Because this paradigm specifies which neurons control an action, it circumvents the neural complexity associated with natural behavior. Its use goes, therefore, beyond BMI applications; it can serve as a research tool that facilitates dissecting the involved plasticity mechanisms (Sadtler et al., 2014). A closed-loop implementation in which the feedback takes the form of direct cortical activation (i.e., an artificial sensory channel) would offer the advantage of having full control of the neurons directly involved in the association to be learned. Implementing such an “artificial position sense” (Lebedev et al., 2011) in a BMI setting could not only offer more flexible and faster feedback mappings, but, more importantly, substitute the position sense of a neuroprosthetic device when natural sensation is lost.

Stimulation of sensory cortex can create behaviorally distinguishable percepts (Romo et al., 1998, Romo et al., 2000, Tabot et al., 2013); bias (Salzman et al., 1990), mimic (O’Connor et al., 2013, Sachidhanandam et al., 2013, Tabot et al., 2013), or even augment (Thomson et al., 2013) natural perception; and cue goal-directed movements (Fitzsimmons et al., 2007, Huber et al., 2008, Jazayeri et al., 2012). It can also be optimally integrated with natural sensory cues during real reaching movements (Dadarlat et al., 2015). However, in conjunction with volitional neural control, cortical stimulation has only been able to provide cues for choosing (O’Doherty et al., 2009) or for discriminating (O’Doherty et al., 2011) between virtual targets, while natural sensory feedback was used to guide learning and the actual execution. The feasibility of artificial sensations to not only cue, but to actually instruct in real time the generation of conditioned neural patterns, remains currently unexplored. This would mean that the brain can create an association between an action and its real-time sensory consequence when both are restricted to activities of different sets of cortical neurons.

Stable activity patterns of local populations of motor cortex neurons have been found to emerge during motor learning (Peters et al., 2014), and highly interconnected cortical ensembles (Harris and Mrsic-Flogel, 2013) might be responsible for the emergence of such population dynamics where individual neurons matter little. Operant conditioning of a single motor cortex neuron might thus be expected to entrain its ensemble during learning and just be a participant of a changing population code. Recent findings, however, suggest that learning might be confined to the conditioned neurons rather than involving a cortical ensemble (Arduin et al., 2013, Clancy et al., 2014). Conclusive evidence of such learning specificity requires conditioning single neurons and simultaneous unambiguous tracking of a large number of neighboring non-conditioned neurons.

In the present study, we therefore address the following questions: Can mice learn to use a fabricated feedback channel to control single-neuron activity? How do responses of the conditioned and neighboring cortical neurons change with learning? For this purpose, we developed an all-optical BMI, a system for simultaneous wide-field two-photon population imaging of neurons expressing genetically encoded calcium (Ca) indicators in primary motor cortex (M1) and simultaneous activation of neurons expressing optogenetic actuators in primary somatosensory cortex (S1). Operant conditioning of a single M1 neuron was performed by reading out its activity in real time, transforming it into a rate code of optogenetic stimulation pulses in S1 and reinforcing above-threshold activations with reward. We tested the necessity of the optogenetic feedback signal for identifying the reinforced activations and for learning to produce them more often over time. We then analyzed learning-related changes observed in conditioned neurons in comparison to those in neighboring, longitudinally tracked, non-conditioned neurons. Our results unveil basic properties of L2/3 processing, which may also characterize cortical activity during, but not be delineable with, natural behaviors.

Results

Operant Conditioning of L2/3 M1 Neurons under Artificial Sensory Feedback

To test if artificial sensory feedback can guide operant conditioning of cortical neurons, we first developed a novel two-photon imaging system designed for simultaneous optogenetic stimulation of cortical areas in head-fixed mice (Figures 1A–1D; see STAR Methods). Mice expressed the genetically encoded Ca indicator GCamP6f in forelimb M1 and the optogenetic actuator channelrhodposin-2 (ChR2) in the corresponding somatosensory representation (Figure 1A). We used a Cre-dependent reporter mouse line (Ai32) to ensure a stable level of ChR2 expression over time. To monitor the effect of the optogenetic stimulation, we measured the local field potential in the contralateral S1 with electrocorticograms (ECoGs; Figure 1B). This signal originated from the prominent callosal axonal fibers (Mao et al., 2011, Petreanu et al., 2007). Experimental animals were either classified as ChR2 mice (n = 8) or control mice (n = 11) depending on whether ECoG responses were detectable upon optogenetic stimulation (Figures 1C, S1A, and S1B, available online). The absence of optogenetically driven responses in control mice was due to insufficient or lack of expression of ChR2 (see STAR Methods). Two-photon images of large populations of individual L2/3 neurons (461 ± 164, mean ± SD) were acquired at ≈30Hz and simultaneously streamed to a dedicated computer for real-time processing (Figure 1D).

Figure 1.

Experimental Overview

(A) Dorsal view of the cranial window showing the expression patterns of GCaMP6f and ChR2 (green) and the intrinsic signal (red) of contralateral forepaw stimulation. The blue laser was either aimed at the S1 forepaw site (blue dot) or a control non-ChR2 site (gray dot), while the two-photon imaging plane (red square) covered the forepaw M1, all within the objective’s field of view (white square).

(B) Schematic of our chronic preparation for simultaneous two-photon imaging and optogenetic stimulation. The two-photon infrared laser (red) was scanned, through a cranial window, across a population of neurons expressing GCaMP6f (green), and a blue laser was aimed through the same objective at neurons expressing ChR2 (blue) for optogenetic stimulation. Electrocorticogram (ECoG) electrodes were implanted in the contralateral hemisphere for measuring neural responses evoked by the optogenetic stimulus in the callosal projection neurons.

(C) Event-related potentials (averages of 30 repetitions) evoked by optogenetic stimulation pulses as measured by the ECoG electrodes in an example ChR2 and control mouse.

(D) Schematic of the experimental setup showing the head-fixed mouse under the two-photon microscope. The two laser beams for imaging and stimulation were controlled independently, but focused through the same microscope objective. To protect the sensitive photomultiplier tubes of the imaging system during the optogenetic blue light pulses, we combined optical filters and gated detection electronics. Two-photon images were acquired on a PC running Scanimage and streamed to a control PC that extracted in real time the Ca-dependent neural activity; generated the feedback signal (inset: transfer function between neural activity and feedback pulse rate; see STAR Methods for details); acquired ECoG, lick, and forepaw movement sensor data; and controlled the blue light mask. S1, S2, scanning mirrors; D1, D2, dichroic mirrors; EF, emission filter; gPMT, gated photomultiplier tube. A conditioned neuron (CN) (blue) was chosen among hundreds of simultaneously imaged neurons (gray).

(E) Example Ca-dependent activity of a CN (blue), its real-time transform into a rate signal of optical pulses, reward times, and lick sensor recordings.

To condition the activity of a single neuron, we extracted its fluorescence changes from the image and used it as a proxy for its neuronal activity. The fluorescence transients were transformed directly into a rate code dictating the frequency of the optogenetic stimulation pulses targeting S1. A water reward was delivered whenever the conditioned neuron’s activity and, by extension, the rate of optogenetic stimulation crossed a chosen threshold, thereby reinforcing high activation levels (Figure 1E). We trained the ChR2 and control mice in the operant conditioning task for at least 15 consecutive daily sessions, for 30–40 min per session (Figure S2A; see STAR Methods for details). Potential visual cues from the optogenetic stimulation were effectively masked with blue light flashes (Figure S1C).

Artificial Sensory Feedback Leads to Rapid Learning of Volitional Neuronal Activation

Learning was defined as significant increases in threshold crossing rate (TCR) of the conditioned neuron’s (CN) Ca response between session start and end (two-tailed paired t test, p < 0.05). According to this definition, ChR2 mice learned on average within single sessions (Figure 2A), with six out of eight individuals showing significantly increased TCRs. A significant increase in TCR compared to session start was already observed in the second time bin (p < 10⁻³), suggesting that operant control of the CN ensued after 4.9 ± 1.64 min (mean ± SD) of conditioning. Control mice, which underwent identical experimental procedures and were rewarded at similar rates as ChR2 mice (Figure S2B), did not learn on average, with only one out of eleven showing an increase in the TCR (Figure 2B). Baseline TCR at the start of a session did not differ between mice that learned and the ones that did not learn (non-paired t test, p = 0.15; Figure S2C). Sessions with significant learning (linear regression, p < 0.05) became more frequent across successive days of training in ChR2 mice only (Figure 2C). After more than 10 consecutive days of conditioning, the probability of observing learning in a given ChR2 animal reached ≈40%. During these learning sessions, the number of threshold crossings increased by 0.66 ± 0.08 (mean ± SEM) every minute. Movement artifacts in the two-photon images and activation of the CN by the optogenetic stimulation via direct inputs from S1 were negligible and did not contribute to threshold crossings (Figures S2D–S2G and S3). Learning was also not accompanied by more frequent contralateral forepaw movements (Figure S2H). In previous studies, operant conditioning of cortical neurons in the absence of sensory feedback (reward only) was found to be either not possible (Koralek et al., 2012) or only successful in experienced animals previously trained with visual feedback (Fetz, 1969) or extensively exposed to reward-based learning (Hira et al., 2014). Our results indicate that in naive animals, providing feedback by ChR2-based cortical stimulation can expedite operant conditioning of cortical neurons when compared to reward reinforcement only. Therefore, the brain is able to form associations between the activity of a single neuron and proportional feedback stimulation in S1 within single behavioral sessions.

Figure 2.

Importance of Artificial Sensory Feedback for Execution and Learning

(A) Within-session changes of normalized average threshold crossing rate (TCR) of eight ChR2 mice (n = 127 sessions) calculated within eight equally sized time bins. Non-normalized TCRs were significantly different between the first and last time bins (two-tailed paired t test, p < 10⁻⁷, t(126) = 5.69). Population mean error bars are SEM. Individual mice with significantly different TCRs between the first and last time bins (p < 0.05) were classified as learners and as non-learners otherwise.

(B) Same data as in (A) for eleven control mice (n = 158 sessions).

(C) Fraction of ChR2 and control mice yielding “learning sessions” (linear regression, p < 0.05) over successive training days.

(D) Examples of feedback, feedback OFF, and playback conditions under lick-triggered reward delivery.

(E) Median (with 40% and 60% quantiles) detection probability (proportion of threshold crossings followed by licking and reward) was significantly impaired, compared to the optogenetic feedback condition, only when the feedback was either removed or directed to a non-ChR2 site, in eight ChR2 mice (n = 39 sessions). The eleven control mice showed near-zero detection probability across all conditions (n = 46 sessions). ^∗p < 0.01; n.s., p > 0.05; Kruskal-Wallis test.

Artificial Feedback Is and Remains Necessary for Detecting Conditioned Activity Patterns after Learning

Given that learning occurs surprisingly fast, have the mice truly formed an association between activity increases of the CN and the reward outcome? Is optogenetic feedback a necessary cue for maintaining this association after the initial learning has occurred? Contrary to normal task conditions where rewards were automatically delivered, these questions were addressed with a “lick-triggered reward” condition, during which mice had to initiate licks upon threshold crossings to receive the reward. This condition therefore tested if the mice can behaviorally detect instances of threshold crossings (i.e., the actions that lead to reward) and was introduced for brief intervals of time at the end of the last five experimental sessions (Figure S2A). To learn and maintain an association between above-threshold activation and the reward outcome, mice could also rely on internal predictions related to the generation of the neuron’s activity (i.e., analogous to efference copies). To assess the relative contribution of the involved cues, we uncoupled the optogenetic feedback from the CN’s activity in feedback removal and “playback” conditions.

In the normal feedback condition, detection probability, defined as the proportion of threshold crossings successfully detected by subsequent licking, was close to unity (Figures 2D and 2E). When feedback was transiently removed, detection probability significantly dropped (p < 10⁻⁵, Kruskal-Wallis test) and became no different than would be expected by chance (Figure S4; see STAR Methods for details). These results reveal that the association has indeed been learned, that the optogenetic stimulus remains necessary, and that potential internal cues alone are not sufficient for detecting correct performance. To assess if optogenetic stimulation alone was sufficient for identifying the rewarded actions, we also introduced a “playback” condition, during which the stimulation pattern was transiently decoupled from the CN’s real-time activity and controlled by a recording from a previous day (Figures 2D and 2E). During playback, because rewarded “actions” were not generated anymore by the animal, their identification could only be based on the S1 stimulation pattern. Detection probability during playback was not different compared to normal feedback periods (p = 0.33), demonstrating the importance optogenetic stimulation acquired during learning (Figures 2D and 2E). To exclude reliance on any other sensory cues, such as visual percepts evoked by the blue light pulses, we also delivered playback stimuli to a control area not expressing ChR2 (Figure 1A). Under these conditions, detection ability was significantly compromised (p < 10⁻⁶) and indistinguishable from either the stimulus removal condition (p = 0.84) or chance occurrence (Figure S4). Detection probability in control mice was at chance levels across all conditions. These results indicate that the artificial sensory signal is not only needed for learning to rapidly occur, but also to behaviorally identify correct performance during the task and remain the sole necessary cue for this purpose thereafter.

Neuronal Specificity and Day-to-Day Flexibility of Learning-Related Changes

What leads to increased threshold crossings of the CN’s activity during learning? Ca events might occur more frequently over time and thus the rate of those that cross threshold also increases. Alternatively, the Ca events might become larger and thereby cross threshold more often. To examine this, we looked at activity changes of the CNs in all sessions with significant learning (n = 33). Within-session increases in TCR were guided by increases in the rate of Ca events (i.e., occurrence of spike bursts), as well as increases in their individual amplitudes (i.e., number of spikes in a burst) (Figures 3A and 3B). While both changes in Ca event rate and amplitude contributed to TCR increases, event amplitude changes accounted significantly more on average for the variability in TCR during a session (general linear model; Figure 3B, inset).

Figure 3.

Learning-Related Changes Are Confined to Conditioned Neurons

(A) Ca-dependent activity, average fluorescence images, and reward times of the CN during the course of an example learning session and subsequent feedback removal period.

(B) Within-session changes of mean (±SEM) TCR, Ca event rate, and event amplitude of learning sessions (n = 33, 8 mice) and during feedback removal (shaded region). The TCR (paired two-tailed t test, p < 10⁻⁴, t(32) = 4.6) and event rate (p < 10⁻⁴, t(32) = 5.13) significantly decreased upon feedback removal, whereas event amplitude (p = 0.095, t(32) = 1.72) did not. ^∗p < 0.01; n.s., p > 0.05. Inset: individual (gray) and average (black) estimated parameters (b_Er and b_Ea) of the general linear model with the TCR Z score as the dependent and the Z scores of the event rate (Er) and event amplitude (Ea) as the independent variables for the 33 learning sessions. On average, changes in Ea accounted for more of the variance in TCR than changes in Er (paired two-tailed t test, p = 0.034, t(32) = 2.213). ^∗p < 0.05.

(C) Event rate and event amplitude changes of CNs and INs between the first and last time bins of an example (left) and all learning sessions (right, average IN changes are shown, n = 27 sessions with at least three INs). Dotted contour: 95% confidence ellipse.

(D) Left: median (±quartiles) Δf/f₀ traces aligned to threshold crossings of the CNs (n = 33) and INs (n = 1,314) during feedback and playback conditions of all 33 learning sessions. Right: distribution of cross-correlation lags between event traces of INs/CNs and the reward trace.

(E) Mean (±SEM) Δf/f₀ traces aligned to threshold crossings of the CN, INs (n = 43), and “leading neurons” (n = 27) of an example session and their spatial location in the imaged field of view relative to the CN.

(F) Mean distances of INs (n = 30 sessions with at least one IN) and “leading neurons” (n = 24 sessions with at least one neuron) relative to the CN compared to mean distances of size-matched random samples (color, individual sessions; black, median values). Bold lines are individual sessions with significantly different distances (p < 0.05, two-sided bootstrap test). ^∗p = 0.0048 (z = 0.126); n.s., p = 0.89 (z = 2.82); Wilcoxon two-sided rank-sum test.

How is the animal’s performance affected if optogenetic stimulation is suddenly turned off? A decrease in TCR would confirm the behavioral necessity of this signal. Furthermore, if either increase of the amplitude or rate of the conditioned events still persists, it would indicate that this activity change might be the result of learning-related plasticity. We found that upon feedback removal event rate significantly dropped, leading to a decreased number of threshold crossings, yet the event amplitude remained elevated (Figure 3B). The persistence of the elevated amplitude might therefore be an indication of lasting plasticity changes in the system. The immediate decrease in the rate of events, on the other hand, indicates that feedback remains necessary for maintaining the learned behavior and that the observed increases in CN’s activity are not signs of irreversible Ca accumulation or related to the acquisition of a habit.

We next asked if these learning-related changes are specific to the CN or if they can also be found in simultaneously imaged neighboring neurons. We found that 6.4% (quartiles, 2.3% and 10.6%) of neighboring neurons had significant within-session linear increases in either event rate or amplitude and were therefore labeled as “increasing neurons” (INs) These increases, however, did not match the characteristics of the learning-related activity changes of the CNs (Figures 3C and S5). Whereas the rate changes of INs were similar to those of CNs (two-sided paired t test, p = 0.51, t(26) = 0.661), the mean changes in event amplitude, which are mostly responsible for learning, were statistically different (p < 0.001, t(26) = 4.257). Furthermore, the onset of Ca events in INs lagged both rewarded Ca events of the CN and reward onsets (Figure 3D). Thus, activity increases observed in non-conditioned neurons probably reflect the behavioral consequences (e.g., reward anticipation, collection, or consumption) rather than causes of above-threshold activations of the CN.

It follows that learning-related changes are restricted to the CN and do not involve the collective activation of a local population of M1 neurons. Because the feedback signal represents a pseudo-random stimulation pattern for all neurons other than the CN, the observed neuronal specificity suggests that it is the activity-locked feedback signal, and not just the S1 optogenetic stimulation per se, that is important for learning. The neuronal specificity further suggests that the conditioned activity was not primarily related to other variables such as forelimb movements that would most likely result in driving all forelimb-related neurons in concert.

In spite of an absence of learning-related activity increases in neighboring neurons, it is still possible that the CN is driven by activity in the local network. Indeed, we found that a small fraction of neighboring neurons (2.5%; quartiles, 0% and 8.2%) exhibited Ca events that preceded those of the CN more often than would be expected by chance (p < 0.01, hypergeometric probability) (Figure 3E) and could therefore potentially constitute pre-synaptic inputs. These events occurred sparsely (30% ± 10%, mean ± SD of threshold crossings), and in only 16% of these “leading neurons,” an ideal observer could discriminate whether a conditioned Ca event would cross threshold or not (p < 0.05, permutation test, receiver operating characteristic [ROC] analysis; see STAR Methods for details). We found that the “leading neurons,” but not the INs, were situated in closer spatial proximity to the CN than a size-matched random neuronal sample (Figures 3E and 3F). Small spatially organized functional clusters of neurons might therefore exist in L2/3 of M1, as previously suggested (Hira et al., 2013), and sporadically drive CN’s activation. These putative local pre-synaptic drivers might actually be more numerous, yet be individually weak and therefore below the detection threshold of Ca imaging used in this study. Therefore, even if the animals learned to control a single neuron instead of an M1 subpopulation, the nearby neurons might still play an active role in providing part of the drive to the CN.

Long-term tracking of individual L2/3 neurons has revealed that their population representation might be stable, but their individual responsiveness and/or functional tunings are flexible day to day in motor (Huber et al., 2012, Peters et al., 2014), as well as sensory (unpublished data) cortices. Similarly, consistent cross-day learning of multi-unit neuroprosthetic control has been found to require either daily retraining or iteratively adapting the decoder to the day-to-day variability of neuronal activity (Orsborn et al., 2014, Taylor et al., 2002). Redundant representations of a motor or sensory function across many neurons might allow such flexibility and reflect a necessary adaptability of neural responses. However, operant conditioning of a single M1 neuron attributes a functional role to that neuron only. It might therefore be expected that its responses would become more stable and stereotyped over time. We analyzed the stability of the CNs across multiple days of learning. Despite their exclusive status, CNs were readily found inactive at the start of a new session and often remained silent for several consecutive days (Figures 4A and S6). This prevented further conditioning with the same neuron and forced us to choose a new one. The decision to switch conditioning to a new neuron was based on a baseline recording prior to session start (see STAR Methods for details), whereas the data in Figure 4A depict the neuron’s average activity during the entire session. The distribution of day-to-day changes in the overall Ca event rate of the CNs was not different from that of the non-conditioned neurons (Figure 4B; two-sample Kolmogorov-Smirnov goodness-of-fit test, p = 0.15). Conditioned L2/3 neurons seem therefore not to be immune to flexibility, even when they acquire an exclusive functional role, which further highlights the robustness of this seemingly intrinsic property of L2/3 representations.

Figure 4.

Flexible Day-to-Day Activity of Conditioned Neurons

(A) Cross-session changes of mean event rate relative to the first day of conditioning (session #1) for the 20 neurons (y axis) conditioned across the eight ChR2 mice. Neurons from the same mouse are delineated by the y axis ticks. A value of 0.5 indicates that the event rate has been halved relative to session #1. Red lines denote the last conditioning session with that neuron enforced by either activity drops or experiment end.

(B) Distribution of event rate change ratios between successive sessions of conditioned and median ratios of non-conditioned neurons.

Taking together the remarkable speed and specificity with which the conditioned neuron adapts to the imposed constraints, our results suggest the existence of functionally and spatially specific plasticity mechanisms able to tune the activity of an L2/3 neuron on very short timescales.

Rapid Learning of Multi-neuron Conditioning under Artificial Sensory Feedback

To determine if more complex mappings can also be rapidly learned with artificial feedback, we trained mice to modulate the activity of three neurons simultaneously. ChR2 mice previously trained on the single-neuron task were taught to co-activate two arbitrarily chosen M1 neurons, while silencing a third neuron to gain reward. The amplitude of the fluorescence signal of each neuron was transformed with a logistic function. The outputs of neurons 1 and 2 were summed and that of neuron 3 subtracted to obtain an ensemble signal (Figure 5A). The ensemble signal dictated the frequency of optogenetic stimulation pulses according to the same transfer function as in Figure 1D, and above-threshold crossings triggered reward. The threshold level was chosen so that only simultaneous activations of neurons 1 and 2, and near-baseline activity of neuron 3, led to reward. Any activation, no matter how strong, of either neuron 1 or 2 alone was not sufficient for threshold crossing, and any above-baseline activity of neuron 3 prevented it (Figure 5B). We found that ChR2 mice can also learn this more complex mapping, as indicated by within-session linear increases in TCRs (Figure 5C).

Figure 5.

Multi-neuron Conditioning

(A) The Δf/f₀ activities N₁, N₂, and N₃ of three CNs were first transformed by a logistic function (see STAR Methods) into f(N₁), f(N₂), and f(N₃), and the ensemble activity computed by adding f(N₁) and f(N₂) and subtracting f(N₃). Each activity transform dictated the angular position of a robotic actuator and the ensemble activity was proportional to the effector distance to target.

(B) Example Δf/f₀ traces of the three simultaneously conditioned neurons, their ensemble activity computed as in (A), and its real-time transform into a rate signal of optical pulses encoding the distance to target.

(C) Mean and individual session (n = 8, two mice) target hit rates indicate that mice could learn the multi-neuron conditioning task under the guidance of optogenetic feedback.

As imposed by the mapping rule, neurons 1 and 2 were co-activated at threshold crossings of the ensemble activity (Figure S7B). The probability of these co-activations might fortuitously increase during a session as the rate of Ca events increases in the two neurons, or even in only one of them. Both neurons indeed increased their overall activity throughout a session (Figure S7A). But to test whether mice also generated more co-activation per se, we analyzed if the number of co-activations per produced Ca event changes during learning. We found that this ratio significantly increased between session start and end for both neuron 1 (p = 0.003, Kruskal-Wallis test) and neuron 2 (p = 0.03), demonstrating that more frequent co-activation did not occur fortuitously. A comparison of each neuron’s activity aligned on any Ca event of the other, between the start and end of a session, confirms that they became more co-active during learning (Figure S7B).

To illustrate a practical neuroprosthetic application of this mapping, we also translated the imposed rules into mechanical constraints governing the multi-joint displacement of a robotic arm (Figure 5A; Movie S1). The activity of each neuron was transformed into a joint angle, rendering the ensemble activity proportional to the distance (in joint angle coordinates) from the arm’s endpoint to a target area. The optogenetic stimulation thus provided feedback of the arm’s position relative to the target. The increased number of threshold crossings was therefore akin to more frequent target hits (Figure 5C). Although unbeknownst to the mouse, increases in the rate of successful displacements toward the target illustrate how multiple L2/3 neurons can be conditioned to execute goal-directed movements of a prosthetic device under the guidance of artificial sensory feedback.

Discussion

Practical Relevance of Artificial Cortical Communication

This study reveals that artificial sensory feedback about the current activity state of one or multiple volitionally modulated neurons can guide rapid learning during operant conditioning. We effectively fabricated an optical version of an artificial cortical communication channel (Jackson et al., 2006), which was used by mice to rapidly (i.e., in less than an hour) learn to produce reinforced neural activity patterns in a goal-directed manner, similarly to using natural sensorimotor associations. Schematic depictions of such a channel are readily included in idealized views of neuroprosthetic control where artificial sensory information from the prosthetic device is fed back directly to the brain (Bensmaia and Miller, 2014, Kwok, 2013). Our experiments provide a seminal proof of concept of such a system using an adaptation-based approach (Bensmaia and Miller, 2014) and illustrate that it could in principle be used to modulate multiple neurons in concert for executing goal-directed multi-joint prosthetic movements. A more biomimetic implementation (Bensmaia and Miller, 2014) might also be feasible. It would necessitate that multiple instances of the artificial channel be learned in parallel, each activating and providing “proprioceptive” feedback of a single joint.

Graded stimulation of the cortex has previously only been used to guide natural forelimb reaches (Dadarlat et al., 2015), albeit under the assistance of proprioception and a lengthy (i.e., several months) learning procedure involving pairings with visual cues. In contrast, we demonstrate that both sensory and motor peripheries can be bypassed and that unassisted, arbitrary associations can be fashioned between activities of two different sets of cortical neurons (not necessarily invoking direct anatomical pathways).

What Does the Neural Specificity of Learning-Related Activity Imply?

A key component in our experiments was the wide-field imaging of a large number of cortical neurons and their longitudinal tracking during conditioning. It enabled us to peek into cortical mechanisms underlying volitional neural control and how the brain drives its own activity. Conceptually, the reinforced activity of the conditioned neuron (i.e., the controlled variable) is driven by a “volitional controller” (Fetz, 2007). Our results suggest that this controller is only sparsely represented in local neighboring neurons. This might seem surprising given that cortical networks are locally highly interconnected (Harris and Mrsic-Flogel, 2013). It is, however, consistent with the general sparseness of L2/3 neuronal firing (Petersen and Crochet, 2013) where population activity is thus not expected to reflect a common motor function and might be less correlated compared to the output neurons of the deeper layers of cortex. Previous studies actually report a similar confinement of learning-related activity to conditioned cortical neurons (Arduin et al., 2013, Clancy et al., 2014). The original single-neuron conditioning experiments of Fetz and Baker (Fetz and Baker, 1973) targeted deeper cortical layers (probably layer 5) using microelectrodes to simultaneously record the conditioned and an adjacent neighboring neuron. Their findings revealed that in about half of the recorded pairs, the responses were correlated during threshold crossings. We can therefore hypothesize that spatial specificity is more prominent in L2/3 and might be related to the necessary flexibility of this cortical layer for the purposes of learning new skills and adaptive mechanisms in general (Huber et al., 2012, Rokni et al., 2007). It follows that the “volitional controller” driving L2/3 neurons most likely involves other, more distributed or distant cortical activations or subcortical pathways (Koralek et al., 2012).

Our results also demonstrate that when reward is contingent on the volitional activation of a single neuron, neural mechanisms can home in on that cell and ignore its immediate cortical neighbors. In a way, this mirrors a concept put forward by in vivo single-cell stimulation experiments (Brecht et al., 2004, Houweling and Brecht, 2008). The latter suggests the existence of circuits downstream of the stimulated cortical neuron capable of reading out single-cell activity. Our finding, on the other hand, suggests that activity in circuits upstream of the conditioned neuron can be routed to a single cortical cell. Both emphasize the behavioral relevance of single-neuron activity, which might be more significant than previously thought.

Parallels with Previous Findings

The closest direct precedent to our study is the work of Clancy and colleagues (Clancy et al., 2014). They also used wide-field imaging to condition cortical neurons and were thus also able to unambiguously track activity of a local subpopulation of individual neurons during learning. Instead of conditioning a single neuron, they conditioned ensembles of up to eleven neurons. The auditory feedback and reward delivery were dependent on the summed activity of the ensemble. Additionally, they calibrated the feedback transform coefficients and reward thresholds daily based on baseline activity levels, whereas we kept them the same throughout the experiment for each conditioned neuron. Their approach was hence more immune to day-to-day fluctuations in the activity of single neurons and probably explains why apart from observing within-session learning as we did, their mice also learned across days. Neurons within their conditioned ensembles developed coordinated, synchronous activity during learning. Similar coordination between non-conditioned neurons, those outside the ensembles, was not present and did not increase over time. Such specificity of learning-related changes to conditioned neurons is confirmed in the present study, but our findings go further by demonstrating that this remains true even when conditioning is restricted to just a single neuron. Clancy and colleagues also identified that some non-conditioned cells were activated around threshold crossings, which was more evident for those nearby than those distant from the conditioned ensemble. Although a direct comparison is not possible because they do not report precise temporal relationships, these nearby cells might be similar to our spatially clustered “leading neurons,” which represent a putative pre-synaptic drive of operant activity in the local network.

Neural Plasticity Might Underlie Learning and Does Not Involve Direct Inputs from the Stimulated S1 Site

The finding that increased amplitudes of the Ca events (reflecting stronger bursts) guided learning and persisted despite a performance drop after feedback removal led us to speculate that learning might in part be the result of neuronal plasticity, such as synaptic potentiation or more synchronous inputs to the conditioned neuron. Future experiments could thus be aimed at identifying potentiated synapses and subsequently tracing the source of the related pre-synaptic input (Wickersham et al., 2007). Since the “volitional controller,” again conceptually, uses the feedback signal to modify the controlled variable (Fetz, 2007), the fact that we could artificially constrict the feedback to a set of cortical neurons might further facilitate its identification. A potential site of plasticity could be the CN’s synapses receiving direct input from S1. In the mouse vibrissal system, S1 projections directly target L2/3 neurons in M1 (Mao et al., 2011), and analogous inter-areal connections most likely exist between motor and sensory forelimb representations. The sequential activation of the CN and S1 afferents might have, by design, led to a Hebbian-like spike timing-dependent plasticity (Markram et al., 1997) and have been a contributing factor to producing more post-synaptic spikes over time. This could therefore constitute a potential mechanism of how optogenetic feedback expedites learning. If that were the case, it is expected that S1 stimulation would evoke higher Ca levels in the CN after, as compared to before, learning. However, we were not able to detect any significant fluorescence transients in the CN upon optogenetic activation of S1, neither before nor after learning (Figure S3C). Learning-related activity increases in the CN therefore do not seem to be mediated by direct S1-to-M1 projections, reinforcing the idea that more intricate pathways are involved. Actually, because S1 stimulation does not lead, but follows, the CN’s activation by a few milliseconds, it is more likely that the synapses in question were depressed and not potentiated (Jacob et al., 2007). Indeed, stimulation of a cortical site proportional to real-time activity recorded at another site induces neural plasticity that is consistent with synaptic potentiation of pathways from the recording to the stimulation site, and not vice versa (Jackson et al., 2006). Further work is therefore required to unequivocally identify the recruited pathways and the source of the input driving the conditioned neurons in M1.

Outlook

Our experiments demonstrate that a fabricated cortical communication channel can rapidly gain behavioral relevance. In practical terms, this means that attaching an “artificial position sense” to neuroprosthetic actuators is feasible and can in principle be rapidly learned (Figure 5A; Movie S1). These findings in rodents might also be applicable to humans and pave the way toward developing artificial feedback systems for patients, which could be implemented even with currently available electrode-based approaches. Besides neuroprosthetic applications, our study also provides a novel behavioral tool for in vivo circuit dissection: an “in cerebro” learning paradigm that bypasses sensory and motor peripheries by imposing the learned action and its feedback directly in the cortex. This approach can potentially substitute for natural behavior and thereby facilitate dissecting neural circuits underlying sensorimotor learning. For instance, trans-synaptic tracing (Beier et al., 2011, Wickersham et al., 2007) can identify the presynaptic drivers of the conditioned neuron and the postsynaptic targets of the stimulated neurons, thus revealing how the feedback signal is routed to instruct volitional control and what ultimately drives the output. In addition, structural imaging and functional manipulations of the spines of conditioned neurons or the boutons of stimulated cells can provide insights into the synaptic plasticity mechanisms at play. Finally, the accuracy of circuit dissection can be improved by producing a more specific feedback signal aimed at individual neurons with two-photon targeted activation and simultaneous imaging (Packer et al., 2015, Rickgauer et al., 2014). The latter will also allow exploring multiple channels of graded feedback.

STAR★Methods

Key Resources Table

REAGENT or RESOURCE	SOURCE	IDENTIFIER
Antibodies
Anti-CHR2	MFD Diagnostics	clone 15E2
Biotinylated Goat Anti-mouse IgG	Vector Labs	Cat# BA-9200; RRID: AB_2336171
Bacterial and Virus Strains
AAV2.1-Syn-GCaMP6f.WPRE.SV40	University of Pennsylvania	Cat# CS0201
AAV2.1-hSyn-Cre.WPRE.hGH	University of Pennsylvania	Cat# CS0342
Chemicals, Peptides, and Recombinant Proteins
Sigma Fast DAB tablet	Sigma-Aldrich	Cat# D4293-50SET
Experimental Models: Organisms/Strains
Mouse: Ai32: Rosa-CAG-LSL-ChR2(H134R)-EYFP-WPRE	Jackson Laboratory	RRID: IMSR_JAX:012569
Mouse: Wild type: C57BL/6NCrl	Charles River	Cat# 027

Contact for Reagent and Resource Sharing

Further information and requests for resources and reagents should be directed to and will be fulfilled by the Lead Contact, Daniel Huber (daniel.huber@unige.ch).

Experimental Model and Subject Details

We used 8 to 12 weeks old Ai32 transgenic homozygote male mice carrying a floxed ChR2(H134R)-EYFP fusion gene inserted in the Gt(ROSA)26Sor locus in a C57/Bl6 background mouse strain (Madisen et al., 2012) (Jackson Labs). The animals were housed in an animal facility, maintained on a 12:12 light/dark cycle and were placed under a water restriction regime (1 ml/day) (median weight: 22.5 g, range: 19.2 g to 24.1 g). The experiments were performed during the light phase of the cycle. The animals did not undergo any previous surgery, drug administration or experiments and were housed in groups of maximum 5 animals per cage.

All procedures were approved by the Institutional Animal Care and Use Committee of the University of Geneva and Geneva veterinary offices.

Method Details

Surgeries and viral injections

All surgeries were conducted under isoflurane anesthesia (1.5%–2%) and additional analgesic (0.1 mg/kg buprenorphine intramuscular (i.m.)), local anesthetic (50-100 μL 1% lidocaine subcutaneous (s.c.) under the scalp) and anti-inflammatory drugs (2.5 mg/kg dexamethasone i.m. and 5 mg/kg carprofen s.c.) were administered as necessary. A custom made titanium head bar was implanted in the skull to allow for subsequent head fixation. A craniotomy was performed over the left frontal cortex and two viral injections (30-50 nL each) were delivered (10-20 nl/min) into the forepaw representation in S1 (0.75 mm anterior, 2.25 mm or 1.75 mm lateral to Bregma) and two in the forepaw M1 (1.75 mm anterior and either 2 mm or 1.5 lateral). In 8 mice, the S1 injections consisted of diluted “Cre-virus” (AAV2.1-Syn-Cre, 1:100 for 2 mice and 1:1000 for 6 mice, 0.2% FastGreen in sterile saline, virus stock titer 1.46 × 10¹³ genome copy per ml (GC/ml)). In M1, a “GCamP6f-virus” (AAV2.1-Syn-GCaMP6f, 1:10, 2.96 × 10¹³ GC/ml) was injected. The coordinates for the somatosensory representation of the forepaw were based on a series of intrinsic signal imaging sessions (7 additional Ai32 mice). After virus injection, cortex was rinsed for 1-2 min with dexamethasone (0.03%). Two hand-cut glass coverslips (150 μm thick) that matched the shape of the craniotomy were glued together with optical adhesive (Norland 61). The cranial window was placed on top of the cortex, glued to the bone with cyanoacrylic glue and secured with dental cement. The correspondence between the S1 injection sites and the sensory forepaw representation was confirmed post-operatively with intrinsic signal imaging in each mouse. The M1 injection coordinates were calculated relative to the S1 coordinates and based on a database of cortico-cortical projections in somatic sensorimotor areas (Zingg et al., 2014). In addition, all mice were implanted with cortical surface electrodes (Teflon coated gold wires, AU-3T, Science Products, Germany) in the contralateral somatosensory (0.75 mm anterior, 2 mm lateral to Bregma) and visual cortices (3 mm posterior, 1 mm lateral to Bregma). After a four day recovery period, mice were placed under a water restriction regime (1 ml/day). Experiments began on the 15th day after surgery. Neural responses could be evoked by optogenetic stimulation of S1 in all 8 mice (Figures S1A and S1B). Histological verification of ChR2 expression revealed dense labeling of cell bodies and neuropil in L2/3 and L5 at the S1 site, whereas no neurons were retrogradely labeled in M1; only axonal projections of Chr2 neurons from S1 could be observed (Figure S8A).

11 additional male mice that underwent identical surgery and injection procedures were used as controls as they did not exhibit ECoG responses to the optogenetic stimulus (Figures S1A and S1B) and showed no or only sparse expression of ChR2:

• 4animals were wild-type siblings not carrying the floxed ChR2-EYFP gene.
• 2Ai32 mice were heterozygote siblings
• 4Ai32 homozygote mice received a highly diluted Cre-virus injection in S1 (1:10’000).
• 2Ai32 homozygote mice received a diluted Cre-virus injection in S1 (1:1000), but showed only very sparse expression of ChR2 (Figure S8B). We explain the low ChR2 expression levels and lack of evoked responses in these mice by the failed virus delivery during surgery.

Histology

At the end of the experiments, mice were deeply anesthetized with pentobarbital, transcardially perfused with cold saline and paraformaldehyde 4% (PFA) and the brains were removed and stored overnight in PFA 4%. Then, the brains were transferred to a sucrose solution (20% in PBS 0.1M) for at least 24 hr and sliced in 50 μm coronal sections with a freezing microtome. Tissue was stored in a solution of sodium azide (0.01% in PBS 0.1M) until histological processing. We performed immunohistochemical analysis to determine the level of ChR2 expression. Slices were incubated with a mouse primary monoclonal antibody anti-ChR2 (clone 15E2, MFD Diagnostics) and with a biotinylated secondary antibody anti-mouse (Vector Labs). The tissue was finally processed with diaminobenzidine (Sigma-Aldrich) for the colorimetric reaction. Slices were mounted and imaged with a wide field scanner microscope (Olympus VS120) at 10x.

Intrinsic signal imaging

Mice were head-fixed and placed on a heated platform under light isoflurane anesthesia (0.75%). Ten 1 s vibrotactile stimuli consisting of a 100 Hz sinusoidal vibration were delivered to their forepaw with a 10 s inter stimulus interval. The cranial window was illuminated by a collimated red light LED (630nm) and imaged at 10 fps with a 256 by 332 pixels resolution. The average difference image between the stimulation period and a 1.5 s baseline period was processed by a 50 by 50 pixels spatial averaging filter and subsequently smoothed by a 5 by 5 pixels Gaussian low pass filter with 0.5 pixels standard deviation. Stimulus presentation and image acquisition were controlled by Ephus software (scanimage.org).

2-photon imaging and optogenetic stimulation

Imaging was performed with a custom built two-photon microscope (MIMMS; https://openwiki.janelia.org/wiki/display/public/Home) controlled by Scanimage 4.2 (scanimage.org) using a 16x 0.8 NA objective (Nikon) and with excitation wavelength at 940 nm (Ultra II, tunable Ti:Sapphire laser, Coherent). 512 by 512 pixel images covering 626 by 665 μm of cortex were acquired at 29.57 Hz using bidirectional scanning with a resonant scanner system (Thorlabs). The power was modulated with pockels cell (350-80-LA-02, Conoptics) and calibrated with a photodiode (Thorlabs). Optogenetic stimuli were generated with a 473nm laser (DHOM-W473-200mW, Ultralasers) which was custom modified to be gated electronically using TTL pulses. Pulse length was 5ms. The power was calibrated before each session to be ∼20mW at the stimulation site on the brain surface. The beam was positioned with a pair of galvanometric mirrors (Cambridge Technology). The imaging and optogenetic beams were combined using a longpass dichroic mirror (700dcxxr, Chroma) between the scan lenses and the tube lens. The primary mirror for imaging was a custom polychroic (Chroma, zt470/561/nir-trans) transmitting the infrared and blue light, while reflecting the green. Before detection, the remaining IR light was filtered with a colored glass band pass filter (BG39, Chroma), whereas the remaining blue light was removed with a short pass filter (CG475, Chroma). Images were continuously acquired using gated photo multiplier tubes (gPMT) (H11706P-40 SEL, Hamamatsu) and written in 16 bit format to disk in separate files (1110 frames/file). The start/end of file trigger was issued by a separate PC and later used for temporal alignment of Ca traces with behavioral variables. The objective was aligned perpendicular to the imaging window for each mouse using a custom built laser-based alignment device. At the start of each session, the imaging field was manually located using reference images of previous recording days and slow drifts manually corrected for throughout the session.

Image analysis

To correct for lateral movements, a custom MATLAB registration algorithm was used to align each image to a template taken as the average image of a 30 s resting baseline period recorded at the start of each session. The cross-correlation was computed between each image and the template by multiplying the two-dimensional discrete Fourier transform of one with the complex conjugate of the Fourier transform of the other and taking the inverse Fourier transform of the product. The row and column location of the peak cross-correlation value was taken as the vertical and horizontal shift, respectively. 10% of each image was cropped at the boundaries for the purposes of this computation.

Semi-automatic custom MATLAB routines were used to draw regions of interests (ROIs) for individual neurons using the session average image. The drawn ROIs were then automatically aligned and iteratively updated over subsequent sessions. The Ca-dependent fluorescence time series were extracted by averaging pixels within each ROI. The time-varying baseline f₀ of a fluorescence trace was computed as the average value in a 20 s moving time window by excluding values that surpass 20% of this average. At each time point, the updated baseline value f₀ was subtracted from the raw fluorescence value f to yield Δf and compute the relative change to baseline as Δf/f₀.

Technical details and task design

Images of scanned brain tissue acquired with ScanImage were streamed on the local network using the user datagram protocol (UDP). Each end-of-frame signal triggered a custom coded MATLAB routine that sent, in succession, 32 equal-sized lines of the 512 by 512 pixel image (i.e., 16 packets/image) over the network. The packets were read on the appropriate UDP port on a separate PC, the images reassembled and the relative change with respect to baseline of the conditioned neuron(s) computed by extracting the average pixel intensity in a defined region of interest comprising a single cell. The extracted activity was downsampled to ≈0.2 s by averaging blocks of 6 consecutively received images and changes relative to a preceding 20 s moving baseline window were computed (Δf/f₀). Δf/f₀ values exceeding 20% were excluded when updating the baseline value. A second custom written MATLAB instance running on the same PC transformed, at every time sample t, Δf/f₀ into a frequency value f(t) according to $f\left(t\right)=A{e}^{n\left(t\right)}-A$ where n(t) is the Δf/f₀ value at time t. The obtained frequency value was then binned into one of 17 bins ranging from 1 Hz to 15.12 Hz in quarter-octave increments and produced a rate signal for the optogenetic pulses. The pulse rate was updated whenever the binned f(t) differed from the binned f(t-1). Linear changes in neural activity therefore resulted in exponential changes in the feedback signal (Figure 1D, inset). A third custom written MATLAB instance used the rate code to generate an analog output signal on a Data Acquisition Board (National Instruments, Austin, TX) that produced the optogenetic feedback to the mouse. The optical stimulus consisted of 5 ms squared pulses sampled at 44 kHz. The same instance also triggered a 0.35 s auditory beep and a reward pulse that opened a valve to deliver a drop of water to the mouse whenever the signal crossed a threshold level. The water drop was sucked away with a peristatic pump (Minipulse 2, Gilson, Middleton, WI) right upon delivery, thereby requiring immediate consumption and instigating anticipatory licking. Rewards were always delivered automatically upon threshold crossings except in the lick-triggered reward condition of the last 5 sessions (see Behavioral Training section below). The threshold level was initially chosen for each neuron on a trial-and-error basis to yield a minimum of approximately two rewards per minute, based on a baseline recording period, and was kept fixed thereafter. The threshold corresponded to the lowest Δf/f₀ that was binned in the maximum 15.12 Hz bin and was set by adjusting the coefficient A. Only threshold crossings triggered a reward and the 15 Hz stimulation was turned off if a sustained activity above threshold continued in excess of 1 s. The optogenetic feedback was reinstated once such sustained activity came back below threshold. This was done to pair the maximum stimulation frequency (15 Hz) with activity threshold crossings only, as this is what we sought to reinforce. By turning the stimulus off we avoided optogenetically reinforcing high levels of activity sustained above threshold that did not lead to reward. This had to be done in less than 1% of all threshold crossings.

The total median time delay between frame completion and feedback generation was uniformly distributed between 19.2 ms and 219.2 ms (see table).

Median (5% and 95% quantiles) time delays incurred by the different processing stages

Frame transmission	11.2 (9.2, 13.0) ms
Signal extraction	7.1 (6.8, 10.1) ms
Rate code computation	0.84 (0.8, 5.4) ms
Feedback generation	0.052 (0.05, 3.2) ms

The choice of the conditioned neurons was biased toward neurons in the middle of the imaged field of view. Other selection criteria were a clear identifiable morphology and a minimal Ca event rate of 2 events / minute (assessed during initial baseline measurements lasting 2 to 3 min). Once a neuron was chosen, it was used for conditioning on all subsequent sessions as long as it remained active (produced more than 2 Ca events / minute). If the criterion was not met, conditioning was switched to a new neuron meeting the criterion before the start of the session. The functional role (relationship to movements) of the chosen neurons was not assessed and therefore remained unknown.

Because image movement correction was not performed for the real-time data, we quantified the artifactual transitions in the conditioned Δf/f₀ traces that were induced by lateral motions (Figures S2D–S2G). They were found to be negligible and did not contribute to threshold crossings.

The optogenetic feedback was guided by mirrors through the imaging objective and visually aligned on the ChR2 expression site in forepaw S1 using a CCD camera and a reference image of the blood vessel map. The squared pulses were used to electrically switch the laser on and off creating as such a fast and silent shutter for the blue light stimulus. The same pulse signal elongated by 1 ms turned off the gPMT used for imaging during photostimulation periods. This resulted in at most one band of ≈92 lines of the 512 by 512 pixel images to be blacked out over a two frame period for the maximal 15 Hz stimulation level. Eventual blanking of the CN was detected online and the frame in question was ignored in subsequent processing stages. As a visual mask, a collimated blue light LED (473 nm, Roithner) was turned on and the gPMT turned off during flyback periods of scanning (1.5 ms pulses at the 29.57 Hz acquisition rate), thus not affecting the acquired images while providing a perceptually stable visual stimulus. This pulsated blue light illuminated the otherwise dark setup from session start to session end. The feedback signal was delivered continuously without any trial structure but was paused for 1 s at 37.5 s intervals to allow for logging acquired data to separate files. This end/start of file trigger was also used to periodically update the saved image files on the Scanimage PC and later used for aligning extracted Ca traces with behavioral variables.

Behavioral procedures

Mice sat head-fixed in a tube (25 mm inner diameter). The forepaw contralateral to the imaged and stimulated cortical sites was resting on a hold bar to maintain balance while the ipsilateral forepaw was restrained inside the tube. The water reward was delivered through a spout placed at licking distance below the snout. The tube and the mouse were connected to a 5 V node and, when touched, the conductive hold bar and reward spout shunted the 5 V to a circuit that pulled a respective analog input signal to the Data Acquisition Board to high. Licks and forepaw rests were in this manner continuously acquired at a sampling rate of 1 kHz. Releases of the hold bar were used to assess if mice consistently used contralateral forepaw movements to solve the task (Figure S2H).

We used 4 control mice to test the efficacy of a blue visual mask in suppressing visually evoked ECoG responses by the optogenetic stimulus. With the mask turned off, visually evoked potentials were observed in response to the onset and offset of the blue laser pulse train, but were completely suppressed when the mask was turned on (Figures S1C and S1D).

Mice were trained in the operant conditioning task for 15 to 17 consecutive daily sessions. The experimental timeline of each session was divided into condition blocks (Figure S2A). Each conditioning session began with a three minute period of playback where a neural activity recorded on a previous day controlled the optogenetic feedback and reward delivery, in otherwise identical experimental settings. The Δf/f₀ trace of the conditioned neuron saved in a previous session was used for this purpose. Instead of the Δf/f₀ values extracted from the streamed images at each time sample in real-time, the optogenetic feedback and threshold crossings were determined by the Δf/f₀ values of that saved trace. Control was then switched to the conditioned neuron and after an average of 25 min of real-time conditioning, the optogenetic feedback was removed for 3 min and reinstated thereafter. The blue light mask was present throughout.

In the last 5 experimental sessions, lick-triggered rewards were introduced during the first 3 min of playback and 3 min before, during and after optogenetic feedback removal. Instead of being automatically delivered, mice had to initiate licking in a 500 ms time window following threshold crossings to obtain the reward. Because rewards were otherwise always automatically delivered, these brief introductions of the lick-triggered reward condition were not significant enough to incentivize mice to engage in continuous licking as is sometimes observed in such lick-triggered reward paradigms. Additionally, each session ended with 6 min of playback under the lick-triggered reward condition. In the first 3 min, optogenetic stimulation was applied to a control non-ChR2 site and moved back to the S1 ChR2 site in the last 3 min of playback. The positions corresponding to the ChR2 and control sites were located on the blood vessel map using a CCD camera and their image coordinates recorded. The optogenetic beam was then positioned with a pair of galvanometric mirrors on those same image coordinates (using a reflective surface) and the corresponding voltages applied to the two mirrors were recorded for each site. The beam was then moved between ChR2 and control sites, during the experiment, by applying the appropriate voltage levels at relevant times. All transitions between different experimental conditions were not cued by experiment interruptions or additional sensory stimuli.

In the three neuron conditioning experiments, each neuron’s activity n was transformed by a logistic function according to

$$f\left(n\right)=\frac{1}{2}\left[1+\tanh \left(A\left(n-c\right)\right)\right].$$

The ensemble activity $f\left(n_1\right)+f\left(n_2\right)-f\left(n_3\right)$ controlled the rate of optogenetic feedback pulses and reward delivery. A and $c$ were set to 2.2 and 0.6, respectively and the reward threshold to 1.75. This mapping constrained neurons N₁ and N₂ to co-activate and N₃ to remain silent to bring the ensemble activity above threshold and trigger a reward. More specifically, Δf/f₀ values of N₃ above ≈0.36 prevented threshold crossings irrespective of N₁ and N₂ activity. Concomitant Δf/f₀ values of N₁ and N₂ above ≈1.15 resulted in threshold crossings provided the activity of N₃ remained at zero.

Quantification and Statistical Analysis

No statistical methods were used to predetermine sample size and all trained animals were included in the analysis. Two-tailed Student’s t test (for samples with equal variances) or Welch’s t test (for samples with unequal variances) were used where appropriate. Equal variances were determined with the two-sample F-test and the normality assumption was tested with the Kolmogorov-Smirnov test. Non-parametric tests were used when the normality assumption was not met. All data analyses were performed with custom written routines in MATLAB.

The efficacy of ChR2 activation (Figures S1A and S1B) was assessed by measuring differentially between the two contralateral cortical electrode responses evoked by a 0.5 s train of 5 ms optical pulses delivered at 15 Hz. Peak responses in a 20 ms time window following the onset of each 5 ms light pulse were compared across all stimulation trials (n = 30) to peak responses to the same number of simulated light pulses in the 0.5 s baseline period preceding the stimulus train onset. Animals with significant differences (paired t test, p < 0.05) between true and simulated mean peak values were classified as ChR2 mice and as control mice otherwise. No method of randomization was used and the investigator was not blind with respect to the classification of 5 control animals, but was blind for the other 6 and for all ChR2 animals.

Learning curves were calculated for the conditioning period between the end of the initial playback and either the onset of feedback removal or onset of the lick-triggered reward condition that precedes feedback removal. Threshold crossing rate (TCR) was calculated for each of 8 equally sized time bins in this window. Learning sessions were defined as those with significant linear increases of TCR across the 8 bins (linear regression, p < 0.05). Mice with significantly bigger TCR in the last compared to the first bin across all sessions were labeled as learners and as non-learners otherwise. TCR normalization was performed by offsetting individual session learning curves to the same TCR value in the first bin, taken as the mean of all first bin TCR values, and dividing the rate of each bin by it.

Chance levels of detection probability of threshold crossings assessed in the lick-triggered reward condition (Figure S4) were simulated separately for each mouse and each experimental condition as follows: 1. We identified the experimental epoch corresponding to each individual data point included in the mean value calculations of Figure 2E. 2. Within that epoch, we isolated licks that occur only during baseline stimulation (i.e., 1 Hz) and attributed them to spontaneous licking. 3. We then simulated spontaneous licking for the whole epoch by randomly reproducing the isolated lick sequences, thus preserving their temporal statistics (i.e., the inter-lick-interval and lick duration). 4. Chance detection probability was then calculated using these simulated licks and the same neural activity as for the actual data. As such, this procedure yielded for each individual data point a corresponding chance level and allowed paired statistical comparisons.

Ca events were detected by taking the first derivative of the smoothed Δf/f₀ trace (Savitzky-Golay filter, second order polynomial, 15 data points) and an event onset was defined when the z-scored trace crossed a value of 2 and event end when it decreased again below 0. Event amplitude was defined as the difference of the Δf/f₀ values between event end and event onset time points. Event rate and average event amplitude were calculated for the same 8 time bins as the learning curves. A general linear model with the threshold crossing rate (TCR) z score as the dependent and the z scores of the event rate (ER) and event amplitude (EA) as the independent variables was fit to each of the 33 learning sessions as follows:

$$TCR=b_{Er}ER+b_{Ea}EA.$$

The relative contribution of ER and EA changes to TCR increases was evaluated by comparing the distributions of the fitted $b_{Er}$ and $b_{Ea}$ values expressed in terms of number of standard deviations across all learning sessions. Event rate and event amplitude change factors correspond to the ratios between the last and first time bins of the two measures, respectively. Temporal Ca event probability (Figure S3A) was computed by transforming each Δf/f₀ trace into an event trace which was set to 1 between each event onset and end, and to 0 otherwise. The reward-triggered average of all event traces of a given neuron yielded the event probability.

Increasing neurons were defined, in a given learning session, as those having significant linear increases (p < 0.05, linear regression) of either their EAs or ERs between the start and end of learning, evaluated with the 8 time-binned values.

Leading neurons were identified, in a given learning session, by computing the hypergeometric cumulative distribution function value for each non-CN according to

$$p=1-\sum _{i=0}^x\left(\begin{array}{c}K\\ i\end{array}\right)\left(\begin{array}{c}M-K\\ N-i\end{array}\right)/\left(\begin{array}{c}M\\ N\end{array}\right),$$

where x is the number of instances in which the non-CN produced a Ca event in a 1 s interval preceding the onset of the CN’s threshold crossing Ca event. K is the product of the number of time samples in the 1 s interval and the number of threshold crossings in a given session (number of samples drawn), M is the product of the number of time samples in a 4 s interval preceding the CN’s event onset and the number of threshold crossings (size of the population) and N is the number of instances in which non-CN events occurred in the 4 s interval (number of items with the desired characteristic in the population). Non-CNs with p < 0.01 were defined as leading neurons (i.e., those that produce Ca events that precede the CN’s threshold crossing Ca event more often than what would be expected by chance).

A receiver operating characteristic (ROC) analysis was carried out on the leading neurons to assess whether their activity can predict threshold crossings of the CN’s Ca trace. For each learning session, the CN’s Ca events were classified into rewarded (those that crossed threshold, event type X) and unrewarded (those that did not cross threshold, event type Y), and each leading neuron’s Δf/f₀ trace in a 1 s interval preceding the onset of these events was taken into account. A discrimination score (DV_X) was computed for each leading neuron for the i^th event type X as the dot-product of the neuron’s Δf/f₀ trace (X_i) and the mean trace across all type X events (excluding the i^th event, $\overline{X}$) minus the dot-product of X_i and the mean trace across all type Y events $\left(\overline{Y}\right)$. The discrimination score DV_Y was analogously obtained for the i^th event type Y and, thus, according to

$$D{V}_X=X_i\left({\overline{X}}_{\forall j\ne i}-\overline{Y}\right)$$

$$D{V}_Y=Y_i\left(\overline{X}-{\overline{Y}}_{\forall j\ne i}\right).$$

An ROC curve was constructed by plotting, for each criterion value c varied across the range of DV_X and DVy values, $p\left(D{V}_X>c\right)$ (i.e., the fraction of DV_X values exceeding the criterion) against $p\left(D{V}_Y>c\right)$ (i.e., the fraction of DV_Y values exceeding the criterion). The area under the ROC curve (AUC) was computed using trapezoidal numerical integration (trapz() function in MATLAB) and corresponds to the fraction of events correctly discriminated by an ideal observer using the obtained discrimination scores. A permutation test, consisting of shuffling the X and Y event labels and calculating AUC values with 1999 shuffled subsets, yielded a p value for each leading neuron, taken as the fraction of shuffled AUC values that were more extreme than the AUC corresponding to the non-shuffled data. Leading neurons with p < 0.05 were deemed to correctly predict, above chance level, weather a CN’s Ca event would cross threshold.

Bootstrap hypothesis testing of a test statistic being different from zero was performed by taking at random, with replacement, N values from the total set of N measurements of a variable, 1999 times. The two-tailed bootstrap p value was then computed from the 1999 sample measurements of a test statistic x as follows:

$$p=2min\left(\frac{1}{B}\sum _{j=1}^{B}I\left(x_j\le 0\right),\frac{1}{B}\sum _{j=1}^{B}I\left(x_j>0\right)\right),$$

where B = 1999, x_j is the j^th bootstrap sample of x and I() is the indicator function, which is equal to 1 when its argument is true and 0 otherwise.

Author Contributions

M.P. and D.H. designed the experiments and built the experimental setup. G.L.G. performed surgeries and viral injections. M.P. conducted experiments and analyzed data. All authors discussed the results and commented on the manuscript.

Published: February 22, 2017

Supplemental Information

Movie S1. Multi-neuron Conditioning

Related to Figure 5. The head-restrained mouse under the two-photon microscope was required to coactivate two neurons (green and blue) while keeping a third one inactive (red) to bring the ensemble activity (i.e., the rate of the optogenetic stimulus) above reward threshold. The calcium fluorescence of each neuron controlled the joint angle of a robotic arm and the arm’s distance to target was proportional to the feedback rate. The blue light mask pulses (1.5 ms) as well as the optogenetic stimulus pulses (5 ms) were undersampled by the mouse video acquisition rate (120 Hz). The undersampled light flashes apparent in the mouse video emanate from the blue light mask. In reality, the mask produced a perceptually stable illumination of the setup.