The need for calcium imaging in nonhuman primates: new motor neuroscience and brain-machine interfaces

Abstract

A central goal of neuroscience is to understand how populations of neurons coordinate and cooperate in order to give rise to perception, cognition, and action. Nonhuman primates (NHPs) are an attractive model with which to understand these mechanisms in humans, primarily due to the strong homology of their brains and the cognitively sophisticated behaviors they can be trained to perform. Using electrode recordings, the activity of one to a few hundred individual neurons may be measured electrically, which has enabled many scientific findings and the development of brain-machine interfaces. Despite these successes, electrophysiology samples sparsely from neural populations and provide little information about the genetic identity and spatial micro-organization of recorded neurons. These limitations have spurred the development of all-optical methods for neural circuit interrogation. Fluorescent calcium signals serve as a reporter of neuronal responses, and when combined with post-mortem optical clearing techniques such as CLARITY, provide dense recordings of neuronal populations, spatially organized and annotated with genetic and anatomical information. Here, we exhort that this methodology, which has been of tremendous utility in smaller animal models, can and should be designed and developed for use with NHPs. We review here several of the key opportunities and challenges for calcium-based optical imaging in NHPs. We focus on motor neuroscience and brain-machine interface design as representative domains of opportunity within the larger field of NHP neuroscience.

Introduction

Neuroscientists seek to understand the function and dysfunction of the nervous system, with an eye towards ultimately comprehending and supporting the health of the human brain. In order to understand how a system like the brain operates one must measure its internal workings, much like understanding a computer requires measuring voltages and currents throughout its circuitry. For decades, a dominant approach to measuring these signals has been extracellular electrophysiology. Using sharp electrodes inserted into the brain, neuroscientists can record the spiking activity of one or many individual neurons involved in perception, cognition, and action. Decades of discoveries and fundamental insights into brain function have resulted from studying the brain using these types of electrical measurement techniques.

Traditional electrophysiology captures the spiking activity of a sparse sample of neurons, allowing the responses of individual neurons to be measured and of neural populations to be collectively visualized and modeled. However, these models remain at a level of abstraction agnostic to microstructural details of neural circuits. These limitations of electrophysiology have motivated the development of an array of impressive technological advances that have enabled fluorescent labeling and all-optical recording and manipulation of targeted cell types in awake behaving animals. Whereas electrical measurements accurately capture neuronal spiking, optical methods can provide a complementary view of neural activity that is substantially richer in many ways, contextualizing patterns of neural activity within a genetically annotated, spatially localized, dense map linking circuit structure with function (Deisseroth and Schnitzer 2013; Peron, Chen, and Svoboda 2015; Emiliani et al. 2015). These tools have already transformed the study of neural circuits in small animal models, including worms, zebrafish, flies, and rodents; in this paper we highlight the rich experimental opportunities enabled by application of optical technologies to primate systems neuroscience.

Recent technological, viral injection, and surgical advances suggest we are on the cusp of widespread adoption of optical imaging to study the non-human primate brain. The first key advance was the surgical replacement ofs natural dura with translucent artificial dural windows that enabled long-term optical access to cortex in NHPs, enabling researchers to map cortical functional organization using intrinsic signal optical imaging (Grinvald et al. 1991; Chen et al. 2002). Voltage-sensitive dyes have also been successfully imaged through artificial dural windows. These signals provide improved spatial and temporal resolution and can be used to map subthreshold membrane dynamics at the submillimeter scale (Arieli, Grinvald, and Slovin 2002; Grinvald and Hildesheim 2004; Chen, Palmer, and Seidemann 2012). The development of new synthetic calcium indicator dyes have enabled detailed mapping of the functional organization of macaque primary visual cortex (Nauhaus et al. 2012; Ikezoe et al. 2013), although these dyes cannot be used for long-term recording and have been employed primarily in acute experimental contexts. For long-term experiments, the genetically encoded calcium indicator memTNXL has been successfully transduced via AAV1 in macaque primary visual cortex (Heider et al. 2010). Employing two-photon imaging through a small, chronically-implanted cranial window and a narrow micro-lens objective, calcium signals from several neurons were collected at single-cell resolution. More recently, Sadakane et al. (2015) developed an AAV-based TET-inducible system to amplify and regulate expression of GCaMP6f, enabling stable, repeated imaging of many neurons in marmoset neocortex over many months. This last study demonstrates that high signal-to-noise, long-term calcium recording of large neural populations is currently possible in NHPs.

Building upon these successes, we envision and outline an all-optical pipeline for capturing a dense map of neural activity, spatially-localized and combined with multiple types of molecular and structural information in behaving primates. We believe that this model will provide a substantially richer view of neural circuits, transforming state-of-the-art “black box” models of neural computation into models which may be directly, empirically tested and deeply grounded in the details of the neural circuit architecture within which this computation exists. [fig1] outlines experimental and analytical pipelines in which imaging is used to record the activity of many neurons in awake, behaving NHPs, translating similar approaches already successfully deployed in rodent, fly, zebrafish, and worm model systems (Deisseroth and Schnitzer 2013). An imaging chamber would be implanted in lieu of a standard electrophysiology chamber. Existing designs readily permit optical access to cortex for intrinsic optical imaging and optogenetic stimulation (e.g. Grinvald et al. 1991; Chen et al. 2002; Arieli, Grinvald, and Slovin 2002; Ruiz et al. 2013), but changes to the geometry of these chambers ([fig1:chamber]) are likely needed to accommodate the wide objective lenses necessitated by two photon imaging (Trautmann et al. 2015). In this imaging chamber, natural dura is replaced by a silicone artificial dura or a glass window, which provides stable, long-term optical access to cortex ([fig1:access]). Subsequently (or during the implantation procedure if time constraints allow), viral injections would be performed through this imaging chamber ([fig1:injections]), enabling the transfection of all neurons or targeted neural sub-populations with genetically encoded calcium sensors (Chen et al. 2013; Inoue et al. 2014) or voltage indicators (Jin et al. 2012; Hochbaum et al. 2014). The expression profile of these sensors could then be monitored over time using widefield epifluorescence imaging ([fig1:widefield]). Concomitantly, the monkey would be trained to perform a task designed to engage the neural circuit being studied, e.g. a reaching task for studying motor cortical computation ([fig1:task]). During this behavioral task, two-photon or (widefield epifluorescence) microscopy can be used to capture the optical activity readout of neural populations in the field of view ([fig1:2p]). Finally post-mortem histology can be performed using CLARITY ([fig1:clarity]), a clearing technique which preserves the 3d arrangement of cells and provides information about cell type and projection patterns via immunostaining (Chung et al. 2013; Rajasethupathy, Ferenczi, and Deisseroth 2016; Lerner, Ye, and Deisseroth 2016).

Fig. 1.

Experimental and analytical pipelines for employing calcium-imaging and CLARITY for primate systems neuroscience. (a) Schematic of cross-section with high-NA multiphoton-optimized objective lens. (b) Optical chamber implanted over motor cortex. Adapted from (Trautmann et al., 2015). (c) Viral constructs will be injected into cortex to deliver the calcium reporter gene and other fluorescent markers. (d) Widefield epifluorescence microscopy can be used to monitor expression and to record macro-scale functional activity. Adapted from (Sadakane et al., 2015). (e) The monkey will be trained to perform a behavioral task, during which (f) two-photon microscopy can be performed through the artificial dural window. Adapted from (Sadakane et al., 2015). (g) Post-mortem CLARITY can be performed on brain tissue and spinal cord to localize expression and provide more detailed information about the neurons and circuits recorded via immuno-staining. (h) Calcium imaging data can then be analyzed to extract the functional activity of each neuron as a function of time. (i) These traces can be visualized as neural state trajectories using dimensionality reduction techniques. Adapted from (Afshar et al., 2011). (j) Individual cells can be registered to fluorescent markers imaged using two-photon in vivo or collected post hoc via CLARITY. facilitating the annotation of the functional traces with cell type and projection information. (k) Spatial relationships between cells can also be used to embed models of the population activity within their 2D or 3D topography, e.g. by building a graphical model with functional (not synaptic) connectivity. (l) Combined functional, spatial, projection, and genetic information can be used to build models of the population dynamics in which cell types and circuit projections can serve distinct computational roles.

Two photon microscopy and CLARITY images produced in this experimental pipeline can next be sent through an analytical pipeline. Image processing algorithms can be used to identify, segment, and extract the functional activity from individual neurons in the field of view ([fig1:traces]). These activity traces can be collectively visualized as neural trajectories by employing a dimensionality reduction technique ([fig1:trajectories]), as would typically be performed with multielectrode electrophysiology data. However, we can also identify the cell type and information about projection targets or inputs of each imaged neuron by registering the functional data against CLARITY or static fluorescent markers co-injected and imaged in vivo ([fig1:traces_identified]). These data can then be used to build functional models which link the temporal dynamics of neural activity to the spatial organization and relative proximity of the cells ([fig1:spatial]). By utilizing functional data annotated with spatial, projection, and genetic information, neuroscientists can build models of neural computation which describe the dynamics of neural populations subserving behavior in which specific cell types and anatomical projections serve distinct, identifiable roles in shaping the population dynamics ([fig1:dynamics]).

[fig1:chamber] [fig1:access] [fig1:injections] [fig1:widefield] [fig1:task] [fig1:2p] [fig1:clarity] [fig1:traces] [fig1:trajectories] [fig1:traces_identified] [fig1:spatial] [fig1:dynamics]

In the rest of this paper, we enumerate a set of opportunities and challenges in harnessing optical methods like these in NHPs. We focus here on the use of calcium imaging in the study of the cortical motor system, both in the realm of basic science and the development of brain-machine interfaces, but certainly similar opportunities for optical imaging exist in other brain regions and across the spectrum of perception, cognition, and action (Belmonte et al. 2015). We proceed by exploring the limitations of electrophysiology, explore the avenues of scientific inquiry we feel would be especially well-served by optical approaches within the cortical motor system, and highlight some of the challenges to be tackled in developing the primate optical experimental approach we envision.

Opportunities for imaging in motor neuroscience

Limitations of electrophysiology and the development of optical methods

Pioneers in systems neuroscience have made remarkable advancements towards understanding the functional role of brain regions in perception, cognition, and behavior. These advances were enabled by many neurotechnologies for recording electrical and chemical activity within the brain. In particular, since the pioneering work of Hubel and Wiesel (1962) in visual cortex, extracellular electrode recordings of individual neuron’s spiking activity have enabled many seminal discoveries in domains such as the sensory and motor systems, decision-making, attention, object recognition, color processing, and many others. While single neural responses are informative, in the past decade a growing appreciation for the role of neuronal populations and of population dynamics has emerged. Studying neural populations generally necessitates the ability to record from many individual neurons simultaneously; consequently researchers have employed multiple electrodes and multi-electrode arrays to study the spiking activity of up to hundreds of individual neurons simultaneously. Using these recordings, the field has made considerable progress in relating multi-electrode spiking responses to behavior on a single trial basis. For example, in the primate motor system, a monkey’s reach reaction time can be predicted from the population-level dynamical state prior to movement (Afshar et al. 2011). Similar advances combining optical imaging with dynamical analysis have been used to dissect single trial neural population activity during decision-related processing in the leech (Briggman, Abarbanel, and Kristan 2005; Briggman and Kristan Jr 2008) and to relate sequential activation of neurons in the parietal cortex in rodents to specific choices during navigation decision tasks (Harvey, Coen, and Tank 2012).

These studies help us to unravel how the dynamics of a population of neurons drives behaviors, but it remains very difficult to connect dynamical system models to circuit knowledge at a lower level of abstraction. We believe this difficulty fundamentally stems from the information about neural populations not captured by extracellular electrophysiology. State of the art electrode arrays used in primates sample neurons sparsely, recording one or several neurons from a local population of many thousands (Robinson 1968). The sampling is also biased towards highly active cells (Barth and Poulet 2012) and provide very little information about cell identity. While some studies have used spiking waveform shape to distinguish excitatory and inhibitory neurons (Barthó et al. 2004; Kaufman et al. 2010; Kaufman, Churchland, and Shenoy 2013), it is difficult to validate the accuracy of this approach, and it is infeasible to resolve more finely the diverse excitatory and inhibitory neuron types present in cortex (Markram et al. 2004). While it is possible to record spiking activity across multiple cortical areas, it is not currently possible to dissect how axonal projections from one brain region shape the local dynamics in another by observing spiking activity alone. Consequently most efforts to understand circuit function can only regard the many anatomically-observed inputs as a latent unknown rather than a directly observable entity.

Collectively, these limitations of electrophysiology have highlighted the need to study neural circuit function and structure with a more powerful and informative technologies for neural circuit interrogation. Technological innovation has risen to meet this need, making it now possible to optically record activity from fluorescently labeled neurons and neural projections. Genetically encoded calcium indicators (e.g., GCaMP6) allow calcium concentration (related to action potentials) from thousands to tens of thousands of individual neurons to all be optically imaged simultaneously (Ahrens et al. 2012 e.g.). When combined with optogenetics (Yizhar et al. 2011) and optical clearing techniques such as CLARITY (Chung et al. 2013), these optical tools can provide understanding bridging the gap between neural circuit structure and function.

Extending imaging techniques to monkeys presents the possibility of fundamentally transforming our understanding of the primate brain. Imaging methods provide several key advantages over electrophysiology. First, imaging provides single-cell spatial resolution over a large field of view, which could significantly increase the total number of recorded neurons. It is likely that it will be possible to record from the majority of neurons located within the microscope’s focal volume, currently hundreds or thousands of neurons simultaneously. Third, whereas electrophysiology is typically biased towards larger, highly active neurons, optical imaging may more faithfully record activity in more quiescent cells (Dombeck et al. 2007), which comprise the majority of cortical circuitry (Barth and Poulet 2012). Fourth, imaging could potentially identify and record from the same neurons across multiple days (Tian et al. 2009; Huber, Gutnisky, and Peron 2012), which is currently extremely difficult to certify, if even possible, using electrodes (Tolias et al. 2007). These advances in turn enable entire new classes of experiments to address previously unaddressable questions in the primate. Fifth, optical imaging allows recorded neurons to be fluorescently labeled in other color channels with markers to distinguish cell types or identify projection targets. As imaging preserves the spatial onfiguration of recorded cells, we anticipate that it will eventually be possible to identify single neurons in calcium imaging data and later to register these to fine-resolution neuron-scale 3D reconstructions built using CLARITY (Chung et al. 2013; Tomer et al. 2015; Lerner, Ye, and Deisseroth 2016). Finally, it should be possible to exploit cell-type specific, virally transduced labels to selectively excite or inhibit cells using optogenetic stimulation while simultaneously optically recording activity (Grosenick, Marshel, and Deisseroth 2015; Rickgauer, Deisseroth, and Tank 2014).

Approaches integrating these tools have heretofore focused on simple organisms such as worms, flies, and rodents, which are all amenable to short development timescales and rapid iteration (Deisseroth and Schnitzer 2013; Peron, Chen, and Svoboda 2015). Despite the large array of tools and techniques available for rodents, there are two primary motivations we highlight here for applying optical tools using nonhuman primates (Anderson 2008). First, many neural circuits, including the motor system, are finely adapted to the specific requirements of each species. As such, it is unlikely that studying the motor system of simpler organisms like rodents will yield a mechanistic understanding of the principles underlying human motor control. Primates, however, are a close biomechanical analog of the human motor system. A monkey’s motor cortex must perform very similar computations as a human’s, and many aspects of anatomy and function are consequently well conserved. Second, primates exhibit a high degree of cognitive flexibility and are capable of learning a rich repertoire of sophisticated, precision behaviors. In the context of motor control, this enables us to map precisely the relationship between neural activity and precisely controlled movements. While it is possible to train rodents to reach and manipulate an object (e.g. Allred et al. 2008), it is likely that the relatively simpler rodent motor system would not support complex reaches (e.g. to multiple target locations, or avoiding obstacles in their path) or more dexterous manipulation tasks which require precise control of the digits.

We now enumerate a list of key opportunities opened to the field of primate neuroscience by adopting optical methods. We focus here on the domain of motor control with application to brain-machine interface design for the sake of concreteness, but we expect that imaging will present similar opportunities in other branches of systems and translational neuroscience.

Relating dynamics of neural populations to circuit organization

Understanding how cortical circuits prepare, control, and execute volitional arm movements has been a major research focus within neuroscience for several decades. In the past several years, the dynamical systems approach has emerged as a powerful candidate framework for understanding motor cortical function (e.g. Shenoy, Sahani, and Churchland 2013). The development of this conceptual framework was facilitated by sampling large heterogeneous populations of cells for a large number of trials using multielectrode arrays (Afshar et al. 2011) combined with methods for visualizing and modeling population responses (Yu et al. 2009; Shenoy, Sahani, and Churchland 2013; Cunningham and Yu 2014). This perspective treats motor cortical neurons as components of a recurrent neural network trained to generate specific patterns of activity appropriate to drive the spinal cord and muscles to create a desired movement (Churchland et al. 2006; Churchland et al. 2010; Churchland et al. 2012; Sussillo et al. 2015). By focusing on the dynamics of the neural population collectively, this framework has led to a number of advances in our understanding of the functional properties of motor cortex (e.g. Churchland, Afshar, and Shenoy 2006; Afshar et al. 2011; Ames, Ryu, and Shenoy 2014; Kaufman et al. 2014). In turn, these conceptual advances followed directly from the development of new neural measurement technologies which enabled stable, simultaneous recordings of many neurons.

However, electrophysiological recordings are limited in their ability to connect descriptions of population dynamics to the underlying neural circuit structure. In particular, using electrophysiology alone, it is not possible to study how coordinated population activity arises from the constituent neuronal cell types, the fine spatial organization of cells within the circuit, and connectivity of the network (Grewe and Helmchen 2009; Shenoy, Sahani, and Churchland 2013; Peron, Chen, and Svoboda 2015). In contrast, optical functional imaging methods may allow us to connect the computational level descriptions afforded by the dynamical systems view with underlying neural mechanisms. Because imaging captures the activity of all neurons within the field of view and preserves their location in space, the temporal dynamics of population activity can be connected with the spatial micro-organization of the circuit. Used in conjunction with genetic and anatomical targeting tools, optical imaging can enable recording from cells with known genetic identity and projection patterns. By bringing together functional neural response data with anatomical information about cell type and circuit inputs and outputs, optical imaging offers a means to connect the computational and circuit levels of description, grounding the dynamics of neural population responses in the underlying circuit in which they arise.

Precedent for this strategy is provided by numerous studies which employed optical imaging to connect the computational and circuit levels in other model systems. For instance, in the study of navigation, using a combination of anatomical and dynamical systems methods has led to better understanding of how hippocampal and entorhinal cortical circuits facilitate navigation (Domnisoru, Kinkhabwala, and Tank 2013; Schmidt-Hieber and Häusser 2013). In the olfactory system, examination of network dynamics of mitral cells and the spike timing of Kenyon cells have identified how odors are encoded and decoded (Mazor and Laurent 2005; Broome, Jayaraman, and Laurent 2006). Finally, using optical imaging in combination with transgenic mice allowed identification of the role of different excitatory and inhibitory neurons in the medial prefrontal cortex during goal-directed behavior (Pinto and Dan 2015). Similarly, recent work using calcium imaging in the mouse motor system shed light on how neural circuits mediate movement preparation and initiation across different brain areas (Li et al. 2015). Importantly, in each of these studies, the link between circuit dynamics and behavior has been clarified by annotating functional recordings with cell type identity and circuit connectivity.

Within motor cortex, we anticipate that a similar approach could elucidate the circuit-level organization that subserves motor planning and execution. Current dynamical models of motor cortical function provide descriptions of the computations underlying movement generation at a level separated from the spatial organization of the circuit itself. In other words, these models describe the temporal evolution of neural activity during movement (Churchland et al. 2012), but remain necessarily agnostic as to how these patterns are organized spatially within cortex and whether this organization serves a useful role within the neural circuitry in generating these patterns.

Clarifying the functional macroscopic organization of motor cortex

Short duration electrical stimulation, delivered either at the surface (Fritsch and Hitzig 1870; Penfield 1937) or intracortically (Asanuma and Rosen 1972), can evoke twitches of small groups of muscles, which can be used to construct maps of efferent zones. This approach, often combined with EMG measuement of evoked muscle responses, readily produces a rough picture of broad anatomical organization of neural responses in primary motor and premotor cortex. Proximal muscles are activated from a more rostral “horseshoe-shaped” region (extending into premotor cortex), which surrounds a central core, located more caudally, which drives distal musculature (Park and Belhaj-Saif 2001; Park, Belhaj-Saïf, and Cheney 2004). This organization is consistent with the findings of anatomical tracing studies as well (He, Dum, and Strick 1995). Motor cortex also exhibits somatotopic organization along the medial-lateral axis, suggesting that with medial aspects of primary and premotor cortices involved in movements of the trunk and leg and lateral aspects involved in movements of the hands and face (Park and Belhaj-Saif 2001).

A classical interpretation of these results posits that each area of the body is mapped to a specific location on the motor cortical surface (Woolsey et al. 1952). However, other studies demonstrate that twitches of a specific muscle can be elicited by stimulation of multiple disparate regions of cortex, arguing for intermixed, distributed, overlapping many-to many mapping of neurons to muscles (Strick and Preston 1978; Donoghue, Leibovic, and Sanes 1992). Consistent with this functional evidence, transsynaptic tracing with rabies virus demonstrates that individual muscles are controlled by several discrete, populations of corticomotoneuronal neurons within motor cortex (Rathelot and Strick 2006). Moreover, individual stimulation sites typically activate multiple muscles or muscle synergies simultaneously (Asanuma and Rosen 1972; Overduin et al. 2012). Similar coactivation patterns are observed when using spike-triggered averaging to detect the effect of individual corticospinal neurons on EMG (Fetz and Cheney 1980; Cheney and Fetz 1985). This cofacilitation is thought to facilitate and stabilize coordinated, multi-joint movements. Collectively, these results demonstrate the cellular-scale functional organization may be more distributed and intermingled than the simple somatotopic model of the motor homunculus suggests (Strick and Preston 1978; Schieber and Hibbard 1993; Sanes et al. 1995).

Moreover, motor cortex facilitates producing coordinated, voluntary movements, which necessitates coordinated patterns of muscle activation, rather than simple twitches of individual muscles. An important goal for motor neuroscience is therefore to understand the spatial and temporal organization of motor cortex in the context of producing behaviorally relevant movements (Graziano et al. 2002; Shenoy, Sahani, and Churchland 2013). Recent studies have made important strides towards achieiving this understanding using electrical stimulation and electrode recordings. In one set of studies, Graziano and colleagues employed longer duration (e.g. 500 ms) intracortical electrical stimuli to probe motor cortex. These long-train stimuli typically evoke complex, multijoint movements that resemble purposeful actions, such as hand-to-mouth or defensive gestures (Graziano, Taylor, and Moore 2002; Graziano, Aflalo, and Cooke 2005). This work suggested a different picture of motor cortical functional organization in which different categories of actions are generated from local regions of cortex organized as an action or behavioral map, with somatotopy-like features emerging incidentally due to the structure of the behavioral repertoire (Graziano et al. 2002; Aflalo and Graziano 2006; Graziano and Aflalo 2007).

Another direction of research has explored the spatiotemporal layout of motor cortex along the rostral-caudal axis. Measurements of EMG and joint velocity during reaching tasks demonstrate that movements typically require sequential recruitment of muscles in a proximal to distal manner (Hatsopoulos, Olmedo, and Takahashi 2010). This correspondence between proximal to distal muscle recruitment in behavior and a rough proximal to distal organization in motor cortex implies that there may be directions of neural activity flow in motor cortex. In support of such a hypothesis, extracellular recordings using multielectrode arrays have shown that beta band LFP (local field potential, approximately 10-45 Hz) has a travelling wave like characteristic (Rubino, Robbins, and Hatsopoulos 2006). Subsequent analyses suggest these LFP phenomena may exist in single neuron activity (Kim et al. 2011; Takahashi et al. 2015).

We believe that achieiving a holistic, accurate picture of motor cortical organization will require fusing two domains of investigation, the first attempting to map the spatial organization of motor cortex, and the second attempting to elucidate the dynamics of neural population activity in the same areas (Shenoy, Sahani, and Churchland 2013). Understanding the neuroanatomical structure of the circuit in the context of the computational mechanisms by which voluntary movements are prepared and executed will be essential to understanding motor cortex, as well as helpful in contextualizing the collection of experiemntal observations just described. However, we believe that achieving this integrated understanding remains challenging with electrophysiology for serveral reasons. First, the effect of electrical microstimulation on neural populations – essential to correctly interpreting stimulation-based mapping studies – is far from straightforward or well-characterized (Strick 2002; Logothetis et al. 2010; Overduin et al. 2012; Histed, Bonin, and Reid 2009), a topic which we will expand upon later. Second, current multielectrode arrays do not support dense recordings over large regions of cortical space, which hinders capturing a unified, precise, and sufficiently detailed map of motor cortical responses.

Secondly, the maps produced by electrode penetrations or electrode arrays are coarse relative to the density of neurons – only a few neurons may be sampled at each electrode site, and recording sites are spaced at intervals typically on the order of hundreds of microns (Rubino, Robbins, and Hatsopoulos 2006). With this very sparse sample of neurons, it remains difficult to investigate the fine spatial organization of motor cortex and to define the relationship between the microstructural details of the neural circuit with the dynamics of neural activity observed in the population. In contrast, calcium imaging methods can sample many if not all neurons within a field of view, providing the spatial resolution to dissect the mechanisms by which sequential patterns of neural activation arise and organize to generate and control movements (e.g. Harvey, Coen, and Tank 2012). Imaging, being inherently spatiotemporal, can measure the temporal order in which neurons at different spatial locations respond during movement. Measurements in different regions of the precentral gyrus would help clarify the microstructure of observed travelling waves and incorporate them into dynamical models (Wu and Hatsopoulos 2008). Moreover, imaging could shed light on the mechanisms by which motor cortex coordinates different aspects of movement in time, such as the position of the arm and aperture of the hand in a reach to grasp task (Vaidya et al. 2015). Little is known about the coordination of neighboring cells within motor cortical microcircuits.

Imaging at cellular resolution, the patterns of activity in a motor cortex during voluntary movement would help to bridge the gap between spatial and dynamical models of the circuit. These recordings would help identify if there are spatially organized subdomains that project down to individual muscles or groups of muscles, analogous to ocular dominance columns in the visual system. When combined stimulation mapping with EMG, these recordings could provide unprecedented detail about the underlying fine scale organization in motor cortex (Park and Belhaj-Saif 2001; Park, Belhaj-Saïf, and Cheney 2004). Additionally, fine structure spatiotemporal dynamical data obtained by imaging during simple motor tasks would help connect the computational level descriptions offered by dynamical (Shenoy, Sahani, and Churchland 2013) and neural network models (Sussillo et al. 2015) of motor cortex with the anatomical and structural implementations of the computation.

Linking dynamical systems models with neuronal identity and cortical projections

The dynamical models put forth to explain movement generation are necessarily instantiated in the networks of excitatory and inhibitory neurons present in cortical circuits. At present with extracellular recordings, one can weakly classify neurons based on waveform shape (Kaufman et al. 2010; Kaufman, Churchland, and Shenoy 2013). However, the variation in waveforms in cortical circuits (Vigneswaran, Kraskov, and Lemon 2011) precludes finer distinctions beyond broad- and narrow- spiking. Moving towards more concrete, testable models of how population dynamics arise and lead to behavior is critically needed in motor neuroscience, and this will likely require moving beyond datasets comprised of a collection of unlabelled neurons devoid of genetic and connectivity information. Combining calcium imaging with genetically targeted fluorescent co-labeling (Pinto and Dan 2015; Li et al. 2015) or with post-mortem CLARITY should allow precise identification and classification of cells into finer-grained cell types (Chung et al. 2013; Tomer et al. 2014). Furthermore, when combined with appropriate targeting techniques (Lerner, Ye, and Deisseroth 2016; Rajasethupathy, Ferenczi, and Deisseroth 2016), the roles of cell types and anatomical projections in shaping the neural population dynamics underlying the preparation and execution of movement may be elucidated.

Another limitation of electrophysiological datasets is that typical recordings are biased towards high firing rate cells (Barth and Poulet 2012). However, there is evidence that a majority of cortical neurons have low firing rates, often firing in sparse bursts in cortical circuits (Shoham, O’Connor, and Segev 2006). These types of responses are rarely sampled in extracellular recording methods due to the inherently low likelihood of observing and successfully isolating the spiking waveforms of quiescent cells. Imaging, in contrast, facilitates largely unbiased sampling of neurons (Dombeck et al. 2007). This in turn would help us restructure our dynamical models of motor cortex to better understand and characterize sparsely active neurons, which are known to play critical roles in motor output circuits (Hahnloser, Kozhevnikov, and Fee 2002; Fiete et al. 2004). Furthermore, because the same population of cells can be stably imaged over a period of several months (Ziv et al. 2013), the responses of these quiescent cells could potentially be characterized over a broader, more inclusive range of behavior.

Finally, imaging would better characterize the interconnected roles of different cortical regions in producing behavior. Together with parietal cortex, motor and premotor cortices help utilize sensory cues (visual, proprioceptive, auditory, and somatosensory) in guiding the control of movement (Asanuma and Rosen 1972; Archambault, Caminiti, and Battaglia-Mayer 2009; Cluff, Crevecoeur, and Scott 2014; Scott et al. 2015). Extensive sensory inputs to premotor and motor cortex terminate predominantly in superficial layers (Strick 1975; Wise et al. 1997; Arikuni, Watanabe, and Kubota 1988). Characterizing the source, organization, and timing of these inputs and how they shape the evolution of population activity in motor cortex remains an important open question. Imaging the activity of sensory afferents in layer 2/3 of primate motor and premotor cortex could reveal their influence on the activity of the local population preceding and during movement. The functional content of these projections could also be used to localize the types of computations being performed by different parts of a cortical circuit (Gunaydin et al. 2014). Direct observation of the information being conveyed between the numerous brain regions involved in motor control would greatly clarify our understanding of the roles of each region and the heretofore unobserved influence that each region has on other regions? population dynamics (Dum and Strick 2002; Sussillo et al. 2015).

Optical dissection of therapeutic neural stimulation and sensory write-in

Electrical stimulation is increasingly being used as a therapeutic tool to rescue deficits in Parkinsonism (Holtzheimer and Mayberg 2011) and depression (Mayberg et al. 2005), and cortical microstimulation shows growing promise as a means to write-in sensory information for visual (Tehovnik et al. 2009; Jepson et al. 2014) and motor prostheses (O’Doherty et al. 2011; Dadarlat, O’Doherty, and Sabes 2015). In the future, optogenetic tools may also be used in a clinical context to enable genetically targetable neural stimulation (Gradinaru et al. 2009) and finer control over spatiotemporal patterning for writing information into the nervous system (Nirenberg and Pandarinath 2012; Grosenick, Marshel, and Deisseroth 2015).

Current clinical applications for neural write-in are limited by our lack of understanding of how electrical microstimulation is perturbing a neural circuit. One long-standing hypothesis is that microstimulation precisely activates a local sphere of cells surrounding the electrode tip (Tehovnik et al. 2006). However, the effect of electrical stimulation can often produce an excitatory and inhibitory effect on neural activity as a function of time (Seidemann et al. 2002; Logothetis et al. 2010), likely through activation of a fast, electrically-coupled inhibitory network (Butovas et al. 2006). The spatiotemporal dynamics of these bidirectional effects depends on local connectivity patterns, affecting both cortico-cortical and cortico-subcortico-cortical pathways (Logothetis et al. 2010). These effects also vary as a function of stimulation frequency and amplitude (Butovas and Schwarz 2003). Additionally, recent work has argued that microstimulation activates a sparse, distributed population of cells in the vicinity of the electrode, presumably through activation of collateral fibers (Histed, Bonin, and Reid 2009). Consequently, when the electrode tip is translated by only a few tens of microns a different sparse subset of cells is activated, hinting that the effect of microstimulation on neural tissue may be unpredictable and potentially unstable. This creates uncertainty both in probing neural circuit function via electrical perturbation (Histed, Ni, and Maunsell 2013) and in using microstimulation to restore sensory percepts in visual, auditory, and motor prostheses (e.g. Otto, Rousche, and Kipke 2005; Tehovnik et al. 2009; O’Doherty et al. 2011; Bensmaia and Miller 2014; Dadarlat, O’Doherty, and Sabes 2015).

Historically, it has been difficult to study how electrical stimulation impacts circuits for two reasons. First, stimulation creates electrical artifacts that impede simultaneous electrical stimulation and recording (Wagenaar and Potter 2002). Second, as described above, even high-density microelectrode arrays do not provide sufficient spatial resolution to characterize the impacts of electrical stimulation on the whole population of cells neighboring the stimulating electrode (Butovas and Schwarz 2003). Imaging methods directly address both of these limitations, enabling densely sampled recordings free of electrical stimulation artifacts (Seidemann et al. 2002; Histed, Bonin, and Reid 2009; Tehovnik and Slocum 2013). Along these lines, Adelsberger et al. (2014) recently combined dual fiber optic imaging with bulk loading of a synthetic calcium indicator (OGB-1 AM) and intracortical electrical microstimulation. These fiber photometry measurements demonstrated that stimulation strongly activated the pool of neurons near the stimulation source, but had no detectable effect on the signal recorded at the second fiber, located only 4 mm away. We anticipate that two-photon calcium imaging will provide insights into the effects of electrical stimulation on motor cortex at cellular resolution, where it has already enabled recordings of electrically evoked transients in visual cortex of mice (Histed, Bonin, and Reid 2009).

The effects of optogenetic stimulation are similarly poorly characterized, though it is thought to directly activate a similar local sphere of cells close to the stimulation source (Yizhar et al. 2011; Ozden et al. 2013). Recent demonstrations of optogenetic modification of behavior in monkeys have been promising (Cavanaugh et al. 2012; Gerits et al. 2012; Jazayeri, Lindbloom-Brown, and Horwitz 2012), though its effects on motor behavior are subtler than those typically elicited by electrical microstimulation. Optogenetic excitation of motor cortex can successfully impair motor preparation similarly to subthreshold electrical microstimulation (Churchland and Shenoy 2007) but fails to evoke twitches in the resting arm (Diester et al. 2011), even when stimulation was targeted primarily to excitatory neurons (O’Shea et al. 2014). Understanding the differences and advantages of each stimulation modality to drive desired patterns of neural activity to achieve desired behavioral outputs in primates remains a key open question for researchers and also key to develop circuit level therapeutics.

Insights from optical methods for brain-machine interface design

In addition to providing a clearer understanding of the problem space for researchers interested in designing systems for sensory write-in, calcium imaging can provide key insights into the problem of reading out intention from motor cortex for the purpose of motor brain-machine interfaces (BMIs). BMIs seek to restore lost function to people with neurological motor injury or disease by decoding neural activity from the brain to drive a prosthesis device, such as a computer cursor on a screen or a robotic arm (Bensmaia and Miller 2014; Ethier, Gallego, and Miller 2015; Homer et al. 2013; Kao et al. 2014; Tsu et al. 2015). To date, substantial progress has been made in intracortical BMIs in which movement intention is inferred from electrical recordings of neurons in PMd and M1, enabling recent phase I clinical trials for translating this technology to humans (Collinger et al. 2013; Gilja et al. 2015; Hochberg et al. 2006; Hochberg et al. 2012; Wodlinger et al. 2015). While recent algorithmic advances have enabled clinically-relevant performance using existing multi-electrode arrays (Gilja et al. 2012; Gilja et al. 2015), it is widely accepted that larger numbers of electrodes are necessary for restoring near-natural levels of motor control of e.g., prosthetic arms. The field therefore eagerly awaits revolutionary advances in the numbers of electrodes that can be implanted in the brain (e.g., on the order of 10,000 thousands compared to today’s order of as well as marked improvement in recording stability.

An optical-imaging-driven BMI (o-BMI) may allow BMI researchers to preview the opportunities and challenges of this much-anticipated high neuron count future by recording from many neurons optically in a non-human primate animal model. O-BMIs can be used to evaluate how well BMIs can perform with large numbers of stably recorded neurons, thus providing a test platform for developing BMI techniques that can then be transferred to clinical BMI systems once comparable electrode counts are available for implantation in people with paralysis. To be explicit, we do not propose that imaging, which requires introducing reporters (e.g. calcium reporters) and optical access to the brain, is a recording modality appropriate for use in clinical BMIs, at least in the foreseeable future. Rather, we believe that engaging optical imaging in basic research and pre-clinical primate studies allows tackling important scientific and real-time signal processing challenges that will likely be encountered when very high channel-count electrode sensors become available.

O-BMIs are particularly well suited for characterizing how BMI performance will vary with the number and implant location of electrodes. By optically recording from potentially many hundreds of neurons simultaneously, one could study how BMI performance scales with the number of neurons available to the decoder. This approach can potentially sample larger neural populations – recent studies have simultaneously imaged many hundreds (Ziv et al. 2013) to hundreds of thousands (Sofroniew et al. 2016) of cells. Even the most advanced electrode arrays currently available have yet to exceed simultaneous decoding from 500 neurons (Schwarz et al. 2014). A larger pool of neurons available for BMI decoding would enable within-day, same-animal comparisons that are not possible with arrays. O-BMIs also provide a means to test how performance varies not only with how many, but also with which, neurons are recorded. While there have been prior electrode-based investigations of how movement-related information varies by cell type (Kaufman, Churchland, and Shenoy 2013) or cortical depth (Parikh, Marzullo, and Kipke 2009; Markowitz et al. 2011), this approach is limited to either small ensembles of neurons recorded by movable electrodes, or larger but fixed ensembles from higher electrode-count arrays. We’ve recently reported a cortical viewing chamber with a 1 cm diameter usable area (Trautmann et al. 2015). With such a chamber, the o-BMI would have access to a relatively large region of motor cortex, from which neurons could be sampled with the much higher density available to optical imaging compared to multielectrode arrays. Thus, o-BMIs could compare closed-loop decoding using different simultaneously recorded large neural populations within a given experimental session. An even greater variety of ensembles could be surveyed across multiple recording sessions.. This would enable an optimized design of future clinically viable sensors by specifying, for a desired degree of performance, how many multi-electrode arrays are needed, what electrode geometry they should have, and where they should be implanted. These design specifications and proof of performance metrics would lay the foundation for a directed push to then develop the requisite electrode arrays for clinical use.

Decoding algorithms for large neuron counts

A large increase in the number of simultaneously recorded neurons would give rise to a wealth of opportunities for algorithmic and signal processing advances. To date, BMI algorithms have been designed and optimized in a regime where approximately 100 neurons are recorded simultaneously. However, the resulting design assumptions may be poor in a higher neuron count regime. For example, state-of-the-art kinematic Kalman filter based techniques model the firing rates of recorded neurons as a time-invariant linear combination of a few kinematic variables, such as the endpoint and velocity of a robotic arm (Collinger et al. 2013; Hochberg et al. 2012). These models are formulated on the assumption that kinematics are causal to neural activity and implicitly smooth neural responses via a model of how these low-dimensional kinematics, rather than neural activity, evolves over time. It is currently unclear that this approach would scale well when information from more neurons is available. In particular, the simplified neural tuning assumptions inherent to most existing decoders may fail to make full use of a large and heterogeneous neural population’s complex relationship with kinematic parameters (M. Churchland and Shenoy 2007; Shenoy, Sahani, and Churchland 2013). If so, performance will saturate below near-natural levels despite having access to more neurons.

It may instead be that to achieve near-natural performance, very high neuron count BMIs should employ algorithms with fundamentally different modeling assumptions. For example, a recent study demonstrates that modeling the dynamics of how the neural population activity at one time depends on the neural population activity a moment earlier can denoise and smooth neural observations and substantially increase BMI performance (Kao et al. 2015). Optical-imaging recording from more neurons will allow more accurately identifying the lawful dynamics of neural activity and could enable even better performance. There exists a variety of other decoding algorithm classes whose as-of-yet unknown performance when scaled to large neuron counts can be revealed with o-BMI experiments. These include Wiener filters (Carmena et al. 2003), particle filters (Brockwell, Rojas, and Kass 2004), point-process filters (Truccolo et al. 2005), artificial neural networks (Rao et al. 2005), and recurrent neural networks (Sussillo et al. 2012). These algorithms can model more complex (e.g. time-varying or nonlinear) relationships between neural activity and kinematics, which may prove critical for exploiting the opportunities provided by future larger neuron count arrays. Optical imaging provides the capability to assess sooner rather than later the strengths and weaknesses of these different decode algorithms under large neuron count regimes.

Tracking individual neurons through time for the principled design of co-adaptive BMIs

O-BMI also promises to be a particularly effective tool for investigating how to design, in a principled way, a so-called co-adaptive BMI. Co-adaptive refers to a closed-loop BMI setup in which how both the response properties of individual neurons are adapting (possibly via plasticity) to improve the BMI user’s performance, and the decode algorithm is adapting to try to make the BMI easier to control ((e.g. Orsborn et al. 2014; Taylor, Tillery, and Schwartz 2002), (reviewed in Shenoy and Carmena 2014)). Understanding and facilitating co-adaptation is generally believed to be an important next step in BMI design, but progress is stunted by our inability to reliably track individual neurons using current multi-electrode arrays across days.

Individual neurons enter and leave the recording range of electrodes due to today’s rigid arrays moving with respect to the surrounding tissue, as well as other factors (e.g. Santhanam et al. 2007; Chestek et al. 2007; Barrese et al. 2013; Nolta et al. 2015). One cannot predict a priori which neurons will remain in the recorded population, and so in practice only a handful of neurons can be retrospectively tracked for several weeks. Furthermore, it is difficult to definitively tell apart neurons using their action potential waveforms alone. The neurons? functional properties (e.g., tuning curves) provide an additional signature, but these properties may themselves change as part of adaptation. Consequently, direct and assured measurement of neural adaptation is scant (e.g. Ganguly and Carmena 2009; Stevenson et al. 2011; Fraser and Schwartz 2012; Santhanam et al. 2007; Tolias et al. 2007) and BMI learning experiments are often limited in scope to observing changes over short durations (Sadtler et al. 2014).

Optical imaging provides, for the first time, the ability to definitively record the activity of the same population of cortical neurons across weeks or even months (e.g. Ziv et al. 2013) in an awake, behaving NHP. Individual neurons can be identified by their anatomical location and potentially by their morphology as part of daily optical imaging characterization and setup. This will enable tracking the properties of the neuronal ensemble over time as an animal uses a truly constant BMI decoder, i.e. one in which the recorded neurons and their decoder weights are unambiguously constant. Such studies can effectively ask whether and under what conditions large-scale plasticity occurs during long-term use of a BMI. Their answers would in turn provide principled guidance as to whether a biomimetic approach (e.g. Gilja et al. 2012), a neural adaptation approach (e.g. Ganguly and Carmena 2009), or a combined (co-adaptive) approach to BMI design should be embraced (e.g. Shenoy and Carmena 2014). If adaptation is observed, optical imaging (possibly combined with later CLARITY) may identify distinct roles for different subpopulations (e.g. cortical layer, cell type, downstream projection targets) in this process, and thus inform what subpopulations? activity should directly drive the BMI. This same setup also provides a platform by which to systematically test adaptive and co-adaptive control algorithms to promote advantageous neural adaptation and contend with unanticipated neural response changes.

Finally, it may be possible to use this closed-loop o-BMI experimental paradigm to discover what happens in the larger neural network to enable volitional control of one, or a handful of, “volitional-neurons” whose activity determines feedback (and reward) provided to the animal. This fundamental question has remained unanswered since the first demonstration that an animal can learn to modulate the activity of a particular neuron using operant conditioning (Fetz 1969). A related question with particular relevance to motor prosthesis applications is whether an arbitrary number of neurons can be independently controlled, or whether anatomical and/or functional connections between neurons create difficult-to-break correlations. Recent studies using microelectrode arrays (Ganguly et al. 2011) have attempted to answer these questions but are limited by the aforementioned difficulty in recording from many neurons across days. Using an o-BMI, one would be able to arbitrarily assign modulation rules to specific neurons of the experimenter’s choosing and observe this population, and their surrounding network, for weeks. Several such o-BMI studies have recently been performed in rodents (Clancy et al. 2014; Koralek, Costa, and Carmena 2013), and we anticipate that o-BMI studies in NHPs will enable scaling up the complexity and duration of these studies to provide insight more directly applicable to clinical BMI use. The number of surrounding neurons that can be sampled is potentially very large, since the “volitional-neurons” can be held constant while recording from different parts of the surrounding network each day. In addition to providing new insight into fundamental questions about learning, these experiments will add to the nascent neuroscientific understanding of the mechanisms and limitations of BMI learning (Koralek, Costa, and Carmena 2013; Koralek et al. 2012). For example, quantifying how much of the cortical network must be engaged to shape one voluntarily-controlled neural degree of freedom may reveal whether there is a fundamental limit to how many independent control signals the user of a BMI can generate.

Challenges of optical measurement in NHPs

Having discussed the merits of optical methods for basic motor neuroscience and BMI applications, we now turn to the challenges that must be overcome to apply these methods in NHPs. The broad array of tools developed in recent years for genetic targeting of neural subpopulations with genetically-encoded calcium sensors (Deisseroth and Schnitzer 2013) has been most widely utilized and transformative with small animal models, particularly in rodents. These fundamental advances lay the groundwork for developing an optical approach to neuroscience in primate models, however, differences in physical scale and requirements for experimental longevity necessitate re-engineering in order to perform calcium imaging in awake, behaving primates. Additionally, image processing and statistical inference techniques are required in order to distill and understand population activity from noisy optical signals. Some of these challenges are shared with researchers using small animal models, but some arise from the desire to compare optical recordings to the dynamical systems models employed to understand primate neural circuit function. Here we briefly highlight some of these challenges as well as the strategies that we and other groups are pursuing to address them (Trautmann et al. 2015; Sadakane et al. 2015).

Imaging in monkeys: data collection challenges

Chamber and microscope geometry

Successful translation of calcium imaging approaches to awake, behaving primates requires adapting existing techniques to the larger scale of primates and primate brain regions. First, in order to gain optical access to primate cortex, we can build upon existing approaches which use silicone artificial dura or glass windows to maintain optical access to cortex (e.g. Grinvald et al. 1991; Arieli, Grinvald, and Slovin 2002; Ruiz et al. 2013). These chamber designs were optimized for capturing the intrinsic optical signal or voltage sensitive dye loading and employ wide-field or single photon fluorescence microscopy. While these imaging modalities are also a viable approach for calcium imaging as well, obtaining high signal to noise measurements at depths beyond order of 100 microns in scattering brain tissue requires multiphoton microscopy (Helmchen and Denk 2005), which in turn necessitates the use of large body, high numerical aperture objective lenses. For an objective lens of a given size and working distance, the geometry of the imaging chamber must accommodate the lens sufficiently close to the desired imaging plane. Achieving this proximity is also hindered by the thickness of primate cranium, which in macaques is typically 3-5 mm, in the same range as the working distance of a typical objective lens.

Depending on the brain region of interest, it is also likely that the objective lens will need to be rotated so as to match the orientation normal to the brain surface. Several commercially available movable objective microscope (MOM) designs enable adjustable orientation of the objective lens, though the photon collection efficiency of the different approaches taken may vary.

Stabilization

When imaging in awake, behaving primates, there are several sources of motion of the brain relative to the objective lens that are drastically increased in scale relative to imaging in smaller animals. First, cardiac and respiratory rhythms can translate the surface of the brain by several millimeters within a large craniotomy spanned by silicone, necessitating some form of mechanical stabilization to reduce this motion. Second, the performance of the behavioral task itself may induce motion. This issue has been largely addressed in the context of a visually-guided reaching task by utilizing a multipoint head-stabilization system (Aaron Batista, personal communication). Lastly, high frequency vibration may be amplified in some MOM designs where some optical components are cantilevered over the subject. A careful consideration of numerous factors should be conducted in order to minimize mechanical vibration, including microscope design, background vibration levels, behavioral task, optical table selection (e.g. rigid vs. floating), and vibration dampening materials (e.g. floor mats).

Longevity of optical access

Once optical access to cortex is established by an initial durotomy and installation of artificial dura, maintaining a clear view of the brain is essential to long-term imaging experiments. Prior work has reported clear optical access for several months to a year (Arieli, Grinvald, and Slovin 2002; Ruiz et al. 2013). However, the requirements for optical clarity demanded by optogenetic stimulation or single photon imaging may be less stringent than those of two-photon imaging at depth. Maintaining ideal conditions for multiphoton microscopy may require adjustments to existing chamber maintenance approaches, potentially involving better dural tissue barriers, frequent flushing or debriding of regrown tissue, antibiotic regimens and tight seals to prevent infection, etc.

Targeting of optical reporters

The targeting precision of virally-transfected optical reporters such as GCaMP depends on a combination of factors, including the tropisms of different viruses and viral serotypes and the specificity of the promoter used (Yizhar et al. 2011). Some studies have compared the efficiency and specificity of viral serotypes (Watakabe et al. 2014; Markakis et al. 2010) and promoters (Diester et al. 2011; Han et al. 2009; Nathanson et al. 2009) in transfecting primate neurons, but more systematic, quantitative studies are needed. Additionally, the repertoire of genetically-defined cell types that can be targeted at present in primates is relatively quite small in comparison to mice and rats. However, as interest in genetic targeting and circuit tools for primate neuroscience and human gene therapy grows (Han 2012; Gerits and Vanduffel 2013; Belmonte et al. 2015), we anticipate that the array of targeting strategies will continue to expand and improve. Lastly, recent advances in genome editing technology have spurred a renewed interest in the generation of genetically-modified primate lines, particularly using common marmosets (Sasaki 2015; Kishi et al. 2014; Belmonte et al. 2015). These approaches could allow for direct expression of GCaMP in certain neurons without viral transfection, or alternatively, the development of recombinase lines that would restrict expression of a transgene like GCaMP to a specific subset of cells, an approach that has been very successful in the rodent community (Witten et al. 2011).

Longevity of functional signals

If the calcium reporter is delivered to neurons via viral transfection, the time window in which useful images may be collected is also dependent on the expression profile. Inadequate expression will yield low signal measurements, and overexpression typically results in bright but constant fluorescence signals with reduced dynamic range (Tian et al. 2009; Chen et al. 2013). If the transition from underexpression to overexpression takes place over the course of a few weeks, the time window for useful imaging may be too brief to fully sample the brain region of interest. Consequently, we anticipate the need for careful characterization of expression timelines as a function of viral serotype, genetic promoter, and reporter protein to identify the best approaches for stable, long-term imaging in primates. Additionally, techniques to gate or modulate transgene expression, e.g. via tetracycline dependent expression systems (Sadakane et al. 2015), offer a promising direction forward in order to carefully titrate expression and extend the time window where successful imaging is possible.

From images to models of neural population dynamics: data-processing challenges

Fluorescence-based calcium signals report neuronal activity indirectly, and thus the translation of raw light measurements into direct measurements of neuronal firing and dynamics depends on a series of signal-processing and modeling steps. The typical processing pipeline involves some or all of the following: (1) Image processing: image acquisition, image normalization, and image registration; (2) Identifying neurons: image segmentation, neuron and/or regions of interest identification, neuron registration; (3) Time series extraction: fluorescence time-series extraction, fluorescence time-series normalization; (4) Spike- or burst-detection; and (5) Direct dynamical modeling. Below, we review and discuss each of these topics in order to convey why these are current and critical research questions of particular importance in the context of larger NHPs.

Image processing

Current in vivo measurements of fluorescent cellular-calcium signaling molecules depend on scanning two-photon illumination methods. Thus, the large field-of-view images that capture more neurons require faster two-photon scanning, and so return fewer fluorescence photons per pixel. This leads to a fundamental compromise between neuron count and image quality. Image quality is also strongly affected by variable uptake or expression of fluorescent indicator molecules, making some neurons much more difficult to detect than others.

This means that a crucial first stage of processing for high-neuron-count imaging depends on denoising and normalization algorithms. Effective approaches embody appropriate domain knowledge regarding the signal-generation and imaging processes, and thus require some adaptation of generic image processing algorithms.

Successive image frames may be affected by movement of the brain under the microscope associated with heartbeat, respiration or movement of the subject. This is of particular potential concern in larger animals, such as rhesus monkeys. Indeed, movement during the acquisition of a single wide-area image frame may appear as a non-rigid deformation. Furthermore, images of the same brain volume taken on different days may be misaligned due to micron-scale differences in microscope position. Rigid registration issues can be resolved by standard correlation-based methods when displacements occur in horizontal position alone. However non-rigid deformation, and displacement or tilt in depth can introduce more challenging artifacts as neurons come in and out of focus, and so new imaging technologies and algorithms will likely be essential to fully achieving within- and between-session registration.

Identifying neurons

Each two-photon-scanned micrograph represents a slice of imaged tissue transecting the cortical neuropil, typically with a resolution of microns (in the XY plane) and a thickness on the order of a few to tens of microns. This optical section may thus contain some complete cell bodies spread over multiple pixels, but will also reflect fluorescence from partially-transected somata, dendrites, and spines. The result is a densely-packed image from which the signals corresponding to individual neurons must be extracted by segmenting individual somata.

The current state of the art is heavily dependent on human operator input, but this practice raises obvious concerns about efficiency and scalability (notably in brain-machine interface applications) as well as reproducibility across laboratories. Automation of the process is thus a crucial part of extending calcium imaging methods to very high neuron counts. Attempts at automated methods generally fall into two classes. One group (e.g. Mukamel, Nimmerjahn, and Schnitzer 2009) searches for linear projections of pixels that change fluorescence over time coherently, but independently of other pixels, using time-series algorithms such as Independent Components Analysis (ICA). The second approach ignores time, starting with a single image (often the temporal mean of the individual frames) and using thresholded matches of a spatial template to pick out the cells. Neither approach by itself is able to handle very large-field images with high neuron counts: correlation-based algorithms scale relatively poorly and are confused by correlations in the underlying neural activity, whereas template-based methods have difficulty handling the variability in appearance caused by differences in cell-type, in the location of the soma relative to the imaging plane and in the uptake or expression of the calcium indicator.

Some recent work (Pachitariu et al. 2013) has extended template-based methods to allow for the image of each cell to be formed of a potentially different linear combination of basis vectors, while maintaining numerical efficiency. It may well be possible to adapt such an approach to exploit temporal information for segmentation, either by requiring that higher-order temporal moments (particularly covariance and kurtosis) align with the same basis combination, or by pursuing simultaneous reconstruction of the entire spatio-temporal image sequence from space-time basis vectors.

Time series extraction Once the spatial extent of each neuron is identified, the signals from the corresponding pixels must be integrated to obtain a time series of fluorescence in the cell. Pure correlation-based methods return such a time series as an intrinsic part of the segmentation. Spatial methods, however, yield a set of regions of interest (ROIs), from which the time series must be obtained as a second step. Most simply, the signal is just summed within the ROI, possibly after applying a set of template-derived weights. This approach risks incorporating fluorescence from non-somatic neuropil or other sources of noise that fall within the same imaging volume. A more sophisticated approach might be to run an ICA-like correlation algorithm within each ROI to find weights optimal for noise rejection. Alternatively the contribution of the neuropil might be estimated by assuming a longer-range spatial coherence in its activation (Chen et al. 2013).

An interesting open problem involves the identification of the same neurons in image sequences collected over multiple days. If the images themselves can be registered accurately then this identification may be achieved simply by position within the image. However for images distorted by tilting in the depth of the imaging plane or changes in indicator expression, where perfect image-level registration may be impossible, there is a need to pursue more model-based approaches at the time-series, and possibly the 3D spatial reconstruction, level.

Spike or burst detection

Models of the dynamical activity of a neuronal circuit must ultimately be linked to the action potentials that represent communication both within and between the vast majority of neuronal populations. The intrinsic dynamics with which intracellular calcium levels respond to spiking, as well as the kinetics of the indicator biochemistry, mean that the transient fluorescence changes associated with individual spikes extend over hundreds of milliseconds and will thus overlap unless those spikes are well isolated in time. This means that the recovery of spikes from the fluorescence signal is an exercise in (possibly non-linear) deconvolution.

One approach exploiting fast exponential deconvolution methods with a non-linear saturation model (Vogelstein et al. 2010) has been applied successfully to time series obtained using dyes such as Oregon Green Bapta-1 (OGB1). Even at moderate firing rates of tens of spikes per second, however, such approaches depend crucially on the very rapid post-spike rise in fluorescence signal obtained with such dyes. Although the best currently available genetically-encoded calcium indicators (i.e., GCaMP6 family) now match OGB1 for total fluorescence change, and for offset kinetics, they still show substantially slower spike-driven transient rise times (Chen et al. 2013; Sun et al. 2013). This complicates such deconvolution-based approaches, and the successful in vivo recovery of moderate-rate spiking from such fluorescence time series remains an open challenge that must be tackled either by new analysis tools, or by the development of more responsive indicator molecules, or both.

Direct dynamical modeling

Ultimately, of course, an important goal is to recover the dynamical activity of the network as a whole, which is both created by, and reflected by, the spiking of the cells of the network and thence may be mapped to the intracellular calcium signals. Thus, an alternative to precise spike recovery may be available if the essential elements of the network dynamics could be recovered using models that map the population dynamics through the unobserved spikes directly to calcium indicator activity. One obvious challenge to such an approach would be the need to separate the dynamics of the calcium indicator from those of the network: made the more difficult by the fact that the dynamical time-scales contributed by both sources are similar given current indicator technology.

Fortunately, two structural features of the data may provide the necessary modeling leverage to solve this challenge. First, indicator kinetics must evolve separately within each cell’s response, whereas the network activity is fundamentally shared across the neurons of the network. Second, in vitro or small-scale in vivo experiments may provide an accurate enough characterization of the indicator kinetics (and any dependence on cell-type or expression patterns) to allow for strong constraints to be set on the structure of its contribution to the eventual signal, in the same way that balloon models of hemodynamics are used in fMRI analysis. Thus such direct dynamical modeling provides an important area of future work.

Ultimately, conducting behavioral and BMI experiments with behaving monkeys will enable optical-imaging measurements to be made from the same brain areas (e.g., primary motor cortex and/or premotor cortex) as spiking-activity electrical measurements. This provides an “apples to apples” test bed for cross-validating the neuronal dynamics inferred from optical imaging with the neuronal dynamics measured and assessed with electrode arrays (e.g. Sussillo et al. 2015; Pandarinath et al. 2015; Kao et al. 2014; Kao et al. 2015; Kaufman et al. 2014; Ames, Ryu, and Shenoy 2014; Churchland et al. 2012).

Concluding remarks

Electrical measurements of neurons have enabled numerous fundamental scientific insights into the neural computations that underlie perception, cognition, and action by capturing spiking activity in awake animals engaged in a growing array of behavioral tasks. Electrophysiology provides a wealth of information about the types of responses neurons exhibit and the oscillatory electrical rhythms that accompany neural computation in neural circuits. These data enable systems neuroscientists to devise computational models that can capture the dynamical evolution of the neural population activity and link this activity quantitatively to sensation, deliberation, and behavior. This approach offers an understanding of neural circuits at a particular level of description, but it remains very challenging to connect these mathematical formulation of neural population dynamics with lower levels of abstraction, which is likely to be critical to untangling circuit-level mechanisms, causally testing theoretical predictions, and developing effective therapeutic interventions.

Optical methods for recording neural activity enable the experimental capabilities that can begin to bridge the gap between computational and circuit levels of description. Because tools like calcium imaging sample nearly completely from the local neural population and preserve the spatial arrangement of the cells, optical measurements provided a means to link models of neural dynamics with the spatial microstructure of neural circuit organization. Cells recorded using functional imaging can be registered against anatomical fluorescence images and post-mortem CLARITY data, functional datasets can be annotated with genetically-defined cell type, information about where cells project their output or from where they receive their inputs. Lastly, functional imaging can be performed on the same population of cells repeatedly for many months, opening entirely new scientific avenues for understanding how neural computations change through learning and plasticity, or for understanding how these computations support a vast repertoire of behaviors, whose cardinality far exceeds what could feasibly be studied in a single experimental session.

Optical tools like calcium imaging and CLARITY, combined with rapid advances in viral targeting and genome editing, have rapidly transformed neuroscience with rodents, flies, and other small animal models. Rodents and other small animals serve critically important roles in many domains of neuroscience and biomedical research. Rodents differ in several significant, well-established capacities from humans, including in overall brain organization, cognitive and behavioral flexibility, social cognition, and manual dexterity. NHP models possess greater anatomical, genetic, cognitive, and social similarity with humans, and consequently offer complementary value to the study of the nervous system in health and disease. NHP researchers in other have and will continue to benefit tremendously by borrowing and adapting novel tools for neural circuit interrogation and manipulation from the rodent and small animal communities. We have highlighted an array of scientific and translational opportunities afforded by application of optical methods to NHPs. We have focused primarily on scientific questions within the cortical motor system and geared towards improving the design of future brain machine interfaces, as reflects our particular domains of expertise. However, we reiterate that the experimental capabilities enabled by calcium imaging will certainly empower all branches of neuroscience working with NHPs. Optical tools provide a means to link the mechanistic and organizational details of neural circuits with dense, annotated functional measurements in awake, behaving NHPs. In concert with the broad and growing toolkit of neurotechnologies, these tools will be immensely powerful in understanding the mechanisms of neural computation and in restoring healthy neural function in neurological disease.

Highlights.

• Calcium imaging in non-human primates would enable new neuroscience

• Imaging and CLARITY could provide novel insights into motor cortical organization

• Large cell count recordings could inform brain-machine interface design

• Multiple, surmountable challenges lie in translating these tools from small organisms