Highlights from the 2017 meeting of the Society for Neural Control of Movement (Dublin, Ireland)

Introduction

Over 450 researchers, clinicians, and students attended the 27th annual meeting for the Society for the Neural Control of Movement held in Dublin, Ireland (May 1–5th, 2017). A satellite meeting focused on “The Roles of Proprioception and Vision in Perception and Action,” while during the main meeting topics ranged from learning sensorimotor maps from scratch to presynaptic inhibition for sensorimotor control to vestibular reflex pathways. Approaches ranged from behavioral assays to single neuron recordings to neural manifolds. Model systems ranged from fruit flies to mice to primates. There was a wide range of interests represented this year (Figure 1). Here we present a brief summary of the meeting, highlighting talks that generated extended discussion and posters that featured a novel approach or question. We focus on three themes that emerged across many sessions: learning, motor control of hand movements and holding still, and the advantages of a neural population activity perspective.

Figure 1.

Self-reported interests of NCM meeting attendees

The roles of proprioception and vision in perception and action

This year’s NCM began with a satellite session centered on sensory control of movement, which covered proprioception, vision, the combination of proprioception and vision, and the clinical application of basic knowledge on these topics in a variety of disorders.

Jonathan Cole (Bournemouth University, England) and Fabrice Sarlegna (Aix-Marseille University, France) talked about the importance of proprioception using the dramatic example of deafferented patients. Cole presented a number of studies he did with Ian Waterman, a patient who lost all proprioception and touch below level C3, but still had intact motor function (Cole, 1995). He also retained temperature, pain and fatigue sensation. Remarkably, Mr. Waterman soon realized that he could control his limbs based on vision, learning to walk and hold heavy objects in the absence of proprioceptive or tactile feedback. He can even perform such challenging tasks as matching the force exerted with both hands, in the presence of visual feedback (Cole and Sedgwick, 1992). He can also perceive weight differences, most likely by detecting the level of muscle fatigue. An alternative explanation, which Cole and his collaborators have not been able to rule out, is that he is detecting changes in central drive. Interestingly, Waterman is better at purely visually-guided motor tasks than control subjects. For example, he outperformed controls during a mirror drawing task, probably because he did not suffer from incongruent visual and proprioceptive input (Miall and Cole, 2007). Besides residual sensation and visual feedback, he seems to rely heavily on cognitive strategies for moving and interacting with the environment. Accordingly, his ability to perform rhythmic movements degrades when asked to simultaneously perform another attention-demanding task (Lajoie et al. 1996). Cole’s talk highlighted how the study of deafferented patients is a very powerful way to examine the role of sensory pathways in motor control and motor learning.

Additional proof of the integral role of sensory pathways in motor learning was presented by Sarlegna. He focused his talk on motor learning in deafferented patients. It had been shown previously that visual feedback is not necessary to learn new dynamics in a force field task (Franklin et al., 2007), but Sarlegna asked whether proprioceptive feedback is needed (Sarlegna et al., 2010). Participants sat on a rotating platform and were required to reach to visual targets while a novel Coriolis force perturbed their movements. The deafferented patient learned the task as well as controls, showing similar time-course and magnitude of adaptation. This shows that proprioceptive input helps improve movement quality, but is not critical for sensorimotor adaptation. Indeed, proprioception may even be detrimental in the case of incongruent feedback, as shown by Cole. There still remain open questions about how different sensory pathways interact during motor learning.

Simon Gandevia (Prince of Wales Medical Research Institute, Australia) continued the discussion of the role of sensory input for the control of movement. First, he summarized how the perceived position of the wrist during anesthesia and ischemia changes depending on the voluntary command (Gandevia et al., 2006). Then, he introduced the mechanisms underlying different proprioceptive illusions. He described how for the popular rubber hand illusion, anesthetizing cutaneous receptors from different combinations of fingers does not eliminate the sense of ownership; vision and proprioception suffice (Walsh et al., 2011). Gandevia also presented a new “plastic finger illusion” that allowed his group to study the role of proprioception and touch without vision (Héroux et al., 2013). They use a screen to remove visual input, and couple the subject’s index finger to an artificial finger positioned above it. With this illusion, they confirmed previous results from other groups showing that vision is not necessary for body ownership. They also found that congruent afferent muscle signals (between the two hands) give a strong feeling of body ownership. This feeling of ownership became stronger once touch and joint receptors were anesthetized, possibly because they decreased incongruent feedback. Importantly, this result indicates that one channel of somatic information (e.g., vision or touch) is enough for embodiment, in contrast to classic illusions that manipulate the correlation between sensory modalities. Therefore, the combination of different sensory modalities with regards to body ownership is more flexible than traditionally thought, and there remain open questions about how a feeling of limb ownership is generated by the brain.

Continuing with the theme of the interaction between sensory modalities, several talks discussed the role of different visual pathways in guiding reaching and grasping movements, and how vision and proprioception are integrated. Stephen Scott (Queen’s University, Canada) presented a series of studies aimed at understanding the relative contributions of visual and proprioceptive inputs from the perspective of optimal feedback control. He recorded EMG activity while subjects performed a reaching task in a virtual reality environment with their arm supported by a robotic exoskeleton. Some trials included a random mechanical perturbation of the arm. He focused on the EMG activity evoked by this perturbation. He separated the activity into short (<50 ms) and long-latency (50–100 ms) reflex responses, and a voluntary period (after 100 ms), during which visuomotor responses play a role. He observed that when we move fast, during the first part of a reach, visual information does not contribute significantly; it only matters as the target is approached. Notably, from a control theory standpoint, sensory inputs with different delays, like vison and proprioception, pose a big challenge, because they are hard to integrate. But Scott’s group has proposed a way by which the brain may solve this challenge. They have shown that dynamic Bayesian models not only account for error size, but also sensory delays (Crevecoeur, Munoz, and Scott, 2016).

An example of clinical application was given by Sean Dukelow (University of Calgary, Canada), who talked about loss of proprioception after stroke, and the neural mechanisms underlying recovery. Dukelow first presented a method to assess proprioception based on a position matching task that overcomes the lack of accuracy of classic clinical scales (Dukelow et al., 2010). In this method, a robotic exoskeleton moves the patient’s affected arm to a location in space, then the patient has to mirror the position with the unaffected arm. Using this and other robotic tests in a longitudinal study with over a hundred patients, Dukelow and colleagues showed that the time-course of motor and sensory recovery are different (Semrau, et al., 2015a). This is an important consideration for the design of rehabilitation programs. To further understand recovery after stroke, they studied the relationship between visuospatial and kinesthetic (sense of limb motion, a component of proprioception) deficits (Semrau, et al., 2015b). They found that visuospatial deficits are largely predictive of kinesthetic deficits, whereas the opposite is not true: the presence of kinesthetic deficits does not necessarily imply impaired visuospatial function. The work by Dukelow and colleagues provides an example of how basic research on sensory and sensorimotor integration can aid the development of rehabilitation therapies. It also highlights one of the recurring themes of the satellite meeting: the need to study the integration of different modalities when trying to understand the sensory control of movement.

Learning

Two core themes were present across many of the presentations on learning at the meeting: the integration of multiple signals in learning, and learning novel motor patterns that compete with previously well-established actions.

The theme of multisensory integration in learning was prominent in a talk by Philip Sabes (University of California San Francisco, USA), who presented data on learning a de novo mapping using both traditional actions and artificial sensory feedback (Dadarlat, et. al., 2015). Macaque monkeys learned to interact with a visual display under two conditions: by performing reaching actions with their arm or by using sensory information from novel signals provided by intracortical microstimulation (ICMS) of primary somatosensory cortex. Notably, while learning occurred in both conditions, performance was best when both arm movements and ICMS were used simultaneously. Thus, integration of the natural and artificial information allowed the greatest level of performance. Modeling indicated that the ICMS signal was optimally combined with vision, suggesting that the brain utilized the novel learning signal in a manner consistent with natural sensory feedback signals such as proprioception.

In a similar vein, Megan Thompson (University of California San Francisco, USA) examined the ability of participants to combine motor and proprioceptive information to acquire a novel audiomotor map for producing vowel sounds. A complete array of vowel sounds was mapped to different physical locations on a touch screen. Participants interacted with the touch screen display, receiving auditory feedback of the map by pressing in different locations. Notably, once this novel mapping had been acquired, the learned behavior showed similar behavioral properties to vocal sound production, including generalization of the learned map to produce untrained, novel sounds, and adaptation of the mapping in response to the introduction of perturbations. Although the mapping involved learning to produce vowel sounds using the arm, MEG data indicated changes occurring in areas involved in vocal speech production following acquisition of the new mapping. These data indicate a potential overlap between the newly acquired audiomotor mapping for using the tablet and the existing mapping used for vocal speech production.

A standout poster integrated data from two learning tasks to probe for multimodal signatures of motor learning. Adam Steel (University of Oxford, England) used a serial response time task and a visuomotor force tracking task to probe the effects of reward and punishment on functional connectivity. Data indicate that reward increases reliance on cerebellar-premotor networks, while punishment increases premotor-hippocampal connectivity, but impairs caudate activity related to retention. Critically, integrating data from multiple learning sources provided results with less idiosyncratic, task specific activity, and were therefore more reflective of the effects of motor learning. Thus, while the previous talks had focused on learning through multiple sources of sensory information, this poster inverted the concept by integrating multiple sources of information to enhance our understanding of the learning process. Together these presentations highlighted the importance of signal integration not only in the process of learning, but also in the study of the learning process itself.

Around the theme of learning new movement patterns that compete with established actions, Max Donelan (Simon Fraser University, Canada) presented data examining participants’ abilities to adapt their walking gait to minimize energy expenditure. While it is well established that people prefer walking patterns that are energetically optimal, Donelan asked whether this process is the result of extensive previous experience accumulated over one’s lifetime, or whether the optimization of gait can occur continuously over a short timescale. Participants walked on a treadmill while robotic exoskeletons applied torques at the knee to either increase or decrease the energetically optimal step frequency. Brief exposure to the perturbation alone did not result in adaptation for most participants. However, when participants first completed a condition in which they explored the new energetic landscape imposed by the exoskeletons, they were later able to rapidly settle into the new energetically optimal gait pattern. Surprisingly, participants showed such adaptation even when it resulted in relatively minimal improvements in energy expenditure (Selinger and Donelan, 2015). Notably, such short-term optimization could be captured when conceptualized using a reinforcement learning model; participants select and execute a minimum cost step frequency from a predicted cost landscape, then iteratively compare and update the predicted cost to the measured cost. Using such an approach, the inherent variability in the predicted and measured costs of walking allows a local search strategy to converge on an energetically optimal gait within hundreds of steps.

The effects of prior practice on learning and performance were further explored in a talk that examined the acquisition of novel stimulus-response associations over multiple weeks of training. Robert Hardwick (Johns Hopkins University, USA) examined the relationship between skill acquisition and habit formation in motor learning. Participants were required to learn an original visuomotor mapping between visual stimuli and button press responses. Skill in the task was measured by examining the speed- accuracy trade-off between reaction time and response accuracy. Once this original mapping had been acquired, specific stimulus-response relationships were revised; this allowed assessment of both the level of skill participants had in the task (when responding to stimuli that had consistent responses between the two mappings), and the degree to which learning the original associations led to habitual errors in performance (when responding to stimuli that were revised between the two mappings). Data indicated that both skill acquisition and the likelihood of committing habitual errors increased with practice, being greatest when participants practiced over a month-long timescale. However, both the changes in skill acquisition and habit expression could be captured using a model that considered only the level of skill participants had acquired in the task, with no need for additional parameters to explain habit strength. These data indicate that while skill increased continuously with practice, habits could be considered as an all-or-nothing process, the magnitude of expression of which was skill-dependent.

Similarly, a poster by Nicola Popp (University of Western Ontario, Canada) examined the effects of chunking sequence acquisition. In the first two days of the experiment, participants learned two or three item chunks, and then on later days combined these chunks into longer 11 press sequences. Notably, even after three weeks of training, participants were still faster to initiate the component chunks than they were in transitioning between the different chunks that comprised longer sequences, indicating that the structure of the initial training had long term implications for subsequent performance.

Together, these presentations emphasized the approach of studying how prior practice (over the course of multiple sessions of learning or over the course of a lifetime) affects subsequent learning. The identification and study of habitual behaviors (such as failing to move away from an established walking pattern once it becomes less efficient, or producing hand movements in a stereotyped way), and how they can be overcome (through the introduction of increased variability, or through exploration) provides an interesting avenue for future research.

Relatively few studies have examined the neural mechanisms underlying motor learning. Matthew Perich (Northwestern University, USA) presented evidence for a novel mechanism that explains how monkeys rapidly adapt their behavior during a classic force field adaptation center-out reaching task (Shadmehr and Mussa-Ivaldi, 1994). Using simultaneous population recordings from monkey dorsal premotor (PMd) and primary motor (M1) cortices, he showed that during motor adaptation there are no changes in functional connectivity within these areas, although there was an apparent change in the PMd- M1 mapping. To understand this change, he separated the PMd activity into “potent components” that capture PMd outputs to M1, and “null components,” which reflect PMd changes that do not directly affect M1 (Kaufman et al., 2014). Interestingly, the mapping between PMd potent components and M1 activity remained unaltered during adaptation, whereas the mapping between PMd null components and M1 changed with a time course similar to that of behavior. Together with the lack of functional changes within PMd and M1, the preserved PMd potent to M1 mapping rules out neural plasticity as an adaptation mechanism. The intriguing changes in the relationship between the PMd null components and M1 activity that paralleled behavioral adaptation indicate that PMd recruits M1 in a new way to compensate for the altered limb dynamics. Remarkably, when Perich and colleagues instead considered a classic visuomotor rotation paradigm (Krakauer et al., 1999; 2000), they saw no functional changes within or between areas, in agreement with evidence that adaptation recruits upstream regions of the brain (Tanaka, et al., 2009). Thus, this work proposes a novel learning mechanism in which neural population activity explores existing activity patterns to quickly adapt to new task demands (Perich et al., 2017).

In her poster, Mackenzie Mathis (Harvard University, USA) shed further light into the neural structures involved in motor learning during the same force field task (Mathis et al., 2017). First, she replicated the classic force field reaching paradigm in mice. She trained mice to perform a one target center out reaching task, and showed that like monkeys and humans, mice can also adapt their behavior to successfully perform this task. By fitting several standard motor learning models to the data, Mathis found that learning was best explained by sensory prediction errors. To mechanistically test this observation, in another set of experiments they photoinhibited the forelimb area of the primary somatosensory cortex, and saw that mice could no longer adapt their motor commands and reduce errors. Remarkably, S1 photoinhibition did not impair basic movements or post-perturbation performance of the motor task, suggesting that S1 is critically involved in motor adaptation. This study, along with Perich’s, highlights that monitoring or manipulating neural activity during carefully designed behavioral tasks has the potential to illuminate new neural mechanisms, and thus lead to a better understanding of how the brain controls movement.

Diana Mitchell and Kathleen Cullen (McGill University, Canada) described their work aimed at understanding how the brain adapts to a vestibular neuroprosthesis. Current vestibular neuroprostheses consist of gyroscopes that measure orientation and a stimulator that applies frequency-modulated stimulation to the vestibular nerve. These neuroprostheses replicate the main effects of natural activation of the vestibular pathways (Cullen, 2012), namely postural and gaze stabilization by eliciting vestibulo-spinal and vestibulo-ocular reflexes, respectively. However, it is well-established that the gain of the vestibulo-ocular reflex elicited with the neuroprosthesis is lower than natural values. Thus, they sought to understand whether this difference was due to plastic changes following long-term stimulation. They found that the probability of detecting spontaneous activity decreased as stimulation frequency increased. A detailed examination of the underlying mechanisms revealed that there are no changes in the efficacy of any particular pathway, but rather the response of indirect inhibitory neurons increases. Interestingly, they observed that natural active motion accelerated the recovery of normal function of the afferent pathways, compared to when the subject remained stationary, perhaps because of ongoing efferent drive. This work provides a good example of the need to combine basic science and neural engineering methods to build and refine neuroprostheses to restore lost neurological functions.

Recovering dexterous hand movements

The recovery of hand function after CNS injury likely takes advantage of the redundancy of multiple pathways that project to the spinal cord. This session explored the pathways involved in hand control in mice, monkeys, and humans. Robert Brownstone (University College London, England) reviewed the role of inputs from cutaneous low-threshold mechanoreceptors on grasping behavior in mice. He showed that dI3 interneurons receive low threshold sensory inputs from primary afferents involved in the cutaneous control of grasping and project to spinal motoneurons (Bui et al., 2016). The ability to maintain grip in response to changing or increasing load is lost after genetic silencing of dI3 interneurons. He then proposed that dI3 interneurons comprise a spinal circuit involved in grasp that receives predictive and instructive inputs and function as comparator neurons. This circuit is involved in short term adaptation as well as long-term plasticity. This shows that spinal circuits do not simply relay information from other circuits, but are critical for normal motor function and for inducing plasticity in motor microcircuits following injury.

Then, Tadashi Isa (Kyoto University, Japan) discussed how monkeys recover hand dexterity following partial spinal cord injury. Recovery of precision grip in non-human primates takes one month following a pyramidal lesion in SCI (C4/C5 lesion). As long as a small number of corticospinal tract fibers remain intact the monkey can control movements fairly well. Pyramidal tract neurons are necessary for the initiation of recovery. The ipsilateral motor cortex has a contribution in the early recovery stage, while the premotor cortex contributes during the late stage. During recovery, the nucleus accumbens, which regulates motivation-driven effort, is directly involved in the control of finger movements (Sawada et al., 2015). These pathways each make a different contribution to the reacquisition of dexterity following a spinal cord injury. Stuart Baker (Newcastle University, England) focused on the reticulospinal pathway. He countered a number of misconceptions and showed that the reticulospinal pathway makes mono- and di-synaptic connections to motoneurons. In addition, both distal and proximal motoneurons receive reticulospinal input. There is significant convergence of the corticospinal and reticulospinal pathways, in particular as related to reaching and grasping. He suggested that functional recovery from stroke or spinal cord injury could be the strengthening of reticulospinal connections. The imbalance in reticulospinal functional plasticity might also explain the imbalance observed in patients who can hold things but have trouble releasing them (Zaaimi et al., 2012). Monica Perez (University of Miami, USA) brought the session full circle and showed similar convergence of corticospinal and reticulospinal involvement in recovery in human spinal cord injury patients. She showed altered corticospinal and intracortical activity during precision and power grasping, with contributions from the reticulospinal during the power grasp. Understanding the contributions of these pathways to hand control is critical to the rehabilitation goal of recovering of hand function.

Towards recovering hand function, Ed Hodkin, Andrew Jackson and colleagues (Newcastle University, England) presented an interesting low-cost functional electrical stimulation (FES) system to restore hand function to stroke and spinal cord injury patients. The system is designed to induce Hebbian plasticity (Hebb, 1949), by associating the patient’s motor intent and a grasp movement assisted with FES (Ethier, et al., 2015; Jackson and Zimmermann, 2012). The system operates by stimulating two muscles, allowing the patient to grasp a manipulandum and move it along a rail. Hand position is detected with proximity sensors. Preliminary results in two stroke patients show a significantly greater improvement in the Action Research Arm Test (ARAT) for the treated arm when compared to the non-treated one, suggesting the clinical potential of this type of interventions.

As highlighted in the satellite session, dexterous object manipulation requires not only a sophisticated motor system but also sensory feedback. Sliman Bensmaia (University of Chicago, USA) explored this in healthy monkeys and Sharlene Flesher (University of Pittsburgh, USA) explored our ability to restore sensory feedback in a person with spinal cord injury. Bensmaia investigated hand proprioception in cortex by simultaneously recording time-varying joint kinematics of the hand and neural activity from primary somatosensory cortex (S1) while a monkey performed a range of natural grasping movements. He found that individual S1 neurons encode the posture and movement of the hand distributed over multiple joints spanning the entire hand. This is consistent with the hypothesis that sensorimotor cortices represent whole-hand configurations rather than individual joints or digits. However, they found no relationship between the proprioceptive fields and kinematic synergies, implying a greater level of complexity. Finally, Flesher applied our understanding of tactile percepts in the hand to a neuroprosthesis. While brain-computer interfaces (BCIs) have enabled users to have high degree of freedom control of a prosthesis or robotic arm (Bensmaia and Miller 2014), object manipulation could benefit from tactile sensory feedback. Flesher described pioneering work in which she used intracortical microstimulation (ICMS) of the primary somatosensory cortex to evoke such tactile percepts in a person with spinal cord injury (Flesher et al., 2016). She showed that ICMS could elicit spatially selective tactile percepts that are graded over a range of perceived intensities, and could elicit sensations like pressure and touch. As the satellite meeting discussed at length, restoring tactile feedback in this way is critical for natural control of movements and could improve a BCI user’s control and experience with a prosthetic device.

On the topic of cortical representations of the hand, a poster by Daan Wesselink (University College London, England) and colleagues showed that digit representations in primary sensory cortex are not influenced by experience. This group had previously shown with fMRI data that single digit representations in S1 are preserved even 25 years after amputation (Kikkert et al., 2016). Here, they presented data indicating that congenital amputees do not have digit representations. Therefore, sensory experience is not necessary to maintain a sensory representation but is critical to establish it. Moreover, they found a positive correlation between the separability of single digit representations in S1 and M1. When they examined the relationship between phantom experience and digit representations, they found no correlation between the separability of digit representations in S1 and phantom sensation. In contrast, there was a positive correlation between the subject’s motor control of the phantom and the separability and structure of the single digit representations. This group’s work challenges the accepted notion that adjacent representations take over the missing limb’s representation after amputation, highlighting that much is yet to be learned about the basic principles of neural plasticity.

Distinct neural circuits for control of movement vs. holding still

While most of the meeting focused on the neural control of movement, as one might expect, one session highlighted the neural circuits involved in holding still. While one might consider holding still to “come for free,” Reza Shadmehr (Johns Hopkins University, USA), Yifat Prut (The Hebrew University of Jerusalem, Israel) and Robert Scheidt (Marquette University, USA) discussed the distinct neural circuits that are involved in movement and holding still, in the ocular motor system as well as arm reaches.

Shadmehr started by reviewing evidence that saccades are controlled by two different brainstem circuits: one that makes the eyes move, and another circuit, sitting downstream, that integrates this activity during the hold period. Inspired by this observation, he asked whether a similar separation of control exists for moving the arm (Shadmehr, 2017). It is well established, based on anatomical tracing and ICMS studies, that primary motor cortex is divided into two regions: a rostral region (old M1), and a caudal region (new M1). In a classic study, Crammond and Kalaska (Crammond and Kalaska, 1996) showed that caudal M1 neurons exhibit a phasic burst of activity during movement, followed by a tonic discharge during posture. In contrast, rostral M1 neurons lack the tonic activity during the hold. Most corticomotoneuronal cells, cortical neurons that synapse directly onto motoneurons, are located in caudal M1 (Rathelot and Strick, 2009). Therefore, the electrophysiological differences are paralleled by neuroanatomical differences. A consequence of these observations is that caudal M1 neurons would need a downstream integrator circuit in order to be able to hold a posture. Interestingly, Bizzi and colleagues (Giszter et al., 1993) found evidence of what could be a spinal integrator in frogs, as there were interneurons that when stimulated caused sustained force.

Further evidence of a neural integrator in the corticospinal system was presented by Prut. During an isometric task with a hold phase, the activity patterns of M1 neurons are more phasic than the activity patterns of spinal neurons (Shalit et al., 2012), which suggests the existence of a subcortical integrator. They investigated whether this integration happens because of the termination patterns of corticospinal fibers, which may help shape the descending commands (Zinger et al., 2013). By delivering microstimulation to M1, they found that sites that projected to finger muscles were less likely to be connected to spinal interneurons than sites that projected to wrist muscles. Therefore, there are finger- related sites whose activity is less likely to be processed by the rich interneuron circuitry. This observation is consistent with the emergence of the corticomotoneuronal pathway for the control of dexterous finger movements. Prut ended by presenting ongoing work trying to understand the role of cerebellar input in shaping the motor command. Her group has found that the cerebellar input acts as a differentiator, and that this effect is maximal in caudal M1 and deeper layers, hinting at another difference between moving and holding still. Together this session showed that across multiple modalities of motor control, circuits that are responsible for producing a burst of activity characteristic of movement commands are distinct from those circuits that produce maintained activity characteristic of holding still

A new focus on neural population activity

An increasing number of studies on many species across different brain areas have begun to move away from single neuron level analyses to focus on population level analyses. These studies reveal that the activity of populations of neurons can be described by a surprisingly small number of neural covariation patterns (Cunningham and Yu, 2014; Gallego et al., 2017). Mathematically, these covariation patterns define a “neural manifold” (Sadtler et al., 2014; Stopfer et al., 2003), a low- dimensional subspace within the high-dimensional “neural state space” in which each dimension represents the activity of one recorded neuron. The structure of the neural manifold, how it emerges, and how the temporal evolution of the population activity within it (the “neural population dynamics”) relates to behavior was the cornerstone of many posters and talks, and a session entitled “From single neurons to neural manifolds.”

The session began with an elegant theoretical argument by Surya Ganguli (Stanford University, USA) describing how the notion of a neural manifold arises when studying neural population activity during a task, and how the dimensionality of such a manifold is determined by the complexity of the behavioral task (Gao and Ganguli 2015). Because of this, recording from more neurons does not add any information; one instead needs to focus on more complex tasks. He emphasized the need to study neural population dynamics instead of single neuron activity as key to understanding how the brain works, stating that “single neurons are mere shadows of the population dynamics.”

Juan Gallego and colleagues (Northwestern University, USA) investigated the relationship between neural manifolds in M1 during different motor tasks. They hypothesized that if motor behavior is generated “from the manifold,” there should be similarities between task-specific manifolds, even though the activity patterns of single neurons change quite drastically across tasks (see, for example Sergio et al., 2005). Indeed, Gallego showed that the low-dimensional neural manifolds identified during different tasks have very similar orientations in high-dimensional neural space. Upon further examination, he found that population dynamics along some of the manifold dimensions were virtually identical across different tasks, despite the differences between single neuron activity patterns. Moreover, some of these task-independent dimensions captured a constant mapping to muscle commands. These observations support arguments that the neural manifold represents a relevant neural mechanism, and suggest that M1 may cause movement during different behaviors by combining the covariation patterns that span the manifold. To further explore neural manifolds in M1, Emily Oby (University of Pittsburgh, USA) used a brain-computer interface paradigm to explore how a neural manifold constrains learning. She created BCI mappings for a monkey to learn that required unique neural activity patterns either within the neural manifold or outside the neural manifold. Previously, her group showed that the neural manifold shapes learning: monkeys could learn BCI mappings that lie with the manifold, conforming to existing patterns of neural covariation, but usually could not learn to generate novel neural covariation patterns outside the neural manifold (Sadtler et al., 2014). However, she showed that given more time, monkeys can learn outside manifold BCI mappings, and they do so by generating novel covariation patterns, outside the neural manifold. Here the perspective of the neural manifold helps understand why some behaviors are more difficult to learn than others.

Next, Christian Machens (Champalimaud Centre for the Unknown, Portugal) presented a new method for identifying neural manifolds whose dimensions relate to a relevant behavioral variable through a linear readout. This method is called demixed principal component analysis (dPCA) (Kobak et al., 2016). To showcase the application of this method, he showed that, during a decision-making task (Romo et al., 1999), population activity in monkey prefrontal cortex is mostly condition-independent, and that there are some manifold dimensions along which population dynamics depend on the stimulus and other manifold dimensions along which population dynamics depend on the decision. To highlight the difference between neural manifolds in motor areas and sensory areas, Marius Pachitariu (University College London, England) discussed population level activity in mouse visual cortex. In this study, he had an impressive dataset consisting of ~10,000 neurons simultaneously recorded with calcium imaging during different conditions (Pachitariu et al., 2017). Using these recordings, he first showed that the neural manifold capturing population responses to spontaneous activity has a few hundred dimensions, far more that the few tens normally described in more constrained experiments. When trying to characterize the neural manifold in response to ~3,000 stimuli, linear dimensionality reduction methods such as principal component analysis failed. In contrast, when he applied a nonlinear transformation to the neural activity, he could identify a non-linear manifold underlying the population responses. Interestingly, the spontaneous and response manifolds were almost perpendicular to each other. Through simultaneous recordings with ~1,000 contact “Neuropixel” probes and detailed measures of arousal (pupil dilation), he was able to show that a lot of presumed spontaneous activity is explained by the mouse’s arousal level. This study raised interesting questions about the dimensionality of neural manifolds in different brain areas, their geometry as a function of the complexity of the behavior or the task parameters, and the methods that will be needed for future research.

In addition to the team session, there were several interesting posters and talks that focused on population dynamics in the neural manifold. Mehrdad Jazayeri (Massachusetts Institute of Technology, USA) examined how neural population dynamics of the medial frontal cortex (supplementary eye field, supplementary motor area, and pre-supplementary motor area) can encode timing (Wang et al., 2017). Primates were trained to produce movements with both differing time intervals and differing effectors. While individual neurons had heterogeneous activity, stretching or shortening their response profiles in accordance with the observed interval revealed an invariant pattern of activity. Moreover, speed in this scaling subspace could predict behavior; the timing between actions could be encoded as the speed at which neural population dynamics unfolded over time. These data indicate that the temporal control of voluntary action is achieved via flexible control of the speed of cortical population dynamics along invariant neural trajectories.

Abigail Russo (Columbia University, USA) presented work that further explored the view that movement generation in motor cortex results from the evolution of neural population dynamics that produce muscle-like commands (Churchland et al., 2012; Shenoy et al., 2013). In particular, she hypothesized that neural activity in motor cortex defines a smooth trajectory, in agreement with Ganguli’s theory of neural dimensionality (Gao and Ganguli, 2015). She quantified smoothness in terms of the tangling of the trajectories describing the neural population dynamics, and tested whether such trajectories in motor cortex exhibit tangling. Notably, she found that neural population dynamics in motor cortex are not very tangled when monkeys reach or pedal a “hand-actuated bicycle,” and also when mice reach. In contrast, neural population dynamics in V1 exhibit a much greater tangling, which points at interesting differences between sensory and motor areas. Finally, even though muscle activity (EMG) is consistently more tangled than motor cortical activity, linear readouts of the neural population dynamics provide a good description of EMG.

In a poster, Jonathan Michaels (German Primate Center, Germany) and colleagues presented a simulation of cortical activity in three areas that are critically involved in the planning and execution of grasping movements: the anterior intraparietal area (AIP), ventral premotor cortex (F5), and primary motor cortex (M1). They first trained a convolutional neural network (CNN) to extract relevant features about 49 objects that monkeys grasped (Schaffelhofer and Scherberger, 2016). They then used these features as inputs to a recurrent neural network (RNN) with three sparsely connected modules, each representing one of the three cortical areas, which was trained to reproduce the recorded limb kinematics (Michaels et al., 2016). The RNN was able to reproduce single trial kinematic data, with the generated activity patterns qualitatively resembling recorded neuron activity patterns. Furthermore, the structure of the neural population dynamics of the RNN was strikingly similar to the population data. This poster presented a powerful simulation tool that can be applied to many studies of neural population dynamics. Together with the work presented by Jazayeri, Russo, and other groups, their findings corroborate seminal work by Sussillo (Sussillo et al., 2015) showing that RNNs reproduce neural population activity during laboratory tasks quite well.

Andrew Jackson (Newcastle University, England) presented his work comparing neural population activity during sleep, voluntary movement, and while learning a new skill consisting of a BCI task. Building on previous work from his group (Hall et al., 2014), he showed that the way neurons covary with respect to each other and their sequential activation is preserved between voluntary movement and sleep. This implies that the neural manifold is preserved across these two conditions. He then trained monkeys to learn abstract BCI paradigms in which they had to modify the synchronization or sequential activation of pairs of neurons. Remarkably, monkeys learned these paradigms by exploring a neural manifold that was very similar to the existing sleep manifold. Therefore, new neural strategies reflect extrinsic rules and the intrinsic constraints captured by the neural manifold, similar to what had been reported in (Sadtler et al., 2014).

In summary, the neural manifold framework furthers the understanding of the neural control of movement, sensory processing, and motor learning. Although it is an ongoing and lively debate in the community whether the manifold represents a real physiological entity, these studies highlight the potential of the framework, as most of these observations could not have been made analyzing only single neuron activity.

Take home messages

A key take-home message is that meetings like these are crucial for opening the communication and facilitating conversation between experimentalists that work on different model systems, and theorists as well, trying to address similar questions in motor control. During a discussion after one session this was explicitly brought to the forefront with the question “how can mice researchers and monkey researchers communicate and learn from one another?” The answer came in two parts: 1. Monkey researchers need better promoters for monkey genetics. Without those tools, the kind of intricate, cell specific experiments that are possible in mice are impossible in monkeys. 2. Alternatively, researchers that use a mouse model need better, more complex behavioral assays. It is also important to remark, as Roger Lemon did in his plenary talk, that primates have pathways that make them unique as model animals. For example, corticomotoneuronal projections, monosynaptic connections between motor cortical neurons and spinal motoneurons, are very important for dexterous function, and are only present in more evolved primates (Lemon, 2008). To this end, a few sessions brought together animal and human researchers, and tried to emphasize how findings in each model can inform the other. We advocate that increasing interactions and collaborations between people working in these related but somewhat separate areas is critical for the advance of the field.