What is the biological basis of sensorimotor integration?

Abstract

This Prospects presents the problems that must be solved by the vertebrate nervous system in the process of sensorimotor integration and motor control. The concepts of efference copy and inverse model are defined, and multiple biological mechanisms are described, including those that form the basis of integration, extrapolation, and comparison/cancellation operations. Open questions for future research include the biological basis of continuous and distributed vs. modular control, and somatosensory-motor coordination.

Introduction

Over the last few decades, a great deal of research effort has been devoted to gaining a better understanding of the neural control of movement. In the 1970s, engineering control systems approaches allowed physiologists to get a handle on processes involving feedback, such as smooth pursuit, where the input is movement (slip) of a target across the retina and the output is an eye movement that negates this retinal slip. In the 1980s and 1990s, these engineering advances in the oculomotor field gradually evolved into a nearly complete understanding of saccadic eye movements. This was accomplished by incorporating, into what originally began as an engineering approach, neural recordings from subcortical structures, and neurobiological concepts such as the alignment of sensory and motor maps in the superior colliculus. However, at the turn of the century, decade 2000 seemed to host a redirection of research focus, away from basic biological mechanisms and toward issues such as which sensorimotor correspondences can be learned and remembered, and how best to decode neural information for prosthetic applications.

While learning is always a topic of interest, and it is important to translate neurobiological research results into advances in prosthetics, the purpose of this opinion paper is to return attention to questions about biological mechanism. First, I will list the basic physiological processes that must be implemented by the nervous system in sensoirmotor integration. Then, I will show that these processes can be best understood in light of the types of biological operations that are known to be available to the system. I will give examples of these biological operations and then I will give examples of lines of investigation that, in my opinion, have not helped in answering questions about the neurobiology of sensoriomotor integration. Finally I will suggest more fruitful approaches for the future.

Problems that must be solved by the nervous system

Interpreting sensory targets

All sensory input is filtered by the nervous system according to its destination and use. If the use is perception, the filtering may take the form of feature extraction and combination of inputs from different sensory sources, such as the integration of visual and auditory information. When sensory input is instead used to instruct the generation of motor commands, it must be interpreted in terms of the current state of the motor system. For example, when the visual system provides a target position that requires an arm movement, the motor command must be appropriate for the hand’s initial position. Obviously, to move the hand in the mid-sagittal plane, motor commands and muscle activations would differ depending on whether the hand began in front of, or beyond, the visual target. Therefore the first step in the process must be to re-express the neural representation of the target state, in terms of the difference between the current state and the target state (Fig 1A).

Fig 1.

A diagram of the general process of transforming target information into motor commands for movements. A) The current state of the system is subtracted from the target state to create a motor error, or desired movement signal, which is then used to produce a motor command to the muscles. B) Efference copy is used to keep track of the expected result of the motor command, and this is used to update the current state. C) Delayed sensory feedback is used to update the association (box) between the desired movement signal and the motor command.

Employing efference copy

The needed representation of current state could potentially be provided by sensory systems. But using sensory input sometimes takes too much time. Therefore, to keep track of the current state, the system appears to use a form of internal monitoring called efference copy (Fig 1B). Efference copy simply means an internal copy of outgoing information. It is a term for the idea that the motor system forms a representation of its own current state, here used as a baseline upon which to add the appropriate new motor command to move in the direction of the new target state (Fig 1A). This classic concept was fully described by von Holst and Mittelstadt (1950) in relation to a wide variety of motor behaviors, ranging from orienting movements in fish and insects, to spinal reflexes in mammals.

Using memory as a filter

Memories may be old ones that have been previously stored by synaptic modifications, or new ones that are continuously updated and maintained by physiological activity. Sensory feedback can be used to modify the old memories, thus forming new memories. For example, an incorrect estimation of the mechanical properties of a hand-held object is sensed and then corrected, and the corrected mechanical properties are stored in memory. As indicated in Fig 1C, the correction may be based on a comparison between the sensory state that is associated with the efference copy of the motor command (Fig 1B) and the true sensory state, as indicated by the delayed sensory feedback (dashed arrows).

Producing motor commands

Thus, as diagramed in Fig 1 (thick solid arrows), subtracting the current state from the target state, and filtering this signal through memory, results in the motor commands that give rise to muscle activation, force and movement. This filtering is essentially a transformation from sensory (movement) parameters to motor (force) parameters, the inverse of the causal relation of force to movement, and the memory filter has been referred to as the “inverse model.” As mentioned above, there has been a great deal of research on what sensorimotor correspondences can be stored in the inverse model. There has also been an ongoing debate about how the motor command maps into muscle activity. Before discussing these issues, we will first consider some of the biological operations that are available to the system to solve problems like these.

Neural filtering mechanisms

Feature extraction

The series of neuronal activations/inhibitions that is associated with sensation can be characterized as a process of feature extraction. For example, in the visual system, neurons may be connected to one another in a manner that makes them particularly sensitive to certain features of the visual image. As cartooned in Fig 2A, the neural network, through genetic and activity dependent mechanisms, may establish weighting coefficients (a, b, c) to accentuate certain features of the input (the circle) and downplay others (e.g., the rectangles). Importantly, this simple example highlights the fact that most neural operations are accomplished through the distributed activity of groups of interconnected neurons, with little evidence of distinct modularity.

Fig 2.

Two types of biological filtering operations: A) feature extraction, and B) extrapolation of a smoothly changing sensory signal forward in time. In each case, the input/output associations are made by establishing weighting coefficients (a-f) at individual synapses. The learning might also involve intrinsic cell properties such as spike frequency adaptation (SFA) and short-term depression (STD).

Integrating, differentiating and extrapolating

Other key features of neural processing are phenomena similar to low-pass and high-pass filtering. When action potentials are recorded from neurons in the sensorimotor system, these spike trains are often characterized as tonic or phasic, and the neurons are then classified as slowly adapting (low-pass) or rapidly adapting (high-pass), respectively. A neuron that responds to a transient input with a sustained discharge can form the basis for neural integration. Conversely, a neuron can be regarded as a high-pass filter if it responds transiently to a change in the level of a maintained input; this neuron may be sensitive to the time derivative of a signal (such as velocity rather than position).

In the vestibular sensory system, there has been some debate about whether neural integration is a cellular phenomenon or a network function or both. The vestibular semicircular canals use fluid inertia and tissue stiffness to create a sensory signal proportional to head rotational velocity. Thus the sensory input from the semicircular canals is transient during maintained rotations. However, a network of neurons involving the vestibular nucleus integrates its head velocity-related inputs to provide the a signal used to control eye-in-head position (Cannon et al. 1983). A recent experimental and modeling study indicated that this mathematical integration is accomplished through a combination of several biological mechanisms (Aksay et al. 2007). First, consistent with the Robinson model (Arnold and Robinson 1991), the system appears to take advantage of the push/pull opponency between left and right ear inputs to form a bilateral positive feedback system based on inhibition. Furthermore, there is evidence for recurrent network activity on each side, to help maintain the activity initiated by excitatory phasic inputs. Finally, intrinsic cellular mechanisms such as plateau potentials may allow individual neurons to integrate synaptic inputs. Although not as fully studied, it appears that similar mechanisms may allow for integration of somatosensory input for memory operations (Romo et al. 1999), and signals related to navigation for path integration (Taube and Bassett 2003).

Whereas the use of multiple mechanisms for neural integration is relatively well established, neural differentiation seems a bit more controversial and unresolved (Krekelberg and Lappe 2002). In visual motion processing, it is desirable to make up for time delays by extrapolating the target trajectory. This could potentially be done using phasic, velocity-related signals. Classic research (Pellionisz and Llinas 1979) suggested that neural circuits essentially implement a Taylor series expansion, combining position- and velocity-related inputs to output a signal related to the position expected at a future time. In more recent work, Puccini and colleagues (2007) showed that this velocity sensitivity and the associated phase advance could be achieved with a combination of spike frequency adaptation and short-term synaptic depression (SFA and STD in Fig 2B). Interestingly, this brain slice recording and simulation study showed that adding SFA to STD decreased presynaptic firing rates to reduce postsynaptic fluctuations, thus boosting the phase lead while also reducing the random variability in the output signal.

Alternatively, or perhaps in addition, extrapolation of visual target position could be a network operation. A neural network trained with smooth target trajectories (weighting coefficients d, e, f, in Fig 2B) could transform current target position into the future target position that usually results from this type of familiar trajectory. A modeling study by Kawato and colleagues (Shibata et al. 2005) demonstrated that this is potentially an operation performed by neurons in visual cortical area MST. As is the case for neural integration, it seems possible that differentiation is achieved through a combination of intrinsic cellular and network operations.

Comparison and cancellation

There are two classic examples of cases where inputs to particular groups of neurons are known to be either added or subtracted to perform the behavioral functions of comparison and cancellation, respectively. The first is illustrated in Fig 3A, which schematically shows neurons in an avian brainstem auditory nucleus getting convergent input from the two ears. The incoming action potential spike trains are phase-locked to the sound wave and due to the time it takes for axons to conduct action potentials, a sound that first enters the right ear causes action potentials to reach some neurons sooner than others. In the barn owl, the anatomical arrangement of the axons creates the proper pattern of delay to allow the neurons to act as coincidence detectors (Carr and Konishi 1990). This allows the system to use timing signals to encode the relative right/left location of the sound source. A more complex type of binaural convergence can allow the system to combine sound intensities between the two ears (e.g., Park et al. 1997).

Fig 3.

Examples of biological mechanisms that might be used to combine signals: A) comparison, and B) cancellation. In the avian auditory system (A), inputs from the two ears are known to be combined for sound localization. In the somatosensory system (B), it is hypothesized that the efference copy may be used to cancel the sensory input that would be expected to result from the motor command. This may involve presynaptic inhibition of cutaneous afferent input (primary afferent depolarization, PAD), as shown in more detail in Fig 4A.

As a second classic example of a system using a convergence of inputs, the electrosensory lateral line nucleus of some mormyrid fish is known to perform a filtering operation more akin to cancellation. Another, nearby brainstem nucleus provides a motor command to the electric organ and, in electrophysiological experiments, an efference copy of this command was detected in the electrosensory nucleus (Bell 1982). The incoming sensory signal and the efference copy had a subtractive interaction in the nucleus, which could allow the fish to detect electric field patterns that are different from those produced by its own discharge.

A more recent example of this sort of cancellation is shown in Fig 3B. It is well known that the corticospinal tract projects from motor cortex to the ventral horn of the spinal cord, carrying the motor command to the motoneurons. However, a lesser known projection of the corticospinal tract is to the dorsal horn of the spinal cord, where it converges with incoming input from somatosensory receptors. This may serve as an efference copy system to cancel the somatosensory input predictably produced by the associated motor command (Lemon 2008). Seki and colleagues (2003, 2009) showed that this filtering of “expected” somatosensory feedback may be accomplished by primary afferent depolarization (PAD), where signals arising from descending motor commands can selectively reduce firing rates in the intraspinal terminals of muscle spindle, tendon organ or cutaneous afferents (Fig 4A, see Rudomin and Schmidt 1999).

Fig 4.

Possible sites of inhibition of cutaneous afferent input in vertebrates. A) Spinal interneurons reduce the efficacy of synaptic transmission by synapsing on the intraspinal terminals of afferents fibers from various types of peripheral somatosensory receptor. B) Corticofugal projections originating in motor cortex tend to inhibit incoming somatosensory signals in the dorsal column nuclei.

This sort of cancellation of somatosensory input can also occur in the brain (Fig 4B). Both the somatosensory cortex and the motor cortex send corticofugal projections to the dorsal column nuclei in the medulla, an important component of the afferent somatosensory pathway. Although the exact cellular mechanisms are yet to be determined, the net effect of this corticofugal projection appears to be inhibition of the somatosensory input associated with voluntary arm and hand movements (Towe and Jabbur 1961; Shin and Chapin 1989).

The results of a recent psychophysical study (Weiss and Flanders 2011) are consistent with the hypothesis that this cancellation mechanism is used during haptic exploratory movements. When human subjects ran the index finger along a curved surface, they reacted to a change from the expected curvature with a compensatory decrease in the activity of the stretched muscles, at a spinal reflex latency. This response was opposite to the well-known stretch reflex, but consistent with a hypothetical involvement of an afferent inhibitory mechanism. We proposed that this sort of somatosensory comparison/cancellation is a fundamental neural operation that takes place at multiple levels: the cerebral cortex, subcortical structures (including the cerebellum and dorsal column nuclei), and the spinal cord. As illustrated in Fig 5, the current state or the sensory input that is expected (based on efference copy) is essentially subtracted from new sensory input to drive a new or corrective movement.

Fig 5.

Sensory filtering at cortical, subcortical and spinal levels. The gray circles highlight areas of hypothesized neural comparison/cancellation operations, involving activity resulting from efference copy and sensory feedback.

Research in computational motor control

Bayesian inference

As outlined in Figs 1 and 5, basic sensorimotor operations must rely upon the generation of an expected sensory state, which is a learned association between copies of previous motor commands and their resulting sensory states. This expectation based on prior activities is then combined with new information about the current sensory state. Thus, at first glance, it appears that the central nervous system is implementing a Kalman filter or performing a process akin to Bayesian inference, where probabilities derived from prior events are combined with newly sensed parameters. However, while probabilistic statistical methods have been useful in prosthetic decoding algorithms (e.g., Wu et al. 2006) and in predicting behavioral decisions (e.g., Kim and Basso 2010), this approach is not typically directed toward revealing the biological basis of sensorimotor processing. Hopefully, future studies will elucidate the cellular mechanisms used to bias neural output based on the reliability of recent memories.

Inverse models

Compared to the recently popular use of probabilistic analytical tools, studies of the process of internal model learning might seem to be further along in explaining neural operations. As mentioned above, the inverse internal model can be defined as a memory filter used in the transformation from kinematic parameters (e.g., velocity) to kinetic parameters (e.g., force). Numerous studies on this topic have been conducted (see Cisek 2009 for a review). It has been concluded that learning is sometimes in the frame of reference of the body rather than the world, kinematic and kinetic parameters may or may not be learned independently depending on the situation, and impedance (stiffness) is learned by a different process. In one interesting study, Thoroughman and Shadmehr (2000) showed that learning new forces for one movement direction influences movements in neighboring directions in a manner reminiscent of the breadth of directional sensitivities of neurons in the motor system.

One might suppose that this active field of research on inverse models would inspire and direct current research in neuroprosthetics, but this does not seem to be the case. Instead some of the most successful neuroprosthetic studies capitalize on the classic observation of Fetz and Finocchio (1975): that motor cortical neurons exhibit remarkable plasticity. For example, by recording from the same set of identified neurons for multiple days, Ganguly and Carmena (2009) showed that motor cortical neurons could be trained to represent randomly chosen new associations between their activities and the movement of a computer cursor.

Modularity

As mentioned above with regard to Fig 2, from a biological perspective, sensorimotor operations are expected to involve numerous interconnected neurons. Contrary to this assumption, a currently active area of motor control research is based on the premise of the existence a small number of discrete motor control modules, called motor primitives or muscle synergies (see d’Avella 2009 for a review). A muscle synergy is defined as a group of muscles that work together, although experimental work suggests that for primate arm and hand muscles this should be redefined as a multi-muscle group of motor units that work together (Herrmann and Flanders 1998; Weiss and Flanders 2004).

Characterizing a complex set of data by identifying its basis functions, or primitives, is clearly a useful approach. For example, our group showed that a wide range of human hand shapes could be well reconstructed as the weighted sum of just a few patterns of covariation of joint angles (Santello et al. 1998), and this discovery has been used in prosthetic/robotic applications (Ciocarlie et al. 2007). We also showed that an even wider range of human hand shapes and movements could be represented by a small number of muscle synergies (Weiss and Flanders 2004; Klein Bretler et al. 2007).

However, these observations should not be interpreted as support for the premise of discrete modularity. To the contrary, the hand shape and the muscle synergy basis functions described a distributed, not modular, organization: Santello et al (1998) showed that numerous higher order principal components of hand shapes represented important features of the data set, not random variability, and our subsequent studies showed that each additional muscle synergy provided a smoothly increasing amount of variance accounted for. It is not clear what would be the biological advantage of control with a small number of discrete motor control modules, instead of distributed control.

An elegant simulation paper by Chhabra and Jacobs (2006) addressed this question. These authors demonstrated that the existence and advantage of a small number of basis functions/muscle synergies, was at the level of computational theory, which is distinct from the level of hardware implementation (see Marr 1982). Furthermore, a computational analysis by Lewicki and Sejnowski (2000) suggested that distributed, “overcomplete” basis functions, in addition to being inherently more neurobiological, can provide highly efficient solutions to reconstruction problems such as speech generation.

More open questions

What is the biological basis of efference copy?

The idea that the sensorimotor system must use efference copy is well accepted (reviewed by Crapse and Sommer 2008; Poulet and Hedwig 2007). In some invertebrate examples the anatomical basis has been established (Poulet and Hedwig 2006; Webb 2004), but in many other cases efference copy still tends to be an amorphous concept. As initially described by von Holst and Mittelstadt (1950), the idea can be readily applied to automatic systems such as oculomotor and spinal reflexes, and the field of spinal cord/locomotor research is close to unraveling the complex neural substrates (Rudomin and Schmidt 1999; Grillner and Jessell 2009).

As illustrated in Fig 1, efference copy has two main functions: 1) to give rise to a real-time estimate of current state, and 2) to allow a comparison with delayed sensory feedback. We can ask more specific questions about each aspect:

1. How does the system represent its current state? In oculomotor control, the current eye position is constantly represented by the pattern of activity in the motor nuclei. For more complex movements, however, the relation between motor neuronal firing frequencies and effector position is not one to one, and state estimation may rely on the current pattern of activity in the somatosensory system. Johansson and Flanagan (2009) proposed that sequences of hand and arm movement are broken into discrete segments so that particular somatosensory events can trigger the next “action phase.” This would correspond to the longest dashed feedback loop in Fig 5. However, fluent hand movements sometimes involve significant blending of one segment into the next (Jerde et al. 2003), suggesting that current state must be continuously represented and used to tailor ongoing actions (the efference copy/current state feedback loop in Fig 4).

2. How is sensory feedback evaluated? As mentioned above, we have hypothesized that a cancellation mechanism is used during haptic exploration (Weiss and Flanders 2010). However the locations and forms of this cancellation are yet to be determined. In general, active sensing must involve some sort of comparison or cancellation, and this phenomenon is often referred to as sensory gating (reviewed by Chapman 2009). The biological implementation of this type control algorithm has yet to be determined, although clear progress is being made in studies of rat whisker movements (e.g., Hentschke et al. 2006).

To what extent is sensory input extrapolated?

The best examples of target trajectory extrapolation come from research on the visual system (e.g., Berry et al. 1999). Collectively, this research has shown that derivative operations are basic biological phenomena that occur at many different sites (from retina to cortex) and via many different mechanism (from cellular adaptation to network operations). However, more work is needed to parameterize this process for the visuomotor system (e.g., Soechting et al. 2010). For example, what is the time delay of the visual signal in the smooth pursuit eye movement system under various conditions? It seems reasonable to assume that similar extrapolation processes are available to the auditory and somatosensory systems, but much less is known.

Can hardware models help solve these problems?

There are several types of approach to analysis, modeling and simulation. One possibility is to build a hardware model, such as a robot programmed to perform a neural function in the same manner that the nervous system is thought to do so (reviewed by Pfeifer et al. 2007). A recent success in the field of active sensing is robots programmed to gather and interpret the same sensory parameters as rat whiskers (Solomon and Hartmann 2006; Fox et al. 2009). Numerous other issues could be approached. For example, a tendency in robotic research is to break a task into modules, but perhaps it would be possible to explicitly test whether a more continuous, distributed control strategy would be advantageous in some situations. Although ideal robotic and human solutions are not necessarily the same, robotic research potentially provides a suitable testing ground to quantify the relative merits of different biological control algorithms.

Which questions are appropriate for simulation?

Sensorimotor research inherently focuses on processes that involve multiple parameters and multiple processing algorithms. For example an important unexplored issue is the interplay between somatosensory and motor systems. Even at the periphery, these systems are intertwined. For example, muscle spindle sensory receptors contain tiny muscles! Other somatosensory receptors also respond differently depending on the actively generated state of the surrounding musculature. How are inputs from the multiple types of somatosensory receptors evaluated in light of their active vs. passive components, and how are they combined? How can this somatosensory signal be extrapolated forward in time to make up for sensory processing delays? How does this representation interact with the representation of motor commands? These complex questions will require multiple experimental approaches, undoubtedly including computer simulation.