Toward a Proprioceptive Neural Interface That Mimics Natural Cortical Activity

Abstract

The dramatic advances in efferent neural interfaces over the past decade are remarkable, with cortical signals used to allow paralyzed patients to control the movement of a prosthetic limb or even their own hand. However, this success has thrown into relief, the relative lack of progress in our ability to restore somatosensation to these same patients. Somatosensation, including proprioception, the sense of limb position and movement, plays a crucial role in even basic motor tasks like reaching and walking. Its loss results in crippling deficits. Historical work dating back decades and even centuries has demonstrated that modality-specific sensations can be elicited by activating the central nervous system electrically. Recent work has focused on the challenge of refining these sensations by stimulating the somatosensory cortex (S1) directly. Animals are able to detect particular patterns of stimulation and even associate those patterns with particular sensory cues. Most of this work has involved areas of the somatosensory cortex that mediate the sense of touch. Very little corresponding work has been done for proprioception. Here we describe the effort to develop afferent neural interfaces through spatiotemporally precise intracortical microstimulation (ICMS). We review what is known of the cortical representation of proprioception, and describe recent work in our lab that demonstrates for the first time, that sensations like those of natural proprioception may be evoked by ICMS in S1. These preliminary findings are an important first step to the development of an afferent cortical interface to restore proprioception.

The 15 years since brain–computer interfaces (BCIs) became a reality have seen a host of technical improvements that have moved the field from two- or three-dimensional control of a cursor to multidimensional control of a robotic limb (Hochberg et al. 2006; Collinger et al. 2013), to the controlled activation of multiple paralyzed muscles (Ethier et al. 2012). However, in nearly all BCIs, the feedback guiding these movements is exclusively visual. The only exceptions we are aware of is a study that used passive movements of a monkeys limb to supply proprioceptive feedback (Suminski et al. 2010), and a recent study in which stimulation in S1 of monkeys encoded the vector from hand to target, allowing the monkey’s to guide reaching in the absence of other cues (Dadarlat et al. 2015). Researchers are increasingly recognizing that supplying somatosensory input directly to the user’s brain is as important, and arguably more difficult a problem, than is extracting motor command information from the brain (Weber et al. 2012; Bensmaia and Miller 2014). Most of the experimental work within this area has focused on the sense of touch in monkeys (Romo et al. 1998; Fitzsimmons et al. 2007; Tabot et al. 2013) or whisking in rodents (Venkatraman and Carmena 2011). In contrast very little progress has been made in reproducing proprioception to provide kinesthetic feedback.

There are multiple points at which a somatosensory afferent neural interface might be implemented. For patients with intact CNS, such as amputees, the nerve stump and dorsal roots are possible implant sites. One advantage of a peripheral nerve interface is the further processing of these signals as they propagate centrally. The relative simplicity of the information carried by peripheral afferent fibers might be considered another advantage. Since both muscle spindles and Golgi tendon organs (GTOs) have been accurately modeled (Mileusnic et al. 2006; Mileusnic and Loeb 2006), in principle, their firing rates can be mimicked. However, mimicking these inputs is complicated by the unknown state of the descending gamma system that normally serves to alter both the static sensitivity of spindles to muscle length, and their dynamic sensitivity to length changes (Eldred et al. 1953; Lennerstrand 1968). Inferring muscle lengths and forces based on measured limb state would also be a substantial challenge.

Central interfaces on the other hand, present the obvious advantage of applying not only to persons with limb amputation, but also those with spinal cord injury, for whom peripheral interface sites are of no use. Furthermore, neurons in S1 integrate multimodal afferent signals to form a higher-level representation of limb state. Neurons within area 2 of the primary somatosensory cortex (S1) appear to signal the direction of hand movement, whether those movements are active reaching movements, or perturbations of the hand (Soso and Fetz 1980; Prud’homme and Kalaska 1994; London and Miller 2013). Cortical interfaces have the potential to tap into this higher-level kinematic representation. However, these same neurons are influenced by movement-related forces as well as kinematics (Jennings et al. 1983; Prud’homme and Kalaska 1994; London et al. 2011), making independent control of the two modalities an additional challenge.

In the rest of this chapter we will focus on the implementation of cortical, rather than peripheral interfaces and we direct the reader to the following additional sources for information on the latter approaches (Navarro et al. 2005; Raspopovic et al. 2014; Tan et al. 2014; Saal and Bensmaia 2015). We will survey the underlying neurophysiology and anatomy that gives rise to the sense of proprioception, followed by a discussion of prior and current attempts using electrical stimulation to recruit these areas in the restoration of sensation.

Proprioception Is an Essential Part of Normal of Motor Control

Sherrington first used the term proprioception to define the sense of body position (Sherrington 1906). Proprioception is now commonly defined as the sensory information that contributes to a sense of joint position and motion. It arises primarily from muscle spindles and GTOs that respond to muscle length and force, respectively. However, joint receptors and cutaneous afferents responding to skin stretch also likely contribute (Burke et al. 1988; Macefield et al. 1990; Weerakkody et al. 2009). Sir Charles Bell called proprioception the “sixth sense” (Bell and Shaw 1865), as it is often overlooked compared to the five main senses of which we are more consciously aware.

Beyond its somewhat limited role in our conscious sense of limb position and motion, proprioception plays a critical subconscious role in the planning and control of limb movement. Chronic loss of proprioception produces cortical remapping (Sanes et al. 1988) and profound changes in motor ability (Sanes et al. 1984; Sainburg et al. 1993; Gordon et al. 1995). Patients suffering chronic loss of proprioception due to large fiber neuropathy make looping, discoordinated reaching movements (Fig. 1). While these patients can compensate to a certain extent through vision, this compensation is effortful, and normal ease of motion is never restored. Claude Ghez and his colleagues showed that these deficits arise from a loss of coordination between musculature at the elbow and shoulder that prevents these patients from compensating for the intersegmental inertial dynamics of the limb (Sainburg et al. 1993). They hypothesized that this compensation relies on a model of limb dynamics, the accuracy of which depends on proprioceptive feedback. Interestingly, in some respects the movements of robotic limbs controlled by current BCIs look similar to those of these deafferented patients. This similarity suggests that some of the remaining functional inadequacy of BCIs may result from the lack of somatosensory feedback, including both touch and proprioception.

Fig. 1.

Reaching trajectories in normal subjects and subjects with peripheral deafferentation. Plots show the trajectories of a series of rapid out and back reaches to targets at several angles. Control subjects (left) were able to direct their movements accurately and make rapid reversals of direction, while deafferented subjects made errors in the initial reach direction and large looping motions (Adapted from Sainburg et al. 1995, Fig. 3)

Cortical Representation of Proprioception

Primary somatosensory cortex (S1) is composed of four distinct regions, two of which (areas 3a and 2) receive thalamic inputs originating from muscle receptors and are considered to be proprioceptive in nature (Pons et al. 1985; Krubitzer et al. 2004). Area 3a is a functionally earlier cortical processing stage for proprioceptive signals than area 2, in that its thalamic input arises primarily from muscles, with a small number of cutaneous responses perhaps due to the indeterminate boundary between 3a and 3b (Friedman and Jones 1981). Additionally, 3a has only weak corticocortical connections wtih cutaneous areas 3b and 1 (Huffman and Krubitzer 2001). In contrast, area 2 receives a combination of muscle and cutaneous inputs from the thalamus, as well as strong corticocortical inputs from cortical areas 3a and 3b (Huffman and Krubitzer 2001). Both areas have a weak somatotopy with the proximal limb represented medially, and the hand more lateral, but the spatial resolution of these maps is nothing like that of the detailed maps in the cutaneous areas 3b and 1. In some sense this is inevitable given inputs that arise from muscles, including those that span more than one joint. Despite the lack of large-scale structure, neighboring neurons do tend to share similar responses to limb motion (Soso and Fetz 1980; Weber et al. 2011).

Despite these anatomical differences between the two areas, there has been relatively little attention paid to the potential functional differences between them. Neurons located in the proximal limb parts of both 3a and 2 generally fire in bursts during arm movements, typically with firing rates that vary approximately sinusoidally with the direction of hand motion and increase with the speed of motion, much like the similar behavior of neurons in M1 (Prud’homme and Kalaska 1994; London and Miller 2013). It is thus useful to represent a given neuron with a “preferred direction” (PD). These neurons typically discharge during both active and passive movement, the relative magnitude of the two components varying from neuron to neuron (Soso and Fetz 1980; Prud’homme and Kalaska 1994; London and Miller 2013). Neurons that respond strongly during both active and passive motions typically have PDs for the two types of movement that are well aligned (London and Miller 2013), thus simplifying the representation of limb movement, and the requisite mapping from limb state to patterns of ICMS.

On the other hand, in addition to the phasic, movement-related discharge, many proprioceptive neurons also have a tonic component that varies approximately with the instantaneous position of the hand. This combined movement and postural tuning also occurs in varying proportions across neurons (Gardner and Costanzo 1981; Prud’homme and Kalaska 1994; London and Miller 2013). The directionality of the postural tuning appears to be similar to the movement-related PDs (Prud’homme and Kalaska 1994). Restoring a full sense of proprioception will presumably require supplying both these components.

Beyond the single neuron studies described above, we have recently used chronically implanted arrays to combine information from many simultaneously recorded neurons in order to reconstruct detailed information about kinematic limb state. Figure 2a is a scatter plot of the mean firing rate for two of 41 area 2 neurons measured in a 300 ms window following limb perturbations in four different directions (indicated by the symbol color). The structure in this plot is caused by the differing directional sensitivity of these two neurons. Although it is impossible to depict the corresponding 41-dimensional plot for the entire 41-neuron sample population directly, there are methods to reduce the dimensionality of this “neural space” to a much smaller number. Figure 2b shows the results of a factor analysis, which can be thought of as the particular projection of these 41-dimensional points onto a plane that retains as much of the original information as possible. Not only were points corresponding to a particular target tightly clustered in this 2-D neural space, the geometry of the two spaces was approximately isomorphic. This suggests that the neurons in area 2 carry information that conveys, approximately linearly, the position of the hand following perturbations in different directions.

Fig. 2.

Neural discharge related to limb movements imposed by force perturbations on the monkey’s hand in different directions. a Each circle represents the mean firing rate of two (of 41) neurons during a 300 ms window following a limb perturbation. Perturbation directions are coded by color, as shown in the inset diagram. Some separation of the neuronal response by reaching target is evident, but there is considerable overlap for the four reach directions. b Factor analysis applied to the full 41-dimensional space of concurrently measured firing rates. The perturbation directions are clearly separated in this 2D factor space, which has a geometry very much like that of the perturbation directions

The dynamics of limb movements can also be captured accurately using the “decoder” methods that have much more commonly been applied to M1 neurons (Chapin et al. 1999; Serruya et al. 2002; Vargas-Irwin et al. 2010; Wodlinger et al. 2015). Trajectories during planar, random-target reaching movements were predicted from area 2 neurons with mean R² between measured and predicted signals of 0.59 for position and 0.66 for velocity (Weber et al. 2011). Acceleration was predicted more poorly with R² = 0.44. In that study, a neuron dropping analysis revealed that these predictions were as accurate for neurons that had primarily cutaneous receptive fields as those with more muscle-like responses. This counterintuitive result is nonetheless consistent with earlier observations of the reliable modulation of area 1 and 3b neurons during center-out reaching movements (Cohen et al. 1994), the discharge of cutaneous peripheral afferents during rotations of the knee (Edin 2001), and of the perception of externally imposed finger movements in the presence of local anesthesia (Edin and Johansson 1995).

In addition to purely kinematic responses, S1 neurons also modulate during isometric force production with directionally tuned firing rates (Jennings et al. 1983; Prud’homme and Kalaska 1994; London et al. 2011). During reaching while compensating for loads in different directions, firing rates in S1 are well fit by sinusoidal functions of force direction as well as movement (Fig. 3). Under these conditions, the force PDs (“load axis” in that study) tend to be roughly anti-aligned with the kinematic PDs. Although the depth of modulation during movement and isometric tasks co-varies significantly across neurons (Jennings et al. 1983; London et al. 2011), neurons with strong tonic kinematic tuning to position tend to have stronger force representation (Prud’homme and Kalaska 1994). This differential sensitivity to force might account for some of the differing responses of S1 neurons during kinematically similar active and passive movements. (Soso and Fetz 1980; Prud’homme and Kalaska 1994; London and Miller 2013) These observations raise important questions about how the CNS resolves these differences to evoke perceptually similar kinesthetic sensations during active and passive movements. It also raises questions of the impact this sensitivity to muscle activity might have on attempts to elicit kinesthetic sensations by intracortical microstimulation (ICMS).

Fig. 3.

Representation of the combined effect of reach direction and external loads on the activity of proprioceptive neurons in S1. The plotted surface represents the response of 93 neurons, averaged after alignment to the movement and force PDs. The marginal curves represent the neural responses to unloaded reaching and to isometric force (Adapted from Prud’homme and Kalaska 1994, Fig. 11A)

While there have been quite a number of studies that have attempted to distinguish between different kinematic coordinate representations within M1 (Evarts 1969; Caminiti et al. 1991; Scott and Kalaska 1995; Kakei et al. 1999; Morrow et al. 2007; Oby et al. 2013), the analogous studies have not been done in S1. Existing studies have largely assumed a hand-centered, Cartesian representation and simply described the resulting mappings. It remains possible that neurons in 3a or 2 might actually be better described in terms of muscle lengths. Alternately, given the multimodal nature of the receptive fields in Area 2, it may be that a transformation occurs between the representations of kinematics in these two areas. Only with an accurate model of the relation between limb state and the cortical activity in S1, can we hope to recreate naturalistic patterns of cortical activity through electrical stimulation.

Similar uncertainty exists for the force representation in S1. Although contact forces applied to the hand and digits are represented in cutaneous areas 3b and 1 (Sinclair and Burton 1991; Tremblay et al. 1996), it seems very unlikely that they would also be represented in the proximal limb regions of areas 3a and 2. Instead, the endpoint force-related tuning in these areas seems more likely to represent muscle forces. Under conditions in which endpoint force is well correlated with muscle force (for example, isometric contractions) it can be predicted quite accurately from the activity of multiple area 2 neurons (London et al. 2011). Even during reaching movements against purely inertial loads, predictions remain reasonably accurate. However, when additional random forces are added to dissociate the direction of endpoint motion and endpoint force, the predictions become much worse, suggesting the modulation of area 2 neurons is more directly related to muscle forces, rather than the measured endpoint force.

Use of Intracortical Microstimulation to Restore Sensation

Anyone who has placed their tongue on the poles of a 9-V battery knows something of the odd sensation that can be evoked when the nervous system is activated electrically. In the late eighteenth century, Alessandro Volta is credited with the first electrical activation of the auditory system when he inserted two metal rods into his ears and connected them with a battery. He described a very unpleasant “boom” within his head. Substantially refined two centuries later, the cochlear implant has been used to restore hearing to 300,000 deaf patients worldwide as of 2012 (Yawn et al. 2015).

ICMS has been shown to evoke detectable percepts from a variety of cortical areas, including human visual cortex (Bak et al. 1990; Schmidt et al. 1996), rat (Otto et al. 2005a; Koivuniemi and Otto 2012) and cat (Wang et al. 2012) auditory cortex, rat barrel fields (Talwar et al. 2002; Houweling and Brecht 2008; Venkatraman and Carmena 2011; Bari et al. 2013; Thomson et al. 2013), and monkey somatosensory cortex (O’Doherty et al. 2009; Zaaimi et al. 2013; Dadarlat et al. 2015; Kim et al. 2015b). There is currently tremendous interest in using this stimulation to replace natural sensation when it has been lost.

The earliest application of direct cortical stimulation to induce sensation was probably in the primary visual cortex, resulting in the perception of small dots of light dubbed “phosphenes.” Phosphenes were first produced experimentally in the late 1920s by neurosurgeons in Germany, Lowenstein and Borchardt (Löwenstein and Borchardt 1918), Foerster (1929) and Fedor Krause and Heinrich Schum (Krause and Schum 1932), who described localized sensations of light, the position of which depended on the location of the stimulus near the occipital pole (Lewis and Rosenfeld 2016). The possibility that this electrically elicited sensation might be exploited to provide a form of artificial vision was pursued independently 40 years later in England by Giles Brindley at Cambridge University (Brindley and Lewin 1968) and in the Untied States at the University of Utah by William Dobelle (Dobelle and Mladejovsky 1974). These experiments used approximately 1 mm-sized electrodes placed on the surface of the visual cortex, and required currents as large as 10 mA to evoke the visual effects (Fig. 4a, b).

Fig. 4.

Cortical interfaces used to restore sensation. a Radiograph of electrodes implanted on the surface of V1 in a blind human patient (Figure a adapted from Dobelle et al. 1976, Fig. 1). b Map of the location of phosphenes generated by the electrodes in (a). (Figure b adapted from Dobelle et al. 1974, Fig. 3). c, d Stimulus trains delivered by penetrating electrodes in S1 allow monkeys to discriminate frequency as they would with vibratory stimulation of the fingers. c Frequency discrimination task using either mechanical vibration of the fingertip or ICMS in tactile area 3b. The monkey was presented a baseline mechanical stimulus of fixed frequency and a subsequent comparison stimulus and asked to report whether the latter was higher or lower frequency. d Psychophysical responses for mechanical (filled) and ICMS (open) comparisons, which were not significantly different (Figures c and d adapted from Romo et al. 1998, Figs. 1 and 2). e Color coded receptive field locations of recordings made by a Utah array implanted in area 2. f Psychometric performance comparing the relative intensity of two sequential indentations of the skin on the finger (blue curves) or two sequential ICMS trains applied to the corresponding electrode (red curves). As in the frequency discrimination task, these curves were statistically indistinguishable. The ICMS current was computed from indentation amplitude using an empirically determined mapping function between the natural and artificial percepts (Figures e and f adapted from Tabot et al. 2013, Figs. 1 and 3)

Artificial vision was pursued further through the neuroprosthesis program at the NIH, using penetrating intracortical electrodes. Experiments included intraoperative stimulation of three normally sighted patients undergoing surgical treatment of epilepsy (Bak et al. 1990) and chronic stimulation over a period of four months in a patient who had been blind for 22 years due to glaucoma (Schmidt et al. 1996). Phosphenes were produced in all these patients with threshold currents ranging from 1 to 10 μA, much lower than those necessary for the surface stimulation of Brindley and Dobelle. Further work led to development of two different types of high-density electrode arrays made from silicon, one effort led by Ken Wise at the University of Michigan (Hoogerwerf and Wise 1994) and another by Richard Normann at the University of Utah (Jones et al. 1992). The resultant “Michigan” and “Utah” electrode arrays, currently available from NeuroNexus and Blackrock microsystems, respectively, have undergone considerable further development, and continue to be used in a wide range of recording and stimulation studies.

Stimulus intensity can be graded by frequency, pulse width, or current and used to modulate the reliability with which the subject can detect the stimulus (Fridman et al. 2010). The increase in detection rate with current intensity (pulse amplitude or duration) results from the activation of a larger tissue volume, whereas increases in frequency presumably lead to higher firing rates in recruited neurons. Studies report detection thresholds between 2 and 40 μA in sensory areas ranging from visual areas in humans (Bak et al. 1990; Schmidt et al. 1996) auditory areas in rat and cat (Koivuniemi and Otto 2012; Wang et al. 2012) and somatosensory areas in rats (Houweling and Brecht 2008; Bari et al. 2013) and monkeys (Tabot et al. 2013; Zaaimi et al. 2013; Kim et al. 2015a). There is typically a gradual increase in the detection rate with increased current. This monotonic relation between stimulus and perceptual intensity provides a basis for modulating the stimulation to achieve percepts of different magnitudes.

Kevin Otto and colleagues trained rats to discriminate between ICMS delivered on different electrodes in auditory cortex. With 68 μA stimuli, rats were unable to distinguish electrodes with less than 750 μm separation (Otto et al. 2005b). The effective electrode density is thus coupled to the maximum current, potentially limiting the variety of distinct sensations that may be delivered. Further, some researchers have reported loss of efficacy at high stimulus currents. In V1 phosphenes lost color and shrank in size as currents were increased (Schmidt et al. 1996). These investigators also noted that high current on a single electrode occasionally produced multiple phosphenes. Similarly, in experiments stimulating area MT, effects on perceived visual motion were lost above 80 μA (Murasugi et al. 1993). These effects may reflect a saturation in the recruitment of neurons near the electrode combined with the recruitment of a more distant, and likely less functionally homogeneous populations. Combined, these findings set a limit on the maximum currents that may be used for increasing the strength of artificial percepts.

To add to the complexity, electrical stimulation activates not only the soma of nearby neurons but also axons passing near the electrode. As a consequence, the evoked activity is not focused exclusively near the electrode, but instead includes sparse activation of neurons within a several millimeter radius (Histed et al. 2009). This suggests that the problem of activating a functionally homogeneous population of neurons may be more severe than that suggested by a simple model of current spread.

An adjunct method for increasing the robustness of the evoked sensation may be to stimulate multiple electrodes. In stimulus detection studies, detection threshold current at each electrode within a group decreased modestly with the use of two or four electrodes in tactile areas of S1 (Kim et al. 2015b), and more dramatically in a separate study of area 2, (Zaaimi et al. 2013). In the latter study the effects were particularly large, even supralinear, with five and seven electrodes. Distributing stimulation across many electrodes and reducing the current at each one has the twin advantages of potentially increasing the functional homogeneity of the activated neurons within a smaller volume (Weber et al. 2011), and reducing the potential for damage to the surrounding tissue or electrodes. A recent study found no histologic damage or functional deficits following six months of stimulation on multiple electrodes with currents up to 100 μA (Rajan et al. 2015). These results suggest that easily detected currents are well below those expected to cause damage, but it is difficult to extrapolate to the cumulative effect of decades of stimulation that would be required from a useful prosthesis.

Beyond demonstrating the ability to detect stimulation, a number of animal studies have shown that ICMS can also be used to provide salient information. Temporally patterned stimulation in rat auditory cortex-enabled rats to perform a frequency discrimination task in which ICMS replaced an auditory cue (Otto et al. 2005b). In somatosensory cortex, ICMS has been used to provide salient information in a number of different tasks, allowing animals to distinguish between patterns of stimulation in cortex and to compare the percepts due to ICMS and natural stimuli. In an early study, cortical barrel field stimulation was used to guide a rat’s open field locomotion, stimulation in right or left S1 signaling the corresponding rewarded turn (Talwar et al. 2002). Another study showed that rats performing an object detection task were able to locate virtual objects on the basis of the timing of ICMS delivered to barrel as the whisker passed through the virtual object (Venkatraman and Carmena 2011). The rats explored the virtual object by whisking back and forth, much as they would against an actual object, suggesting that the sensation produced by ICMS was qualitatively similar to that driven by deflection of the whiskers.

In experiments with monkeys, stimulus frequency within proprioceptive area 3a provided cues that allowed monkeys to distinguish between rewarded and unrewarded targets in a simple cued reaching task (London et al. 2008). The group of Miguel Nicolelis has performed more elaborate experiments in which monkeys used brain control to move an avatar hand among multiple virtual targets (O’Doherty et al. 2009). Target contact triggered ICMS within area 1. Two targets were associated with different temporal patterns of stimulation referred to as “virtual textures.” Monkeys were able to explore the targets and learn the pattern associated with the rewarded target.

In contrast to the discrimination experiments described above, in at least two other studies, animals learned to interpret ICMS that provided continuous feedback during motion. In one case, ICMS was delivered to barrel cortex of rats and modulated in intensity according to the rat’s distance from an infrared light above a water port. ICMS intensity increased as rats drew closer to the port, allowing them to locate it through this artificial somatosensory percept (Thomson et al. 2013). Having learned the basic task with a visible light cue, the rats learned to use the IR-driven ICMS cue in about a month. In another experiment, monkeys learned to reach to invisible targets guided by feedback that represented an error vector from the current hand position to the unseen target (Dadarlat et al. 2015). Feedback was provided either by a random moving-dot flow field or the strength of somatosensory ICMS delivered across eight electrodes. The frequency of simulation was determined by the projection of the error vector onto eight evenly spaced unit vectors assigned arbitrarily to the eight electrodes. After 3–4 months of paired visual and ICMS feedback, the monkeys learned to use ICMS to guide reaches in the absence of the visual information.

Biomimicry in Afferent Cortical Interfaces

Work in patients with limb amputation has shown that physiologically appropriate cutaneous feedback improves the patient’s embodiment of the prosthesis (Cincotti et al. 2007; Marasco et al. 2011). Likewise, it is reasonable to suspect that learning to use an ICMS interface that mimics natural sensations would be faster, and ultimately perhaps more effective than learning arbitrary associations with unnatural sensations or arbitrarily modulated ICMS (Bensmaia and Miller 2014). However, the extent to which artificially evoked activity must mimic that of the natural afferent inputs in order to be useful remains a critical question.

Ranulfo Romo performed a now classic series of experiments in the somatosensory system, in which monkeys identified which of two sequential mechanical stimuli applied to the fingertip had a higher frequency (Romo et al. 1998). The monkeys were able to complete the task when one or both of the mechanical stimuli was replaced with an ICMS train of the same frequency in the tactile area 3b (Fig. 4c, d). This experiment was the first to demonstrate that ICMS in somatosensory areas can evoke percepts with some naturalistic components. Similar experiments have been done more recently using punctate pressure stimuli (Tabot et al. 2013). The monkeys in that experiment were able to report both the location and magnitude of effects evoked by area 1 or 3b stimulation, and to compare them to their mechanical analogs (Fig. 4e, f). ICMS with higher currents led to a sensation of higher pressure, allowing construction of mapping functions between the desired perceptual effects and ICMS. These maps, and the receptive field locations characteristic of each electrode, were ultimately used to generate discriminable sensations through ICMS based on forces applied to sensors on a prosthetic limb, demonstrating a plausible method to provide cutaneous feedback to a patient. These effects rely on the somatotopy of S1, and the fact that neighboring neurons in both areas tend to have largely overlapping receptive fields.

Extrapolaind from these studies, a reasonable approach to reproducing a sense of limb movement in a particular direction would be to electrically activate the neurons that naturally signal movement in that direction. Figure 5 suggests the design of such an interface, using an array of electrodes that activate groups of neurons with identified PDs. By activating neurons in an appropriate spatiotemporal sequence, it may be possible to reproduce a sensation of the natural movement that elicits the same sequence of neural activity.

Fig. 5.

Schematic representation of a biomimetic neural prosthesis for proprioception. Each electrode, indicated by a black circle, activates a volume of neural tissue with an experimentally-determined, multi-unit PD, indicated by the black arrow. We speculate that stimulating an appropriate set of electrodes (red sparks) with PDs similar to the instantaneous direction of limb movement (red arrow on hand path) will evoke a corresponding sense of limb motion (green arrow). A complex spatiotemporal pattern of stimulation might then be used to convey feedback of the entire movement trajectory. Our experiments, described later, suggest that stimulating a set of electrodes with a single, well-defined PD causes a sensation of hand movement in that direction. The effect of stimulating electrodes with dissimilar PDs has yet to be determined

A potential challenge to using the natural behavior of neurons to design biomimetic interfaces is that the sensations resulting from ICMS do not always correspond to those normally signaled by the activated neurons. Stimulation of V1, for instance, does not produce the perceptual experience that might be anticipated based on the physiological characteristics of single neurons recorded in V1. Rather than the oriented bars that might be anticipated based on the response properties of complex cells (Hubel and Wiesel 1962), ICMS gives rise primarily to the sensation of fixed spots of colored light. Perhaps this is the consequence of the combined activation of many cells with different orientation selectivity. On the other hand, stimulation of the higher visual areas MT and MST can, in some conditions, elicit a sensation of visual motion that is consistent with the visual motion sensitivity of the neighboring neurons (Salzman et al. 1992; Celebrini and Newsome 1995). These experiments combined ICMS with the moving-dot stimuli in manner that biased the monkey’s judgment of the actual dot motion toward the directional preference of the electrically activated neurons. This effect was present only with currents well below 80 μA (Salzman et al. 1992) and required the electrode to be positioned at a depth within a column in which neurons encountered over 150 μm of electrode travel had similar preferred directions (Salzman et al. 1992).

Generation of Artificial Proprioception Through ICMS

The strong representation of hand velocity by neurons in area 2 and the similarity between these neurons and the motion sensitivity of neurons in MT/MST suggests a useful experimental paradigm that we have begun to explore, examining the hypothesis that stimulating in area 2 will alter the monkey’s perception of hand motion. More specifically, just as ICMS in MT and MST biases perception of motion, ICMS in S1 should increase the activity in the neurons near the stimulated electrodes, biasing the monkey’s perceived hand motion toward the PD of the stimulated neurons.

We trained a monkey to perform two reaching tasks, while grasping the handle of a planar robotic manipulandum. Movement of the handle controlled the position of a cursor on a screen and servomotors could be used to apply forces to the handle. The first task required the monkey to reach to a sequence of randomly positioned targets in order to generate movements with different speeds and in many directions. We recorded multi-unit activity and used these data to compute preferred directions for each electrode. We used multi-unit, rather than discriminated single units, to better represent the set of neurons that would be activated by ICMS.

The monkey also learned a two alternative forced choice task designed to test his ability to discriminate the direction of force pulses applied to the handle (Fig. 6). After moving the cursor to a central target, an additional pair of targets appeared, located symmetrically on a line passing through the central target. After a random hold period, the robot applied a 500 ms force pulse that displaced the monkey’s hand a few centimeters. The monkey then reached to the outer target that was closer to the direction of the force pulse. The monkey was rewarded for a correct choice and given an audible error signal if the reach was to the wrong target.

Fig. 6.

Direction identification task. The monkey held its hand in a central location (green circle) while a physical perturbation (black arrow) was applied to the hand. The monkey was required to reach to the red target that was nearer to the pertubation direction (blue arrow). If the monkey reached to the correct target he received a liquid reward. This task increased in difficulty as the angle of the perturbation (θ) approached 90°, midway between the two targets

To evaluate the monkey’s performance in the direction identification task, we computed the proportion of trials in which he chose one of the targets and fit these responses as a sigmoidal function of the perturbation direction (black curve in Fig. 7a). The monkey could judge perturbations near either target quite accurately. Between these extremes, the psychometric function passed through the point of subjective equality (PSE), the angle for which the monkey perceived the perturbation to be intermediate between the two targets. In this example, PSE fell very close to 90° in the coordinate system aligned to the target axis.

Fig. 7.

Biased perception of perturbation direction resulting from concurrent ICMS in area 2. a Psychometric curves showing the monkey’s performance on the direction identification task of Fig. 6. Perturbation direction is relative to the target axis, where 0° corresponds to the PD of the stimulated electrodes. In this example, data were collected over seven days, with the PD target at 211° in absolute coordinates. Black circles and curve indicate the proportion of the monkey’s choice of the PD (0°) target as a function of the direction of the mechanical perturbation. Red curves indicate the monkey’s responses with progressively increasing stimulus currents. Stimulation caused more frequent reaches to the PD target, shifting the psychometric curve up and to the right. b Summary of the PD groupings of the seven electrode sets tested during these experiments. Filled circles around the perimeter indicate sets for which stimulation biased the monkey’s movements toward the corresponding PD target. A single set of electrodes (open cyan circle) failed to produce the predicted result

On any given ICMS session, we selected four electrodes that had similar PDs and aligned the target axis to their mean PD. Thus one target was in the direction of the PD and the second in the anti-PD direction. We delivered ICMS to all four electrodes concurrently with the perturbation, using currents of 5, 10, 15, or 20 μA. The sensation caused by the ICMS appeared to combine with the natural sensation of the force pulse, producing a biased perception of the force direction (red curves and symbols in Fig. 6a). The psychometric curves shifted up and to the right, introducing a large bias toward the zero-degree PD target for 90° perturbations, and shifting the monkey’s PSE toward the PD. Furthermore, the bias was graded with current. 5 μA stimulation had no significant effect, while 20 μA was strong enough that the monkey was never fully confident in selecting the anti-PD target, even when the perturbation was aimed directly toward it.

In a series of experiments over a period of 18 weeks we tested seven different groups of electrodes (Fig. 7b). Six of these resulted in an effect essentially like that shown in Fig. 7a, with the bias direction congruent with the PD of the stimulated electrodes. The seventh set of electrodes produced no detectable effect. Subsequent analysis showed that these electrodes had larger PD variability over time than the other sets, suggesting that the failure to induce a bias may have been the result of instability across the several days of testing.

It is important to note that the ICMS-induced bias was not the result of training, as was thye case in several other studies. We rewarded the monkey based on accurate discrimination of the physical perturbation, rather than any feature of the stimulation. Since the monkey’s performance was roughly optimal prior to stimulation, the bias had the effect of increasing the monkey’s error rate, thereby lowering the amount of reward. This is the opposite of studies that train monkeys to associate arbitrary ICMS patterns with specific actions in order to receive rewards. The fact that reward rate decreased, suggests that the monkey was unable to learn to ignore the sensation due to ICMS, as doing so would have allowed him to return to the baseline level of reward. These observations suggest that the ICMS generated sensations that were sufficiently naturalistic that they combined predictably with the mechanical stimuli.

Future Directions

Forty years ago, Dobelle and colleagues implanted an 8 × 8 grid of electrodes in a patient blinded 10 years earlier by a gunshot wound (Dobelle et al. 1976). They identified a set of six (among 60 some) phosphenes that formed a braille cell, which allowed the patient to read at a rate of 30 characters per second, “much” faster than he could read tactile braille. However, Dobelle emphasized that “cortical braille [should be viewed] primarily as a technique to begin investigation of dynamic pattern presentation, rather than as a basis for clinically useful prostheses.” Dobelle’s statement reflected the recognition that his prosthesis could transmit only the simplest of spatial patterns. The prospect of scaling it up to reproduce an approximation of a visual scene is daunting, to say the least. Not only is the density of existing electrode arrays insufficient to pixelate an entire scene, but even existing spacing can cause interactions between adjacent electrodes (Dobelle et al. 1976; Otto et al. 2005b).

The cortical representation of touch and proprioception suggest that each may have an advantage over the cortical visual prosthesis. Unlike the highly artificial visual phosphenes, small punctate stimuli that activate only a small number of neurons make up a good portion of natural tactile stimuli. It is plausible that an array or arrays with hundreds of electrodes might be used to deliver clinically useful feedback about object contact timing, location, and pressure to the user of a prosthetic limb (Tabot et al. 2013). On the other hand, the broad directional tuning of proprioception means that a large proportion of these neurons are activated to greater or lesser extent for most arm movements. The sense of limb movement results from their overall integrated activity. Here the considerable challenge will be in selecting and activating many electrodes with an appropriate time-varying intensity (see Fig. 5).

The biases shown in Fig. 7 suggest that ICMS can cause activation that combines meaningfully with naturally-evoked activity. In order to move beyond these initial experiments to larger groups of electrodes, we intend to test a vector summation model, in which the predicted sensation is the sum of the individual sensations from each electrode. Alternatively, ICMS is known to entrain neurons to the stimulus (Griffin et al. 2011), which may preclude simple linear integration of the effect across electrodes. In that case, stimulation in multiple areas with different functional properties could lead to a winner take all scenario in which the strongest signal dominates. The eventual desire to provide not just a sense of limb motion, but also limb position and movement-related forces presents challenges well beyond our current experimental or modeling efforts.

Another important difference between touch and proprioception is that the former is very much a part of our conscious perception, while the latter is less so. Perhaps the most important function of a proprioceptive prosthesis will be in the control of movement, a function that was not addressed at all by our preliminary experiments. We elected to stimulate in area 2 rather than area 3a, in large part because of its accessibility on the cortical surface. However, its multimodal inputs are an additional reason that area 2 is an appealing target, as the sense proprioception relies on all these inputs (Collins et al. 2005). It is interesting to speculate whether area 3a, which receives inputs primarily from the deep muscle and joint receptors, might make a more appropriate site for a prosthesis that specifically addresses the subconscious control of limb movement.

A potential confound of ICMS used for artificial sensation is the observation that the standard symmetric biphasic current pulse tends to recruit axons at lower threshold currents than cell bodies. This effect causes distributed recruitment of neurons and complicates biomimetic approaches based on recording the activity of neurons near the electrode. A potential way to mitigate this effect may be to use pulses with reduced cathodal amplitude, which instead biases recruitment toward the soma (McIntyre and Grill 2000, 2002). A recent stimulus detection study in rat barrel field found that this asymmetric waveform yielded lower detection thresholds than did symmetric pulses, despite the smaller cortical area it recruited (Bari et al. 2013). By limiting recruitment to smaller clusters of neurons, this approach might recruit populations with more uniform functional properties.

While ICMS represents the current state of the art for afferent somatosensory interfaces in large animals and humans, optogenetic techniques may eventually provide a more selective method (Fenno et al. 2011; Yizhar et al. 2011). Optogenetics uses viral transfection to alter the DNA of neurons, causing them to produce photosensitive ion channels, allowing neurons to be activated by pulses of light. While current methods for delivering focal sources of light do not match the spatial resolution of ICMS, this technique allows cells with particular phenotypes to be targeted, providing the potential to engage only inhibitory neurons or those projecting only to or from particular areas. It also eliminates the challenges imposed on concurrent recordings by electrical stimulus artifacts and can eliminate the problem of the activation of axons of passage. While these techniques have been used primarily in rodents, there is considerable interest in translating them to primate models where they show great promise for future application to afferent interfaces (Diester et al. 2011).

Take Home Message

Only recently, the focus of neural interfaces turned to the critical need to restore somatosensation, which is lost along with the ability to execute movement, as a consequence of SCI or limb amputation. Most efforts to restore somatosensation at the cortical level have relied on learned associations between arbitrary stimulation patterns and either reward (O’Doherty et al. 2011; Venkatraman and Carmena 2011; Thomson et al. 2013) or other feedback modalities (London et al. 2008; Dadarlat et al. 2015). In a few studies attempting to restore cutaneous sensation, ICMS has been shown to reproduce naturalistic sensations (Romo et al. 1998; Tabot et al. 2013). However, no such evidence has previously been presented for proprioception. By delivering ICMS coincidently with force pulses applied to a monkey’s hand, we biased the monkey’s perception of the direction of the resulting movement. We suggest that the combination of ICMS-driven cortical activity and that due to the actual perturbation, altered the monkey’s perception in a predictable manner. The biases shown in these experiments required no learning, and appear to reflect naturalistic sensations of arm movement. These preliminary findings are a first step toward the development of an afferent cortical interface to restore proprioception.