Learning from the spinal cord: How the study of spinal cord plasticity informs our view of learning

Abstract

The paper reviews research examining whether and how training can induce a lasting change in spinal cord function. A framework for the study of learning, and some essential issues in experimental design, are discussed. A core element involves delayed assessment under common conditions. Research has shown that brain systems can induce a lasting (memory-like) alteration in spinal function. Neurons within the lower (lumbosacral) spinal cord can also adapt when isolated from the brain by means of a thoracic transection. Using traditional learning paradigms, evidence suggests that spinal neurons support habituation and sensitization as well as Pavlovian and instrumental conditioning. At a neurobiological level, spinal systems support phenomena (e.g., long-term potentiation), and involve mechanisms (e.g., NMDA mediated plasticity, protein synthesis) implicated in brain-dependent learning and memory. Spinal learning also induces modulatory effects that alter the capacity for learning. Uncontrollable/unpredictable stimulation disables the capacity for instrumental learning and this effect has been linked to the cytokine tumor necrosis factor (TNF). Predictable/controllable stimulation enables learning and counters the adverse effects of uncontrollable simulation through a process that depends upon brain-derived neurotrophic factor (BDNF). Finally, uncontrollable, but not controllable, nociceptive stimulation impairs recovery after a contusion injury. A process-oriented approach (neurofunctionalism) is outlined that encourages a broader view of learning phenomena.

1. Introduction

For nearly 25 years, we have explored the plastic potential of the lower (lumbosacral) spinal cord asking: Can it learn; what are the mechanisms that regulate spinal plasticity; and how do these systems affect recovery after injury (for reviews, see Ferguson et al., 2012b; Grau et al., 2006, 2012)? In the course of conducting these studies, we have been forced to grapple with the definition of learning and the methods used to demonstrate it (reviewed in Allen, Grau, & Meagher, 2009; Grau & Joynes, 2001, 2005a, 2005b, 2006). The work has implications for our description of spinal function, recovery after spinal injury and, we suggest here, how we characterize brain-dependent learning.

In the sections that follow, I outline a framework for learning and then explore whether spinal mechanisms meet these criteria. As we will see, our findings forced us to re-examine how we characterize learning about stimulus-stimulus (S-S) and response-outcome (R-O) relations, in both cases encouraging a view that assumes environmental puzzles can be solved in multiple ways. Such a view fits well with a comparative approach (Papini, 2002), but instead of asking how the mechanisms that underlie learning vary across species, I ask how they vary across different levels of the nervous system (Jackson, 1931). I will push a process-oriented approach (neurofunctionalism) that focuses on detailing the functional properties of the underlying mechanism(s) (Grau & Joynes, 2005a, 2005b) and will suggest that this framework provides a useful vehicle for linking neurobiological observations with behavior.

My first aim is to convince the reader that spinal neurons can learn. Beyond this, I hope to encourage a broader view of what constitutes learning and will argue that doing so will enhance the relevance of the field to neurobiologists and those attempting clinical application.

2. Defining Learning

Elucidating whether spinal mechanisms can learn requires a workable definition of this process. While this issue rose to the fore in our studies of instrumental learning (Grau, Barstow, & Joynes, 1998), the framework we derived can be applied more broadly. It builds upon a straight-forward classification scheme, outlined by Rescorla (1988) and implicitly adopted by others (e.g., Domjan, 2010) . It asks whether an experience at time 1 has a lasting effect (at time 2). Here, we use the term experience in the simplest sense, to indicate that the organism encountered an event(s) and/or a behavioral relation. From this perspective, whether the ‘experience’ gives rise to a conscious percept is irrelevant.

We first sought some common criteria for learning (Grau et al., 1998). While we recognized non-neuronal cells can influence neural function (Perea, Navarrete, & Araque, 2009; Vichaya, Baumbauer, Carcoba, Grau, & Meagher, 2009), we limited our definition of learning to behavioral changes linked to neural plasticity (Table 1, Criterion 1). Neural function may be altered as a result of development, injury, or experience. We limited learning to the last of these possibilities (Criterion 2). Finally, we required that the consequence of learning (the memory) extend beyond the contingencies used to induce it—that the experience has a lasting effect on performance (Criterion 3).

Table 1.

Common Criteria

The behavioral modification depends on a form of neural plasticity. The modification depends on the organism's experiential history. (a) The modification outlasts (extends beyond) the environmental contingencies used to induce it; (b) The experience has a lasting effect on performance.

Single Stimulus Learning

4. Exposure to a stimulus alters the response elicited by the target event.

Stimulus-Stimulus Learning

4. Imposing a temporal relationship between two stimuli alters the response elicited by one, or both, stimuli.

Response-Stimulus Learning

4. Imposing a temporal relationship between a response and a stimulus alters the response.

Building on these common criteria, we can distinguish the three most widely-studied forms of learning with an additional criterion (4). We state these in terms of observable events involving exposure to a single stimulus (S), the relation between two stimuli, and the relation between an organism generated response (R) and a S. In each case, the experience can generate either an alteration in an observable response or have a more subtle effect that requires additional manipulations to infer (e.g., as in latent inhibition and sensory preconditioning).

For each category of learning, researchers have developed some routine methodologies. Single stimulus learning is typically studied by examining the effect of stimulus preexposure, with the result being a decrement (habituation) or increment (sensitization) in its behavioral and/or psychological effect. S-S learning is generally explored using the procedures of Pavlov (Pavlovian conditioning), wherein the presentation of the unconditioned stimulus (US) is made conditional upon another cue (the conditioned stimulus [CS]; Staddon, 2005). Instituting a temporal relation can affect the magnitude of the response elicited by the paired CS (the CS+) relative to an unpaired cue (the CS-). Similarly, learning about R-S relations is usually studied using a biologically significant S (e.g., food or an aversive shock). In instrumental conditioning, a contingency is established between the performance of a particular R and the presentation of the S, which is often referred to as the outcome (O). When a R-O relation exists, the O is controllable.

As Rescorla (1988) notes, when described in this abstract way (Criterion 4), these three types of time-1 experience encompass the majority of behavioral studies on basic learning processes:

“They involve teaching the organism about the existence of a stimulus, about the relation of that stimulus to other stimuli in its environment, and about the relation of that stimulus to the animal's own behavior. One might argue that if we can understand how organisms learn these three things about a stimulus, we will have close to a complete characterization of how they learn about events in their environment.”

Rescorla (1988) also reminds us that a demonstration of learning requires that we address two basic issues. First, we must employ an experimental control that equates subjects on every factor except the target dimension (Criterion 4). For example, if the aim is to demonstrate that subjects have encoded a relationship between two stimuli (S1 and S2), compare conditions that equate exposure to the stimuli and vary the temporal relation. In this case, one group might be exposed to these events in a paired manner while another receives these stimuli explicitly unpaired. If these training conditions (time-1) yield differential performance at time-2, the S1-S2 relation must matter.

The second basic issue concerns our inference of learning; to demonstrate that experimental treatments at time-1 matter, we must test groups under common conditions at time-2. To illustrate the importance of this factor, let's consider how stimulus intensity affects the development of habituation. If stimulus intensity is manipulated across groups, those exposed to weaker levels of stimulation will likely exhibit a greater decline in response magnitude during training (time-1; Thompson & Spencer, 1966). From this, it might be concluded that the magnitude of habituation declines as stimulus intensity is increased. But notice that we are comparing performance across groups that differ in two ways: their prior experience on earlier trials and the intensity of the eliciting stimulus. To evaluate how the time-1 experience affects habituation, subjects must be tested at time-2 under common conditions. In this case, subjects could be tested with both weak and strong stimuli. When this is done (Davis & Wagner, 1968), it is generally found that intense stimulation yields greater habituation. When tested in the appropriate manner, we reach the opposite conclusion. A further advantage of this approach is that the time-2 test, by necessity, must occur after the time-1 experience, providing some evidence that the time-1 experience has a lasting effect (Criterion 3).

For all of the behavioral phenomena we will discuss, researchers have confirmed that spinal neurons play a pivotal role (Criterion 1; e.g., see Crown, Ferguson, & Joynes, 2002a; Durkovic, 2001; Groves & Thompson, 1970; Joynes, Ferguson, Crown, Patton, & Grau, 2003; Patterson, 1976). Likewise, controls have been included to demonstrate that the behavioral change observed at time-2 is related to the subject's experience at time-1 (Criterion 2). And in seeking evidence that training affects performance when subjects are tested under common conditions, we gain some indication that the experience had a lasting effect (Criterion 3). We acknowledge that the term lasting will remain ill-defined. At a minimum, we will look for evidence that a behavioral effect lasts hours, and will be most comfortable when it endures for a day or more.

3. Structural Organization of the Spinal Cord

Before we examine whether spinal neurons can learn, we need a basic understanding of how this system is organized. Key components are illustrated in Fig. 1. Anatomists have grouped the segments of the spinal cord into four sections: cervical, thoracic, lumbar, and sacral (Fig. 1A). Within each section, segments are numbered along the rostral-caudal axis. A cross-section of the spinal cord (Fig. 1B) reveals an outer ring of axons (the white matter) that relay signals between the brain and spinal cord. Communication with the periphery is organized by neurons within the inner region (the central gray). Afferent (sensory) input projects to the dorsal horn and efferent (motor) output exits from the ventral horn. Both sensory and motor processing is regulated by brain systems through descending fibers.

Fig. 1.

Gross anatomy of the spinal cord. (A) The human spine is covered by bony segments (vertebrae) and is grouped the segments into 4 sections: cervical, thoracic, lumbar and sacral. Within each section, the segments are numbered along the rostral-caudal axis. During development, the bony covering grows faster than the underlying tissue, which is accommodated by lengthening the sensory/motor fibers. This yields a bundle of fibers at the caudal tip known as the cauda equina. (B) A cross-section section of the spinal tissue illustrating the major components of the white (outer band) and gray (inner region) matter. Sensory neurons enter through the dorsal root and project to neurons within the dorsal horn of the gray matter. Neurons carrying motor commands from the ventral horn exit through the ventral root. (C) Cells within the central gray are organized into layers known as laminae. The substantia gelatinosa plays a central role in processing sensory signals related to pain. (Adapted from Grau et al., 2006.)

Intermediate regions (laminae; Fig. 1C) of the central gray contain neural networks that can organize some simple behavioral units. For example, noxious (potentially tissue damaging) stimuli engage pain (nociceptive) fibers that elicit a protective withdrawal response. This can be demonstrated in rodents by applying a noxious thermal stimulus to the tail, which elicits a reflexive tail-flick response. Because this response can be evoked after communication with the brain has been disrupted through a thoracic spinal cord transection, it is considered a spinal reflex. This behavior is often used to assess whether an experimental manipulation affected nociceptive processing within the spinal cord (the tail-flick test).

Neurons within the central gray can also organize relatively complex behaviors. A dramatic example of this is seen in cats after a complete thoracic transection. As expected, this injury induces a paraplegia that disrupts voluntary hindlimb motor behavior. But if the cat is then placed in a harness and suspended over a moving treadmill, it can be trained to step (Edgerton, Roy, de Leon, Tillakaratne, & Hodgson, 1997; Edgerton, Tillakaratne, Bigbee, de Leon, & Roy, 2004). Moreover, with training, hindlimb stepping improves and becomes sensitive to treadmill speed. If an obstacle is then positioned so that the cat's paw strikes the object as it lifts the paw forward (swing phase), neurons within the spinal cord will adjust step height to minimize contact with the obstacle. Notice too that the maintenance of rhythmic behavior suggests that the lower (lumbosacral) spinal cord contains a central pattern generator (CPG; Grillner & Zangger, 1979).

Many are surprised to learn that spinal neurons can organize relatively complex behaviors. There are, however, two ways in which these observations are not surprising. First, by current estimates, the human spinal cord contains over ten million neurons (Kalat, 1998). We are accustomed to the idea that an invertebrate (e.g., Aplysia) can exhibit some relatively complex behaviors with just 20,000 neurons (Hawkins, Kandel, & Bailey, 2006). Given this, it should not be surprising that a system that contains 500 times more neurons can organize behavioral units. A second consideration concerns the need for local control. As the number of neural connections increases, so too does processing time and accumulated error. For this reason, the brain gains efficiency by off-loading processing tasks to spinal neurons that locally regulate and structure neural activity. Brain circuits may guide the parameters of stepping, rather than its microstructure. The same is true for the sensory/motor modules that help to structure arm and hand movements. Neurally, the additional processing requirements yield an enlargement of the central gray in the cervical and lumbosacral regions (Fig. 1A). Our studies focus on functional capacities of neurons within the lumbosacral region using rodent (rat) subjects.

It is well known that brain systems can modulate neural processing within the spinal cord. Is this regulatory influence rigidly determined (immutable) or is it affected by experience? Can engaging descending fiber systems bring about a lasting alteration (a form of memory) in how spinal systems operate? Evidence for spinal memory was first provided by DiGiorgio (1929), who showed that lesions of one cerebellar hemisphere can produce a postural asymmetry in the hind legs, causing one leg to be flexed while the other is extended. If the thoracic spinal cord is cut soon after (within 45 min) the cerebellar lesion, the asymmetry vanishes. If, however, the legs remain in an asymmetrical position for several hours prior to spinal transection, the asymmetry remains (spinal fixation) and can last for 2-4 days (Patterson, 2001). Evidence for spinal memory has also been provided by Wolpaw and his colleagues (Wolpaw, 2001; Wolpaw & Carp, 1990) using an electrical analog of the stretch reflex (the Hoffman [H-] reflex). A standard operant procedure with food reward is used to train animals to exhibit an increase or decrease in the H-reflex. After extensive training, the behavioral modification survives a spinal transection. In this case, brain-dependent instrumental learning brings about a lasting behavioral change by altering a spinal circuit—the effective memory is stored within the spinal cord.

Our work extends these observations to explore whether spinal neurons have the capacity to adapt in the absence of input from the brain. Do neurons within the lumbospinal cord support both learning and memory?

4. Single Stimulus Learning

While the neurobiology of learning now focuses on the brain, this was not always the case. Indeed, when Richard Thompson published the first edition of his classic text in 1967, he introduced the section on learning within the mammalian central nervous system by noting “the most common mammalian CNS preparation for the study of behavioral plasticity has been the neurally isolated spinal cord” (Thompson, 1967). And it was, of course, this work that laid the foundation for the dual-process theory of habituation and sensitization (Groves & Thompson, 1970). I briefly review this work below and discuss how current research on nociceptive processing has brought a resurgence in interest in spinal cord plasticity, revealing that spinal neurons support physiological phenomena associated with learning and memory within the hippocampus.

4.1. Spinal Reflexes Exhibit Habituation and Sensitization

Over a century ago, Sherrington (1906) described how stimulation can affect reflex vigor in spinally transected subjects. In a typical experiment, animals underwent a mid-thoracic transection and electrical stimulation to the skin or a nerve was used to elicit a hind limb flexion response. With moderate stimulation, response vigor wanes (habituates) when the stimulus is repeatedly applied. If a much stronger shock is applied elsewhere on the limb, a more vigorous (sensitized) response is typically observed.

As detailed by Thompson and Spencer (1966), spinally-mediated habituation and sensitization exhibit many of the phenomena observed in intact subjects. There are, however, some key differences. For example, spinally-mediated habituation appears limited to a short-term form that develops more rapidly when the inter-stimulus interval (ISI) is decreased (Groves, Lee, & Thompson, 1969; Joynes & Grau, 1996). In contrast, long-term habituation in intact animals is generally most robust with spaced (a long ISI) rather than massed (a short ISI) practice (Whitlow & Wagner, 1984). We suggest that this divergence mirrors a theme that will be pushed in subsequent sections; that a behavioral phenomenon can be mediated by more than one process and these processes can have distinct functional properties. For example, in some cases a context-stimulus association may contribute to habituation, producing a long-term decrement that is strengthened by spaced practice (Wagner, 1981; Whitlow & Wagner, 1984). In contrast, spinally-mediated habituation is generally characterized as involving a non-associative process (Groves & Thompson, 1970).

4.2. Discoveries from the Pain Literature

Since the 1980s, some of the most interesting work on spinal cord plasticity has been conducted within the pain literature (for recent reviews, see Malcangio, 2009). Using tail withdrawal from radiant heat to assay the regulation of spinal nociceptive mechanisms, researchers have shown that exposure to a cutaneous nociceptive stimulus (e.g., electric shock) can induce an antinociception that inhibits behavioral reactivity to noxious stimuli (e.g., Lewis, Cannon & Liebeskind, 1980; Grau, Hyson, Maier, Madden, & Barchas, 1981). With mild forms of stimulation, this antinociception depends on brain systems that modulate nociceptive processing through descending fibers (Grau, 1987). With more intense and prolonged stimulation, intraspinal mechanisms are engaged that reduce nociceptive reactivity (Meagher, Chen, Salinas, & Grau, 1993; Meagher, Grau, & King, 1990). In many cases, this antinociception has been linked to the release of an opioid peptide (Basbaum & Fields, 1984; Watkins & Mayer, 1986).

Peripheral inflammation and nerve injury often have the opposite effect on behavioral reactivity, facilitating rather than inhibiting responsiveness. For example, application of capsaicin (the active ingredient from chili peppers) to a hind paw causes an increase in behavioral reactivity, leading subjects to exhibit a nociceptive-like withdrawal response to a tactile stimulus (von Frey filament) applied to the rat's paw (LaMotte, Shain, & Tsai, 1991; Willis, 2001). Interestingly, this effect extends to other dermatomes and may enhance responsiveness to stimuli applied to the contralateral limb (Huang & Yu, 2010). This enhanced mechanical reactivity (EMR) has been linked to the (central) sensitization of nociceptive processing within the spinal cord (Latremoliere & Woolf, 2009). Central sensitization likely contributes to the pain elicited in humans by innocuous tactile stimulation of an inflamed and/or injured region, a phenomenon known as allodynia (Simone, Baumann, & LaMotte, 1989). Peripheral inflammation can sensitize spinal nociceptive circuits without input from the brain (Ferguson, Huie, Crown, & Grau, 2012a; Ji, Kohno, Moore, & Woolf, 2003). Indeed, if anything, descending fibers tend to dampen nociceptive sensitization (Gjerstad, Tjolsen, & Hole, 2001; Sandkühler & Liu, 1998). The loss of these fibers may contribute to the development of neuropathic pain after SCI (Bardin, Schmidt, Alloui, Eschalier, 2000).

The sensitization of spinal nociceptive circuits can have a lasting effect, yielding a form of memory that has been linked to NMDA receptor (NMDAR) mediated plasticity (Coderre, Vaccarion, & Melzack, 1993; Dickenson & Sullivan, 1987; Ma & Woolf, 1995; Willis, 2001, 2009). This work has fueled interest in the signal pathways and shown that spinal plasticity is mediated by neurochemical/neurophysiological systems similar (if not identical) to those implicated in brain-dependent learning and memory (Ji et al. 2003). Moreover, spinal systems support both long-term potentiation (LTP) and depression (LTD) (Sandkühler, 2000; Sandkühler & Liu, 1998; Sandkühler, Chen, Cheng, & Randi). This windfall of cellular and electrophysiological data has transformed how many perceive the topic of spinal learning; taking the physiological/cellular data as having greater weight, there is a growing assumption that spinal systems should be capable of learning.

5. Pavlovian Conditioning

While the idea that spinal neurons can exhibit habituation and sensitization is widely accepted, their capacity to support Pavlovian conditioning has raised greater skepticism. As we will see, there is little doubt that spinal neurons are sensitive to S-S relations. What has proven more controversial is whether the underlying process should be considered an instance of learning.

5.1. Evidence Spinal Neurons are Sensitive to S-S Relations

Culler was the first to examine whether spinal neurons can support Pavlovian conditioning (Culler, 1937; Shurrager & Culler, 1940). Using tailshock as a CS, and hindpaw shock as the US, evidence was obtained for both the acquisition and extinction of a conditioned response in spinally transected animals. This work proved controversial on two grounds (Patterson, 1976, 2001). First, others reported difficulty replicating this effect (Kellogg, Pronko, & Deese, 1946). While there were a number of differences across laboratories the most important may have been the interval between spinal transection and testing (Patterson, 2001). Experiments yielding positive results tested subjects soon after spinal injury (acute) whereas those obtaining negative results generally trained and tested weeks after surgery (chronic). Subsequent research has confirmed that the capacity for Pavlovian conditioning declines over time after injury (Durcovic, 2001). This is important for clinical application because it suggests: 1) treatments soon after injury may have a greater impact (e.g., Brown, Woller, Moreno, Grau, & Hook, 2011); and 2) inducing alterations in the chronic state may require supplemental treatments (e.g., application of a neurotrophin) to enable learning (Huie, Garraway, Hoy, & Grau, 2012). Increased plasticity soon after injury may be related to a loss of descending inhibition (Crown & Grau, 2005). What is less clear is why this capacity to change subsequently wanes.

The second controversial issue concerned claims that the CS had no capacity to generate a CR-like behavior prior to training (Shurrager & Culler, 1940). The implication was that training established a new (de novo) association. This aspect of the work has not proven replicable and subsequent research has routinely used CSs that generate a CR-like (alpha) response prior to training. Acknowledging this limit, Thompson and his colleagues refined the paradigm with the aim of developing a model system to study the neurobiology of learning within the mammalian central nervous system (CNS; Fitzgerald & Thompson, 1967; Patterson, 1976, 2001; Patterson, Cegavske, & Thompson, 1973). In a typical experiment, electrical stimulation of the thigh or a nerve (e.g., the saphenous) served as the CS. The US was a stronger stimulus applied to the paw, or a nerve (e.g., the superficial peroneal), that elicited a flexion response. Using this procedure, research has shown that pairing the CS with the US alters the behavioral response elicited by the CS, enhancing its capacity to generate a flexion response (Durkovic, 1986, 2001; Patterson, 1976, 2001). Importantly, the appropriate controls were used to show that exposure to the CS and US does not generate a CS-elicited response when the events are explicitly unpaired. Furthermore, exposure to paired (versus unpaired) presentation of the CS and US has an effect that lasts hours and is evident when subjects are tested under common conditions (Durcovic, 2001). As expected, presenting the CS alone after training causes the CR to decline (extinguish). Moreover, a lasting effect is observed when the CS precedes the US (forward conditioning), but not when it is presented after the US (backward conditioning) (Durcovic, 2001; Patterson, 2001).

Subsequent research showed that optimal conditioning emerges when CS intensity is adjusted to a level that engages both A-beta and A-delta fibers and that performance deteriorates when stimulus intensity is increased to a level that engages C-fibers (Durcovic, 2001). In contrast, a monotonic relation is observed for US intensity, with only A-delta and C-fiber level stimulation being effective and CR magnitude increasing as a function of US intensity. Durcovic (2001) has also explored whether this learning depends upon a form of NMDAR-mediated plasticity. Pretreatment with a NMDA antagonist (APV) had little effect on the development of a CR during training, but eliminated the CR when subjects were tested hours later under common conditions (Durkovic & Prokowich, 1998). This suggests that training can affect the CS-elicited response through a process that does not involve the NMDAR and that establishing a lasting memory depends on NMDAR-mediated plasticity.

5.2. Conditioned Antinociception and the Regulation of Pain

Like most, I began my research career studying the brain. Early studies asked, how does learning affect pain (e.g., Grau, 1987; Maier, Drugan, & Grau, 1982)? Physiological and anatomical studies at the time had made clear that pain (nociceptive) fibers are regulated at multiple levels of the nervous system. One key site lies within the spinal cord, where descending fibers can modulate the incoming sensory signal, providing a form of top-down regulation (Basbaum & Fileds, 1984; Millan, 2002). By attenuating the afferent signal, this type of mechanism could reduce the effectiveness of an aversive event (McNally, Johansen, & Blair, 2011). At a behavioral level, treatments that block the transmission of nociceptive signals within the spinal cord also inhibit spinal nociceptive reflexes (e.g., tail withdrawal from radiant heat).

We and others showed that pairing a cue (the CS) with an aversive shock (the US) endows the CS with the capacity to generate a conditioned antinociception that inhibits tail withdrawal from radiant heat (Chance, White, Krynock, & Rosecrans, 1977; Illich & Grau, 1991; Watkins, Cobelli, & Mayer, 1982a). We assumed that this learning depended upon higher neural systems within the brain (Grau, 1987). Of course, if the CS engages a brain-dependent sensory system, this must be true. We also recognized that exposure to an intense shock could cause a spinally-mediated inhibition of the tail-flick response (Watkins, Cobelli, & Mayer, 1982b), but we assumed that this reflected an unconditioned process that was insensitive to S-S relations.

An undergraduate (Juan Salinas) questioned whether spinal mechanisms operate in such a mechanical manner, noting that years before Thompson and his colleagues had provided evidence of spinal conditioning (Fitzgerald & Thompson, 1967; Patterson et al., 1973). Salinas' hypothesis was that S-S (Pavlovian) relations might influence the development of antinoception without input from the brain. To explore this possibility, rats were spinally transected at the second thoracic (T2) region and given a day to recover. Weak (1 mA) shocks applied to the left or right hind leg were used as CSs. A tailshock given at an intensity (3 mA) known to produce an unconditioned antinociception served as the US. One CS (the CS+) was paired with the US while the other (the CS-) was presented alone (Fig. 2A). After 30 training trials, we assessed tail-flick latencies during the CS+ and CS-. We found that subjects exhibited longer latencies during the CS+, a form of conditioned antinociception (Fig. 2B; Grau, Salinas, Illich, & Meagher, 1990). Further work showed that this mechanism exhibited a number of Pavlovian phenomena, including latent inhibition, blocking, and overshadowing (Illich, Salinas, & Grau, 1994).

Fig. 2.

(A) The stimuli used to examine conditioned antinoception in spinally transected rats. Moderate shock to the left or right hind leg served as the conditioned stimuli (CS1 and CS2). An intense tailshock served as the US. (B) Tail-flick latencies before (Baseline) and after spinally transected rats had received differential conditioning in which one CS was paired with the US (CS+) while the other (CS-) was explicitly unpaired. At the start of testing, subjects exhibited longer tail-flick latencies during the CS+ (conditioned antinociception). The CS+/CS- difference waned (extinguished) over the course of testing. (C) Mean tail-flick latencies in subject that had received the CS paired with the US (CS+) or the CS presented explicitly unpaired (CS-). At the end of training, tail-flick latencies were assessed in the presence of the pretrained CS and a novel CS (provided by stimulation of the contralateral leg). Training produced a CS+/CS- difference. Tail-flick latencies during the novel CS (CS-N) were comparable to those observed during the CS+ and significantly greater than the CS-. This pattern of results suggests that a form of protection from habituation may have generated the CS+/CS- difference. (Adapted from Grau & Joynes, 2001.)

It was clear, however, that conditioned antinociception in spinally transected (spinal) rats differed in a quantitative manner from that observed in intact rats. First, more intense stimulation was needed (c.f., Illich & Grau, 1991). Second, the magnitude of the behavioral change produced was smaller. And third, our CSs were aversive. Indeed, the stimulus used for our CS had an intensity comparable to a typical US in intact rats (Chance et al., 1977; Illich & Grau, 1991). Not surprisingly, these CSs had some capacity to generate an unconditioned antinociception (Joynes & Grau, 1996).

5.3. Multiple Processes Can Encode S-S Relations

Our studies provided further evidence that spinal mechanisms are sensitive to S-S relations and, if we accept the criteria outlined in Table 1, suggest that this system supports a form of Pavlovian conditioning. It is here where we part company with some, for we assumed that S-S relations can be encoded in multiple ways. Traditionally, Pavlovian conditioning has been linked to associative learning. Under ideal circumstances, an associative process endows a neutral cue (one that has no capacity to generate a CR-like response prior to training) with the capacity to produce a new response (Gormezano & Kehoe, 1975; Kimble, 1961). At the end of training, presentation of the CS+ would generate the CR, while unpaired cues (either a CS- or a novel stimulus) that are discriminably different (i.e., exhibit little generalization) would not (Fig. 3Ai).

Fig. 3.

(A) Three mechanisms that could generate differential performance to a paired CS (+) relative to an unpaired CS (-). The graphs illustrate the pattern of results expected for each mechanism, working in isolation, during training and testing. Because each mechanism can produce a comparable CS+/CS- difference at testing, a novel CS (N) is needed to help uncover the underlying mechanism. These idealized results assume zero stimulus generalization. For associative learning (i), only the paired cue (+) generates a response at the time of testing. In pairing specific enhanced sensitization (ii), both the unpaired (-) and novel (N) CS would generate a response, but it would be weaker than that produced by paired CS. In protection from habituation (iii), the unpaired CS generates a weaker response than both the paired and novel CS. (B) The proposed framework assumes that a Pavlovian (CS-US) relation can be encoded by multiple mechanisms. We suggest that detailing the functional process that underlies the learning will simplify the derivation of linking hypotheses by providing a formal map of how the system operates. It is assumed that multiple biological processes may generate similar functional outcomes and contribute to multiple processes. (Adapted from Grau & Joynes, 2005a.)

We recognized that our example of spinal learning did not meet the associative requirement of neutrality, because our CS (legshock) produced a CR-like antinociception prior to training. In traditional terms, it reflects an instance of alpha conditioning. While a traditionalist might reject the paradigm on this basis, I believe that most today would take a softer stance. After all, many of the most used paradigms employ a CS that has some capacity to generate a CR-like behavior. For example, the stimuli used as phasic cues in rodent conditioning (e.g., a light, tone or noise) are not behaviorally neutral and at moderate intensities elicit an orienting response. In appetitive conditioning, the nature of this pre-existing response predicts the form of the CR (Holland, 1977). In studies of fear conditioning using a conditioned suppression paradigm, the CS often elicits some suppression before it is paired with the US (Mackintosh, 1974). Likewise, rats will avoid a novel taste before it is paired with illness (neophobia; Rozin, 1976). If a complete absence of alpha conditioning is seen as a criterion for Pavlovian conditioning, the list of Pavlovian phenomena would be quite small. Further, clinical relevance would also be diminished. Researchers frequently suggest that Pavlovian conditioning contributes to the development of phobic behavior (Bouton, Mineka, & Barlow, 2001). Yet the stimuli that support common phobias (e.g., to heights, snakes) often innately elicit some fear. Here too, S-S learning appears to reveal itself through a form of alpha conditioning.

For these reasons, we did not discount our demonstration of S-S learning simply because the CS produced a CR-like response prior to training. Instead we asked how might a S-S relation affect the development of a pre-existing response? There were two obvious possibilities. Kandel and his colleagues had shown that tail-shock sensitizes the gill withdrawal response to tactile stimulation in Aplysia (Kandel & Schwartz, 1982). When a touch to one region (e.g., the siphon) is paired with shock, a stronger sensitization develops within the siphon circuit, a phenomenon known as pairing-specific enhanced sensitization (Hawkins, Abrams, Carew, & Kandel, 1983; Walters & Bryne, 1983). As a result, pairing a touch at one locus (the CS+) with shock (the US) causes the CS+ to elicit a stronger response, relative to both an unpaired (CS-) and a novel cue (Fig. 3Aii).

The second possibility involved a pairing-specific alteration in the development of habituation (Humphrey, 1933; Mitchell, Scott, & Mitchell, 1977). Prior work has shown that the rate at which a cue (the CS) loses its capacity to generate a response (habituation) may be affected by the presentation of another event (the US). In this case, the US slows the rate at which the CS-elicited response habituates. What is important is that the magnitude of this effect may depend upon the CS-US relation; less habituation is observed when the CS is paired with the US, a phenomenon known as protection from habituation. Conceptually, this could be described as a form of pairing-specific diminished habituation. Behaviorally, this phenomenon can be distinguished from pairing enhanced sensitization using a novel CS. Both phenomena produce a CS+/CS- difference. What differs is the relative response elicited by a novel cue; in pairing-specific enhanced sensitization, the response elicited by the CS+ should be stronger than the novel stimulus (Fig. 3Aii) whereas in protection from habituation the two stimuli will generate a comparable response (Fig. 3Aiii).

Relative to the baseline performance, the pattern of results observed in Fig. 2B appears to suggest that a form of pairing-specific enhanced sensitization is at work. However, we could be misled here, because we do not know how our trained CSs compare to an untrained (novel) stimulus. Only this comparison will reveal whether training affected the CS+ or the CS-. Given that many training regimens likely affect the response elicited by both stimuli, and that a novel CS comparison can help reveal the type of mechanism at work, it is surprising that this control condition is seldom included. To examine the processes involved in spinally-mediated conditioned antinociception, Robin Joynes first assessed whether repeated exposure to a CS alone affects response magnitude, relative to a novel cue (Joynes & Grau, 1996). She found that the preexposed CS generated a weaker antinociception, which suggests that prior exposure weakened (habituated) its behavioral effect. She then trained two groups of subjects, one that experienced the CS paired with the US while the other experienced these events unpaired. At the end of training, she tested nociceptive reactivity during the pretrained CSs and a novel stimulus (CS-N). As usual, training produced a CS+/CS- difference indicative of conditioned antinociception (Fig. 2C). Most importantly, the CS- generated a weaker response relative to both the CS+ and the CS-N, which implies that the learning reflects a form of protection from habitation (Fig. 3Aiii).

Further support for this conclusion was obtained by assessing the impact of varying the number of training trials and inter-stimulus interval (ISI). We reasoned that a protection from habituation mechanism would slow the rate of habituation, but not prevent it. If that is true, increasing the number of training (CS+) trials five-fold might weaken the CR. That is what we found (Joynes & Grau, 1996). While seemingly odd, a similar effect has been observed in other paradigms and is anticipated by an older model of learning (Hull, 1943; Overmier, Payne, Brackbill, Linder, & Lawry, 1979).

Increasing the ISI generally weakens non-associative habituation (Groves et al., 1969). Given this, we assessed the impact of increasing the ISI 5 fold, from 2 to 10 min. If the learning reflects a form of protection from habituation, increasing the ISI should weaken habituation to the CS-, attenuating the difference between the CS+ and CS-. Increasing the ISI had exactly this effect (Joynes & Grau, 1996).

5.4. Is Spinally-Mediated Conditioned Antinociception a Form of Learning?

Our results suggest that conditioned antinociception reflects a form of protection from habituation. Given this, a number of questions come to the fore. First, can spinal neurons support other forms of conditioning? Though not systematically tested, the pattern of results obtained by researchers using Thompson's paradigm suggest pairing specific enhanced sensitization can contribute to the development of the CR. A similar conclusion follows from our Pavlovian analysis of instrumental conditioning (see below).

Second, limiting our attention to the two non-associative mechanisms being considered, is there any reason to treat one as superior to the other? Both are capable of producing a comparable CS+/CS- difference. On these grounds, and the criteria listed in Table 1, the answer would appear to be no. Further, they appear to share a form of conceptual equivalence; what differs is simply the process affected (habituation versus sensitization) and the nature of the modification (reduced habituation versus enhanced sensitization). Nor are there any obvious differences in the range of Pavlovian phenomena supported (e.g., Illich et al., 1994).

The third and more controversial issue concerns the relation of these non-associative mechanisms to associative learning. By one account, we would be wise to reject both as contributors to Pavlovian conditioning (Machado, 2005; Reilly & Schachtman, 2005). Taking a strong stance on this perspective, one could argue that these non-associative mechanisms are best viewed as artifacts to be controlled for in our studies of true learning (Gormezano & Kehoe, 1975). Not surprisingly, we suggested an alternative perspective that focuses on the criteria used to define Pavlovian conditioning (Table 1). From this perspective, both the spinal cord and Aplysia can encode S-S relations. By our view, it is best to remain agnostic regarding mechanism and leave it to nature, and data, to decide the relative importance of each. Further, it seems unlikely that conditioning of any type remains pure across development and training (Callaghan & Richardson, 2013; Joynes & Grau, 1996, 2005; Sahley & Crow, 1998). Just as spinal neurons rely on a seemingly simpler process, so too might young organisms. Likewise, during the early stages of fear conditioning, the response elicited by a paired CS could reflect the combined effect of protection from habituation and pairing-specific enhanced sensitization. An associative process may contribute too, with its relative role increasing as a function of training. What is important, at first pass, is whether our conditioning procedure generates a CS+/CS- difference. If it does, further work can be conducted to uncover the underlying mechanism(s) and there is little a priori reason to anoint one as superior to the rest. From this perspective, identification of a learning process involves a set of embedded methodologies, starting with the derivation of the type of learning (e.g., does it depend on a S-S or R-O relation?), followed by further tests to uncover the nature of the underlying process.

5.5. A Process-Oriented Approach

The notion that Pavlovian relations can be encoded in multiple ways has implications for neurobiological studies of learning. One is a note of caution. A demonstration of conditioning does not necessarily imply a demonstration of associative learning. Whether or not that is true will require further study. The view also encourages an open-mindedness regarding new observations and mechanisms. Following others, we have suggested two alternatives to associative learning, but other mechanisms and forms of description likely exist. In the end, relative merit will be determined by explanatory value, clinical relevance, and neurobiological observations.

We suggest that a process-oriented approach to the study of learning can provide a useful structure in which to understand new biological observations. As neurobiologists, we seek linking hypotheses that tie neurobiological substrates with the mechanisms that underlie learning. Deriving these links requires a detailed map of how a process operates, a functional description that details the operational properties of the system and its temporal dynamics. In the absence of such a model (ideally, mathematically defined), our linking hypotheses are often vague, poorly defined, and difficult to test. If it is the case, as we have suggested, that the same environmental puzzle can be solved in multiple ways, linking biological observations with behavior will require elucidation of the type of mechanism that is at work (Fig. 3B). Consider the contrast between an associative and non-associative mechanism. How these processes are affected by training and ISI may differ and, so too, would the pattern of changes expected in the underlying neurobiological substrates. Identification of the processes involved, coupled with a detailed model of its operation, can give neurobiologists the structure needed to understand their biological observations. Conversely, those who focus on behavior and process would be wise to follow the neurobiological work, for this can reveal distinctions that are difficult to derive given behavior alone.

6. Instrumental Learning

We have also explored whether spinal neurons are sensitive to R-O (instrumental) relations. Again, we conclude that spinal systems can encode these relations, but it does so using a mechanism that lacks the flexibility exhibited by brain-dependent forms of instrumental learning. The work has also changed the way in which we think about this process, suggesting some R-O relations may be encoded by sensory mechanisms, allowing the relation to be “directly” perceived. Finally, work in this domain suggests that some processing capacities (e.g., deriving the relation between events; abstracting temporal regularity) may be inherent qualities of neural assemblies.

There were 2 factors that sparked our interest in instrumental learning. One is that this form of learning seemed more relevant to the types of tasks used in physical therapy to promote recovery after spinal cord injury (SCI; Grau et al., 2012). Another concerned the status of past research in this domain. From prior work, it was not clear whether spinal mechanisms are sensitive to R-O relations (c.f., Buerger & Chopin, 1976; Chopin & Buerger, 1976; Church, 1989; Church & Lerner, 1976).

6.1. Evidence Spinal Neurons are Sensitive to R-O relations

The apparatus we use to study instrumental behavior in spinally transected rats is illustrated in Fig. 4A. Leg position is monitored by taping a contact electrode to the rat's paw. At the start of training, the underlying salt solution is adjusted so that the electrode tip is submerged. A flexion response is elicited by applying shock to the tibialis anterior muscle. A R-O relation can then be established by administering shock (the O) whenever the leg is extended (the R). Subjects given controllable shock (master) typically exhibit an increase in flexion duration over the course of training (Fig. 4B). Recognizing that shock per se could sensitize behavioral reactivity, a second group (yoked) is given shock at the same time, but independent of leg position. This is accomplished by experimentally coupling each yoked subject to a master rat; whenever the master subject receives shock, its yoked partner is given shock. In the absence of a R-O relation, yoked subjects do not exhibit an increase in flexion duration (Fig. 4B).

Fig. 4.

(A) The apparatus used for instrumental training with spinally transected rats. Shock is provided to the tibialis anterior muscle, which elicits a flexion response. An insulated rod is taped to the rat's paw and used to monitor limb position. Whenever the tip of the rod contacts the underlying salt solution, it completes a computer-monitored circuit. Tape is used to help stabilize the leg. (B) Rats given response-contingent shock (Master) exhibit a progressive increase in flexion duration over the course of 30 min of training. Rats that are experimentally coupled (Yoked) to subjects in the Master group, and receive shock at the same time, do not exhibit an increase in flexion duration. (C) Testing under common conditions with response-contingent shock. Prior to testing, flexion force was equated and subjects then received training with controllable stimulation. Previously trained (Master) subjects learned faster than subjects that were naïve (Unshocked). Rats that had previously received uncontrollable shock (Yoked) failed to learn. (D) Yoked rats exhibited the highest level of responding during testing. (Adapted from Grau et al., 2006.)

While the master-yoke difference observed during training helps to show that the R-O relation matters, it does not necessarily demonstrate that learning has occurred. The problem is that the experimental contingency can drive differential performance in the absence of learning (Church & Lerner, 1976). To see this, let's assume that the system operates in a mechanical manner (a reactive model) with some statistical variation in the rate at which the leg falls. Whenever a master rat's leg falls, it receives a shock that drives the leg up, which minimizes solution contact. Roughly half the time, the yoked rat's leg would reach the solution first. It would then remain in the down position until the master rat receives shock. As a result, the yoked rat would accrue more time with its leg in a down position, generating a master-yoke difference. As Church (1964, 1989) has noted, this is a general problem with the master-yoke paradigm. Once again we are reminded that inferences drawn from acquisition curves can be misleading (Rescorla, 1988). We addressed this problem in two ways: 1) we assessed the impact of training at time-1 by testing subjects at time-2 under common conditions; and 2) we followed Church's recommendation (Church, 1964) and assessed the impact of experimentally manipulating the R-O relation (by disrupting R-O contiguity).

When subjects were tested under common conditions with controllable stimulation, rats that had previously received controllable shock (master) learned more rapidly (a savings effect) than subjects that were previously untreated (unshocked). What was most surprising is that rats in the yoked group failed to learn, a behavioral deficit reminiscent of learned helplessness (Maier & Seligman, 1976). This did not reflect an inability to perform the target response. Indeed, rats that had previously received uncontrollable shock responded at the highest rate (Fig. 4D); they repeatedly experienced the R-O relation, but this did not produce an increase in flexion duration.

We subsequently showed that uncontrollable stimulation impairs spinal learning for 24-48 hrs. Here, and in subsequent studies, we applied shock using a computer program that emulated the variable stimulation produced by a typical master rat. We found that just 180 shocks to the tail or leg had a long-term effect (Crown, Joynes, Ferguson, & Grau, 2002b). This inhibition of learning appears tied to enhanced GABAergic activity (Ferguson, Washburn, Crown, & Grau, 2003). The long-term effect requires protein synthesis and has been linked to the cytokine tumor necrosis factor (TNF; Huie, Garraway, Hoy, & Grau, 2012; Patton, Hook, Crown, Ferguson, & Grau, 2004). In awake intact subjects, the induction of this learning impairment is inhibited by descending serotonergic fibers (Crown & Grau, 2005).

While uncontrollable stimulation disables learning, controllable stimulation has an enabling effect and can counter the adverse effect of uncontrollable shock. The enabling effect was demonstrated by training rats with a normal (4 mm) contact electrode depth. We then tested subjects on the same (ipsilateral) or opposite (contralateral) leg with a higher (8 mm) response criterion (Crown et al., 2002a). Raising the criterion made the task so difficult that naïve controls could not learn. However, rats that had previously received controllable stimulation were able to learn and this was true independent of whether they were tested on the ipsilateral or contralateral leg. Training with controllable stimulation also has a protective effect that blocks the induction of the learning deficit (Crown & Grau, 2001). Conversely, we can restore the capacity for learning by giving subjects controllable stimulation in the presence of a drug (the opiate antagonist naltrexone [Joynes & Grau, 2004a]) that temporarily blocks the expression of the learning deficit (Crown & Grau, 2001).

Instrumental learning depends on a form of NMDAR-mediated plasticity and is associated with the up-regulation of plasticity related genes (e.g., CREB and CaMKII; Gómez-Pinilla et al., 2007; Joynes, Janjua, & Grau, 2004a). The enabling and protective effects have been linked to the release of brain derived neurotrophic factor (BDNF). Supporting this, pretreatment with BDNF enables learning (Gómez-Pinilla et al., 2007). Similarly, administration of BDNF can both prevent and reverse the learning deficit (Huie et al., 2012b). Conversely, inhibiting BDNF function (with TrkB-IgG) blocks the protective effect of instrumental training. A summary model is provided in Fig. 5.

Fig. 5.

A model summarizing key features of spinally-mediated instrumental learning. The model assumes that nociceptive signals the occur within a regular proprioceptive context are interpreted as controllable, producing a constellation of effects that include an adaptive increase in response duration and the induction of a BDNF-dependent process that enables learning and attenuates the adverse consequences of uncontrollable stimulation. Exposure to uncontrollable stimulation induces a central sensitization-like state that inhibits learning, enhances mechanical reactivity and pain, and disrupts recovery. The adverse effect of uncontrollable stimulation has been linked to an alteration in GABA and the cytokine TNF. The initial state assumes incoming nociceptive signals are related to behavior, providing a bias in favor of behavioral control (indicated by the angle of the nociceptive gate). We propose that spinally-mediated instrumental learning involves learning the relation between proprioceptive signals (an index of position [the response]) and the onset of nociceptive stimulation (the outcome). This learning is fostered by the brain-derived neurotrophic factor (BDNF). Descending serotonergic (5HT) fibers provide a physiological brake that inhibits the over-excitation of spinal neurons and counters the adverse effect of uncontrollable stimulation. (Adapted from Grau et al., 2012.)

6.2. Learning is Reinforced by Shock Onset

What type of process underlies learning that shock is controllable? Is the response reinforced by shock onset (a form of punishment) or shock termination (escape). To explore these issues, and to gain further evidence that the R-O relation mattered (Church, 1964), we examined the impact of experimentally manipulating response (leg position)-outcome (shock) contiguity. First, we tested whether delaying both the onset and offset of shock would affect learning. We found that a delay of just 100 msec eliminated learning (Grau & Joynes, 1998). Next, we delayed onset or offset alone. We found that delaying offset by 100 msec had no effect, whereas delaying onset by 100 msec disrupted learning. This suggests that the change in response duration is reinforced by shock onset; that the learning reflects a form punishment (passive avoidance), not escape learning. From this, it is tempting to conclude that escape learning requires brain-dependent systems.

Our work suggests that the abstraction of behavioral control by spinal systems is linked to the onset of the nociceptive stimulus. What distinguishes master and yoke rats in this situation is leg position at the time of shock onset. For this difference to matter, the system must have an index of leg position, which we assume is provided by proprioceptive cues (Fig. 6A). For the master rat, shock always occurs within the same proprioceptive context, whereas for the yoked rat this relation varies. While it is natural to think of R-O learning in terms of a motoric effect, our work suggests that much of the work may occur on the sensory side. From this view, if a stimulus (the O) regularly occurs within a constant proprioceptive context (the R), it is registered as controllable. If the proprioceptive context varies, it is classified as uncontrollable. As a casual example, notice what you perceive when you tap your finger against a table top. The response (which might be described as either finger position or a vector specifying the finger movement) is integrally linked with the sensation of touching the table top (the O). Here, a Gibsonian would suggest that the sensory signal allows us to directly perceive the R-O relation (Gibson, 1979).

Fig. 6.

A Pavlovian analysis of spinally-mediated instrumental learning. (A) It is assumed that proprioceptive cues (P1-P8) provide an index of leg position (the response). Learning is initiated by the onset of leg shock (the outcome [O]) which occurs at a regular position (P6). (B) It is proposed that the proprioceptive cue (P6) functions as a Pavlovian CS, which is paired with shock onset (the US). As a result of this pairing, the CS acquires the capacity to generate a flexion response (the CR). (Adapted from Grau et al., 2012.)

Mechanistically, the suggestion is that a sensory cue provides an index of leg position which, when paired with the onset of a nociceptive stimulus, gains the capacity to drive a flexion response. The reader will recognize that we have provided a Pavlovian account of the instrumental behavior, with the proprioceptive cue serving as the CS and shock onset acting as a US (Fig. 6B; Konorski & Miller, 1937). We have suggested that, as a default, spinal systems operate on the assumption that external stimulation is related to movement and/or position (Grau et al., 2012). In this sense, our biology may bias (prepare) us to perceive the world as controllable. This bias is represented within our model (Fig. 5) by assuming that the nociceptive signal is normally integrated with the proprioceptive input.

Our account of instrumental learning naturally extends the characterization of Pavlovian mechanisms suggested above. In this case, we are suggesting that alternative environmental puzzles (the encoding of a S-S versus R-O relation) may be solved, in part, using a common process. And if our Pavlovian account of spinal instrumental behavior has merit, it would constitute an example of pairing-specific enhanced sensitization. At a mechanistic level, the US used in studies of Pavlovian conditioning with electrophysiological stimulation, like the outcome used in our work on instrumental conditioning, elicits a flexion response. The sole difference may be in whether the experimenter (Pavlovian) or subject (instrumental) controls its occurrence. Likewise, the CS in a Pavlovian paradigm involves an experimenter-controlled cue whereas an interoceptive cue (leg position) may serve as the CS in an instrumental paradigm. Notice that, if a common mechanism is involved, we should be able to entrain an instrumental-like response modification by taking control of the stimuli, pairing an experimenter-driven change in limb position with shock onset. We are currently exploring this possibility.

If a Pavlovian mechanism underlies spinal instrumental behavior, why maintain the reference to R-O learning? At a behavioral level, what can be experimentally manipulated is the R-O relation. Similarly, in a clinical setting, physical therapists can institute training paradigms that promote behavioral control, by assuring that stimulation occurs in a response-contingent manner. In both cases, the relation that matters is best described as a form of R-O learning and that remains true even if the underlying mechanism builds upon a process that can also generate a Pavlovian CR.

6.3. Multiple Forms of Instrumental Behavior

Spinal instrumental learning has an odd status. On the one hand, some may doubt whether such a low-level system is capable of R-O learning.Yet, most seem comfortable with the Pavlovian analysis of R-O learning provided above. Why then does the claim of instrumental learning prove controversial? I believe that the difficulty lies with a form of over-generalization which, once again, has implications regarding the nature of the underlying process. And here too, parallels can be seen with the invertebrate literature. The over-generalization emerges because a broad range of behavioral phenomena, interchangeably referred to as either instrumental or operant behavior, are classified together. Historically, however, the terms were derived from different perspectives and this has conceptual implications. At the time Skinner (1938) coined the term operant behavior, theorists typically described instrumental behavior in stimulus-response (S-R) terms (Hilgard & Marquis, 1940). From this perspective, instrumental behavior is elicited and may be understood in terms of the laws of reflex function. Skinner noted that, in many situations (e.g., a rat bar-pressing for food or traversing a maze), it was not obvious what stimuli (if any) elicited the response. One could hypothesize that a stimulus cue existed, but in many instances, the account seemed post hoc and had a dubious flavor. Given this, he famously suggested that operant behavior is emitted, not elicited. Operant behavior was contrasted with elicited responses of the sort studied by Pavlov, which Skinner called respondents.

While one could quarrel over whether operant responses are truly ‘emitted’, it seems clear that spinally-mediated instrumental responding involves a shock elicited response (a respondent). It is also clear that the learning depends on the R-O relation and, in this way, we have met the criteria for instrumental behavior. Yet, if the behavior is respondent in nature, it should not be considered an operant. More generally, this suggests that the terms instrumental and operant should not be treated as synonyms (Fig. 7); that operant behavior involves additional criteria and is best viewed as a subset of instrumental behavior (Grau, 2003, 2010; Grau & Joynes, 1996, 2005a, 2005b). Elsewhere we have noted how an ideal operant could be trained using a variety of reinforcers. Conversely, the same reinforcer (e.g., food) could be used to train many different behaviors. In this case, both the response and reinforcer are relatively unconstrained at a biological level. Of course, we now know that even our prototypes of operant behavior (e.g., bar pressing for food) are biologically constrained in some important ways (Timberlake, 1999; Timberlake & Lucas, 1989). Nonetheless, brain-dependent operant behavior has a level of flexibility that spinal systems cannot match. We have, for example, no evidence that we can train a spinal rat to exhibit either an increase or decrease in flexion magnitude using the same outcome. Nor is there evidence that the same response can be established using a range of reinforcers (outcomes). Our conclusion is that spinal learning is biologically constrained and relies on a prepared system.

Fig. 7.

The proposed relation between instrumental and operant behavior. The four criteria for learning about a response-stimulus (a.k.a. response-outcome [R-O]) relation (instrumental behavior) were listed earlier in Table 1. It is assumed that R-O learning can affect both biologically prepared behaviors, involving an elicited response (a respondent), and unprepared systems. The latter support a wider range of behavioral responses and learning can be reinforced using a variety of outcomes (reinforcers) (Advanced Criteria 5 and 6). A behavior system that meets Criteria 5 and 6 allows the organism to operate on its environment in a flexible manner, as implied by Skinner's definition of operant behavior (Skinner, 1938). From this perspective, operant behavior represents a subset of instrumental learning and both depend on a common set of core criteria (1-4). While this is represented in terms of an embedded Venn diagram, it is recognized that the boundary is fuzzy and a continuum of possibilities exist. Spinal learning would seem to lie on one end of this continuum, meeting the basic criteria (1-4) for instrumental behavior within a biological prepared system, but lacking the flexibility (Criteria 5 and 6) exhibited by brain-dependent operant behavior.

Preparedness, of course, also relates to our earlier discussion of Pavlovian mechanisms. There it was suggested that non-associative mechanisms rely on a pre-existing link, whereas associative learning was traditionally seen as building upon a de novo link. The former is prepared, while the latter is less so. From this view, those examples of instrumental behavior that allow for flexibility, that are sensitive to alterations in reinforcer value (Colwill & Rescorla, 1986), and are readily inhibited (Hearst, 1975), naturally align with the associative view.

Our analysis has another mechanistic implication. At a behavioral level, we have described the learning as involving a form of punishment (passive avoidance). From this perspective, rats are less likely to exhibit a downward leg movement because it causes nociceptive stimulation. Yet, at a mechanistic level, it seems unlikely that the learning involves a loss (or inhibition) of a behavioral response (leg extension). Instead, our Pavlovian analysis posits that the instrumental contingency is effective because an index of leg position gains the capacity to drive a flexion response. This mirrors a common account of passive avoidance in intact rats. For example, if a rat is given shock whenever it enters the dark (preferred) side of a chamber, it soon learns to not enter the dark side. Again, in behavioral terms, the learning can be described as a loss of a response (entry). But at a mechanistic level, it is typically assumed that an active process (the Pavlovian conditioning of fear to the dark context) underlies the behavioral change (Estes, 1944, 1969).

6.4. Introducing Temporal Regularity Can Counter the Effect of Uncontrollable Stimulation

At the heart of Pavlovian conditioning is a form of temporal predictability—that the CS predicts the occurrence of the US. The implication is that spinal systems are sensitive to temporal relations. Recently, Kyle Baumbauer discovered that spinal systems can discriminate whether a stimulus occurs in a temporally regular or irregular manner (Baumbauer, Huie, Hughes, & Grau, 2009). He further found that predictable stimulation affects spinal plasticity in a manner that parallels the effect of instrumental training, producing a beneficial effect that enables learning and blocks the adverse effects of uncontrollable stimulation and inflammation.

The intermittent shock schedule we have used to induce a learning impairment was modeled after the pattern of stimulation observed in a typical master subject (Crown et al., 2002b). Inspection of the data from individual subjects revealed that master rats exhibited a flexion response within 80 msec of shock onset. Over the first 5-10 min of training, subjects typically respond at a rate of roughly 30 responses per min (0.5 Hz). Given these parameters, we developed a program that would present 80 msec shocks at a frequency of 0.5 Hz (an ISI of 2″). Because response duration varies as master rats acquire the instrumental response, we presented the shocks on variable time (VT) schedule (0.2-3.8 s, rectangular distribution). Under these conditions, the shock is both uncontrollable and unpredictable. Using this shock paradigm, we showed that just 6 min of stimulation (180 shocks) has a lasting, inhibitory, effect on spinal plasticity.

Baumbauer wished to relate these effects to studies within the pain literature, where researches have used electrophysiological stimulation of the sciatic nerve to examine spinal plasticity (Sandkuhler et al., 1997). To simplify his analysis, he presented shocks in a regular manner, spaced 2 s apart (a fixed time [FT] 2″ schedule). As expected, he found that 180 shocks, given at an intensity that engages pain (C-fibers), induces a robust learning impairment (Baumbauer et al., 2008). But when he examined the impact of varying shock number, a surprising outcome emerged. He knew from prior using cutaneous stimuli that increasing shock number enhances the learning deficit observed after VT stimulation (Crown et al., 2002b). In contrast, when the duration of electrophysiological stimulation was increased from 180 to 900 FT shocks the deficit disappeared (Baumbauer et al., 2008). Exactly the same pattern was observed when shocks were given on a FT 2″ schedule using cutaneous electrodes (Baumbauer et al., 2009). In both cases, an equal number of VT shocks produced the usual learning deficit.

Because 180 FT shocks induced a learning deficit, the fact no deficit was observed after 900 shocks suggests that the additional stimulation (720 FT shocks) had a restorative effect. Supporting this Baumbauer et al. (2009) showed that exposure to 720 FT shocks can reverse the learning deficit induced by 180 VT shocks. Conversely, 720 FT shocks given prior to VT stimulation blocks the induction of the learning deficit, and this protective effect lasts at least 24 hrs. Thus, just as behavioral control can prevent and reverse the learning deficit, so too can a period (24 min [720 FT shocks]) of FT stimulation.

The fact extended training was needed to induce an effect of regular stimulation suggested that a form of learning might be involved. As an initial test of this hypothesis, Baumbauer et al. (2009) examined the impact of drug manipulations known to influence neural plasticity. He had already established that exposure to 720 FT shocks has a lasting protective effect, that blocks the adverse effect of VT stimulation given 24 hrs later. He hypothesized that this lasting effect depends upon a structural change and protein synthesis. Supporting this, he showed that administration of the protein synthesis inhibitor cycloheximide immediately after FT stimulation blocked its long-term effect. So too did pretreatment within the NMDA antagonist MK-801. And like behavioral control, the protective effect of FT stimulation was eliminated by pretreatment with TrkB-IgG, which again implicates the neurotrophin BDNF.

In intact rats, it is well known that subjects can learn that stimuli occur at regular intervals. Here, it is thought that a cue indicative of temporal duration functions as a kind of Pavlovian CS, a form of learning known as temporal conditioning (Pavlov, 1927). Our work suggests that spinal mechanisms are also sensitive to temporal relations and, while further work is needed to reinforce this hypothesis, may support temporal conditioning. The capacity to detect temporal regularity could be linked to the central pattern generator (CPG) used to support stepping behavior. If this is true, then FT stimulation may bolster CPG activity and promote stepping after injury. This hypothesis also fits well with data demonstrating that step-training can benefit recovery and promote BDNF expression (Edgerton et al., 2004; Gomez-Pinilla, Ying, Roy, Molteni, & Edgerton, 2002).

Independent of how spinal mechanisms discriminate regular and irregular stimulation, it is clear that these two forms of stimulation can have divergent effects on spinal plasticity and that these effects mirror the consequences of controllable/uncontrollable stimulation. Predictable/controllable stimulation engages a BDNF-dependent protective/restorative effect whereas unpredictable/uncontrollable stimulation inhibits spinal plasticity.

6.5. Uncontrollable Stimulation May Induce a Form of Central Sensitization

Why does uncontrollable/unpredictable nociceptive stimulation inhibit spinal instrumental learning? Our first hypothesis was that it induces an antinociception that reduces outcome (shock) effectiveness. We examined this possibility by exposing transected rats to variable intermittent shock and testing tail-withdrawal from radiant heat (Crown et al., 2002a). We found no evidence of antinociception. When we then tested reactivity to mechanical stimulation applied to the hind paws, we observed that variably shocked rats were more responsive than the unshocked controls. As discussed above, enhanced mechanical reactivity (EMR) is a hallmark of another phenomenon, the diffuse over-excitation (central sensitization) induced by peripheral inflammation (LaMotte et al., 1991; Willis, 2001).

Because variable intermittent shock produces EMR, we hypothesized that it induces a state akin to central sensitization. This could disrupt the acquisition of selective response modifications by broadly saturating NMDA-receptor mediated plasticity. If this account is true, pretreatment within an NMDA antagonist (MK-801) should block the development of the learning deficit. Adam Ferguson showed this was true (Ferguson, Crown, & Grau, 2006). Likewise, inducing peripheral inflammation using a treatment (intradermal capsaicin) that leads to central sensitization should, like variable shock, impair learning. That too is true (Hook, Huie, & Grau, 2008).

Above, we showed that behavioral control, and an extended exposure to regular stimulation, can prevent and reverse the learning deficit produced by variable shock. If variable stimulation impairs learning because it induces a central sensitization-like state, these manipulations should also prevent, and reverse, the learning impairment induced by peripheral capsaicin. That too is true (Baumbauer & Grau, 2011; Baumbauer et al., 2012; Hook et al., 2008). More important from a clinical perspective, both behavioral control and temporal regularity attenuate the EMR induced by capsaicin.

Our work shows that controllable/predictable stimulation fosters spinal plasticity and inhibits the learning impairment and EMR induced by peripheral inflammation. In these ways, the stimulation appears to have an adaptive effect. Conversely, exposure to uncontrollable/unpredictable stimulation induces behavioral signs of central sensitization (EMR) and impairs instrumental learning. Because these effects could foster the development of chronic pain in humans, and undermine recovery after spinal injury, the consequence of stimulation appears maladaptive (Ferguson et al., 2012b). Yet, from an evolutionary perspective, the most adaptive response to stimuli that are both uncontrollable and unpredictable may be an inhibition of plasticity. In this sense, the biological reaction to unpredictable/uncontrollable stimulation seems adaptive.

Our focus has been less on the specifics of instrumental conditioning (how training alters the S-R circuit) and more on the general consequences of training. Our work suggests that exposure to controllable/predictable stimulation generally enables spinal plasticity. For example, instrumental training bolsters learning when subjects are tested on the opposite leg with a higher criterion and blocks the learning deficit induced by variable intermittent tailshock (Crown et al., 2002a). Likewise, variable intermittent shock to one leg inhibits learning when subjects are tested on the opposite leg (Joynes et al., 2003) and variable shock to the tail has the same effect (Crown et al., 20002b). These effects are both general and long lasting (24 hrs or longer). What seems modified is the capacity for change; whether plasticity is enabled or inhibited. Regulatory processes of this sort, that involve general changes in plasticity (i.e., the plasticity of plasticity), reflect what is sometimes called metaplasticity (Abraham & Bear, 1996).

6.6. Uncontrollable Stimulation Impairs Recovery After Spinal Cord Injury

We have also examined whether uncontrollable stimulation affects recovery after spinal cord injury (Grau et al., 2004). This is clinically important because spinal injuries are often accompanied by tissue damage that could provide a source of noxious input. To explore this issue, we exposed the lower thoracic spinal cord and applied a force that bruised the spinal tissue, producing a contusion injury that emulates the type of injury most often observed in humans. This injury normally produces a near-complete paralysis that begins to wane after 3 to 4 days. Locomotor behavior (as assessed by the BBB locomotor scale [Basso, Beattie, & Bresnahan, 1995; Ferguson et al., 2004]), and other indices of physiological function, generally shows some improvement over the next week, reaching an asymptote 2-3 weeks after injury. Using this paradigm, we found that just 6 min of variable intermittent tail shock given 24 hrs after injury profoundly undermines recovery, producing an effect that is evident 6 weeks later (Grau et al., 2004). Uncontrollable stimulation also impaired the recovery of bladder function, increased the incidence of limb rigidity (spasticity), and led to greater tissue loss at the site of injury. Importantly, allowing rats behavioral control over the nociceptive stimulation eliminated these adverse effects. Uncontrollable stimulation appears to impair recovery, in party, be down-regulating BDNF signaling (Garraway et al., 2011).

Given these results, we hypothesized that inhibiting nociceptive signaling with morphine could have a protective effect. Contrary to our hypothesis, morphine given at a dose that blocked behavioral reactivity to uncontrollable shock did not prevent its adverse effects on recovery (Hook et al., 2007). Moreover, and clinically alarming, morphine treatment per se impaired locomotor recovery and general health. Hook and her colleagues are currently exploring the mechanisms that underlie this paradoxical effect of morphine treatment. Research suggests that it may be linked to glial activation, and specifically to the pro-inflammatory cytokine interleukin-1. Supporting this, Hook et al. (2011) demonstrated that pre-administration of an IL-1 receptor antagonist blocked the morphine-induced attenuation of locomotor recovery.

Further work is needed to explore how learning variables interact with behavioral treatments designed to promote recovery after injury. Research suggests that step-training can engage a spinal CPG to foster locomotor behavior after injury (Edgerton et al., 2004). Application to humans has proven challenging, in part, because we do not fully understand the factors that engage/promote this behavioral system (Galvez, Budovitch, Harkema, & Reinkensmeyer, 2011). Our work suggests that exercise alone may not be sufficient; that for training to have a long-term beneficial effect, it must involve response-contingent stimulation. Behavioral control and temporal predictability are also relevant to treatments designed to minimize muscle atrophy; if noxious stimulation is given in an unpredictable and/or uncontrollable manner, it could have an unanticipated adverse effect. Finally, recent work has shown how the introduction of brain-dependent instrumental reinforcement can foster active recuperative processes that take advantage of surviving circuits to promote recovery (van den Brand et al., 2012). These forms of physical therapy seek to produce lasting behavioral changes, an aim shared with traditional research within the field of learning.

7. Summary

My review has focused on studies of spinal plasticity, asking whether these examples of plasticity count as learning. On the one hand, few would contest that this work is clinically important. But on the other, not all are at ease with the idea that spinal neurons can learn. I have suggested that this stems, in part, from a view that tacitly assumes learning is associative in nature. If a strong stance is taken on this issue, spinal mechanisms cannot learn. The behavioral changes observed in a host of invertebrate paradigms would also fail on this criterion.

7.1. Spinal Plasticity and the Criteria For Learning

I have suggested an alternative perspective that remains agnostic regarding the processes that underlie behavioral change. I began with a set of criteria that can be used to classify behavioral phenomena as instances of single stimulus, S-S, or R-O learning (Rescorla, 1988). I briefly reviewed work on single stimulus learning, noting recent advances demonstrating forms of plasticity analogous to those studied within the hippocampus and other brain structures (Ji et al., 2003). I then explored the mechanisms that underlie an example of spinally-mediated Pavlovian conditioning, suggesting that it reflects a form of protection from habituation. Is this an instance of learning? If one assumes Pavlovian conditioning can be encoded in multiple ways, then non-associative mechanisms would seemingly count. Further, if one is willing to grant non-associative mechanisms can produce simpler forms of learning (e.g., habituation and sensitization) it would seem odd to deny their potential role in more sophisticated learning systems.

I then examined whether spinal mechanisms are sensitive to R-O (instrumental relations). Using converging manipulations (master/yoke paradigm; experimentally manipulating R-O contiguity) I showed that the R-O relation was important. Research has shown that the behavioral change (an increase in response duration) is reinforced by shock onset, not offset (Grau et al., 1998). From this, I argued that, to register an event is controllable, the system must have an index of leg position (the R), which I suggested is provided by proprioceptive cues. From this perspective, a stimulus that is presented in a regular proprioceptive context is registered as controllable whereas an outcome given in a variable context is classified as uncontrollable. This analysis moves the detection of behavioral control to sensory processes, which may allow the organism to directly perceive some R-O relations (Grau et al., 2012). This hypothesis is consistent with an emerging view within the spinal cord literature, which posits that proprioceptive input is integrated online to form a temporal-spatial “image” to guide limb dynamics (Edgerton et al., 2004). Within this system, motoneuron function can be adjusted through a form of back-propagation within the dendritic tree, mediated by NMDAR-dependent/Hebbian synaptic plasticity (Windorst, 2007). The dynamic updating of this internal map can be seen as a form of instrumental learning.

Further work revealed that spinal mechanisms are also sensitive to temporal relations. When spinally transected rats are exposed to an extended series of shocks (e.g., 720-900), only variable shock induces a learning impairment; regularly spaced shock does not (Baumbauer et al., 2009). Further, exposure to regular stimulation has both a protective and restorative effect, that counters the learning impairment observed after variable stimulation. These findings are consistent with other work demonstrating that temporal regularity can affect neural function. For example, low frequency electrophysiological stimulation typically induces LTD. But when stimuli are presented in an irregular (Poisson) manner, LTP emerges (Perrett, Dudek, Eagleman, Montague, & Friedlander, 2001).

Importantly, we have shown that our work is clinically relevant (Baumbauer et al., 2012; Grau et al., 2004; Hook et al., 2008). Both behavioral control, and regular stimulation, inhibit the learning impairment and EMR produced by peripheral inflammation. Likewise, uncontrollable, but not controllable, stimulation impairs recovery after a contusion injury.

Just as our work suggests that S-S relations can be learned about in multiple ways, I argued that R-O relations can be encoded by different mechanisms. Instrumental learning within the spinal cord appears to reflect a modification of an elicited behavior, and in this way reflects a form of respondent conditioning. Brain-dependent systems allow a level of flexibility that far surpasses the functional capacities of spinal mechanisms. This difference can be characterized in terms of biological preparedness; R-O learning within the spinal cord modifies a pre-existing (prepared) behavioral response to stimulation whereas brain-mediated performance seems relatively unconstrained. Likewise, Pavlovian conditioning in intact rats generally develops over a range of circumstances and can be established using a variety of CSs (lights, odors, auditory cues).

7.2. A Neurofunctionalist Approach

We have labeled the process-oriented approach adopted here neurofunctionalism (Grau & Joynes, 2005a, 2005b). Functionalism because we assume that behavioral and neurobiological studies require a detailed description of the functional properties of the underlying behavioral/psychological process. In the absence of such a description, I would argue that attempts to link neurobiological observations with behavior will be fraught with problems.

The approach is, of course, consistent with the methods of cognitive neuroscience. However, that label implies a focus on “higher” cognitive processes that seems outside the realm of spinal learning. Similarly, while those who study lower invertebrates can adopt a process-oriented view, it is doubtful that most would be comfortable characterizing this endeavor as a version of cognitive neuroscience. Within the current framework, cognitive neuroscience represents an example of a neurofunctionalist approach, one that focuses on the role of higher cognitive systems.

Function also has additional connotations with merit (Benjamin, 1988). A full understanding of behavioral/psychological phenomenon should define more than the efficient, formal, and material causes (Kileen, 2001). It should also speak to the final, or ultimate cause— why organisms evolved a particular capacity and how it works within a behavioral system. I have suggested that a major difference between spinal and brain-dependent learning stems from the relative level of biological preparedness. In making this claim, I am assuming that our theories of learning (at a behavioral, formal, and neurobiological level) must reference the limits and constraints imposed by the organism's genetic history.

7.3. Promoting a Broader Perspective

In the end, the key issue is whether an experience at time-1 impacts behavior at time-2. If the experience is behaviorally/psychologically interesting, it would seem worthy of study and relevant to those interested in learning. The acquisition of singing in birds has never fit well within the associative view. Thankfully, there were researchers for whom this was unimportant and the study of song learning has flourished.

If there is a fault, it is that the vestiges of associationism has promoted a singular focus that has led us to neglect the role of other processes. If there is surprise, it is that this influence remains unexamined, unconsciously accepted and fueling an air of superiority and dismissal—as if a map of the mind was in hand. To neurobiologists I would say, if you find associative learning an attractive concept, adopt an established behavioral model. If, however, you seek to chart new territory, attend to core methodological issues, but do not fret too much as to whether the process under study is associative or not. If the phenomenon is interesting and clinically relevant, ignore the detractors. I know that those with a SCI are not concerned with whether our work counts as (associative) learning or not. If the research helps us design more effective treatment protocols, they see value.

Our work suggests that low-level mechanisms within the spinal cord are sensitive to S-S and R-O relations, that learning has a lasting effect on behavioral potential (a metaplastic effect), and that spinal systems can discriminate whether stimuli occur in a regular or irregular manner. Relative to the brain, the lumbosacral spinal cord is a simple structure. Given this, it could be argued that it's mechanistic capacities reflect processing abilities inherent to any neural assembly; that it reveals the features and qualities a neural network is designed to extract and store. Such a view suggests an alternative perspective regarding the evolution of function within higher brain systems—specialization of function may generally reflect a quantitative enhancement in a pre-existing capacity rather than the emergence of a qualitatively distinct process.