Disentangling indirect versus direct effects of somatosensory cortex microstimulation on neurons in primary motor and ventral premotor cortex

Abstract

Objective.

Intracortical microstimulation in the primary somatosensory cortex (S1-ICMS) is being developed to provide on-line feedback for bidirectional brain-machine interfaces. Because S1-ICMS can alter the discharge of the motor cortex neurons used to decode motor intent, successful application of S1-ICMS feedback requires understanding the modulation it produces in motor cortex neuron activity.

Approach.

We investigated the effects of S1-ICMS on neurons in both the primary motor cortex (M1) and the ventral premotor cortex (PMv) during a task in which some trials were instructed with visual cues and other trials with S1-ICMS.

Main Results.

We observed both indirect modulation during and/or after ICMS trains, as well as direct modulation time-locked to the individual S1-ICMS pulses within trains, with all possible combinations of the two types of modulation found among the majority of M1 and PMv neurons. Indirect effects were more prevalent and larger than direct effects. When S1-ICMS produced both indirect and direct modulation in the same neuron, the effects could both be excitatory, both inhibitory, or one excitatory and the other inhibitory. By simulating direct effects, we isolated the concurrent indirect effects, revealing that isolated direct effects failed to account for isolated indirect effects. Furthermore, indirect effects could be present 1 second or more after ICMS trains had terminated, when no direct effects could have occurred. Although the performance of movement decoders trained on visually-instructed trials was poor when applied to ICMS-instructed trials, decoders trained on ICMS-instructed trials performed well on ICMS-instructed trials, indicating that S1-ICMS altered the discharge of M1 and PMv neurons but did not degrade the decodable information available.

Significance.

When decoding movement intent from neural activity in M1 and/or PMv, accounting for indirect and direct modulation may improve the ability of bidirectional brain-machine interfaces to incorporate artificial somatosensory feedback delivered with S1-ICMS and restore functional movement.

Introduction

Brain-machine interfaces (BMIs) designed for movement restoration seek to translate neural activity into commands for controlling external devices such as prosthetic limbs or computer cursors. Primary motor cortex (M1) is a logical choice for decoding intended movements because M1 has been shown to control voluntary execution of native limb movements, with neural activity varying in relation to a variety of kinematic and kinetic parameters including direction, position, speed, and force (Georgopoulos et al., 1982; Schwartz et al., 1988; Georgopoulos et al., 1992; Johnson et al., 1999; Moran and Schwartz, 1999; Sergio et al., 2005). Premotor cortex (PM) is another logical choice for decoding neural activity to control BMIs given that PM neurons encode movement planning for reaching and grasping, as well as cognitive functions including space perception, and learning conditional associations (Mitz et al., 1991; Rizzolatti et al., 2002; Cisek and Kalaska, 2004; Abe and Hanakawa, 2009). Even though BMIs have been successful in decoding movement intent from cortical neurons in M1 and PM to control external devices (Hochberg et al., 2012; Collinger et al., 2013), the movements generated are still not as precise, accurate, fast, or dexterous as natural movements.

One factor preventing current BMIs from achieving such performance is a lack of sufficient somatosensory feedback, which is crucial for dexterous grasping and manipulation of objects (Rothwell et al., 1982; Brochier et al., 1999). Intracortical microstimulation (ICMS) delivered in the primary somatosensory cortex (S1) is known to elicit somatosensory percepts both in macaque monkeys and in humans (Romo et al., 1998; Flesher et al., 2016; Callier et al., 2020; Hughes et al., 2021; Greenspon et al., 2024). ICMS in S1 (S1-ICMS) therefore is being developed as a means of providing somatosensory feedback for use in bidirectional BMIs that control dexterous neuroprosthetic hands (Hughes et al., 2020; Flesher et al., 2021; Pandarinath and Bensmaia, 2022). However, a recent study in human participants with spinal cord injuries found that S1-ICMS, in addition to evoking somatosensory percepts, modulated the discharge of many M1 neurons both directly (time-locked to the individual ICMS pulses) and indirectly (without such time-locking), also noting that such modulatory effects could degrade the performance of algorithms trained to decode movement intent based on the activity of M1 neurons in the absence of S1-ICMS (Shelchkova et al., 2023).

Here, we extended investigations of the extent, prevalence, and nature of the indirect and direct effects of S1-ICMS by applying new analyses to previously recorded datasets from neurologically normal Rhesus macaques performing randomly interleaved trials of reach-grasp-manipulate movements with or without trains of S1-ICMS. We addressed four primary questions: Do the indirect and direct effects of S1-ICMS occur only in M1, or in the more distant ventral premotor cortex (PMv) as well? Can direct effects be inhibitory as well as excitatory? Whereas direct effects, by definition, occur only while ICMS is being delivered, can indirect effects occur after the ICMS train has ended? Are indirect and direct effects, which may involve at least partially different pathways of transsynaptic transmission (see Discussion), independent of one another? Furthermore, by simulating direct effects, we were able to estimate isolated indirect versus direct effects and compare them in sign and magnitude. Finally, we investigated how indirect versus direct effects impacted movement decoding using classifiers trained on trials either without or with ICMS. As S1-ICMS is being developed to provide somatosensory feedback for bidirectional BMIs, a more complete understanding of indirect and direct effects of S1-ICMS on neuron activity in M1 and PM may improve the ability of decoders to decipher movement intent and generate the desired output even as somatosensory feedback is being delivered with S1-ICMS.

Methods

We performed new analyses on data recorded during a previously reported study in which two male Rhesus macaque monkeys (L and X) initially were trained to perform 4 reach-grasp-manipulate (RGM) movements instructed with visual cues, and subsequently were trained to perform the same RGM task instructed instead with ICMS (Mazurek and Schieber, 2017). Each of the 4 different RGM movements was instructed by ICMS delivered simultaneously through multiple microelectrodes on a different array implanted in S1, with different movements instructed by stimulation through different arrays. Methods used for behavioral training, neural recording, and delivering ICMS have been described in detail previously (Mollazadeh et al., 2011, Mazurek and Schieber, 2017), and are summarized here only briefly. All procedures for the care and use of nonhuman primates followed the Guide for the Care and Use of Laboratory Animals and were approved by the University Committee on Animal Resources at the University of Rochester, Rochester, New York.

Initial behavioral training for the reach-grasp-manipulate task with visual instructions

Each monkey initially was trained to perform the RGM task instructed by visual stimuli (LEDs), as described previously (Mazurek and Schieber, 2017). The monkey initiated each trial by reaching to a central home object and pulling it for a variable initial hold period lasting 700 to 1,500 ms. Then the monkey was instructed to reach to, grasp, and manipulate 1 of 4 peripheral objects by illumination of a ring of blue LEDs around the base of that object. The 4 objects were arranged at 45⁰ intervals 13 cm radially from the home object. The monkey pushed a button, turned a sphere, pulled a perpendicular cylinder, or pulled a coaxial cylinder. Upon manipulating an object, a ring of green LEDs around the base of that object immediately was illuminated to indicate to the monkey that the manipulation was complete. The monkey then was required to hold the object in the manipulated position for a final hold period of 700–1000 ms. After the end of the final hold period the monkey’s behavior was not strictly controlled; nevertheless, the monkeys typically held the peripheral object in the manipulated position until the reward delivery had been completed and only thereafter returned their hand to the center object to begin another trial.

Because our task paradigm involved no delay period, we chose to leave the blue instruction LEDs on until the end of the final hold period. The conjunction of green and blue LEDs then enabled the monkey to know immediately that he had manipulated the correct object, encouraging him to maintain the object in the manipulated position throughout the required final hold period. Then the ring of blue instruction LEDs turned off and 300 ms later a water reward was delivered, after which the trial ended. The ring of green LEDs turned off whenever the monkey released the object from its fully manipulated position. A 1000 ms inter-trial interval occurred before another trial could be initiated.

Trials for each target object were presented pseudo-randomly in blocks. Each block included one trial for each target object, and the order of the 4 instructed objects within a block was re-randomized between blocks. Errors occurred if the monkey failed to hold the home object for the required duration, failed to release the home object within 1000 ms of the instruction onset, contacted the wrong target object, failed to contact a peripheral object within 1000 ms of releasing the center object, or failed to hold the target object fully manipulated for the required final hold duration. When an error occurred, trials were aborted immediately, no reward was given, and the same object was instructed again on the next trial to prevent the monkey from avoiding trials involving a particular peripheral object. Therefore, trials following an error were excluded from analysis because the monkey could have known which object would be instructed. The RGM task was controlled by custom software written in TEMPO (Reflective Computing, Olympia, WA). This software also generated codes during each trial marking the final 700 ms of the initial hold period, instruction onset, movement onset, full object manipulation, end of the final hold, and end of the trial. These markers divided each trial into five task epochs: i) initial hold (IH), ii) reaction time (R), iii) movement time (M), iv) final hold (FH), and end-of-trial (EoT).

Neural recordings

Once trained to perform the RGM task instructed with visual cues, each monkey was implanted with multiple floating microelectrode arrays (FMAs, Microprobes for Life Sciences, Gaithersburg, MD) in the primary somatosensory cortex (S1), primary motor cortex (M1), and premotor cortex (PM) of the left hemisphere (Figure 1), as described previously (Mollazadeh et al., 2011). Each of these cortical areas was implanted with four 16-electrode FMAs (impedance ~0.5MΩ, 70% Pt, 30% Ir). In monkey L, all 4 PM FMAs were located in the ventral premotor cortex, whereas in monkey X only two PMv arrays were functional, and because no arcuate spur was present, the medial array (D) may have been in the dorsal premotor cortex. Nevertheless, for present purposes we will refer to all PM arrays as having been in PMv. The intracortical locations of electrode tips were confirmed by the presence of single- or multi-unit activity. In both monkeys, unit activity in the S1 arrays was evoked primarily by lightly stroking the skin (glabrous or hairy) in locations from the digits (for more lateral arrays) to the shoulder (for more medial arrays) and less often by passive joint rotations.

Figure 1. Location of microelectrode arrays implanted in Monkey L (A) and Monkey X (B).

Arrays (~2×2 mm) were implanted in the primary motor cortex (M1, blue), ventral premotor cortex (PMv, green), and primary somatosensory cortex (S1, yellow). Letters next to each array are used to identify that array. In Monkey X, data from ICMS delivered via array M (pale yellow) were not used in the present study because the high frequency (225 Hz) resulted in inter-pulse intervals (4.4 ms) too short for reliable assessment of time-locked changes in neuron spike probability. AS – arcuate sulcus, CS – central sulcus, IPS – intraparietal sulcus. Orientation arrows indicate medial (M) and caudal (C) directions. Modified with permission from Mazurek & Schieber, 2017.

Neural data were recorded with a Multi Acquisition Processor and Sort Client software (Plexon, Inc., Dallas, TX). After 1,000 – 32,000x amplification, waveforms that crossed a threshold set online by the user were sampled at 40 kHz from −0.2 to +0.6 ms around the threshold crossing and saved for offline sorting along with behavioral event markers generated by the TEMPO system. After off-line sorting (Off-line Sorter, Plexon, Inc., Dallas, TX), the signal-to-noise ratio (SNR) and the estimated percentage of inter-spike interval (ISI) violations (<1 ms) were used to classify each sorted unit as a definite single-unit (SNR ≥ 3 and no ISIs <1 ms), a probable single-unit (SNR ≥ 2.5 and <10% ISIs <1 ms), a multi-unit (SNR ≥ 1.5 and <90% ISIs <1 ms), or noise (SNR < 1.5 or ≥ 90% ISIs <1 ms) (Rouse and Schieber, 2016).

Behavioral training with ICMS instructions

Several months after the microelectrode arrays had been implanted, each monkey was trained to perform the RGM task using ICMS pulse trains delivered in S1 as instructions instead of the LED cues (Mazurek and Schieber, 2017). ICMS trains consisted of symmetric, biphasic, cathodic-leading, 200 μs per phase, 1–64 μA pulses delivered at a constant frequency. For monkey L, ICMS pulse frequency was 100 Hz for all target objects, whereas for monkey X, different pulse frequencies were used for each object: button – 75 Hz, sphere – 100 Hz, perpendicular cylinder – 150 Hz, coaxial cylinder – 225 Hz. ICMS pulses were produced by a constant-current IZ2 Neural Stimulator controlled by an RZ5 BioAmp Processor hosted by a PC running the OpenEx Software Suite (Tucker-Davis Technologies, Gainesville, FL).

Each of the 4 objects was instructed with stimulation delivered through electrodes on a different S1 array. Electrodes were selected based on recorded spike activity confirming their intracortical location. The current delivered through each electrode typically was limited by the combination of the electrode’s impedance and the compliance voltage available from the stimulator. Because we did not know in advance which electrodes or how much total current would be needed for the monkey to experience S1-ICMS, for each instruction current pulses were delivered simultaneously through a set of 3–7 electrodes on the same S1 array. At first, ICMS trains were delivered in conjunction with the LED cues. Stimulation on the S1 array assigned to the instructed target began simultaneously with illumination of the blue LED ring surrounding the instructed target object. Given that we did not know a priori what ICMS train duration would be required, and because previous studies had used S1-ICMS trains lasting 500 ms to evoke tactile percepts (Romo et al., 1998; Romo et al., 2000; Romo et al., 2004), we chose to continue ICMS trains throughout the reaction and movement task epochs until the monkey had contacted and manipulated an object, beginning the final hold epoch.

Over multiple subsequent daily sessions the LEDs were gradually dimmed until they could remain completely off while the monkey continued to perform well. To minimize frustration as the monkeys learned to use S1-ICMS instructions, the allowed reaction and movement times were increased. For monkey L, these times were increased to 10,000 ms each and then reduced gradually to 4,000 ms each as performance improved. Nevertheless, monkey L’s reaction time rarely exceeded 1,000 ms and movement time rarely exceeded 350 ms. For monkey X, allowed reaction and movement times were increased to 5,000 and 7,000 ms, respectively, and subsequently were reduced to 3,000 ms, although reaction time rarely exceeded 2,000 ms and movement time rarely exceeded 300 ms. At the end of training, S1-ICMS trains served as the only instruction for the correct target object on each trial, with no visual cue.

Ultimately, trials with only visual instructions or only S1-ICMS instructions could be interleaved randomly and the monkeys each continued to perform well. The limits on reaction and movement times given above then were applied in both visually- and ICMS-instructed trials. Whether visual or S1-ICMS, we refer to the instruction for the button as Instruction 1, for the sphere as Instruction 2, for the perpendicular cylinder as Instruction 3, and for the coaxial cylinder as Instruction 4. All data used in the present analyses were recorded in 4 sessions from monkey L and 3 sessions from monkey X consisting entirely of such randomly interleaved visually- or ICMS-instructed trials. Note that ICMS instructions were delivered only during the reaction and movement epochs whereas the visual-cue instructions persisted additionally through the final hold epoch.

Experimental design and statistical analysis

All analyses presented here were performed off-line using MATLAB (MathWorks, Natick, MA).

Identifying direct modulation in peristimulus time histograms

ICMS pulses delivered in one cortical region can modulate the discharge of neurons in other cortical regions transsynaptically (Kraskov et al., 2011, Zarzecki and Strick, 1978). We used peristimulus time histograms (PSTHs) triggered on the individual pulses of each S1-ICMS instruction to determine which M1 and PMv units were directly modulated at short latency by the ICMS pulses. Units were recorded during all four ICMS instructions, each delivered via electrodes on a different array in S1, leading to four different “neuron-array pairs” for each recorded neuron. We therefore constructed four PSTHs for each unit—one PSTH triggered on pulse artifacts from each of the four different ICMS instructions. Our stimulation and recording systems did not allow us to directly record the time of pulses from the four different S1-IMCS instructions, so we obtained timestamps for individual ICMS pulses by discriminating the waveforms of stimulation artifacts offline from M1 and/or PMv recording channels on which any spikes were much smaller than the ICMS artifacts. Each rhythmic train of artifacts sorted offline then could be unambiguously assigned to ICMS pulses from a given S1 array based on knowledge of which array was stimulated on each trial. The time window of each PSTH was chosen to span from 20% of the inter-pulse interval before the timestamp of the triggering ICMS pulse artifact until the timestamp of the next ICMS pulse artifact. A PSTH bin-width of 0.1 ms was chosen to provide high temporal resolution. Raw spike counts in each bin were divided by the total number of ICMS trigger pulses (nTriggers), such that the PSTH ordinate represented the probability of the neuron firing a spike in each 0.1 ms bin (Guggenmos et al., 2013).

In forming these PSTHs, we blanked stimulation artifacts that might have been mistaken for neuron spikes. Because our recording system did not include hardware artifact blanking, we developed a method for defining artifact blanking windows separately for each neuron-array pair that minimized blanking duration. We first chose a minimum blanking duration of −0.2 ms to +0.6 ms (the duration of snipped waveforms) surrounding the timestamp assigned to each stimulus artifact by Sort Client, blanking all waveforms within that window to ensure no artifact waveforms were included in the spike counts. On some channels, we observed that artifact waveforms could begin more than −0.2 ms before the timestamp and/or last longer than +0.6 ms after the timestamp (see Supplementary Figures S8B and S10). In these cases, the artifact would have distorted spike waveforms occurring immediately before and/or after the −0.2 to +0.6 ms window, artificially lowering spike counts in the bins adjacent to that window. We therefore expanded the duration of the blanking window by proceeding backward in time from −0.2 ms until reaching the first bin exceeding the 15^th percentile of all bins in the PSTH, and forward in time from +0.6 ms until reaching the first bin exceeding the 15^th percentile. Those 15^th percentile bins were assigned to be the start and end of the final adjusted artifact blanking window. Any spikes within this adjusted artifact blanking window (shaded pink in all PSTH figures) were excluded from subsequent analyses of the PSTH, though spike counts in those bins are still shown to illustrate the 15^th percentile thresholds. While adjusting the artifact window in this way may have caused us to miss some short-latency inhibition time-locked to ICMS pulses, it also conservatively prevented those bins as being falsely identified as short-latency inhibition time-locked to ICMS pulses. Though some of our blanking windows were longer than 2.0 ms (Schelchkova et al., 2023), the majority were shorter, allowing us to detect earlier onset effects (Figure 3 and Supplementary Figures S8B, S9A, and S10).

Figure 3. Peak and trough characteristics.

Peaks and troughs were considered significant if the PSTH crossed 2 standard deviations (horizontal dashed lines) beyond the mean (horizontal solid line) of the baseline period before the artifact window (left pink rectangle). This neuron-array pair showed a brief, early peak followed by a long, shallow trough. The onset latency (ON), offset latency (OFF), full width at half maximum (FWHM), and amplitude of each peak (brown) or trough (gold) were calculated as described in the text. Note that the early peak in the PSTH of this neuron-array pair would not have been detected had the artifact blanking window been extended to 2.0 ms as used by Schelchkova et al. (2023). (Ch109U1, L, 20150729)

A neuron-array pair was considered directly modulated by the individual ICMS pulses of a given instruction if the PSTH contained a significant peak or trough in spike probability. Excitatory peaks and/or inhibitory troughs were considered significant when ≥ 3 consecutive bins went beyond ± 2 standard deviations of the baseline mean (bins preceding the artifact window). In cases for which the mean minus 2 standard deviations was less than zero, we required 7 consecutive bins with spike probabilities of zero to identify a significant trough. For 59 neuron-array pairs we did not attempt to identify troughs because the total spike count was less than the number of bins in the PSTH (e.g. <100 spikes for 100 Hz pulse trains leading to 100 bins, or <133 spikes for 75 Hz pulse trains leading to 133 bins, etc.), resulting in many bins that contained zero spike probability by chance alone. Figure 2 shows example PSTHs in which the same recorded neuron was directly modulated (DM+) by ICMS pulses delivered via one S1 array for instruction 1 (Figure 2A) which produced a significant peak, but was not directly modulated (DM-) by ICMS pulses delivered via another S1 array for instruction 2 (Figure 2B).

Figure 2. Example peristimulus time histograms.

A) A PSTH from a definite single unit showing direct modulation (DM+) produced by S1-ICMS pulses delivered via electrodes on one array. B) A PSTH from the same unit showing no direct modulation effect (DM-) when S1-ICMS was delivered via electrodes on a different array. Across multiple RGM trials, this M1 unit fired a total of 1011 spikes (nSpikes) during ICMS Instruction 1 (A, button) and 455 spikes during ICMS Instruction 2 (B, sphere). Spike counts in each 0.1ms bin were converted to probabilities by dividing the count in each bin by the number of ICMS pulse artifacts used as triggers to form the PSTH (nTriggers). Pink rectangles show the artifact window during which spikes could not be reliably discriminated. An inset within each pink artifact window shows the 4 S1 arrays located along the central sulcus (CS), with the stimulating array colored red and the other arrays yellow. Horizontal black lines represent the mean (solid line) ± 2 standard deviations (dashed lines) of the baseline bins preceding the first artifact window. Inset at upper right shows the average waveform of the unit (dark gray), with overlapped individual spikes (light gray) and a 1 ms timescale bar below (black). (Ch78U2, L, 20150729)

We estimated the false-discovery rates of peaks and troughs by recompiling each PSTH after jittering the time of each spike by a random value drawn from a uniform distribution twice the duration of the PSTH (e.g. −10 ms to 10 ms for the examples shown in Figure 2). Then applying the same criteria for identifying direct modulation, a peak or trough was falsely discovered in 5.5% of these jittered PSTHs, which we deemed acceptable because the probability of data lying beyond 2 standard deviations of a normal distribution is 5%.

Measuring the latency, full width at half maximum, and amplitude of peaks and troughs.

We quantified the latency, full width at half maximum (FWHM), and amplitude of each significant peak or trough, as illustrated in Figure 3. Latencies were calculated as the time from the beginning of the artifact window to the first bin in which a peak or trough crossed ±2 standard deviations from the mean of the baseline spike probability. FWHM was calculated as the difference in time between the bins in which a peak or trough crossed 50% of its maximum or minimum value with respect to the baseline mean. We quantified amplitude as the maximum of a peak or the minimum of a trough minus the baseline mean, normalized as a percentage of the baseline mean. For peaks, we also calculated the cumulative probability of a spike as the sum of the probabilities in each bin from onset to offset of the peak.

Applying blanking windows to emulate ICMS artifacts in visually-instructed trials

On ICMS-instructed trials, but not visually-instructed trials, ICMS pulse artifacts could collide with and distort action potentials, preventing them from being discriminated offline and ultimately causing them to be lost from analysis. To permit comparison of firing rates during visually- versus ICMS-instructed trials, we therefore emulated collisions between ICMS artifacts and action potentials recorded during visually-instructed trials as follows: For each neuron-array pair separately, we constructed trains of blanking windows identical to the artifact blanking window applied to that pair’s PSTH. For each visually-instructed trial, the train of blanking windows began at the time of instruction onset and ended at the beginning of the final hold, with blanking windows occurring at the same frequency as the ICMS pulses used for the corresponding ICMS instruction. The train of blanking windows applied to visually-instructed trials thus emulated the train of ICMS artifacts that would have occurred during ICMS-instructed trials involving the same object. Any spike recorded during a visually-instructed trial that fell within a blanking window then was deleted, as if it had collided with an ICMS artifact. This blanking procedure led to the removal of 18.4% of spikes occurring during visually-instructed trials for monkey L, and 16.2% for monkey X. The resulting spike trains recorded during visually-instructed trials and filtered offline with blanking windows then were used in all subsequent analyses.

We also constructed PSTHs triggered on those same blanking windows to confirm that our ± 2 standard deviation criteria for identifying peaks and troughs in PSTHs provided an appropriate false discovery rate. For each neuron-array pair, we formed a PSTH of spikes from visually-instructed trials triggered on the individual blanking windows (visPSTH). We then applied our ± 2 standard deviation criteria to these visPSTHs in which no relationship could exist between the blanking windows created offline and the spike trains recorded during visually-instructed trials. A peak or trough was falsely discovered in 5.6% of these visPSTHs, which was similar to the false-discovery rate in PSTHs constructed using jittered spikes described above (5.5%).

Comparing neural activity during visually- vs. ICMS-instructed trials

In addition to direct modulation, time-locked to individual ICMS pulses in the range of several milliseconds, trains of S1-ICMS can also produce modulation of neuron firing rates over hundreds of milliseconds that is not time-locked to the individual pulses, referred to as indirect modulation (Shelchkova et al., 2023). We initially evaluated such indirect modulation (IM) by comparing the firing rates of each neuron-array pair during visually- versus ICMS-instructed trials involving the same object. For each neuron-array pair, we compared the discharge of the neuron during visually- versus ICMS-instructed trials in spike rasters and firing rate histograms aligned at the time of instruction onset. For statistical comparison, firing rates were calculated in each trial by dividing the spike count during each task epoch (IH, R, M, FH, or EoT) by the duration of that epoch. Mean firing rates across trials in the five task epochs of visually- versus ICMS-instructed trials then were compared using repeated measures analysis of variance (rmANOVA, p<0.01), followed by Tukey-Kramer post-hoc tests (p<0.01) to identify the specific epochs in which differences occurred. For additional visualization, firing rate histograms were formed by assigning the aligned spike times in each trial to 1 ms bins, dividing the spike count in each bin by the total number of trials, multiplying by 1000 bins/second to convert to Hz, and smoothing by convolution with a 50 ms Gaussian kernel.

In monkey X, ICMS instruction 4 (coaxial cylinder, array M) consisted of pulses delivered at 225 Hz, i.e. with a 4.4 ms inter-pulse interval. Given the time occupied by artifact blanking windows, we considered this inter-pulse interval too brief to permit adequate assessment of spike probabilities in PSTHs or firing rates during task epochs. Therefore, for monkey X, data from instruction 4 were excluded from subsequent analyses.

Simulating direct modulation effects

During the reaction and movement task epochs, while ICMS trains were being delivered, indirect and direct modulation could occur concurrently. To disentangle indirect and direct effects, we simulated the change in spike trains recorded during visually-instructed trials that would have resulted from direct effects in the absence of any indirect effect. For each neuron-array pair with direct modulation (DM+), we probabilistically simulated the times at which ICMS pulses would have caused a neuron to fire or to be inhibited had ICMS been delivered on visually-instructed trials. For each visually-instructed trial, we simulated ICMS pulses at the same frequency as the real ICMS pulses delivered on ICMS-instructed trials involving the same object (monkey L, 100 Hz for all four instructions; monkey X, 75, 100, or 150 Hz for instructions 1, 2, or 3, respectively) from the beginning of the reaction time epoch until the end of the movement epoch of the visually-instructed trial. Following each simulated ICMS pulse, we probabilistically inserted spikes into and/or removed spikes from the spike train recorded during the visually-instructed trial based on the peak(s) and/or trough(s), respectively, present in the PSTH of that neuron-array pair compiled from real ICMS pulses during ICMS instructed trials (e.g. Figure 2A), which we refer to here as the icmsPSTH.

We determined the times at which to insert a simulated spike as follows: We subtracted the baseline mean of the icmsPSTH from all bins in that PSTH to remove the component of spike probability that was not time-locked to ICMS pulses. We then set all time bins of spike probability outside of identified peaks (i.e. before peak onset or after peak offset) to 0, producing a peaks-only PSTH (poPSTH) of increased spike probabilities. Based on these isolated peaks, we constructed a piecewise probability density function (PDF) that we could sample following each simulated ICMS pulse. To ensure each PDF summed to 1, we first summed all bins in the poPSTH and subtracted that total from 1 to calculate the outstanding probability. We assigned that outstanding probability to the bin immediately preceding 0 ms (t = −0.1 ms). Explicitly, we defined the PDF derived for each icmsPSTH with one or more peaks as follows:

$$f_{\mathit{\text{poPSTH}}}\left(t\right)=\{\begin{array}{ll}\hfill \mathit{\text{poPSTH}}\left(t\right),& t>0\mathit{\text{and within peak}}\hfill \\ 1-\sum \mathit{\text{poPSTH}}\left(t\right),\hfill & t=-0.1\hfill \\ \hfill 0,& \mathit{\text{otherwise}}\hfill \end{array}$$

where t varies from −0.1 to the inter-pulse interval of simulated ICMS pulses. At the time of each simulated ICMS pulse, we sampled a random time from this PDF. If a time was drawn with a negative latency (-0.1 ≤ t < 0 ms), no spike was inserted. Otherwise, a spike was inserted at the time of the simulated ICMS pulse plus the randomly chosen time. If the inserted spike occurred within 1 ms of an existing spike, we shifted the inserted spike slightly earlier or later in time such that a 1 ms refractory period was no longer violated.

The same approach was not applicable to troughs. Instead, we determined when to remove a spike from the spike train recorded during each visually-instructed trial as follows: Beginning at the time of each simulated ICMS pulse, we identified any spikes occurring within troughs identified in the icmsPSTH. For each such spike, we found the spike probability in the corresponding bin of the icmsPSTH at that specific latency, and calculated the difference between the baseline mean spike probability of the icmsPSTH and the spike probability in that bin. We divided that difference by the baseline mean to capture the normalized probability that the ICMS pulse would have prevented a spike from occurring at that particular latency. We then sampled a binomial distribution using that normalized probability value as the probability of success and removed the spike if the Bernoulli trial returned positive. Consider, for example, that a spike recorded during a visually-instructed trial occurred 3.2 ms after a simulated ICMS pulse and that a trough was present in the icmsPSTH from t = 2.1 to 4.5 ms following real ICMS pulses. If the baseline mean spike probability of the icmsPSTH was 0.1 and the spike probability at t = 3.2 ms in the icmsPSTH was 0.02, we would subtract 0.1 – 0.02 = 0.08 and normalize as 0.08/0.1 = 0.8 = 80%, estimating that real ICMS pulses inhibited spikes occurring at a latency of 3.2 ms 80% of the time. We then would perform one Bernoulli trial sampling from a binomial distribution with the probability of success being 0.8. If the outcome was a success, we would remove the spike. If the outcome was a failure, we would keep the spike.

We assessed the simulation by compiling a PSTH from the simulated trials in which spikes had been added to and/or subtracted from visually-instructed trials (simPSTH), triggering at the times of the simulated ICMS pulses. We then compared the simPSTH to the icmsPSTH. An example is shown in Figure 7A. The baseline means of the simPSTH and icmsPSTH differ slightly, reflecting different average firing rates of the neuron during visually- versus ICMS-instructed trials, which in part may represent indirect modulation. But the timing and amplitude of the peaks and/or troughs in the simPSTH are similar to those in the icmsPSTH.

Figure 7. Isolating indirect versus direct effects.

We created trials simulating the direct effects of ICMS during spike trains recorded in visually-instructed trials. A) For an example neuron-array pair, PSTHs constructed from i) the original visually-instructed spike trains (visPSTH, blue) triggered on blanking windows (see Methods), ii) ICMS-instructed trials (icmsPSTH, red) triggered on ICMS artifacts, and iii) simulated trials (simPSTH, purple) triggered on blanking windows. The three peaks in the simPSTH closely emulate the peaks in the icmsPSTH, but outside the peaks the simPSTH is identical to the visPSTH. For each PSTH, nTriggers is the number of triggers and nSpikes is the number of spikes in the PSTH. B) For the same neuron-array pair, rasters show spike trains from visually-instructed (blue), ICMS-instructed (red), and simulated (purple) trials with the corresponding firing rate (FR) histograms overlapped in the panel below. In that panel the solid black line represents the isolated indirect effect calculated by subtracting the mean of the simulated FR histogram from that of the ICMS FR histogram; the dotted black line represents the isolated direct effect calculated by subtracting the mean of the visual FR histogram from that of the simulated FR histogram. Bar graphs in the bottom panel show average firing rates in each task epoch for visual, ICMS, and simulated trials. PSTHs, raster plots, and bar graphs were constructed as described in relation to Figure 4. (Ch33U1, L, 20150729, PMv, PSU.) C) Scatter plots of average firing rates during the reaction (top) and movement (bottom) task epochs of neuron-array pairs in M1 (blue) and PMv (green). For each neuron-array pair, a point in the left panels (i) represents its average firing rate on visual vs. ICMS trials, in the middle panels (ii) visual vs. simulated trials, and in the right panels (iii) simulated vs. ICMS trials. A vertical vector drawn from the line of unity slope to a given point in the left panels (i) then would represent the sign and amplitude of that neuron-array pair’s combined indirect and direct effects, middle panels (ii) the sign and amplitude of that pair’s isolated direct effect, and in the right panels (iii) its isolated indirect effect. Dots represent monkey L and + represent monkey X. Red circles surround points representing the example neuron-array pair illustrated in A and B. D) Scatter plots show the isolated indirect effect versus the isolated direct effect for each DM+ neuron-array pair. C and D include only those pairs for which the timing and amplitude of the peaks and troughs in the simPSTH were statistically similar to those in the icmsPSTH (p<0.01, see Methods).

To assess this similarity quantitatively, we first subtracted their respective baseline means from the simPSTH and the icmsPSTH. Then we subtracted the remaining spike probability in each bin of the simPSTH from that in the corresponding bin of the icmsPSTH and took the absolute value of those differences. We summed these absolute differences across all bins from the end of the first artifact window to the beginning of the next to give a total distance, d, between the two mean-subtracted PSTHs. We then randomly shuffled the spike probabilities among the bins in the mean-subtracted simPSTH and recomputed d, repeating this shuffling 1000 times to produce a distribution of d under the null hypothesis that the two mean-subtracted PSTHs were no more similar than expected from chance alone. If the value of d from the unshuffled, mean-subtracted PSTHs was below the 1^st percentile of this distribution, we concluded that the timing and amplitude of the peaks and troughs in the simPSTH were more similar to those in the icmsPSTH than could be attributed to chance alone. Such was the case for 178/213 (84%) of the DM+ neuron-array pairs (Monk L, 81/89 (91%) M1 pairs, 36/45 (80%) PMv pairs; Monk X, 25/35 (71%) M1 pairs, 36/44 (82%) PMv pairs).

Using only those 178 DM+ neuron-array pairs, we compared their simulated spike trains to those recorded on visually-instructed trials, which could have no indirect or direct modulation effects, and to ICMS instructed trials, which could have both direct and indirect modulation effects. Subtracting the firing rate during visually-instructed trials from that during simulated trials leaves the isolated direct effect; subtracting the firing rate during simulated trials from that during ICMS-instructed trials leaves the isolated indirect effect.

Decoding Analysis

We decoded which RGM movement was being performed using visually-instructed trials, ICMS-instructed trials, and simulated trials from each monkey, separately. The firing rates of all neurons recorded from both M1 and PMv in the first session from each monkey were used regardless of whether each neuron showed indirect modulation, direct modulation, both, or neither. Decoding was performed using linear discriminant analysis (LDA) in 100 ms non-overlapping intervals based on neural activity realigned separately on each behavioral event (IH, R, M, FH, EoT), beginning 800 ms prior to and ending approximately 3000 ms after instruction onset. Neural firing rates were calculated as the number of spikes in the 100 ms prior to the current time step divided by the duration of the bin, 0.1 sec. Principal component analysis (PCA) was performed to reduce the dimensionality of the firing rates to the minimum number of PCs needed to capture 95% of the variance, capping the maximum number of PCs used at 30. Because we used only trials following a success, different number of trials were available for the four different objects. The LDA classifier then could learn to perform above chance by predicting the object with the most trials more often. Prior to training, we therefore balanced the classes by finding the object with the fewest trials and randomly choosing an equivalent number of trials involving the other objects. We trained separate classifiers using data from visually-instructed trials, from ICMS-instructed trials, or from simulated trials. Each trained classifier then was tested separately on trials of each type. We additionally trained a fourth classifier with randomly shuffled object labels to evaluate chance decoding performance. To provide an estimate of the variance in classification accuracy, each model was trained 1000 times using a randomly chosen 80% of the data for training and the remaining 20% for testing (with replacement) on each iteration.

Results

During each session, neuron spiking data were collected from all 16-channel microelectrode arrays implanted in M1 and PMv of each monkey. After offline sorting, each waveform was classified as a definite single unit (DSU), probable single unit (PSU), or multi-unit (MU) as described in the Methods. Units were tracked across consecutive days based on their mean waveforms, mean firing rates, autocorrelograms, and cross-correlograms with other units (Fraser and Schwartz, 2012). Units found to be present in multiple sessions were analyzed only once from the first session in which they appeared. In Monkey L, 83 units were analyzed—41 from M1 and 42 from PMv. The 41 M1 units included 9 DSUs, 8 PSUs, and 24 MUs. The 42 PMv units included 5 DSUs, 12 PSUs, and 25 MUs. In Monkey X, 69 units were analyzed—36 from M1 and 33 from PMv. The 36 M1 units included 6 DSUs, 14 PSUs, and 16 MUs. The 33 PMv units included 2 DSUs, 6 PSUs, and 25 MUs. We included all three types of units in the following analyses. ICMS instructions delivered through electrodes on the four different S1 arrays in each monkey provided 164 M1 neuron-array pairs and 168 PMv pairs in monkey L; 144 M1 pairs and 132 PMv pairs in monkey X. After excluding the neuron-array pairs involving instruction 4 for monkey X due to the high stimulation frequency (see Methods), only the remaining 108 M1 and 99 PMv neuron-array pairs in monkey X were included in subsequent analyses.

In neurons from both M1 and PMv, we observed that S1-ICMS produced modulation of two different types: 1) modulation of firing rate over hundreds of milliseconds not time-locked to ICMS pulses, which we refer to as indirect modulation (IM), and 2) modulation of spike probability on a millisecond time scale time-locked to the individual pulses of S1-ICMS, which we refer to as direct modulation (DM). Whereas both indirect and direct modulation could occur during the reaction and movement task epochs while ICMS was being delivered, indirect modulation could also occur during the final hold and end-of-trial epochs, after ICMS had terminated.

Direct modulation does not account for indirect modulation of M1 and PMv neurons

Figure 4 illustrates an M1 probable single unit during trials involving instruction 3. The PSTH of the neuron-array pair (Figure 4A) demonstrated a significant trough, representing an inhibitory direct effect time-locked to the individual ICMS pulses. Although this unit’s spike probability was reduced by individual ICMS pulses, its firing rate was higher throughout the reaction and movement task epochs while ICMS was being delivered compared to its firing rate throughout the corresponding epochs on visually-instructed trials with no ICMS (Figure 4B). During the subsequent final hold and end-of-trial epochs, however, the unit’s firing rate became lower during ICMS-instructed trials compared to visually-instructed trials. This neuron-array pair thus showed indirect modulation (p=1.0e-125, rmANOVA) not attributable to its direct effect, with significant differences in firing rate that changed, reversing sign from excitatory to inhibitory after the ICMS train had ended, and persisting for more than 1 second through the end-of-trial epoch, when neither ICMS nor visual instructions were present (Figure 4C, p<0.01, Tukey-Kramer post-hoc tests).

Figure 4. A neuron-array pair showing both excitatory indirect and inhibitory direct modulation.

A) PSTH for a probable single unit from M1 triggered on S1-ICMS pulses of Instruction 3 showing a significant trough. nTriggers is the number of ICMS pulses used as triggers; nSpikes is the number of spikes accumulated in the PSTH. The pink rectangle represents the artifact window during which spikes could not be reliably discriminated. An inset within the pink artifact window shows the 4 S1 arrays located along the central sulcus (CS), with the stimulating array colored red and the other arrays yellow. Horizontal black lines represent the mean (solid line) ± 2 standard deviations (dashed lines) of the baseline bins preceding the first artifact window. Inset at upper right shows the average waveform of the unit (dark gray), with overlapped individual spikes (light gray) and a 1 ms timescale bar below (black). B) Raster plot and corresponding firing rate histograms of unit activity during visually- (blue) and ICMS- (red) instructed trials. Note that although visually- versus ICMS-instructed trials have been sorted separately in the raster, the two were randomly interleaved during recording. Trial numbers therefore indicate only the cumulative number of trials in the raster and not the sequential order of the trials. Black dots in the rasters indicate 6 events during each trial: 1) Inital hold 700 ms prior to Instruction onset, 2) Instruction Onset, 3) Movement Onset, 4) Final Hold Onset, 5) Final Hold Offset, and 6) End of Trial. Those events separate 5 trial epochs: 1–2) Initial Hold (IH), 2–3) Reaction (R), 3–4) Movement (M), 4–5) Final Hold (FH), and 5–6) End of Trial (EoT). In the firing rate histograms, solid lines represent the mean firing rate while the surrounding shading indicate ± 1 standard error of the mean (SEM). Horizontal blue and red bars between the rasters and firing rate histograms represent the time during which the visual and ICMS instructions were present, respectively, with the wider segments indicating the range from minimum to maximum across trials. C) Averaged firing rates during each task epoch of visually- (blue) and ICMS- (red) instructed trials. Error bars represent ±1 SEM. Asterisks indicate epochs with significant differences on post-hoc analysis (p<0.01, Tukey-Kramer tests) performed after the global rmANOVA was found to be significant (p = 1.0e-125). (CH124U1, L, 20150730)

In different neuron-array pairs we found all possible combinations of indirect and direct effects, regarding their presence versus absence as well as their sign, excitatory versus inhibitory. Additional examples illustrated in the Supplementary Material include: a neuron-array pair with an inhibitory indirect effect, but no direct effect (Supplementary Figure S1); a pair with indirect modulation that changed sign over the course of the behavioral trials, and an excitatory direct effect (Supplementary Figure S2); a pair with indirect modulation that changed sign, but no direct effect (Supplementary Figure S3); a pair with no indirect modulation, but with an inhibitory direct effect (Supplementary Figure S4); a pair with neither indirect nor direct effect (Supplementary Figure S5); a pair with only indirect effects appearing late, after the ICMS train had ended (Supplementary Figure S6); and all four pairs involving the same neuron that display a variety of excitatory and inhibitory indirect and direct effects produced by ICMS delivered on the four different S1 arrays (Supplementary Figure S7).

Direct modulation of M1 and PMv neurons

For different neuron-array pairs, direct modulation time-locked to the individual pulses of S1-ICMS could be either an excitatory peak (e.g. Figures 2A and Supplementary Figure S2A) or an inhibitory trough (e.g. Figure 4A and Supplementary Figures S4A, and S7A,D). Direct effects also could consist of multiple peaks and/or troughs (e.g. Figures 3, 7A, and Supplementary Figure S7G,J). Combining the data from the two monkeys and counting each of multiple peaks and troughs separately, we identified 46 peaks in M1 neuron-array pairs and 44 in PMv; 160 troughs in M1 and 77 in PMv. Overall, inhibitory direct effects were more common than excitatory, more so in M1 than in PMv (χ² test, p = 5.2e-3). The likelihood of multiple peaks and/or troughs did not differ depending on whether the unit was an MU, PSU, or DSU (χ² test, p=0.12), but the frequencies of single peak vs. single trough vs. multiple effects differed in the two cortical areas (χ² test, p=1.7e-05), with troughs and multiple effects being more frequent in M1 and peaks more frequent in PMv (Supplementary Table 1). Based on their FWHM, all the present peaks were transsynaptic (Supplementary Figure S8). The distributions of latency, full width at half maximum, and amplitude of peaks and troughs in each cortical area are presented in Supplementary Figure S9.

Indirect modulation is more prevalent than direct modulation

Indirect modulation affected the majority of neurons in both M1 and PMv, as did direct modulation. Of M1 neurons, 37/41 (90%) in monkey L and 32/36 (89%) in monkey X showed indirect modulation from at least one S1-ICMS instruction, and 36/41 (88%) in monkey L and 28/36 (78%) in monkey X showed direct modulation from at least one S1-ICMS instruction. Of PMv neurons, 38/42 (90%) in monkey L and 30/33 (91%) in monkey X showed IM from at least one S1-ICMS instruction, and 26/42 (62%) in monkey L and 29/33 (88%) in monkey X showed DM from at least one S1-ICMS instruction. In each monkey and in each cortical area the prevalence of neurons showing indirect modulation by ICMS delivered on at least one S1 array was not significantly different than the prevalence of direct modulation, except for PMv neurons in monkey L (M1: L, p=1.00; X, p=0.34. PMv: L, p=0.004; X, p=1.00, Fisher’s Exact tests).

Many neurons, however, were modulated by S1-ICMS on more than one array (e.g. Supplementary Figure S7). Table 1 shows the numbers of neurons that were indirectly modulated (IM) by ICMS delivered through 0, 1, 2, 3, or 4 different S1 arrays. Because each neuron was tested with multiple arrays, here we counted only those arrays for which indirect modulation remained significant at p<0.01 after making Bonferroni corrections to the rmANOVA p-values for the multiple testing of each neuron (4 S1 arrays in monkey L or 3 in monkey X). For both monkeys, substantial numbers of neurons in both M1 and PMv were modulated indirectly by stimulation on more than one S1 array (M1: L, 34/41 (83%); X, 23/36 (64%). PMv: L, 27/42 (64%); X, 24/33 (73%)). Similarly, we counted the number of neurons that showed direct modulation from ICMS on 0, 1, 2, 3, or 4 S1 arrays (Table 1, DM). Again for both monkeys, substantial numbers of neurons were modulated directly by stimulation on more than one S1 array (M1: L, 25/41 (61%); X, 13/36 (36%). PMv: L, 13/42 (31%); X, 5/33 (15%)).

Table 1.

Number of neurons modulated by stimulation through 0, 1, 2, 3, or 4 S1 arrays.

Mod Type	Cortical area (monkey)	0	1	2	3	4
IM	M1 (L)	4	3	11	9	14
	M1 (X)	4	9	13	10	N/A
	PMv (L)	4	11	9	12	6
	PMv (X)	3	6	16	8	N/A
DM	M1 (L)	5	11	8	6	11
	M1 (X)	8	15	10	3	N/A
	PMv (L)	16	13	8	4	1
	PMv (X)	4	24	4	1	N/A

Figure 5 graphically depicts this convergence of indirect modulation (Figure 5A) or direct modulation (Figure 5B) on neurons recorded on individual arrays in M1 and PMv evoked by ICMS delivered via each of the four S1 arrays in monkey L (top) and the three S1 arrays in monkey X (bottom). The thickness and grayscale color of straight lines corresponds to the percent of neurons on the M1 or PMv array that were modulated by ICMS pulses delivered on the connected S1 array. Indirect and direct effects both converged on neurons recorded in each M1 or PMv array from wide territories in S1. And conversely, this illustration also shows that both indirect and direct effects from ICMS delivered on each S1 array diverged to a wide territory in both M1 and PMv.

Figure 5. Topography of A) indirect and B) direct modulation from each S1 array.

Each square represents an FMA implanted in M1 (blue), PMv (green), or S1 (yellow). The thickness and grayscale color (legend at center) of each solid line is proportional to the percentage of units recorded from each M1 or PMv array that showed an indirect A) or a direct B) effect evoked by ICMS delivered through the connected S1 array. The letters next to the arrays correspond to those in Figure 1. Brown dotted lines indicate the Arcuate Sulcus (AS) and Central Sulcus (CS).

The thickness of the grayscale lines in Figure 5 also reveals that among neuron-array pairs (rather than among neurons), indirect effects were more prevalent than direct effects. We first counted the number of neuron-array pairs with indirect modulation (IM+). For M1 pairs, 108/164 (66%) in monkey L and 65/108 (60%) in X showed indirect modulation; for PMv pairs, 89/168 (53%) in monkey L and 62/99 (63%) in monkey X. More than half of neuron-array pairs showed indirect modulation. We similarly assessed the prevalence of direct modulation. For M1 neuron-array pairs, the PSTHs of 89/164 (54%) from monkey L and 44/108 (41%) from monkey X showed direct modulation; for PMv, 45/168 (27%) pairs from monkey L and 35/99 (35%) pairs from monkey X. On average, less than half of neuron-array pairs showed direct modulation. In each monkey and in each cortical area, the prevalence of indirect modulation was greater than the prevalence of direct modulation, more so in PMv than in M1 (M1: L, p=0.043, X, p=0.0064; PMv: L, p=1.4e-6; X, p=2.0e-4, Fisher’s Exact tests). In summary, both indirect and direct effects were produced in substantial fractions of individual neurons and neuron-array pairs by ICMS on multiple different S1 arrays. Although the proportions of neurons receiving at least one indirect versus at least one direct effect was similar, the proportion of neuron-array pairs showing indirect effects was greater than the proportion showing direct effects.

Whereas direct modulation could occur only during the reaction and movement task epochs while stimulation was being delivered, we observed indirect modulation not only during those epochs but, for some neuron-array pairs, also during the final hold and end-of-trial epochs after the ICMS trains had terminated (Figures 4 and Supplementary Figures S1, S2, S3, S6 and S7C,I,L). We therefore quantified the prevalence of indirect modulation during the different task epochs. For each IM+ neuron-array pair (rmANOVA p<0.01), we conducted post-hoc tests to identify the epoch(s) in which firing rates were significantly different during visually- versus ICMS-instructed trials (Tukey-Kramer tests, p<0.01). For DM- and DM+ pairs separately, we plotted the percentage of IM+ pairs that displayed IM effects during each epoch for M1 (Figure 6A) and PMv (Figure 6B). For both IM+/ DM− (orange bars) and IM+/ DM+ (brown bars) neuron-array pairs in M1 and PMv, indirect effects were most prevalent in the reaction and movement epochs during which ICMS was being delivered on ICMS-instructed trials. In the final hold epoch, after ICMS had ended in ICMS-instructed trials but visual cues still were present in visually-instructed trials, indirect modulation remained present in substantial percentages of neuron-array pairs in both M1 and PMv (M1: L, 26/108 (24%); X, 16/65 (25%). PMv: L, 25/89 (28%); X, 19/62 (31%)). Even in the end-of-trial epoch, when neither type of instruction was present and task conditions for visually- versus ICMS-instructed trials were indistinguishable, indirect modulation still was found in small percentages of neuron-array pairs in both cortical areas [(M1: L, 12/108 (11%); X, 17/65 (26%)) (PMv: L, 11/89 (12%); X, 9/62 (15%))], yet still above the false discovery rate of 1%. Moreover, in 18/324 (6%) of IM+ pairs, indirect modulation was absent during the reaction and/or movement epoch, yet nevertheless present during the final hold epoch (10 pairs), the end-of-trial epoch (2 pairs), or both (6 pairs, e.g. Supplementary Figure S6).

Figure 6. Percentages of neuron-array pairs either without (DM-, orange) or with (DM+, brown) direct modulation showing indirect modulation during each task epoch.

For each indirectly modulated (IM+) neuron-array pair (p<0.01, rmANOVA) in A) M1 or B) PMv, firing rates were averaged during each task epoch and compared between visually- and ICMS-instructed trials. Neuron-array pairs with significantly different firing rates according to the global comparison of all epochs underwent post-hoc analysis to identify the specific epochs during which differential firing occurred (p<0.01, Tukey-Kramer). We then counted the DM− (orange) and DM+ (brown) pairs indirectly modulated during each epoch and normalized the counts to a percentage of the total number of DM- or DM+ neuron-array pairs, respectively, that had globally significant indirect modulation. The distribution of indirect modulation prevalence across task epochs for DM− versus DM+ neuron-array pairs was not significantly different in either cortical area in either monkey (p≥0.05, χ²-tests, see graph insets at upper left).

Relationship between indirect modulation and direct modulation

For a given neuron-array pair, indirect modulation could be found with or without direct modulation and vice versa. When both were present, the indirect modulation could have the same or opposite sign (excitation or inhibition) as the direct modulation. And indirect modulation could change sign between task epochs.

Was the presence of indirect modulation independent of the presence of direct modulation? For each monkey and each cortical area separately, we counted the number of neuron-array pairs with each of the four possible combinations: i) IM+/DM+; ii) IM+/DM-; iii) IM-/DM+; and iv) IM-/DM (Table 2). Except for PMv neuron-array pairs in monkey X, the presence versus absence of indirect and direct modulation were not independent of one another (p<0.01, Fisher’s Exact tests). Overall, there appeared to be a relative paucity of IM-/DM+ neuron-array pairs.

Table 2.

Counts of neuron-array pairs with each category of modulation combination.

Cortical area (monkey)	IM+DM+	IM+ DM−	IM− DM+	IM− DM−	Fisher test p-value
M1 (L)	74	34	15	41	4.6e-7*
M1 (X)	34	31	10	33	2.9e-3*
PMv (L)	33	56	12	67	1.6e-3*
PMv (X)	26	36	9	28	0.086
Total	167	157	46	169	1.4e-12*

Disentangling indirect and direct effects

During the reaction and movement task epochs, direct effects often were present at the same time as indirect effects. Thus far, we have emphasized examples in which direct effects could not have accounted for indirect effects. But to what extent might direct effects have accounted for indirect effects during these task epochs? We addressed this question by synthesizing simulated spike trains in which only the direct effects of S1-ICMS were added to spike trains recorded during visually-instructed trials, without any indirect effects (see Methods). We reasoned that to the extent the simulated spike activity approximated the real spike activity recorded during ICMS-instructed trials, the direct effect accounted for any indirect effect. And conversely, to the extent the simulated spike activity differed from the real spike activity recorded during ICMS-instructed trials, the direct effect did not account for the indirect effect.

Figure 7 illustrates the results of this simulation. Figure 7A shows three PSTHs for an example neuron-array pair. The PSTH derived from visually-instructed trials using simulated ICMS pulses (blue, visPSTH, see Methods) shows no peaks, as would be expected given that no ICMS was delivered on visually-instructed trials. The standard PSTH derived from ICMS instructed trials (red, icmsPSTH) shows three distinct peaks. The PSTH derived from the simulated spike trains (purple, simPSTH, see Methods) shows three peaks which closely resemble those of the PSTH derived from ICMS-instructed trials. Because simulated spikes were added only at the latencies of bins within the peaks of the icmsPSTH (see Methods), the bins outside of the peaks in the simPSTH are identical to those in the visPSTH. The rasters and firing rate histograms for this example neuron-array pair are shown in Figure 7B. In comparison to visually-instructed trials (blue), the simulated spike trains (purple) show that the simulated excitatory direct effect resulted in increased firing rates throughout the reaction and movement epochs. This increase, however, was not enough to account for the increase in firing rate recorded during the reaction epoch of ICMS-instructed trials (red), indicating an additional excitatory indirect effect. Furthermore, during the movement epoch the simulated firing rate exceeded that during ICMS-instructed trials, indicating that the indirect effect became inhibitory during the movement epoch. And because ICMS had stopped at the end of the movement epoch, the direct effect could not account for the lower firing rate on ICMS- (red) as compared to visually- (blue) instructed trials during the end-of-trial epoch. In the firing rate histograms, subtracting the mean firing rate during visually-instructed trials from that during simulated trials leaves the isolated direct effect (dotted black trace), and subtracting the mean firing rate during simulated trials from that during ICMS-instructed trials leaves the isolated indirect effect (solid black trace).

Figure 7C compares average firing rates during visually-instructed, ICMS-instructed, and simulated trials during the reaction (upper) and movement (lower) epochs for all DM+ neuron-array pairs from M1 (blue) and PMv (green) in which the simPSTH direct effects were statistically similar to the icmsPSTH (p < 0.01, see Methods). As ICMS was being delivered throughout these two epochs, the firing rates on ICMS-instructed trials included both indirect and direct effects, whereas the firing rates in the simulated trials included only the direct effects added to the activity during visually-instructed trials. In the left column of panels (i) showing firing rates in ICMS- versus visually-instructed trials, the points lying below the line of unity slope indicate that firing rates most often were lower on ICMS- than on visually-instructed trials. In the middle panels (ii) showing firing rates in simulated versus visually-instructed trials, such points indicate that simulated direct effects likewise generally lowered firing rates as compared to visually-instructed trials, though most often not to the extent seen in ICMS-instructed trials. In these panels, a vertical vector from the line of unity slope to each point represents the sign and magnitude of the isolated direct effect. In the right panels (iii) showing firing rates in ICMS versus simulated trials, a vertical vector from the line of unity slope to each point then represents the sign and magnitude of the isolated indirect effect. We calculated the magnitude (absolute value) of the isolated direct effect and of the isolated indirect effect for each of these DM+ neuron-array pairs. Across the population pooled from both monkeys, we found that in both M1 and PMv, during both the reaction and the movement epochs, isolated indirect effects were larger than isolated direct effects (M1: Reaction, isolated indirect effect median (IQR) 2.5 (1.2 – 4.4) Hz, isolated direct 1.4 (0.8 – 3.2) Hz, p=4.0e-3; Movement, isolated indirect 4.1 (1.7 – 8.0) Hz, isolated direct 1.2 (0.5 – 2.4) Hz, p=3.9e-11. PMv: Reaction, isolated indirect 2.6 (0.6 – 4.6) Hz, isolated direct 0.9 (0.5 – 1.7) Hz, p=8.7e-7; Movement, isolated indirect 2.1 (1.1 – 5.1) Hz, isolated direct 0.8 (0.4 – 1.8) Hz. p=3.9e-11. Mann-Whitney U-tests).

Figure 7D plots the sign and magnitude of the isolated direct effect versus the isolated indirect effect for each of these DM+ neuron-array pairs. All possible combinations of excitatory versus inhibitory isolated direct and isolated indirect effects were found. Moreover, as shown by the example neuron-array pair (red circles), one combination could occur during the reaction epoch and another during the movement epoch. Overall, however, during the reaction epoch and during the movement epoch, most neuron-array pairs had both a negative (inhibitory) isolated direct effect and a negative isolated indirect effect. Whereas in M1 the isolated direct and indirect effects were correlated during both the reaction (Pearson’s r = 0.35, n = 117, p = 1.0e-04) and movement (r = 0.34, n = 117, p = 2.1e-04) epochs, in PMv they were not (reaction: r = −0.067, n = 61, p = 0.61; movement: r = 0.059, n = 61, p = 0.65).

Decoding movements during S1-ICMS

Development of bidirectional BMIs for controlling neuroprosthetic upper extremities may incorporate S1-ICMS to provide on-line somatosensory feedback from the prosthetic limb. However, the indirect and direct effects of S1-ICMS on motor cortex neurons then might degrade the performance of the algorithms that decode intent from motor cortex to drive movement of the limb (Shelchkova et al., 2023). We therefore evaluated the impact of S1-ICMS on decoding the present RGM movements using pooled M1 and PMv neurons regardless of whether each had any indirect and/or direct effects.

Figure 8 illustrates decoding performance, which was generally better in monkey L (upper panels) than in monkey X (lower panels). We first trained LDA classifiers on visually-instructed trials (Figure 8A) and tested their decoding performance on ICMS-instructed trials. Compared to their performance on visually-instructed trials (blue), the performance of visually-trained classifiers was degraded considerably when used to decode ICMS-instructed trials (red), particularly during the reaction and movement epochs when indirect effects were most prevalent and direct effects occurred as well, but also during the first few hundred milliseconds of the final hold epoch although direct effects were no longer present. Applying visually-trained classifiers to simulated trials (purple), which added direct effects to the activity recorded during visually-instructed trials, showed that direct effects caused little if any degradation of decoding performance. The degraded performance of visually-trained classifiers used to decode ICMS-instructed trials therefore resulted primarily from indirect effects.

Figure 8. Decoding Analysis.

Classification accuracy as a function of time is shown when classifiers trained on A) visually-instructed trials or on B) ICMS-instructed trials were used to decode visually-instructed (blue), ICMS-instructed (red), or simulated (purple) trials. Black traces show decoding of shuffled data (see Methods). Shaded areas represent ± 1 standard deviation from the mean. All data have been aligned separately to the beginning of the 5 epochs – Initial Hold (IH), Reaction (R), Movement (M), Final Hold (FH), and End-of-Trial (EoT) with a 200 ms timescale bar (black).

On the other hand, when classifiers trained on ICMS-instructed trials were used to decode ICMS-instructed trials (red in Figure 8B), decoding performance was comparable to that of visually-trained classifiers used to decode visually-instructed trials (blue in Figure 8A). As might be expected, the performance of ICMS-trained classifiers was substantially degraded when tested on visually-instructed trials (blue in Figure 8B). Applied to simulated trials, ICMS-trained classifiers performed only marginally better, if at all (purple in Figure 8B). We therefore infer that while S1-ICMS changed the patterns of firing rate covariation among M1 and PMv neurons, predominantly via indirect effects, S1-ICMS did not degrade the level of decodable information available in their discharge.

Discussion

We found that ICMS in S1 modulated neuron firing in cortical motor areas in two different ways. Direct modulation was evident in PSTHs as modulation of a unit’s spike probability time-locked to the individual pulses of ICMS trains delivered through a set of electrodes on a particular S1 array. Indirect modulation appeared as a difference in a unit’s firing rate that depended on whether the target object was instructed with a visual cue versus a train of S1-ICMS. Stimulation on different S1 arrays often produced different indirect and/or direct effects in the same neuron. Similar direct and indirect effects of S1-ICMS on motor cortex neurons have been described recently in humans (Shelchkova et al., 2023).

Here, in macaques, we extended such findings by investigating these effects in both M1 and PMv and by simulating direct effects to disentangle the two types of modulation. We found that various neurons in both M1 and PMv showed indirect modulation only, direct modulation only, both indirect and direct modulation, or neither. Indirect effects were more prevalent and generally stronger than direct effects. But in neither cortical area was the presence versus absence of indirect and direct modulation entirely independent of one another. Although both indirect and direct effects were most often inhibitory, all possible combinations of excitatory versus inhibitory indirect and direct effects were found in different neuron-array pairs. Still, the sign and amplitude of isolated indirect and isolated direct effects were correlated during the reaction and/or movement task epochs in the M1 population, though not in the PMv population. Furthermore, in a given neuron-array pair indirect effects could change not only in amplitude but also in sign over the time course of behavioral trials, and some neuron-array pairs showed indirect effects more than 1 second after the trains of ICMS pulses had ended, when no direct effects could have been present. Taken together, all these observations suggest that direct and indirect effects are likely to have differing underlying mechanisms.

Anatomical and physiological bases of direct and indirect modulation

More than 60% of units in both M1 and PMv were directly modulated by the ICMS pulses delivered through at least one S1 microelectrode array. The pathways via which natural somatosensory input reaches M1 and PMv are not entirely understood, though a substantial fraction may be processed through S1. M1 neurons respond to both cutaneous and deep peripheral stimulation (Rosen and Asanuma, 1972; Lemon and Porter, 1976; Wong et al., 1978; Fetz et al., 1980). In PMv, neurons also respond to both cutaneous and deep peripheral stimulation but have larger and more complex somatosensory receptive fields (Rizzolatti et al., 1981a; Gentilucci et al., 1988; Rizzolatti et al., 1988; Graziano and Gandhi, 2000).

We were not able to obtain histological verification of the locations of our microelectrode tips, but based on the distance of our S1 arrays from the central sulcus (<3 mm), their shaft lengths (1.5 – 6.0 mm), and unit responses to natural somatosensory stimulation, the tips of our S1 electrodes were probably located in areas 3b and 1, with some possibly in area 3a. Areas 3a, 1, and 2 project directly to M1 (area 4), but areas 3a, 3b and 1 all project to area 2, which in turn projects to area 4 (Jones et al., 1978; Pons and Kaas, 1986; Ghosh et al., 1987; Ghosh and Porter, 1988; Dariansmith et al., 1993; Tokuno and Tanji, 1993; Padberg et al., 2019). Area 2 projects to area 5 as well, which projects heavily to area 4 (Strick and Kim, 1978; Zarzecki et al., 1978). All these parietal areas also project to the secondary somatosensory area (S2) in the insular cortex, which in turn projects to area 4 (Friedman et al., 1986; Dariansmith et al., 1993). Whereas multiple transsynaptic routes thus are available from S1 to M1, routes from S1 to PMv are more limited. Known somatosensory projections to PMv arise only from area 3a and from S2 (Godschalk et al., 1984; Matelli et al., 1986; Kurata, 1991). Conduction over these cortico-cortical pathways from S1 to M1 and to PMv may account for the direct modulation we observed time-locked to the pulses of S1-ICMS. The FWHMs of these direct effects (Supplementary Figures S8 and S9) suggest that they represent oligo-synaptic, and in some cases possibly monosynaptic, transmission (Kraskov et al., 2011).

Indirect effects from S1-ICMS may also be transmitted to M1 and PMv in part over similar pathways. However, the observation that indirect effects cannot be fully attributed to time-locked direct excitation and/or inhibition suggests greater temporal dispersion of synaptic inputs, most likely resulting from more polysynaptic transmission. In addition to polysynaptic transmission through the pathways enumerated above, longer pathways may involve processing in cortico-thalamo-cortical (Mo and Sherman, 2019; Sherman and Usrey, 2024) and in more circuitous cortico-cortical circuits. For example, during a task requiring comparison of two natural vibrotactile stimuli, the discharge of neurons in multiple frontal cortical areas, including PMv and M1, becomes modulated for up to 3 seconds based on the frequency of both the first and second 500 ms stimuli, as well as their difference (Hernandez et al., 2010). Furthermore, unlike natural sensory stimulation, rhythmic trains of ICMS evoke the synchronous discharge of neural populations (Mazurek and Schieber, 2020; Kumaravelu et al., 2022). We might speculate that such artificially synchronized rhythmic discharge could propagate through the brain differently than neural activation due to natural stimulation, resulting in prolonged barrages of polysynaptic inputs arriving in M1 and PMv. Successive barrages of inhibitory and then excitatory inputs, or vice versa, arriving in some M1 and PMv neurons over different pathways could account for the observation that in some cases indirect effects changed sign in successive task epochs. And prolonged barrages of polysynaptic inputs arriving in M1 and PMv neurons over more circuitous processing pathways might account for cases in which indirect effects were found during the final hold and end-of-trial task epochs, sometimes more than 1 second after the ICMS trains had ended.

Other potential sources of indirect modulation

One could argue that transsynaptic excitation or inhibition produced by the individual S1-ICMS pulses, i.e. their direct effects, may have summed nonlinearly with other synaptic inputs related to the RGM behavior to produce the indirect modulation we observed. Nevertheless, five observations argue against direct modulation being the source of indirect modulation. First, we found indirect effects in a substantial percentage of neuron-array pairs that had no direct effects (157/326, 48%). Second, indirect effects were present in many neurons during the final hold and/or end-of-trial epochs, after the ICMS instructions had terminated. In a few of these cases, indirect modulation was absent during the reaction and movement epochs and appeared only during the final hold and/or end-of-trial epochs (Supplementary Figure S6). Third, neuron-array pairs without and with direct effects showed indirect effects with similar prevalence during the various behavioral task epochs (Figure 6). Fourth, the change in the spike probability due to the isolated direct effect often was in the direction opposite that of the difference in unit firing rates during ICMS- versus visually-instructed trials due to the isolated indirect effect (Figure 7D). And fifth, our simulation of direct effects added to activity during visually-instructed trials (Figure 7) indicated that the isolated direct effect did not account for the isolated indirect effect for many neuron-array pairs. We therefore infer that the indirect effects we observed could not be attributed simply to the direct effects of individual ICMS pulses.

We considered three alternative sources of indirect modulation. First, firing rate differences might be attributable to the monkeys having performed the RGM movements differently depending on whether the instruction was delivered with visual cues versus S1-ICMS. However, success rates were generally similar for the visually- and ICMS-instructed trials, and the movements performed under the two conditions appeared indistinguishable. The neuron-array pair illustrated in Figure 4, for example, showed visual-versus-ICMS average firing rate differences of >10 spikes/sec during the reaction and movement epochs even though the median reaction time differed by only 2 ms and the median movement time by only 8 ms. Furthermore, kinematic differences would not account for differences in firing rate observed in the final hold epoch, during which the arm and hand were stationary. Any minor differences in movement kinematics thus seem unlikely to account for visually- versus ICMS-instructed firing rate differences.

Second, our visual instructions continued through the final hold epoch whereas our ICMS instructions did not. Although responses to visual stimuli in many M1 and dorsal premotor cortex neurons generally are short lived (Weinrich et al., 1984; Wannier et al., 1989), mirror neurons in M1 and PMv show prolonged responses to moving visual stimuli (Kraskov et al., 2009; Vigneswaran et al., 2013) whether biological or not (Albertini et al., 2021). Though our visual cues were not moving, we might speculate that the presence of visual stimuli, but not ICMS, during the final hold period may have contributed to indirect modulation. However, the persistent visual instruction cannot account for indirect modulation during the end-of trial epoch because visual cues were no longer present during visually-instructed trials and ICMS was no longer present during ICMS-instructed trials.

We therefore considered a third possibility. Our LED cues stimulated the visual system, whereas we delivered ICMS in the somatosensory system. M1 neurons have substantial somatosensory responses (Lemon and Porter, 1976; Wong et al., 1978; Fetz et al., 1980) though relatively sparse visual responses (Weinrich et al., 1984; Wannier et al., 1989). PMv neurons have both visual and somatosensory receptive fields (Rizzolatti et al., 1981a, b). Although a prior study in macaques found no difference in the discharge of M1 neurons depending on the natural sensory modality used to trigger movements (Lamarre et al., 1983), the discharge of neurons in multiple frontal areas including PMv and M1 differed depending on whether the same movement was performed based on visual cues versus comparison of vibrotactile stimuli (Hernandez et al., 2010). Furthermore, motor-related cortical potentials over human M1 were found to differ depending on whether an instruction was delivered with natural visual versus natural auditory cues, not only before, but also for several hundred milliseconds after movement onset (Mazurek et al., 2020). We therefore might speculate that differences in the sensory modality used to deliver instructions may have contributed to indirect modulation as well.

Wide spatial distribution of both indirect and direct effects

Extensive convergence and divergence were observed for both indirect and direct effects between the arrays that spanned the upper extremity representation in S1 and M1, as well as between the arrays in S1 and PMv. This might have been expected in PMv given the large somatosensory receptive fields of PMv neurons (Rizzolatti et al., 1981a; Graziano and Gandhi, 2000). Less expected perhaps, was the similarly extensive convergence and divergence between our S1 arrays and those in M1, both of which spanned the M1 upper extremity representation. Interestingly, focal injection of tracers in area 3a, 3b, 1, or 2 results in terminal labeling spread mediolaterally over several millimeters in the same cortical area (Jones et al., 1978). Likewise neurons at any focal location in the M1 upper extremity representation send horizontal collaterals throughout that representation (Huntley and Jones, 1991). Horizontal collaterals in both S1 and M1 thus may provide an anatomical substrate for the extensive convergence and divergence we observed physiologically between S1 and M1.

Preponderance of inhibition in both indirect and direct effects

S1-ICMS produced more inhibition than excitation in both M1 and PMv, both indirect and direct. Somatosensory inputs to cortical neurons typically have been assessed by determining the receptive fields from which neurons in S1, M1, or PMv can be excited. We therefore had not anticipated that the direct effects produced by S1-ICMS would be more often inhibitory than excitatory in both M1 and PMv neurons (Figure 7D, Supplementary Figure S9). In contrast to the present results with S1-ICMS, PMv-ICMS has been found to produce more excitatory than inhibitory modulation of M1 neurons (Kraskov et al., 2011). We speculate, therefore, that while PMv generally facilitates activity in M1 (Shimazu et al., 2004), S1 may provide inhibition of undesired activity in both M1 and PMv.

Limitations of the present study

During the reaction and movement epochs of the present task, indirect and direct modulation often were present concurrently. We therefore simulated isolated direct effects to disentangle them from concurrent isolated indirect effects. Our simulation incorporated the timing statistics represented by the peaks and/or troughs of each neuron-array pair’s PSTH, while removing the baseline that would have incorporated changes in firing rate produced by any concurrent indirect effects. Our simulation assumed, however, that neural activity on visually-instructed trials, isolated direct effects, and isolated indirect effects, all sum linearly, whereas in many cases summation of post-synaptic potentials is known to be non-linear (Muir and Porter, 1973; Hiroyoshi, 2009). Given this limitation, our model may be considered only a first approximation, and we therefore have presented our findings on indirect and direct effects initially from the original recorded neural activity. Future models that incorporate non-linear summation, as well as other factors, may provide more accurate separation of isolated indirect versus isolated direct effects.

The data used in the present study were collected using different numbers of electrodes delivering various currents for S1-ICMS of each of the four arrays. The prevalence and magnitude of both the indirect and direct effects evoked by stimulation on the different arrays thus are not necessarily quantitatively comparable. Future studies may employ equal currents delivered on equal numbers of electrodes or on single electrodes to more quantitatively compare the prevalence, magnitude, and spatial distribution of indirect and direct effects evoked by ICMS.

Conclusion

We found that trains of ICMS delivered in S1 modulate substantial numbers of neurons in M1 and in PMv both directly, i.e. time-locked to the individual ICMS pulses, and indirectly, without such time-locking. Decoders controlling BMIs that were trained to rely on M1 and/or PMv neurons as inputs might be disrupted by such artificial modulation, hindering performance of the BMI (Shelchkova et al., 2023). However, we found that decoders trained on ICMS-instructed trials classified ICMS-instructed trials generally as well as decoders trained on visually-instructed trials classified visually-instructed trials, suggesting that S1-ICMS effects, while altering the activity of M1 and PMv neurons, did not decrease the decodable information available in their firing rates. One might even have expected that delivering S1-ICMS on a different array during trials involving each of the four target objects would have provided even higher classification accuracy of ICMS-instructed trials by decoders trained on ICMS-instructed trials, but such was not the case. Apparently, the divergence and convergence between ICMS effects from different S1 arrays and individual M1 and PMv neurons (Table 1 and Figure 5), together with the preponderantly inhibitory nature of both indirect and direct effects (Figure 7D), failed to change the neural population’s overall differential firing rates across trials involving the four objects enough to increase classification accuracy. Nevertheless, informing decoding algorithms about such indirect and direct effects, and training decoding algorithms to account for them, could provide improved, bidirectional, closed-loop control of neurprosthetic limbs with artificial somatosensory feedback delivered via S1-ICMS.

Data and Software Availability

Analysis-specific code and data are available by request to the Corresponding Author.