Skip to main content
PMC home page
Springer logoLink to Springer
. 2024 Feb 28;242(4):959–970. doi: 10.1007/s00221-024-06790-2

Distinguishing reflex from non-reflex responses elicited by transcutaneous spinal stimulation targeting the lumbosacral cord in healthy individuals

Elizabeth A Gordineer 1,2, Dobrivoje S Stokic 2, Matthias J Krenn 2,3,✉
PMCID: PMC10973043  PMID: 38416179

Abstract

Transcutaneous spinal stimulation (TSS) studies rely on the depolarization of afferent fibers to provide input to the spinal cord; however, this has not been routinely ascertained. Thus, we aimed to characterize the types of responses evoked by TSS and establish paired-pulse ratio cutoffs that distinguish posterior root reflexes, evoked by stimulation of afferent nerve fibers, from motor responses, evoked by stimulation of efferent nerve fibers. Twelve neurologically intact participants (six women) underwent unipolar TSS (cathode over T11-12 spinal processes, anode paraumbilically) while resting supine. In six participants, unipolar TSS was repeated 2–3 months later and also compared to a bipolar TSS configuration (cathode 2.5 cm below T11-12, anode 5 cm above cathode). EMG signals were recorded from 16 leg muscles. A paired-pulse paradigm was applied at interstimulus intervals (ISIs) of 25, 50, 100, 200, and 400 ms. Responses were categorized by three assessors into reflexes, motor responses, or their combination (mixed responses) based on the visual presence/absence of paired-pulse suppression across ISIs. The paired-pulse ratio that best discriminated between response types was derived for each ISI. These cutoffs were validated by repeating unipolar TSS 2–3 months later and with bipolar TSS. Unipolar TSS evoked only reflexes (90%) and mixed responses (10%), which were mainly recorded in the quadriceps muscles (25–42%). Paired-pulse ratios of 0.51 (25-ms ISI) and 0.47 (50-ms ISI) best distinguished reflexes from mixed responses (100% sensitivity, > 99.2% specificity). These cutoffs performed well in the repeated unipolar TSS session (100% sensitivity, > 89% specificity). Bipolar TSS exclusively elicited reflexes which were all correctly classified. These results can be utilized in future studies to ensure that the input to the spinal cord originates from the depolarization of large afferents. This knowledge can be applied to improve the design of future neurophysiological studies and increase the fidelity of neuromodulation interventions.

Keywords: Electrical stimulation, Evoked motor response, Neurophysiology, Post-activation depression, Spinal reflex, Transcutaneous spinal cord stimulation

Introduction

Activation of sensory fibers is a common approach to neuromodulation. Transcutaneous spinal stimulation (TSS) has been introduced as a technique for depolarizing posterior roots over the lumbosacral or cervical segments and providing input to the spinal cord. In neurophysiological studies, TSS is often paired with another spinal or supraspinal input to deduce the magnitude and time course of neuromodulation at the spinal or cortical level (Knikou 2014; Pulverenti et al. 2021; Roy et al. 2014; Shapkova 2004; Shapkova and Schomburg 2001). As a therapeutic modality, TSS can produce short-term changes in spinal excitability that are believed to promote the reorganization of neural pathways (de Freitas et al. 2021). Based on this assumption, TSS has been used in attempts to enable movements (Inanici et al. 2021), reduce spasticity (Hofstoetter et al. 2021), and improve autonomic functions (Samejima et al. 2022) after spinal cord injury in humans, whether used alone (Megía-García et al. 2021; Meyer et al. 2020) or in combination with other rehabilitation strategies (Al’joboori et al. 2020; Gad et al. 2017; Hofstoetter et al. 2021; Inanici et al. 2021).

For TSS to exert neuromodulation effects, the predominant if not exclusive, depolarization of afferent fibers is essential. Yet, lumbosacral and cervical TSS can evoke not only reflex responses but also motor responses due to motor fiber depolarization, as well as mixed responses showing features of both reflex and motor responses (Binder et al. 2021; Danner et al. 2016; Roy et al. 2012; Wu et al. 2020). Reflex and non-reflex responses have been distinguished based on the difference in onset latencies (Danner et al. 2016; Minassian et al. 2007) or with conditioning paradigms (Roy et al. 2012; Saito et al. 2019). Among the latter, a paired-pulse paradigm at different interstimulus intervals (ISIs) is most commonly used. If the second response in the pair is suppressed at short ISIs, the evoked responses are believed to be of reflex origin (Andrews et al. 2015; Courtine et al. 2007; Minassian et al. 2007; Roy et al. 2012).

Consistent with experimental findings, computer simulation studies support the possibility of evoking reflex and motor responses with both epidural stimulation and TSS. An epidural model has proposed that with a posterior midline electrode location, the threshold for depolarizing posterior roots is lower than for depolarizing anterior roots (Rattay et al. 2000). TSS models have suggested that for posterior roots, low threshold sites are located just before they enter into the spinal cord and as they exit the spinal canal where anterior roots can also be activated (Ladenbauer et al. 2010). Another model has suggested that with increasing TSS intensities, the order of activation is posterior roots, anterior roots, and posterior columns (Danner et al. 2011). The latter authors also proposed that identifying TSS responses is essential to properly interpret neurophysiologic studies examining the conducting and processing capabilities of the spinal cord and understanding the neurophysiology behind the putative effects of neuromodulation interventions.

Despite recognizing that TSS may evoke both reflex and motor responses, previous reports did not investigate in detail their prevalence, coexistence, repeatability, associated paired-pulse ratios, or the impact of different electrode montages. Therefore, the main objectives of this study were to identify different response types elicited by unipolar TSS, determine their prevalence, and establish paired-pulse ratio cutoffs at different ISIs that can reliably distinguish reflex from non-reflex responses. We then evaluated the validity of the derived paired-pulse ratio cutoffs when repeated 2–3 months later and when applied to a bipolar electrode configuration.

Materials and methods

Participants

Twelve participants (six women) with no history of neurological disorders were recruited for this study. Their average (standard deviation) age was 30.3 (9.8) years (range 24–59), height was 174.6 (8.3) cm (range 158–188), and body mass was 75.8 (14.1) kg (range 59–95). The study was approved by the Institutional Review Board Committee of the University of Mississippi Medical Center, Jackson, MS (UMMC-IRB 2020-0193). All participants signed written informed consent before their enrollment into the study.

Experimental procedure

The study consisted of two sessions separated by 74 (6) days. All 12 participants were included in the first session (S1), which aimed to identify different response types elicited by unipolar TSS, determine their prevalence, and establish the optimal cutoff point(s) for their discrimination. In the second session (S2), six participants (two women) from the initial cohort were available to validate the developed approach for distinguishing reflex from non-reflex responses using the same unipolar TSS configuration and additionally a bipolar configuration.

During data collection, the participants were lying supine on a hospital bed. The supine position was preferred over the prone position as it predominantly recruits afferent fibers (Danner et al. 2016). We ensured a controlled and comfortable environment that minimized distractions. Participants were instructed not to talk or move during recordings.

TSS procedure

Electrode configurations

For the unipolar configuration (heretofore sessions UPS1 and UPS2), a self-adhesive surface electrode (5 × 5 cm) serving as the cathode was placed midline over the T11-T12 spinal processes, identified by palpating anatomical landmarks. Four interconnected electrodes (5 × 10 cm each) were placed paraumbilically as a large indifferent reference electrode (anode). In the second session (UPS2), the unipolar electrode placement was replicated based on the individual photos taken at UPS1. For the bipolar configuration (BPS2), two 5 × 5-cm self-adhesive surface electrodes were used, with the cathode placed 2.5 cm below the location used for the unipolar stimulation and the anode placed 5 cm above the cathode (Krenn et al. 2020).

Stimulation procedure

Monophasic 1-ms rectangular pulses were delivered by a current-controlled stimulator (DS8R, Digitimer Ltd, Welwyn Garden City, UK). An analog output and acquisition card controlled the stimulation pulse timing and intensity (USB-NI 6003, National Instruments Inc., Austin, TX, USA). The trigger signal for the stimulator was also used as a sync input for the electromyographic (EMG) system. We first selected the stimulation range for each participant using a custom-made real-time viewing program (LabVIEW 2019, National Instruments Inc.).

The stimulation intensities were set based on the range established for each individual. The low end of the range was defined as the intensity that produced a just visible EMG response in any muscle. The high end of the range was defined as the intensity where a 20% increment in intensity produced no further increase in the peak-to-peak amplitude in the soleus muscle. No participant reported discomfort at the selected stimulation intensities. Within this range, we collected 42 data points at incremental intensities to be able to derive the recruitment curve for each muscle.

For the paired-pulse stimulation paradigm, three stimulation intensities were applied corresponding to 25%, 50%, and 75% of the individual stimulation range (for example, in ID03, the established range was 25 to 100 mA; thus, the stimulation intensities applied were 44, 63, and 81 mA). Pulses were delivered at ISIs of 25, 50, 100, 200, and 400 ms with a 5-s pause between consecutive pairs. The order of ISIs was randomized. For each ISI, the stimulation intensity and four repetitions of each intensity were randomized. The protocol deviated for one participant (ID08), in whom ISIs ranged from 50 to 400 ms and were repeated twice.

EMG recordings

The skin was abraded with a gel before bipolar surface electrodes (2.5 cm inter-electrode distance) were centered over the muscle belly. Twelve electrodes were placed on the leg corresponding to the hand-dominant side: vastus medialis (VM1), vastus lateralis (VL1), rectus femoris (RF1), adductors (AD1), lateral hamstrings (LH1), medial hamstrings (MH1), peroneus longus (PL1), tibialis anterior (TA1), lateral gastrocnemius (LG1), medial gastrocnemius (MG1), soleus (SO1), and extensor digitorum brevis (EB1), and on four muscles of the contralateral side (RF2, MH2, TA2, SO2). The decision to account for hand dominance was based on controversy of previous observations of different excitability in the soleus H-reflex (Nativ et al. 1989; Tan 1985). EMG signals were acquired with a wireless data acquisition system (Noraxon U.S.A. Inc., Scottsdale, AZ, USA). The hardware low-pass filter was set to 2 kHz, and signals were digitized with 16-bit resolution at 4,000 samples per second.

Data analysis and statistical analysis

Calculation of the paired-pulse ratio

Response amplitude

The EMG signal was digitally filtered with a second-order, 5-Hz high-pass filter to remove the offset. The peak-to-peak amplitudes of the first and the second response were calculated in the windows individually selected for each participant and muscle. These windows were kept constant over all stimulation intensities, sessions, and electrode configurations. On average, the windows ranged from 9.6 (1.5) to 29.4 (2.8) ms in the proximal muscles and from 15.4 (3.3) to 31.0 (3.2) ms in the distal muscles.

Selection of responses for analysis

Out of the three stimulation intensities applied, we included in the UPS1 analysis the responses evoked by the stimulation intensity (25, 50, or 75% of the established range) that approximated the steepest part of the recruitment curve for each muscle to make results comparable across muscles. The corresponding stimulation intensity was 82.0 (23.1) mA or 1.42 (0.27) times the motor threshold across all muscles. From the second session, we selected for analysis a stimulation intensity that produces the most comparable amplitudes of the first response between UPS1, UPS2, and BPS2 (average of 80.0 (15.2) mA, 83.0 (15.6) mA, and 105.2 (23.6) mA, respectively). The agreement between amplitudes of the first responses was tested by a two-way random effect, absolute agreement, single measurement intraclass correlation (ICC) model (McGraw and Wong 1996) using the Pingouin 0.5.3 Python package (Vallat 2018). The ICC between response amplitudes of UPS1 (2.41 (2.51) mV) and UPS2 (2.62 (2.60) mV) was 0.76, with a 95% confidence interval (CI95%) from 0.66 to 0.84. Similarly, the average response amplitudes for UPS2 and BPS2 were 2.60 (2.60) mV and 2.06 (2.40) mV, with an ICC of 0.77, CI95% [0.65, 0.85].

Outcome measure

The paired-pulse ratio was calculated for all ISIs as the ratio of the amplitude of the second response to the averaged amplitude of the first responses across 50–400 ms ISIs (25-ms ISI not included due to possible contamination by the second stimulation artifact). Before calculating the average amplitude of the first response, outliers were removed if the value exceeded five times the lower or upper semi-interquartile range measured from the median (Dovoedo and Chakraborti 2015; Schwertman et al. 2004), which occurred in 2.9% (165 of 5,594) of all data points.

Missing data

Due to poor signal quality, all recordings for EB1 were discarded. In addition, due to EMG sensor failure in three instances of UPS1, no data were recorded in the LH1 (ID12) and PL1 (ID02), and partial data were recorded in the VM1 (ID11), resulting in a loss of 52 data points. Another 13 data points were lost due to miscellaneous errors in the EMG recording. In total, 6,955 out of 7,020 data points (99.1%) were available for analysis.

Visual classification of response types

All short-latency responses included in the analysis were visually examined by each author independently. The paired-pulse responses in each muscle were displayed for all ISIs and classified based on their appearance and the degree of suppression into reflex, motor, or mixed responses. Reflexes were defined by an apparent suppression of the second response at short ISIs followed by recovery at increasing ISIs. Motor responses were those where the first and second responses were similar in appearance across ISIs (shape, size). The coexistence of the motor and reflex responses, where the compound waveform is changing in size and/or shape across ISIs, was defined as a mixed response. One response type was assigned to each muscle per individual.

The agreement between the assessors in classifying the response types was tested by Fleiss’s kappa analysis in R version 4.2.3 (R Core Team 2023). The agreement was determined based on the alpha values (range -1 to 1) and interpreted as poor (< 0), slight (0 to 0.2), fair (0.2 to 0.4), moderate (0.4 to 0.6), substantial (0.6 to 0.8) or almost perfect (0.8 to 1) (Nichols et al. 2010). The final dataset of the visual classification included responses that at least two assessors categorized in the same way.

The prevalence (counts and percentages) of response types (reflex, motor, mixed) was calculated for each condition from the final visual classification dataset. We then tested whether the proportions of the identified response types differed between legs (UPS1 dominant vs. non-dominant), sessions (UPS1 vs. UPS2), and montages (UPS2 vs. BPS2), using 2 × 2 McNemar’s tests in R version 4.2.3 (Pembury Smith and Ruxton 2020; R Core Team 2023).

Distinguishing reflex from non-reflex responses based on paired-pulse ratios

The paired-pulse ratios from UPS1 were used to establish a numeric criterion for distinguishing between visually classified reflex and non-reflex responses. For this, non-parametric receiver-operating characteristic (ROC) curves were created at each ISI for all muscles combined. The curves were generated using the “ROC Analysis” procedure in SPSS statistical software (version 29, IBM Corp., Armonk, NY, USA). The software provides a true positive rate (sensitivity) and false positive rate (1-specificity) for each ROC model. The discriminative performance of different models at each ISI was compared by the area under the curve (AUC) and CI95%.

The optimal cutoff value of each ROC model was determined as the paired-pulse ratio corresponding to the maximized Youden’s J statistic (Youden 1950). In addition, the macro-average accuracy of response type classifications was calculated based on the decision matrices, which report the true positives and negatives as well as the falsely classified responses in either direction.

Validation of derived cutoff values

The cutoffs from UPS1 were applied to the UPS2 data to assess their validity over time (sessions). The same procedure was repeated with the BPS2 data to determine their validity for the bipolar electrode configuration. The decision matrices and accuracy of classification for UPS2 and BPS2 were reported.

Results

Types and prevalence of responses elicited by unipolar lumbosacral TSS

Out of three defined response types, all three assessors identified only reflex and mixed responses but no sole motor responses. Reflexes showed a clear suppression of the second response at short ISIs with recovery at longer ISIs (Fig. 1, left). Mixed responses showed a more complex waveform. Although the motor (first) and reflex (second) components were largely overlapping, they were visually distinguishable. The size of the reflex component ranged from being comparable to the motor component (Fig. 1, middle) to being relatively small (Fig. 1, right). Still, the reflex component showed apparent suppression at shorter ISIs, whereas the motor component did not.

Fig. 1.

Fig. 1

Open in a new tab

Short-latency response types. Examples of posterior root reflex responses (left) and variations of mixed responses ranging from about equal motor and reflex components (center) to predominant motor component (right) in different participants (ID02, ID12, ID01) and muscles (VL1, RF1). Four responses are superimposed at each interstimulus interval (25–400 ms). The open and closed arrows indicate the timing of delivery of the first and second stimulation pulse (stimulation artifact removed for better visualization). Shaded areas highlight the second response, and the presence of reflex and motor components is shown in grades of orange and blue, respectively. VL1 vastus lateralis, RF1 rectus femoris

In independent visual analysis, the three assessors identically classified 377 out of 384 (98.2%) responses over all muscles, participants, sessions, and montages. Pairwise agreements between assessors ranged from 98.3 to 99.4%. The Fleiss’s kappa for the three assessors was 0.90, suggesting almost perfect agreement.

The result of the final visual classification of responses by all assessors together is presented in Fig. 2. In UPS1, the prevalence of reflexes across 15 muscles and 12 participants was 90% (160 out of 178, responses in 2 muscles were missing due to sensor error). Consequently, the overall prevalence of mixed responses was 10% (18/178). They were observed in only four muscles, of which in two bilaterally (prevalence per muscle: VL1, 33%; VM1, 25%; RF1, 42%; RF2, 33%; TA1, 8%, and TA2, 8%). When analyzing the concordance of response types between bilaterally recorded muscles (RF, MH, TA, SO), the same response type was observed in 45 of 48 instances (94%), whereas only in 3 (6%) instances, response type changed from reflex to mixed responses. McNemar’s test showed that the conversion from reflex to mixed responses was not significant between the bilaterally recorded muscles (X2(1) = 0.33, p = 0.56).

Fig. 2.

Fig. 2

Open in a new tab

Final visual classification of response types by the three assessors. Posterior root reflexes (orange) and mixed responses (blue) are denoted across different muscles for each participant (ID01–ID12). Of the 12 participants included in the first session (S1), 6 also participated in the second session (S2). TSS was applied in unipolar (UP) and bipolar (BP) electrode configurations. VM vastus medialis, VL vastus lateralis, RF rectus femoris, AD adductors, LH lateral hamstrings, MH medial hamstrings, PL peroneus longus, TA tibialis anterior, LG lateral gastrocnemius, MG medial gastrocnemius, SO soleus [1, dominant side; 2, non-dominant side]

Profiles of paired-pulse suppression of reflexes and mixed responses

Figure 3 shows the mean paired-pulse ratios elicited by UPS1 for visually classified reflexes and mixed responses over 25–400-ms ISIs in all recorded muscles. For reflexes, the paired-pulse suppression was profound at shorter ISIs in all muscles, with faster recovery in the proximal than distal muscles. As expected, the greatest suppression of reflexes was observed at short ISIs with median paired-pulse ratios of 0.041 (interquartile range, IQR: 0.002, 0.093) at 25-ms ISI and 0.060 (IQR: 0.023, 0.132) at 50-ms ISI. Mixed responses showed little to no variation in paired-pulse suppression across ISIs with median paired-pulse ratios of 0.889 (IQR: 0.678, 1.028) at 25-ms ISI and 0.889 (IQR: 0.691, 1.035) at 50-ms ISI. In general, the paired-pulse ratios at 100- and 200-ms ISIs were larger than those at 400-ms ISI.

Fig. 3.

Fig. 3

Open in a new tab

Paired-pulse profiles of posterior root reflexes and mixed responses in 12 participants. The group means (circles) and standard deviations (error bars) of the paired-pulse ratios for reflexes (orange) and mixed responses (blue) recorded in UPS1 are shown for different muscles and interstimulus intervals. The number of participants contributing to each type of response is indicated in the top right corner of each panel. Muscle abbreviations as in Fig. 2

Paired-pulse ratio cutoffs for distinguishing reflexes from mixed responses

The ROC analysis provided the best cutoff point that separated reflexes from mixed responses at each ISI (Fig. 4). The best results were obtained at ISIs of 25 ms (AUC = 1.000) and 50 ms (AUC = 0.999), whose confidence intervals overlapped (Table 1). In contrast, the AUC for ISIs of 100, 200, and 400 ms was lower (0.885, 0.772, 0.918), and their confidence intervals were outside of the CI95% of both 25- or 50-ms ISI.

Fig. 4.

Fig. 4

Open in a new tab

The receiver-operating characteristic curves of paired-pulse ratios recorded in UPS1 for each interstimulus interval (ISI). The cutoff point for each curve is indicated by an X. Corresponding ROC metrics are presented in Table 1. The data for all participants and muscles were used to generate the ROCs

Table 1.

Metrics of the receiver-operating characteristic curve for each interstimulus interval for UPS1

ISI (ms) AUROC (CI95%) Optimal cutoff values
Paired-pulse ratio Accuracy (%) Sensitivity (%) Specificity (%)
25 1.000 (1.000, 1.000) 0.512 100 100.0 100.0
50 0.999 (0.997, 1.001) 0.470 97 100.0 99.2
100 0.885 (0.859, 0.912) 0.594 65 100.0 71.0
200 0.772 (0.721, 0.824) 0.899 60 81.7 69.1
400 0.918 (0.888, 0.948) 0.869 77 75.0 93.6
Open in a new tab

AUROC area under the receiver-operating characteristic curve, CI95% 95% confidence interval, ISI interstimulus interval. Accuracy refers to macro-average accuracy

The optimal cutoff value of paired-pulse ratios at 25-ms ISI was 0.512. This cutoff correctly classified all 647 reflex and mixed responses (100% sensitivity and specificity) (Table 1). Similarly, at 50-ms ISI, the cutoff value was 0.470, with 672 correctly classified reflex and mixed responses and only 5 reflexes misclassified as mixed responses (100% sensitivity, 99% specificity). As expected, the sensitivity and specificity values were lower at ≥ 100-ms ISI due to less reflex suppression, resulting in greater overlap with mixed responses. Figure 5 shows the distribution of reflex and mixed responses below and above the derived cutoff values at different ISIs and the corresponding decision matrices.

Fig. 5.

Fig. 5

Open in a new tab

Paired-pulse ratios for reflexes and mixed responses relative to the cutoff values derived from UPS1. Scatterplot of data points (top) and decision matrices (bottom) for different interstimulus intervals (ISIs) for all muscles. Horizontal lines (red) indicate the cutoff value for each ISI

Validation of cutoff values between sessions and electrode configurations

In a repeated session with six participants (UPS2), the prevalence of visually classified reflexes and mixed responses was 93% (84/90) and 7% (6/90) (Fig. 2). Mixed responses were observed in four muscles (VM1, 33%; RF1, 33%; RF2, 17%; and TA2 17%). The concordance of reflexes and mixed responses between the sessions UPS1 and UPS2 was 92% (82/89), with only 8% (7/89) of responses changing from reflex to mixed type and vice versa. This rate of change was not significant (McNemar’s test: X2(1) = 3.57, p = 0.06).

Next, we used the derived cutoff values of 0.512 (25-ms ISI) and 0.470 (50-ms ISI) from UPS1 to determine their validity when applied to UPS2 (Fig. 6A). For the 0.512 (25-ms ISI) cutoff, 356 reflex and mixed responses were correctly classified with only 2 reflex responses misclassified (99% accuracy). For the 0.470 (50-ms ISI) cutoff, 357 reflex and mixed responses were correctly classified and only 3 reflex responses were misclassified (99% accuracy). Both cutoff values performed well and comparably in terms of sensitivity (100% for both) and specificity (> 89%).

Fig. 6.

Fig. 6

Open in a new tab

Paired-pulse ratios for reflexes and mixed responses from UPS2 and BPS2 relative to the cutoff values derived from UPS1. Scatterplot of data points (top) and decision matrices (bottom) for different interstimulus intervals (ISIs) for UPS2 (A) and BPS2 (B) for all recorded muscles. Cutoff values from UPS1 are indicated by the horizontal lines (red) for each ISI

BPS2 exclusively elicited reflex responses (Fig. 2). Thus, six instances (7%) of mixed responses elicited by UPS2 converted to reflexes with BPS2 (example shown in Fig. 7). This conversion from mixed responses in UPS2 to reflex responses in BPS2 was significantly different (McNemar’s test: X2(1) = 6, p = 0.014). As to the discriminative ability of the derived cutoff values, both 0.512 (25-ms ISI) and 0.470 (50-ms ISI) cutoffs correctly identified all responses as reflexes with 100% sensitivity and specificity (Fig. 6B).

Fig. 7.

Fig. 7

Open in a new tab

Example of a recorded mixed response in UPS2 and reflex response in BPS2 for participant ID01 and VM1 muscle. Four responses are superimposed at each interstimulus interval (25–400 ms). The open and closed arrows indicate the timing of delivery of the first and second stimulation pulse (stimulation artifact removed). Shaded areas highlight the second response, and the presence of reflex and motor components is shown in grades of orange and blue, respectively. VM1 vastus medialis

Discussion

The results of this study indicate that unipolar lumbosacral TSS mainly but not exclusively evokes reflex responses. In our sample, motor responses were never elicited in isolation but always in conjunction with reflexes, which were suppressed to various degrees. These mixed responses were identified mainly in the quadriceps muscles. Using the visual classification as a reference, we determined that ISIs of 25 and 50 ms are best suited for distinguishing reflexes from mixed responses. The corresponding paired-pulse ratios of 0.51 (25-ms ISI) and 0.47 (50-ms ISI) can reliably discriminate reflex from non-reflex responses. The established cutoffs performed well between repeated sessions separated by 2–3 months. Bipolar TSS exclusively elicited reflexes that were all correctly classified by the derived cutoff values.

Response types elicited by TSS

Early studies demonstrated the possibility of evoking motor responses besides reflexes with a cathode at different locations over the spinal column (T11 to S1) and the anode over the iliac crest contralateral to the leg being examined with high voltage stimulation pulses (Maertens de Noordhout et al. 1988; Troni et al. 1996). More recently, Minassian et al. (2007) reported the presence of motor responses in the quadriceps muscle (seemingly rectus femoris) in one out of eight participants evaluated in the supine position with unipolar TSS (cathode over T11-T12 interspinous space). Our results somewhat differ from Minassian et al. (2007) because we never identified motor responses in isolation but always overlapped with reflexes. In a few instances, however, the visible suppression of the reflex component was rather small, which could be misinterpreted as a lone motor response.

Previous TSS studies have applied different procedures in an attempt to elicit only reflexes, such as adjusting stimulation intensities (Sayenko et al. 2015), moving the stimulation electrodes (Skiadopoulos et al. 2022), and placing the participant in the supine position (Saito et al. 2019). Sayenko et al. (2015) used lower stimulation intensities to minimize the spread of current to motor fibers; however, if the goal is to elicit prominent responses in the quadriceps muscle, higher stimulation intensities are necessary, possibly resulting in mixed responses, as shown here. Searching for the optimal electrode position during recordings is not necessarily difficult but can be time-consuming. Roy et al. (2012) showed that stimuli over L3 elicited markedly suppressed second responses, whereas the same stimulation over S1 produced identical first and second responses, indicating activation of motor axons. Other studies have adopted this approach of adjusting the electrode placement to ensure clear suppression of the second response, suggesting their reflex origin (Skiadopoulos et al. 2022). As to placing the subject in the supine position, Danner et al. (2016) demonstrated that while the prone position predominantly evoked motor or mixed responses, reflex responses were mainly elicited in the supine. In standing, Binder et al. (2021) reported motor responses in the rectus femoris if the spine was flexed, whereas mixed responses were evoked if the spine was extended or in a neutral position. Collectively, optimizing the study protocol could reduce the occurrence of mixed responses, but eliminating them entirely in some muscles is unlikely with unipolar TSS. Thus, mixed responses may need to be accounted for and ideally excluded from analysis if the intent is to draw conclusions based on modification of reflexes.

Paired-pulse paradigm as a discriminator of response types

Paired-pulse paradigm has been used to assess the nature of responses evoked by TSS and account for the possibility of eliciting motor (Danner et al. 2016; Minassian et al. 2007) and mixed responses (Binder et al. 2021; Danner et al. 2016; Saito et al. 2019) in addition to reflexes. However, most TSS reports have an incomplete description of the approach used for ensuring the collected or analyzed data are limited to reflexes. Typically, this description refers to using paired pulses at ISIs from 30–100 ms, but it lacks specific details or criteria other than observing the suppression of the second response before commencing an experiment. However, relying solely on visual suppression may be insufficient because, as shown here, mixed responses can also be suppressed to a variable degree owing to the suppression of the reflex component. Indeed, when we analyzed mixed responses and reflexes in aggregate, the paired-pulse ratios (e.g., UPS1 50-ms ISI in VL1: 0.47, RF1: 0.43, MH1: 0.10, TA1: 0.20, SO1: 0.04) agree with those reported in many previous studies (Courtine et al. 2007; Danner et al. 2016; Hofstoetter et al. 2019; Sayenko et al. 2015). Lesser suppression of mixed responses, commonly observed in the proximal muscles as reported here and previously (Roy et al. 2012; Saito et al. 2021), may accidentally confound the results and lead to different interpretations compared to profound suppression of reflexes exclusively evoked in other muscles.

A few studies have outlined specific criteria for defining reflex responses, such as “complete attenuation” of the second response at 50-ms ISI in the SO and TA muscles (Andrews et al. 2015) or inclusion of responses if the second one was below 100 µV peak-to-peak at 50-ms ISI in any of the recorded muscles (Saito et al. 2019). The complete attenuation criterion, presumably judged online, may not generally be applicable because even reflexes in our study showed, in aggregate, an incomplete suppression at 50-ms ISI across all studied muscles (median paired-pulse ratio 0.060, IQR: 0.023, 0.132). At the same time, our results suggest that even less than complete attenuation would still be sufficient to differentiate reflexes from mixed responses, thereby validating this rather vague criterion. As to the 100-µV criterion adopted by Saito et al. (2019) based on the results of Minassian et al. (2007), we reanalyzed our data at 50-ms ISI and found that this criterion is rather stringent. This is because 35% (212/606) of the visually identified reflexes would have to be excluded, potentially reducing the available data in some muscles. On the other hand, the same criterion misclassified only 10% of mixed responses as reflexes. Collectively, these two criteria would perform well in discriminating reflexes from mixed responses but at the cost of excluding a sizable proportion of reflexes.

Currently, there is no consensus on which ISI best discriminates reflex from non-reflex responses. Most studies use 50-ms ISI (Hofstoetter et al. 2008; Minassian et al. 2007; Roy et al. 2012; Sayenko et al. 2015) to avoid overlap of responses and stimulus artifacts and, at the same time, ensure the most prominent post-activation depression. Our ROC results support that choice since paired-pulse ratios at 25- and 50-ms ISI were the most capable of differentiating reflexes from mixed responses. To avoid contamination of the first response by the second stimulation artifact at 25 ms, we recommend using an ISI in the 30–50 ms range for this purpose.

The most salient finding of our study is the possibility of distinguishing reflexes from mixed responses based on paired-pulse ratios at 25- and 50-ms ISI. Paired-pulse ratios of 0.512 (25-ms ISI) and 0.470 (50-ms ISI) best discriminated reflexes from mixed responses with high sensitivity (100.0%) and specificity (> 99.2%). As expected, the differentiation between the two response types was less successful at ISIs of 100–400 ms, as evident by the decrease in sensitivity and specificity, which is due to the overall lesser paired-pulse suppression of reflexes at longer ISIs (Fig. 3). The cutoff values at 25- and 50-ms ISI were validated when applied to the second session 2–3 months later in the subgroup of initial participants as demonstrated by the negligible rate of misclassified reflexes and mixed responses (Fig. 6). This suggests that derived paired-pulse cutoffs can be utilized over repeated sessions of unipolar TSS in the configuration used here. Furthermore, the same cutoffs were also validated on reflex responses exclusively evoked by bipolar TSS. The bipolar paired-pulse ratios were well below the derived cutoffs resulting in no misclassification of reflexes as mixed responses (Fig. 6). Thus, responses with paired-pulse ratios less than 0.47 (lower limit of CI95% for both 25- and 50-ms ISI) can be considered reflexes with high confidence, whether collected over time or evoked by unipolar or bipolar TSS, as used here.

Limitations

This study has some potential limitations. The results were based on the unipolar TSS configuration with a cathode placed over T11-12 spinal processes. However, this configuration is commonly used, so the results are expected to be applicable to many TSS studies. Although the second session had a smaller sample size, the participants were recruited based on their availability and without a priori knowledge of the results of the first session. Even in this smaller sample, mixed responses were found mainly in the same subjects and muscles, providing confidence in the results. Moreover, the cutoff values derived from the larger sample were successfully validated in the smaller sample by repeating the recording with the same unipolar configuration and extended to the bipolar configuration. The cutoff values were derived by combining responses recorded in different muscles rather than for each muscle separately. Nonetheless, the distribution of the paired-pulse ratios of both reflex and mixed responses at 25- and 50-ms ISI was similar across muscles (Fig. 3), suggesting that individual cutoffs would not differ much from those obtained after pooling the responses. The response amplitudes were analyzed as peak-to-peak values, yet this is advantageous compared to the area under the curve when it comes to the overlapping responses in which different phases can cancel each other. Finally, the cutoff values are derived from neurological intact individuals, which may be considered too restrictive when applied to an injured population, such as after a spinal cord injury, because of their comparably smaller suppression and faster recovery (de Freitas et al. 2021; Hofstoetter et al. 2019). However, the reported paired-pulse ratios are below 0.1 (Knikou and Murray 2019) and 0.2 (Hofstoetter et al. 2019) at ISI at 60 ms; therefore, the established criterion of 0.5 would still likely identify all reflex responses.

Conclusion and implications

We provide evidence that the commonly used unipolar lumbosacral TSS can elicit not only reflexes but also mixed responses consisting of motor and reflex components, mainly in the quadriceps muscles. Reflexes can be distinguished from mixed responses by paired-pulse ratios rounded to 0.5 at 25- and 50-ms ISI. The paired-pulse cutoff is valid when applied over time and for both unipolar and bipolar TSS configurations.

Our findings translate into a simple quantitative approach for ensuring that the input to the spinal cord provided by TSS originates from the depolarization of large afferents. This is important for neurophysiological studies that use TSS to deliver conditioning or test stimuli as well as for neuromodulation interventions whose goal is to modify excitability and engage circuits at the spinal and supraspinal levels. The depolarization of motor fibers would be counterproductive in both situations, possibly causing the collision of descending and ascending volleys within the motor fibers, antidromic depolarization of motor neurons (like F-wave), and the possibility of producing recurrent inhibition. This could confound the interpretation of results in neurophysiological studies and, regarding neuromodulation, alter the state of excitability of the spinal cord in a non-intended manner.

To apply the proposed approach for identifying reflexes, paired-pulse suppression should be quantified. This can be done in preparation for TSS experiments and interventions to ensure adequate TSS input or by eliminating non-reflex responses during data processing before proceeding with the analysis. As a result, future studies would gain greater confidence in conclusions and increase the fidelity of neuromodulation interventions.

Acknowledgements

We thank Antonio Hayes and William A. Pierce for their technical assistance and the Wilson Research Foundation affiliated with Methodist Rehabilitation Center for continuous support.

Author contributions

E.G., D.S., and M.K. conceived and designed research; E.G. and M.K. performed experiments; E.G. and M.K. analyzed data; E.G., D.S., and M.K. interpreted results of experiments; E.G. and M.K. prepared figures; E.G. drafted manuscript; E.G., D.S., and M.K. edited and revised manuscript; E.G., D.S., and M.K. approved final version of manuscript.

Funding

This study was supported by the Wings for Life Spinal Cord Research Foundation (WFLUS-07/19: 199).

Data availability

The data that support the findings of this study are available from the corresponding author upon reasonable request.

Declarations

Conflict of interest

No conflicts of interest, financial or otherwise, are declared by the authors.

Footnotes

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  1. Al’joboori Y, Massey SJ, Knight SL, DonaldsonDuffell NdNLD. The effects of adding transcutaneous spinal cord stimulation (tSCS) to sit-to-stand training in people with spinal cord injury: a pilot study. J Clin Med. 2020;9:2765. doi: 10.3390/jcm9092765. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Andrews JC, Stein RB, Roy FD. Post-activation depression in the human soleus muscle using peripheral nerve and transcutaneous spinal stimulation. Neurosci Lett. 2015;589:144–149. doi: 10.1016/j.neulet.2015.01.041. [DOI] [PubMed] [Google Scholar]
  3. Binder VE, Hofstoetter US, Rienmüller A, Száva Z, Krenn MJ, Minassian K, Danner SM. Influence of spine curvature on the efficacy of transcutaneous lumbar spinal cord stimulation. J Clin Med. 2021 doi: 10.3390/jcm10235543. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Courtine G, Harkema SJ, Dy CJ, Gerasimenko YP, Dyhre-Poulsen P. Modulation of multisegmental monosynaptic responses in a variety of leg muscles during walking and running in humans. J Physiol. 2007;582:1125–1139. doi: 10.1113/jphysiol.2007.128447. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Danner SM, Hofstoetter US, Ladenbauer J, Rattay F, Minassian K. Can the human lumbar posterior columns be stimulated by transcutaneous spinal cord stimulation? A modeling study. Artif Organs. 2011;35:257–262. doi: 10.1111/j.1525-1594.2011.01213.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Danner SM, Krenn M, Hofstoetter US, Toth A, Mayr W, Minassian K. Body position influences which neural structures are recruited by lumbar transcutaneous spinal cord stimulation. PLoS ONE. 2016;11:1–13. doi: 10.1371/journal.pone.0147479. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. de Freitas RM, Sasaki A, Sayenko DG, Masugi Y, Nomura T, Nakazawa K, Milosevic M. Selectivity and excitability of upper-limb muscle activation during cervical transcutaneous spinal cord stimulation in humans. J Appl Physiol. 1985;131(746–759):2021. doi: 10.1152/japplphysiol.00132.2021. [DOI] [PubMed] [Google Scholar]
  8. Dovoedo YH, Chakraborti S. Boxplot-based outlier detection for the location-scale family. Commun Stat Simul Comput. 2015;44:1492–1513. doi: 10.1080/03610918.2013.813037. [DOI] [Google Scholar]
  9. Gad P, Gerasimenko Y, Zdunowski S, Turner A, Sayenko D, Lu DC, Edgerton VR. Weight bearing over-ground stepping in an exoskeleton with non-invasive spinal cord neuromodulation after motor complete paraplegia. Front Neurosci. 2017;11:333. doi: 10.3389/fnins.2017.00333. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Hofstoetter US, Minassian K, Hofer C, Mayr W, Rattay F, Dimitrijevic MR. Modification of reflex responses to lumbar posterior root stimulation by motor tasks in healthy subjects. In: Artificial organs. Blackwell Publishing Inc., 2008, p. 644–648 [DOI] [PubMed]
  11. Hofstoetter US, Freundl B, Binder H, Minassian K. Recovery cycles of posterior root-muscle reflexes evoked by transcutaneous spinal cord stimulation and of the H reflex in individuals with intact and injured spinal cord. PLoS ONE. 2019;14:1–20. doi: 10.1371/journal.pone.0227057. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Hofstoetter US, Freundl B, Lackner P, Binder H. Transcutaneous spinal cord stimulation enhances walking performance and reduces spasticity in individuals with multiple sclerosis. Brain Sci. 2021 doi: 10.3390/brainsci11040472. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Inanici F, Brighton LN, Samejima S, Hofstetter CP, Moritz CT. Transcutaneous spinal cord stimulation restores hand and arm function after spinal cord injury. IEEE Trans Neural Syst Rehabil Eng. 2021;29:310–319. doi: 10.1109/TNSRE.2021.3049133. [DOI] [PubMed] [Google Scholar]
  14. Knikou M. Transpinal and transcortical stimulation alter corticospinal excitability and increase spinal output. PLoS ONE. 2014;9:1–14. doi: 10.1371/journal.pone.0102313. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Knikou M, Murray LM. Repeated transspinal stimulation decreases soleus H-reflex excitability and restores spinal inhibition in human spinal cord injury. PLoS ONE. 2019;14:1–19. doi: 10.1371/journal.pone.0223135. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Krenn MJ, Vargas Luna JL, Mayr W, Stokic DS. Bipolar transcutaneous spinal stimulation evokes short-latency reflex responses in human lower limbs alike standard unipolar electrode configuration. J Neurophysiol. 2020;124:1072–1082. doi: 10.1152/jn.00433.2020. [DOI] [PubMed] [Google Scholar]
  17. Ladenbauer J, Minassian K, Hofstoetter US, Dimitrijevic MR, Rattay F. Stimulation of the human lumbar spinal cord with implanted and surface electrodes: a computer simulation study. IEEE Trans Neural Syst Rehabil Eng. 2010;18:637–645. doi: 10.1109/TNSRE.2010.2054112. [DOI] [PubMed] [Google Scholar]
  18. Maertens de Noordhout A, Rothwell JC, Thompson PD, Day BL, Marsden CD. Percutaneous electrical stimulation of lumbosacral roots in man. J Neurol Neurosurg Psychiatry. 1988;51:174–181. doi: 10.1136/jnnp.51.2.174. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. McGraw KO, Wong SP. Forming inferences about some intraclass correlation coefficients. Psychol Methods. 1996;1:30–46. doi: 10.1037/1082-989X.1.1.30. [DOI] [Google Scholar]
  20. Megía-García Á, Serrano-Muñoz D, Comino-Suárez N, Del-Ama AJ, Moreno JC, Gil-Agudo A, Taylor J, Gómez-Soriano J. Effect of posture and body weight loading on spinal posterior root reflex responses. Eur J Neurosci. 2021;54:6575–6586. doi: 10.1111/ejn.15448. [DOI] [PubMed] [Google Scholar]
  21. Meyer C, Hofstoetter US, Hubli M, Hassani RH, Rinaldo C, Curt A, Bolliger M. Immediate effects of transcutaneous spinal cord stimulation on motor function in chronic, sensorimotor incomplete spinal cord injury. J Clin Med. 2020;9:1–18. doi: 10.3390/jcm9113541. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Minassian K, Persy I, Rattay F, Dimitrijevic MR, Hofer C, Kern H. Posterior root-muscle preflexes elicited by transcutaneous stimulation of the human lumbosacral cord. Muscle Nerve. 2007;35:327–336. doi: 10.1002/mus.20700. [DOI] [PubMed] [Google Scholar]
  23. Nativ A, Frank J, Allard F. The effect of handedness on spinal and supra-spinal reflex excitability. Electroencephalogr Clin Neurophysiol. 1989;72:157–164. doi: 10.1016/0013-4694(89)90177-6. [DOI] [PubMed] [Google Scholar]
  24. Nichols TR, Wisner PM, Cripe G, Gulabchand L. Putting the kappa statistic to use. Qual Assur J. 2010;13:57–61. doi: 10.1002/qaj.481. [DOI] [Google Scholar]
  25. Pembury Smith MQR, Ruxton GD. Effective use of the McNemar test. Behav Ecol Sociobiol. 2020;74:133. doi: 10.1007/s00265-020-02916-y. [DOI] [Google Scholar]
  26. Pulverenti TS, Zaaya M, Grabowski M, Grabowski E, Islam MdA, Li J, Murray LM, Knikou M. Neurophysiological changes after paired brain and spinal cord stimulation coupled with locomotor training in human spinal cord injury. Front Neurol. 2021 doi: 10.3389/fneur.2021.627975. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. R Core Team. R: a language and environment for statistical computing. R Foundation for Statistical Computing 2023. https://www.R-project.org/ . Accessed 28 Feb 2023
  28. Rattay F, Minassian K, Dimitrijevic MR. Epidural electrical stimulation of posterior structures of the human lumbosacral cord: 2. quantitative analysis by computer modeling. Spinal Cord. 2000;38:473–489. doi: 10.1038/sj.sc.3101039. [DOI] [PubMed] [Google Scholar]
  29. Roy FD, Gibson G, Stein RB. Effect of percutaneous stimulation at different spinal levels on the activation of sensory and motor roots. Exp Brain Res. 2012;223:281–289. doi: 10.1007/s00221-012-3258-6. [DOI] [PubMed] [Google Scholar]
  30. Roy FD, Bosgra D, Stein RB. Interaction of transcutaneous spinal stimulation and transcranial magnetic stimulation in human leg muscles. Exp Brain Res. 2014;232:1717–1728. doi: 10.1007/s00221-014-3864-6. [DOI] [PubMed] [Google Scholar]
  31. Saito A, Masugi Y, Nakagawa K, Obata H, Nakazawa K. Repeatability of spinal reflexes of lower limb muscles evoked by transcutaneous spinal cord stimulation. PLoS ONE. 2019;14:1–10. doi: 10.1371/journal.pone.0214818. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Saito A, Nakagawa K, Masugi Y, Nakazawa K. Inter-muscle differences in modulation of motor evoked potentials and posterior root-muscle reflexes evoked from lower-limb muscles during agonist and antagonist muscle contractions. Exp Brain Res. 2021;239:463–474. doi: 10.1007/s00221-020-05973-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Samejima S, Caskey CD, Inanici F, Shrivastav SR, Brighton LN, Pradarelli J, Martinez V, Steele KM, Saigal R, Moritz CT. Multisite transcutaneous spinal stimulation for walking and autonomic recovery in motor-incomplete tetraplegia: a single-subject design. Phys Ther. 2022 doi: 10.1093/ptj/pzab228. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Sayenko DG, Atkinson DA, Floyd TC, Gorodnichev RM, Moshonkina TR, Harkema SJ, Edgerton VR, Gerasimenko YP. Effects of paired transcutaneous electrical stimulation delivered at single and dual sites over lumbosacral spinal cord. Neurosci Lett. 2015;609:229–234. doi: 10.1016/j.neulet.2015.10.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Schwertman NC, Owens MA, Adnan R. A simple more general boxplot method for identifying outliers. Comput Stat Data Anal. 2004;47:165–174. doi: 10.1016/j.csda.2003.10.012. [DOI] [Google Scholar]
  36. Shapkova EY. Spinal locomotor capability revealed by electrical stimulation of the lumbar enlargement in paraplegic patients. In: Latash M, Levin M (eds) Progress in motor control, Urbana, IL: 2004, p. 253–289
  37. Shapkova EY, Schomburg ED. Two types of motor modulation underlying human stepping evoked by spinal cord electrical stimulation (SCES) Acta Physiol Pharmacol Bulg. 2001;26:155–157. [PubMed] [Google Scholar]
  38. Skiadopoulos A, Pulverenti TS, Knikou M. Physiological effects of cathodal electrode configuration for transspinal stimulation in humans. J Neurophysiol. 2022;128:1663–1682. doi: 10.1152/jn.00342.2022. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Tan U. Relationships between hand skill and the excitability of motoneurons innervating the postural soleus muscle in human subjects. Int J Neurosci. 1985;26:289–300. doi: 10.3109/00207458508985627. [DOI] [PubMed] [Google Scholar]
  40. Troni W, Bianco C, Coletti Moja M, Dotta M. Improved methodology for lumbosacral nerve root stimulation. Muscle Nerve. 1996;19:595–604. doi: 10.1002/(SICI)1097-4598(199605)19:5&#x0003c;595::AID-MUS7&#x0003e;3.0.CO;2-6. [DOI] [PubMed] [Google Scholar]
  41. Vallat R. Pingouin: statistics in Python. J Open Source Softw. 2018;3:1026. doi: 10.21105/joss.01026. [DOI] [Google Scholar]
  42. Wu YK, Levine JM, Wecht JR, Maher MT, LiMonta JM, Saeed S, Santiago TM, Bailey E, Kastuar S, Guber KS, Yung L, Weir JP, Carmel JB, Harel NY. Posteroanterior cervical transcutaneous spinal stimulation targets ventral and dorsal nerve roots. Clin Neurophysiol. 2020;131:451–460. doi: 10.1016/j.clinph.2019.11.056. [DOI] [PubMed] [Google Scholar]
  43. Youden WJ. Index for rating diagnostic tests. Cancer. 1950;3:32–35. doi: 10.1002/1097-0142(1950)3:1&#x0003c;32::AID-CNCR2820030106&#x0003e;3.0.CO;2-3. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

The data that support the findings of this study are available from the corresponding author upon reasonable request.

Articles from Experimental Brain Research are provided here courtesy of Springer

RESOURCES