Interhemispheric Inhibition Measurement Reliability in Stroke: A Pilot Study

Abstract

Objective

Reliable transcranial magnetic stimulation (TMS) measures for probing corticomotor excitability are important when assessing the physiological effects of non-invasive brain stimulation. The primary objective of this study was to examine test-retest reliability of an interhemispheric inhibition (IHI) index measurement in stroke.

Materials and Methods

Ten subjects with chronic stroke (≥ 6 months) completed two IHI testing sessions per week for three weeks (six testing sessions total). A single investigator measured IHI in the contra- to-ipsilesional primary motor cortex direction and in the opposite direction using bilateral paired-pulse TMS. Weekly sessions were separated by 24 hours with a 1-week washout period separating testing weeks. To determine if motor-evoked potential (MEP) quantification method affected measurement reliability, IHI indices computed from both MEP amplitude and area responses were found. Reliability was assessed with two-way, mixed intraclass correlation coefficients (ICC_(3,k)). Standard error of measurement and minimal detectable difference statistics were also determined.

Results

With the exception of the initial testing week, IHI indices measured in the contra-to-ipsilesional hemisphere direction demonstrated moderate to excellent reliability (ICC = 0.725 – 0.913). Ipsi-to-contralesional IHI indices depicted poor or invalid reliability estimates throughout the three-week testing duration (ICC= −1.153 – 0.105). The overlap of ICC 95% confidence intervals suggested that IHI indices using MEP amplitude vs. area measures did not differ with respect to reliability.

Conclusions

IHI indices demonstrated varying magnitudes of reliability irrespective of MEP quantification method. Several strategies for improving IHI index measurement reliability are discussed.

Introduction

The implementation of transcranial magnetic stimulation (TMS) in disease states, specifically stroke, has enriched the examination of underlying neural impairment and subsequent neural reorganization.¹ A TMS pulse of sufficient intensity strategically targeted over the primary motor area (M1) of a preselected muscle generates a downstream muscle response referred to as the motor-evoked potential (MEP). TMS measures of cortical and corticospinal excitability, including but not limited to resting and active motor thresholds and intracortical facilitation and inhibition, are based on MEP amplitude and area responses. The capability of TMS in probing motor tract physiology and neural circuitry in conscious individuals imparts valuable knowledge to researchers and clinicians alike. Further, significant associations between TMS-derived measures of excitability and paretic extremity function pose important clinical implications.^2–10

However, as Schambra et al.¹¹ recently asserted, widespread use of TMS to assess neurophysiology and treatment-induced change in neurophysiology without critical assessment of TMS measurement properties (i.e. reliability, validity, and responsiveness) perpetuates a “false sense of security” that TMS is a sound, unerring instrument. Although the quality of an instrument, as determined by the above measurement properties, influences study design, subject recruitment, and other methodological decisions, there exists a shortage of literature examining the quality of TMS as an instrument to probe neurologic structure and function. Several studies have documented the reliability of TMS measures in healthy^11–20 and stroke populations.^{11,13,15,16,21} Those TMS measurements include MEP amplitude, ^{13,16,17,19,21} MEP area,¹⁹ MEP latency,^13,17,21 cortical silent period,^12,16,21 short-interval intracortical inhibition,^11,14 intracortical facilitation,^11,14 total motor conduction time,¹⁵ resting and active motor thresholds,^{11,16–18,20} stimulus response curves,^11,16,18 and TMS map area, location, volume, and center of gravity.^18,20

One TMS measurement particularly relevant in stroke is GABA_B-mediated^22–24 interhemispheric inhibition (IHI) that involves inhibition projected from one hemisphere to the other through transcallosal fiber connections.^25–27 In healthy individuals, the magnitude of IHI imparted in either direction is typically balanced at rest.^26,28 During movement preparation, IHI balance temporarily shifts, favoring the hemisphere contralateral to the recruited extremity, to ensure the execution of smooth, coordinated movement.^28–30 Following stroke, IHI imbalances often persist^31–34 and negatively impact motor function. Such imbalances pose potential therapeutic targets for repetitive transcranial magnetic stimulation (rTMS) and other forms of non-invasive brain neuromodulation that seek to rebalance IHI by increasing cortical excitability in the ipsilesional hemisphere and/or reducing excitability in the contralesional hemisphere. In contrast to the aforementioned TMS measurements, IHI index reliability is not well documented in the literature. De Gennaro et al.³⁵ reported low test-retest correlations of IHI index measurements in healthy individuals. These findings, however, reflect low association between repeated IHI measurements rather than low test-retest reliability. Therefore, the primary objective of this study was to expand upon prior TMS reliability analyses in stroke by examining the test-retest reliability of an IHI index measurement based on MEP responses from bilateral first dorsal interosseous (FDI) muscles. Additionally, we sought to determine if the quantification method of the MEP size (i.e. area vs. amplitude) affected IHI index reliability. Lastly, we wanted to establish preliminary IHI index measurement characteristics by computing standard error of measurement and minimal detectable difference statistics.

Materials and Methods

Participants

Participants were recruited as part of a larger investigation examining changes in cortical excitability and paretic hand function following different rTMS treatments delivered to the contralesional hemisphere.³⁶ In that five-week crossover study, participants received one session of each of the following treatments in randomized order: 6 Hz primed 1 Hz rTMS, sham 6 Hz primed active 1 Hz rTMS, and 1 Hz primed 1 Hz rTMS. A one-week washout period separated rTMS treatment sessions. Importantly, a separate week-to-week comparison of IHI baseline measurements during rTMS administration using a mixed-effects linear model revealed no treatment carryover or period effects, thus confirming no influence of rTMS on baseline IHI index measurements making the use of these data for reliability assessment appropriate.³⁶

Ten participants that completed the above study were included in the current reliability analysis. Participants were at least 18 years of age with chronic (≥ 6 months) ischemic or hemorrhagic stroke of cortical and/or subcortical involvement. Participants were excluded if they did not possess sufficient cognition per Mini-Mental State Examination (MMSE)³⁷ (score ≤ 24 out of 30), experienced a seizure within the last two years, and demonstrated an absent MEP response from ipsilesional M1. Additional exclusionary criteria included pregnancy, indwelling metal in head, and implanted medical devices. A neurologist reviewed medical records and imaging prior to participant in-person screening procedures that involved an assessment of neurological deficit, handedness, mood, and cortical excitability. Table 1 summarizes participant demographics. The Clinical and Translational Science Institute and the Institutional Review Board of the University of Minnesota approved this study. All participants provided written informed consent.

Table 1.

Participant Demographics

Participant	Sex	Age (years)	Time Since Stroke (months)	Stroke Hemisphere	Stroke Location	Stroke Type	NIHSS (0–44)	UEFM (0–66)	MMSE (0–30)	Contra / Ipsi 0.50 mV Threshold?
1	M	64	118	L	FPL	I	4	52	24	Y / Y
2	M	84	106	L	FPL	I	5	32	25	Y / Y
3	F	71	29	L	PFL DWM	I	3	46	28	Y / N
4	M	72	20	L	CR, PL	I	1	64	30	Y / Y
5	M	48	16	L	T	H	3	58	30	Y / Y
6	F	60	13	L	VM	I	3	30	30	Y / N
7	F	59	95	R	FL	I	3	64	28	Y / Y
8	M	63	12	R	T	H	10	12	29	N / N
9	M	67	34	L	CA FL PL T	I	3	44	18	Y / N
10	M	74	29	R	PFL PLEC	I	2	54	30	Y / Y
		66.2 (9.8)	47.2 (41.8)				3.0 [3.0, 3.8]	49.0 [35.0, 57.0]	28.5 [25.8, 30.0]

Group values presented as mean (standard deviation) or median [interquartile range: Q1, Q3]. CA caudate, Contra contralesional, CR corona radiata, DWM diffuse white matter, FL frontal lobe, FPL frontoparietal lobe, H hemorrhagic, I ischemic, Ipsi ipsilesional, L left, MMSE Mini-Mental State Examination (lower scores indicate greater cognitive impairment), N no, NIHSS National Institute of Health Stroke Scale (higher scores indicate greater severity), PFL posterior frontal lobe, PL parietal lobe, PLEC posterior limb of external capsule, R right, T thalamus, UEFM Upper Extremity Fugl-Meyer (lower scores indicate greater deficit), VM ventral medulla, Y yes

Study Design & Equipment

The current reliability study design mirrors the above-described rTMS study. The crossover design afforded multiple assessments of IHI measurement reliability. Participants completed three weeks of testing with a one-week washout period separating testing weeks. Each testing week consisted of two IHI testing sessions separated by approximately 24 hours with no intervening interventions provided during the 24-hour timeframe. During this time, participants received instruction to refrain from caffeine consumption prior to all visits. Short-interval intracortical inhibition, intracortical facilitation, and cortical silent period measurements from the ipsilesional hemisphere were also collected; however, the primary focus of this current reliability analysis remains the IHI index. Investigators examined the magnitude of IHI imparted by contralesional M1 on ipsilesional M1 (contra-to-ipsilesional IHI) and vice versa (ipsi-to-contralesional IHI) using two 50 mm figure-eight coils, two Magstim 200² stimulators with a Bistim² connecting module, and a Bistim² Trigger Box (Magstim Company Ltd, Spring Gardens, UK).

Threshold Determination

Participants rested in a reclined position wearing earplugs and surface electrodes on their paretic and non-paretic FDI muscles. Electromyography (EMG) signals were collected with a Cadwell Sierra Wave EMG device (Cadwell Laboratories, Kennewick, WA). EMG signals were amplified, bandpass (20 Hz to 2.0 kHz) filtered, and digitized at a sampling rate of 6.4 kHz. A 60-Hz notch filter was also applied. Total recording time for each trial was 300 ms with an embedded pre-trigger duration time of 30 ms. Investigators encouraged participants to maintain a quiet and awake resting state with eyes open during TMS recordings. The EMG signal was visually monitored throughout testing to confirm subjects’ resting state. EMG data was stored on a laptop for offline analysis. MEP amplitude and area were quantified using customized Matlab script. We defined MEP area using the equation $\mathit{Area}=\frac{1}{f_s}{\int }_{t_1}^{t_2}\mid X\mid dt$, where f_s is the sampling frequency and X is the EMG signal. MEP onset (t₁) and offset (t₂) timepoints were the first points on the rectified EMG signal located towards the beginning and end of the MEP that were three standard deviations above the baseline standard deviation.³⁸

The investigator made temporary markings on the participant’s scalp to denote their motor hotspot, known as the region on the scalp where single TMS pulses generated MEPs from the FDI muscle with the lowest stimulation intensity. The stimulation intensities for the conditioning and test pulses during IHI testing (described below) were derived from either the resting motor threshold (RMT) or 0.50 mV threshold. To determine the participant’s RMT and 0.50 mV motor threshold, the investigator positioned the 50 mm figure-eight coil 45 degrees posterolaterally to the cranial mid-sagittal line on the motor hotspot and applied single TMS pulses. The RMT was the lowest stimulation intensity that generated 50 μV MEP responses in at least 3 out of 5 trials.^39,40 The 0.50 mV threshold was the lowest stimulation intensity that produced 0.50 mV MEP responses in at least 3 out of 5 trials. Thresholding procedures were completed for both ipsilesional and contralesional hemispheres and were repeated at each testing session.

Interhemispheric Inhibition Testing

During ipsi-to-contralesional IHI testing, the investigator applied a conditioning pulse to the ipsilesional M1 motor hotspot followed 10 ms later by a test pulse to the contralesional M1 motor hotspot. The stimulation intensities of the conditioning and test pulses were equal to the participant’s ipsilesional and contralesional 0.50 mV thresholds, respectively. If a 0.50 mV threshold was not found, the pulse intensity was set to 120% of the participant’s RMT. The sequence of conditioning and test pulses was reversed to assess IHI in the contra-to-ipsilesional direction. Stimulation intensity of the test pulses during single-pulse trials was also equal to either the participant’s 0.50 mV threshold or 120% of RMT.

One investigator manually held the coils over the ipsi- and contralesional motor hotspots in an identical position as described above during paired and single-pulse TMS delivery. A second investigator handled equipment settings to ensure proper directional delivery of the TMS pulses and visually monitored coil position accuracy and EMG signals. Participants maintained a comfortable resting state during the completion of one block of 40 total trials: 10 trials of single test pulses to contralesional M1, 10 trials of single test pulses to ipsilesional M1, 10 trials of conditioning pulses to contralesional M1 followed by test pulses to ipsilesional M1, and 10 trials of conditioning pulses to ipsilesional M1 followed by test pulses to contralesional M1. Trials were separated by six seconds and delivered in random order.

The IHI index was calculated by dividing the average paired-pulse (conditioned) MEP amplitude and area responses by the average test pulse (non-conditioned) MEP amplitude and area responses. Ratios < 1 indicated an inhibited response. All MEP responses were analyzed with no normalization or filtering for outliers prior to index computation.

Statistics

IHI index measurement reliability was assessed using a two-way, mixed effects model to estimate intraclass correlation coefficients (i.e., ICC_(3,k)) with the tester as the fixed effect and subjects as the random effects.⁴¹P-values ≤ 0.05 were considered statistically significant. An ICC value was computed for each testing week consisting of two sessions separated by 24 hours. ICCs ≥ 0.90 constituted excellent reliability, ≥ 0.75 demonstrated good reliability, ≥ 0.50 indicated moderate reliability, and < 0.50 were classified as poor.⁴² We employed identical statistical procedures to determine test-retest reliability of several secondary dependent measures: non-conditioned MEP responses, RMT, and 0.50 mV thresholds from ipsi- and contralesional hemispheres. Prior to the above calculations, variables were tested for normal distribution using the Shapiro-Wilk test. If a violation of normality occurred, a log transformation was done with back-transformed means and 95% confidence intervals reported.

To assess TMS measurement response stability, we calculated the standard error of measurement $\text{(standard error of measurement}=\text{SD}\sqrt{1-\text{ICC}}$, where SD denotes the standard deviation based on the total sum of squares from the repeated-measures ANOVA.)⁴³ Minimal detectable differences (MDD) were computed ( ${\text{MDD}}_{95}=\sqrt{2}\ast 1.96\ast \text{SEM}$, where SEM is the standard error of measurement⁴²) to determine the smallest amount of measurement change required to represent a true difference.⁴²

Associations between baseline upper-extremity Fugl-Meyer (UEFM) score and bilateral IHI indices computed from MEP amplitude responses were analyzed using Spearman correlation coefficients. P-values ≤ 0.05 were considered statistically significant. All statistical tests were performed using SPSS Version 23.0 software (IBM Corp., Armonk, NY).

Results

Participants

Ten participants completed the study. Amongst those participants, three had a cortical stroke (participants 1, 2, and 7), three had a subcortical stroke (participants 5, 7, and 8), and four had cortical and subcortical involvement (participants 3, 4, 9, and 10; Table 1). All participants tolerated TMS testing procedures with no serious adverse events. Nine participants received contralesional M1 stimulation equal to their 0.50 mV threshold with the remaining participant receiving contralesional M1 stimulation equivalent to 120% of their RMT. Six participants received ipsilesional M1 stimulation equal to their 0.50 mV threshold with four remaining participants receiving stimulation equal to 120% of their RMT (Table 1).

Reliability Analyses

Interhemispheric Inhibition Index

Table 2 provides a detailed summary of all reliability analyses. Test-retest reliability of the contra-to-ipsilesional IHI index varied from poor (ICC = 0.158 – 0.379) during the first week of testing to moderate to excellent during the second and third testing weeks (ICC = 0.725 – 0.913). The ipsi-to-contralesional IHI index displayed poor reliability throughout all three testing weeks (ICC = −1.153 – 0.105). The reliability of non-conditioned MEP amplitude and area responses, based on the average of ten non-conditioned TMS test pulses, ranged from poor to excellent (ICC = 0.293 – 0.911) from the contralesional hemisphere and moderate to excellent (ICC = 0.508 – 0.921) from the ipsilesional hemisphere.

Table 2.

Motor-Evoked Potential and Threshold Reliability Measurements

MEP Amplitude-Based Measurements	ICC	95% CI	p-value	SEM	MDD
Week 1
Ipsi-to-Contralesional IHI	−0.095	−3.408 – 0.728	0.553	0.441	1.223
Contra-to-Ipsilesional IHI	0.158	−2.392 – 0.791	0.401	0.431	1.194
Ipsilesional Hemisphere MEP (mV)	0.650	−0.407 – 0.913	0.067	0.391	1.084
Contralesional Hemisphere MEP (mV)	0.293	−1.848 – 0.824	0.307	0.319	0.885
Week 3
Ipsi-to-Contralesional IHI	−0.383	−4.568 – 0.656	0.682	0.425	1.177
Contra-to-Ipsilesional IHI	0.859	0.432 – 0.965	0.004	0.209	0.580
Log(Contra-to-Ipsilesional IHI)	0.841	0.360 – 0.961	0.006	--	--
Ipsilesional Hemisphere MEP (mV)	0.897	0.584 – 0.974	0.001	0.171	0.475
Contralesional Hemisphere MEP (mV)	0.675	−0.308 – 0.919	0.055	0.213	0.591
Week 5
Ipsi-to-Contralesional IHI	−1.153	−7.669 – 0.465	0.866	0.544	1.508
Contra-to-Ipsilesional IHI	0.822	0.282 – 0.956	0.009	0.210	0.583
Ipsilesional Hemisphere MEP (mV)	0.836	0.341 – 0.959	0.006	0.204	0.566
Contralesional Hemisphere MEP (mV)	0.911	0.643 – 0.978	0.001	0.129	0.356

MEP Area-Based Measurements	ICC	95% CI	p-value	SEM	MDD
Week 1
Ipsi-to-Contralesional IHI	0.005	−3.006 – 0.753	0.497	0.447	1.239
Contra-to-Ipsilesional IHI	0.379	−1.498 – 0.846	0.244	0.371	1.027
Ipsilesional Hemisphere MEP (μV·S)	0.508	−0.908 – 0.878	0.153	0.203	0.562
Contralesional Hemisphere MEP (μV·S)	0.350	−1.616 – 0.839	0.266	0.123	0.341
Week 3
Ipsi-to-Contralesional IHI	0.105	−2.604 – 0.778	0.436	0.356	0.985
Contra-to-Ipsilesional IHI	0.913	0.651 – 0.978	0.001	0.165	0.457
Log(Contra-to-Ipsilesional IHI)	0.933	0.730 – 0.983	< 0.001	--	--
Ipsilesional Hemisphere MEP (μV·S)	0.921	0.683 – 0.980	< 0.001	0.057	0.158
Log(Ipsilesional Hemisphere MEP)	0.945	0.778 – 0.986	< 0.001	--	--
Contralesional Hemisphere MEP (μV·S)	0.542	−0.846 – 0.886	0.130	0.092	0.256
Log(Contralesional Hemisphere MEP)	0.589	−0.655 – 0.898	0.101	--	--
Week 5
Ipsi-to-Contralesional IHI	−0.921	−6.733 – 0.523	0.827	0.552	1.529
Contra-to-Ipsilesional IHI	0.725	−0.108 – 0.932	0.034	0.280	0.777
Ipsilesional Hemisphere MEP (μV·S)	0.800	0.193 – 0.950	0.013	0.084	0.232
Contralesional Hemisphere MEP (μV·S)	0.820	0.276 – 0.955	0.009	0.068	0.187

Threshold Testing	ICC	95% CI	p-value	SEM	MDD
Week 1
Ipsilesional RMT	0.994	0.975 – 0.998	< 0.001	1.725	4.782
Contralesional RMT	0.989	0.956 – 0.997	< 0.001	1.469	4.071
Ipsilesional 0.50 mV Threshold	0.967	0.807 – 0.994	< 0.001	1.684	4.667
Contralesional 0.50 mV Threshold	0.968	0.857 – 0.993	< 0.001	2.752	7.629
Week 3
Ipsilesional RMT	0.987	0.947 – 0.997	< 0.001	3.176	8.802
Log(Ipsilesional RMT)	0.982	0.926 – 0.995	< 0.001	--	--
Contralesional RMT	0.977	0.908 – 0.994	< 0.001	2.471	6.850
Log(Contralesional RMT)	0.968	0.871 – 0.992	< 0.001	--	--
Ipsilesional 0.50 mV Threshold	0.994	0.963 – 0.999	< 0.001	1.403	3.888
Log(Ipsilesional 0.50 mV Threshold)	0.988	0.929 – 0.998	< 0.001	--	--
Contralesional 0.50 mV Threshold	0.956	0.807 – 0.990	< 0.001	2.870	7.954
Log(Contralesional 0.50 mV Threshold)	0.949	0.775 – 0.989	< 0.001
Week 5
Ipsilesional RMT	0.997	0.987 – 0.999	< 0.001	1.593	4.416
Contralesional RMT	0.976	0.904 – 0.994	< 0.001	2.813	7.796
Log(Contralesional RMT)	0.970	0.881 – 0.993	< 0.001	--	--
Ipsilesional 0.50 mV Threshold	0.960	0.765 – 0.993	0.001	3.173	8.794
Log(Ipsilesional 0.50 mV Threshold)	0.952	0.718 – 0.992	0.001	--	--
Contralesional 0.50 mV Threshold	0.957	0.808 – 0.990	< 0.001	2.876	7.973

Log-transformed values presented as back-transformed means (95% confidence interval). Interhemispheric inhibition values represent the ratio of conditioned to non-conditioned motor-evoked potential amplitude and area responses. CI confidence interval, ICC intraclass correlation coefficient, IHI interhemispheric inhibition, MDD 95% minimal detectable difference, MEP motor-evoked potential, RMT resting motor threshold, SEM standard error of measurement

Threshold

Contra- and ipsilesional M1 RMT measurements demonstrated excellent reliability throughout the three testing weeks (ICC = 0.976 – 0.997, Table 2). Contra- and ipsilesional 0.50 mV threshold measurements also showed excellent reliability (ICC = 0.956 – 0.994, Table 2).

Standard Error of Measurement & Minimal Detectable Difference

For IHI indices computed from MEP amplitude measurements, standard error of measurement and MDD values ranged from 0.209 – 0.431 and 0.580 – 1.194, respectively, in the contra-to-ipsilesional direction and 0.425 – 0.544 and 1.177 – 1.508, respectively, in the ipsi-to-contralesional direction (Table 2). For contra-to-ipsilesional IHI indices computed from MEP area measurements, standard error of measurement and MDD values ranged from 0.165 – 0.371 and 0.457 – 1.027, respectively, and 0.356 – 0.552 and 0.985 – 1.529, respectively, in the ipsi-to-contralesional direction (Table 2).

Supplementary materials provide additional reports on group IHI index and threshold measurements during each session (Supplementary Table 1) and individual IHI index measurements (Supplementary Table 2) and non-conditioned MEP measurements (Supplementary Table 3).

Behavioral & Electrophysiological Measurement Association

Baseline UEFM scores and IHI indices computed from MEP amplitude responses at visits 1 and 2 during all testing weeks did not demonstrate significant associations. For contra-to-ipsilesional IHI index measurements, Spearman rho values ranged from −0.182 (p = 0.614) at visit 1 during week 3 to 0.195 (p = 0.590) at visit 1 during week 5. For ipsi-to-contralesional IHI index measurements, Spearman rho values ranged from −0.122 (p = 0.738) at visit 1 during week 3 to 0.620 (p = 0.056) at visit 2 during week 3.

Discussion

The results of this study reveal varying magnitudes of IHI index measurement reliability spanning both contralesional and ipsilesional hemispheres irrespective of MEP size quantification method (i.e. amplitude vs. area). In general, IHI index measurements in the ipsi-to-contralesional direction demonstrated poor reliability, and, with the exception of the initial testing week, IHI index measurements in the contra-to-ipsilesional direction depicted moderate to excellent reliability. This was the first study to evaluate the reliability of bilateral paired-pulse TMS measurements of IHI in stroke and provide preliminary IHI index measurement properties. The following discussion identifies potential factors that influenced measurement reliability and suggests strategies for improving TMS measurement reliability.

IHI Index Reliability

Measures of reliability represent between- and within-subject variability. In several instances, ICC values for ipsi-to-contralesional IHI indices and their corresponding 95% confidence intervals deviated from the theoretical ICC range of 0 to 1 (Table 2). We attributed these findings to a combination large within-subject variability and small between-subject variability. Indeed, the p-values listed in Table 2, representing the between-subjects effect, were not significant in these instances. A lack of significance indicated that subjects were not different from one another on the basis of their IHI index measurement. Small between-subject variability in TMS response may also explain the poor reliability finding of contra-to-ipsilesional IHI index measurements during the first week of testing as compared to later testing weeks.

Because the IHI index is the ratio of conditioned MEP responses to non-conditioned MEP responses, low IHI index measurement reliability might reflect inconsistent paired-pulse and/or single-pulse MEP responses. During the first week of testing, non-conditioned MEP measurements from the contralesional hemisphere demonstrated poor reliability. This finding suggests that low contra-to-ipsilesional IHI index reliability may have arisen from unreliable non-conditioned MEP responses or from a combination of unreliable single- and paired-pulse MEP responses. Throughout testing, non-conditioned MEP responses from the ipsilesional hemisphere demonstrated moderate to excellent reliability that may indicate issues with paired-pulse MEP response reliability. Interestingly, non-conditioned MEP responses from the ipsilesional hemisphere demonstrated higher reliability than non-conditioned MEP responses from the contralesional hemisphere. Higher stimulation intensities applied to the ipsilesional hemisphere may account for greater reliability of ipsilesional non-conditioned MEP responses. However, when formally comparing stimulation intensities for ipsilesional and contralesional hemispheres during visits 1 and 2 across weeks 1, 3, and 5 (not shown) using parametric and non-parametric paired t-tests, hemispheric differences were only statistically significant at visits 1 (p = 0.008) and 2 (p = 0.033) during week 5. In this group of participants with chronic stroke, it appears that hemispheric differences in stimulation intensity may not fully explain the hemispheric differences in non-conditioned MEP response reliability. Lower ICC values for non-conditioned MEP responses from the contralesional hemisphere may reflect lower between-subject variability in MEP responses from the contralesional hemisphere compared to MEP responses from the ipsilesional hemisphere. Restated, greater homogeneity in contralesional non-conditioned MEP response measurements amongst the sample may have resulted in lower ICC values. Our findings of greater ipsilesional non-conditioned MEP response reliability aligns with past work by Koski et al.²¹ and Liu et al.¹⁶ that found greater ICC values for ipsilesional MEP amplitude measurements from the FDI muscle under active muscle recruitment.

Increased within-subject variability in TMS responses and, consequently, low IHI index measurement reliability may have resulted from experimental and/or equipment-related issues. For instance, the small number of single- and paired-pulse trials during IHI testing likely introduced considerable within-subject variability, thereby impacting reliability. During IHI testing, in the majority of participants, suprathreshold conditioning and test pulses were delivered at intensities sufficient enough to produce an MEP response of 0.50 mV. However, MEPs of this magnitude from ipsilesional M1 are frequently absent even at intensities of maximum stimulator output. When the investigator could not determine the 0.50 mV motor threshold, the conditioning and test pulses delivered to the ipsilesional hemisphere were set at an intensity of 120% of the participant’s RMT. Bilateral 0.50 mV threshold and RMT measurements demonstrated excellent reliability (not shown). However, utilizing different thresholding tactics within some individuals (Table 1; participants 3, 6, and 9) may have introduced variability in MEP responses, thus, negatively impacting IHI index reliability.

It is also possible that the conditioning and test pulse intensities along with the preset interstimulus interval may not have been the optimal parameters for generating maximum IHI in all participants. Du et al.¹⁴ found increased variability during short-interval intracortical inhibition and facilitation testing dependent on the interstimulus interval. Investigators recommended individual stimulation profiles to address this variability. A similar solution may also apply to IHI testing that utilizes individual recruitment-like curves to determine pulse intensities and interstimulus intervals that produce the greatest amount of IHI.

Aside from bilateral paired-pulse TMS IHI testing, an alternative method of examining IHI is the ipsilateral cortical silent period.²⁶ Previous work has shown that the ipsilateral cortical silent period, similar to bilateral paired-pulse IHI, is mediated by transcallosal connections.²⁵ This unilateral single-pulse TMS measure may be less vulnerable to experimental error since the procedure involves the handling and positioning of one versus two coils. Active muscle contraction performed during ipsilateral cortical silent period testing may reduce variability in MEP response attributed to rapid fluctuations in cortical and spinal excitability.^44,45 Participants’ fixation on visual feedback of subthreshold muscle contraction levels may also control attention states during testing⁴⁵ and further reduce MEP response variability. Additional research is necessary to directly compare unilateral and bilateral paired-pulse IHI methodology and to confirm that the former approach results in greater measurement reliability.

Coil placement and size may have also negatively impacted measurement reliability and within-subject MEP response variability. In our study, one investigator maintained the positioning of two 50 mm figure-eight coils, and slight shifts and tilting of the coils may have inadvertently occurred. Previous studies collecting IHI measurements have employed different TMS coil shapes²⁶ and sizes.³⁴ One study modified coil orientation during IHI testing to address coil overlap.⁴⁶ Since coil orientation impacts the depth and strength of the TMS-induced electric field,⁴⁷ adjusting coil position during IHI testing may necessitate finding the motor hotspot and corresponding thresholds in this exact coil arrangement. Yet, other work has shown no differences in the amount of IHI after varying coil positions and resulting current direction.⁴⁸

Neuronavigation may alleviate some of the variability in MEP responses and motor hotspot locations associated with inconsistent coil orientation and tilt,^20,49 which could translate to more reliable electrophysiological measures.⁵⁰ However, neuronavigation does not address potential variability attributed to an individual’s physiological brain state and may further explain why other studies have found no improvement in MEP and motor threshold variability between neuronavigation and conventional TMS systems.^51,52

An individual’s brain state is an important consideration when assessing variability in TMS response. Arousal⁵³ and attention⁵⁴ states, for example, affect corticospinal excitability. Standardizing brain states by priming the brain with rTMS prior to application of the intended non-invasive brain stimulation intervention has been hypothesized as a means to address potential variability.⁵⁵ Combining TMS with brain computer interface or electroencephalography^56,57 may also prove helpful in probing one’s brain state prior to formal TMS testing and intervention delivery. In this study, investigators instructed participants to avoid caffeine prior to testing. Additionally, medication-use may have also influenced brain state and heightened within-subject variability in TMS response. This concern, however, is dampened because all medication-use was consistent at both testing sessions throughout the study.

MEP Amplitude vs. MEP Area

A distinct feature of this study was the measurement of TMS single- and paired-pulse measures derived from both MEP amplitude and area responses. Past work concluded no differences in reliability in MEP analysis methods using either MEP amplitude or area in healthy individuals.¹⁹ We found slightly higher ICC values for area-based measures versus amplitude-based measures; however, these differences are likely non-significant due to the overlap of confidence intervals. These findings support the utilization of amplitude and area methodologies in measuring MEP responses.

Standard Error of Measurement & Minimal Detectable Difference

Standard error of measurement and MDD values for both conditioned and non-conditioned TMS measures accompanied the reliability analysis. Contra-to-ipsilesional IHI standard error of measurement values were consistently less than those of ipsi-to-contralesional IHI, suggesting less measurement noise and greater measurement stability in the former. Comparison of standard error of measurement values for non-conditioned MEP responses between contra- and ipsilesional hemispheres revealed less prominent differences, indicating comparable stability in single-pulse MEP responses from both hemispheres during the chronic stroke timeframe.

Issues of insufficient statistical power frequently arise in clinical research. MDD is a valuable statistic to assess treatment effect and to ensure accuracy in statistical power calculations during initial study planning. In this study, if TMS trial-by-trial measurement change surpassed the corresponding MDD value (Table 2), there was a 5% chance of obtaining a measurement demonstrating the same or a greater extent of change relative to the MDD value.⁵⁸ In line with the preceding discussion, measurements that depict greater stability, consistent with small standard error of measurement values, also detect smaller trial-by-trial change.

Brasil-Neto et al.⁵⁹ and Koski et al.²¹ computed coefficients of variation across trials to examine TMS response stability and to determine the number of TMS trials necessary for obtaining mean MEP amplitude measurements within a certain percentage of the true MEP amplitude. In healthy subjects, five trials were sufficient to obtain an accurate MEP amplitude estimation within 20% of the true MEP amplitude.⁵⁹ In individuals with stroke, five trials were adequate to acquire accurate MEP amplitude estimates from the contralesional hemisphere, but at least 10 trials were required to attain accurate MEP amplitude estimates from the ipsilesional hemisphere.²¹ Based on our standard error of measurement values, ipsi-to-contralesional IHI index measurements would also require a greater number of trials compared to identical measures collected in the contra-to-ipsilesional direction. Using similar computations as Brasil-Neto and colleagues⁵⁹ but applying the highest standard error of measurement values to determine the most conservative estimate for both ipsi-to-contralesional and contra-to-ipsilesional IHI indices, we recommend a total of 20 ipsi-to-contralesional IHI trials and 12 contra-to-ipsilesional IHI trials to be within 20% of the true IHI index measurement 90% of the time. Determination of standard error of measurement and MDD metrics in acute and subacute stroke populations and elucidating the medical relevance underlying true TMS measurement changes are important next research steps.

Behavioral & Electrophysiological Measurement Association

We did not observe significant associations between baseline UEFM, an assessment of paretic arm and hand impairment, and IHI indices in either hemisphere direction. Further, the strength of the association or correlation coefficient for UEFM score and contra-to-ipsilesional IHI index did not increase from week 1 to week 5 in parallel with increasing ICC values. Murase et al.³¹ measured IHI during a reaction time paradigm in nine subjects with chronic stroke and found significant associations between contra-to-ipsilesional M1 IHI indices at movement onset and paretic finger-tapping speed, results replicated by Duque and colleagues.³² Active versus resting IHI testing conditions may explain why our behavioral and electrophysiological measurement associations were not significant; whereas, those from Murase and Duque et al. were significant.^31,32 The choice of motor test may also matter. The UEFM exam assesses both proximal and distal upper-extremity impairment. In contrast, finger-tapping speed is solely a fine-motor test that likely provides a greater representation intrinsic muscle function, specifically the FDI muscle, where MEP responses were recorded from during IHI testing. Indeed, Harris-Love and colleagues observed significant associations between UEFM scores in 16 subjects with chronic stroke and IHI measurements with MEP responses recorded from the paretic biceps and triceps.⁶⁰ Notably, the ipsilateral silent period was used to assess IHI.⁶⁰ Consequently, similar ipsilateral silent period testing with MEP responses collected from the paretic FDI in subjects with chronic stroke did not demonstrate significant association with UEFM.⁶¹ Lastly, significant behavioral and electrophysiological measurement associations were likely absent due to our small sample demonstrating a wide range of UEFM baseline scores (minimum to maximum score = 12 to 64 on a 66-point scale; Table 1) but a less wide range of IHI indices (Supplementary Table 2) from either hemisphere.

Limitations

This pilot work contains some limitations. Our small sample size displayed substantial heterogeneity (Table 1) with regards to stroke characteristics. For instance, 30% of the strokes were located in the right hemisphere, 70% extended into subcortical matter, and 20% were hemorrhagic. Despite the chronic timeframe of stroke recovery in all participants, it is possible that such variability in stroke presentation negatively influenced reliability. Participants’ UEFM scores also displayed considerable variability (Table 1). Though IHI index testing did not involve an active motor condition, past work has shown significant associations between UEFM scores in chronic stroke with lesion volume⁶² and white matter integrity.⁶¹ Such anatomical factors may ultimately influence IHI reliability. Notably, a participant with a MMSE score less than our 24-point cutoff was included in this study following physician assessment and consultation. In this instance, post-stroke aphasia impairments primarily contributed to the participant’s low MMSE score. Collectively, less restrictive stroke and behavioral entry criteria resulted in a less homogenous sample. Yet, the lack of heterogeneity or between-subject variability in TMS responses likely perpetuated poor IHI index measurement reliability. A larger sample size may improve IHI measurement reliability by enhancing between-subject variability in TMS response. Our sample size did not possess adequate between-subject variability for certain TMS measurements to satisfy the computation of valid ICCs. The negative ICC values reported should be interpreted as invalid, likely resulting from low dispersion of TMS IHI measurements amongst participants in our sample (i.e. low between-subject variability). Elevated within-subject variability and/or measurement error may have also contributed to the low ICC values. The ICC is the appropriate statistic to assess test-retest reliability. Given the different purposes, other statistical procedures (e.g. Pearson or Spearman correlation coefficients, Cronbach’s alpha, Kappa, and t-test) are not appropriate to assess test-retest reliability.⁴²

Another limitation in this pilot work is the absence of a healthy age-matched control group to differentiate aging effects from post-stroke effects on TMS measurement reliability¹¹ and, most importantly, to confirm and/or dispute prior work that demonstrated low reliability of IHI measurements in healthy individuals.³⁵ Inclusion of a control group may also uncover possible differences in tract recruitment between healthy and stroke populations upon TMS pulse delivery. Post-stroke compensatory recruitment of extrapyramidal tracts may contribute to downstream MEP responses. In rodent stroke models, TMS activated several descending motor tracts, including reticulospinal tract fibers, that subsequently influenced spinal neuron activity and MEP generation.⁶³ These findings support more recent work demonstrating reticulospinal input onto motoneurons in non-human primates^64,65 and reticulospinal connections to intrinsic hand muscles in healthy humans.⁶⁶ Though speculative and beyond the scope of this paper, motor tracts other than the corticospinal tract may assist in the elicitation of MEPs. Whether or not the presence of neural injury such as stroke exploits these tracts has yet to be explored. Likewise, future work may also discover differences in the reliability of various TMS measurements (e.g. MEP latency, amplitude, etc.) dependent on the primary tract of spinal motor fibers stimulated. These alternative motor tracts and their origins could serve as potential therapeutic targets, and establishing the reliability of MEP responses and other TMS measurements from these alternative motor tracts is an important area of study.

Testing parameters may have also impacted measurement reliability. We advise caution when generalizing our findings across different IHI testing scenarios encompassing lengthened inter-stimulus intervals probing long-interval IHI⁴⁶ and active motor conditions.^31,33 We determined participants’ RMT based on MEPs showing peak-to-peak amplitude of 50 μV in 3 of 5 trials. Though previous work showed no difference in motor threshold measurements using criteria of 3 of 6 trials vs. 5 of 10 trials,³⁹ current recommendations advocate criteria of 5 of 10 trials.^67,68 Determining motor hotspot and thresholds using a greater number of trials may have alleviated within-subject variability. Relatedly, our IHI testing protocol entailed only 10 trials of single- and paired-pulses from the ipsi- and contralesional hemispheres. We chose this number based on previous stroke work⁶⁹ and to avoid participant fatigue since additional TMS measurements were also collected. A greater number of trials during threshold determination and IHI testing would positively impact reliability. Lastly, other than instructing participants to avoid caffeine prior to testing, we did not employ additional strategies such as visual target fixation, and/or counting backwards during testing to control for within- and between-subject variability in attention and arousal states.⁷⁰

Improving IHI Index Measurement Reliability

The IHI index is a valuable TMS measurement that could be especially informative in stroke and other diseases. Despite our findings of low measurement reliability, several strategies may enhance IHI index measurement reliability. We previously discussed the implementation of neuronavigation and how potential variability in single- and paired-pulse MEP responses related to coil orientation may be avoided. Neuronavigation would also enable the investigator to deliver TMS pulses to the same location at every subsequent testing session. Additional research is necessary to ascertain if finding a new hotspot location at the beginning of each testing session influences TMS response and measurement variability to a greater degree than maintaining the original hotspot location. For example, despite reusing the original motor hotspot location for later TMS testing sessions, Chen and colleagues⁴⁰ encountered issues of within-subject variability with regards to differences in cortical silent period, intracortical inhibition, and intracortical facilitation measurement response to low-frequency rTMS. Results suggest that neuronavigation does not eliminate all potential sources of within-subject variability.

We also previously acknowledged that a greater number of trials during threshold determination and IHI testing would improve reliability. Because of our low number of trials completed during IHI testing, we retained all trials during offline analysis. Others have systematically analyzed their raw data for potential outliers by either discarding trials residing two standard deviations outside of the overall response mean⁷¹ or eliminating the top and bottom 20% of trials after ranking trials in ascending order.⁷² Normalization procedures may also strengthen measurement reliability. Previous studies have normalized TMS measurements to the maximal MEP amplitude recorded from the muscle tested⁷² or normalized responses to the pre-response mean if an intervention was delivered.⁹ Since baseline EMG activity affects the subsequent MEP response, a second investigator in our study visually monitored baseline EMG activity to ensure that participants remained at rest during testing. Visual monitoring of baseline EMG may have been insufficient; thus, contributing to variability in MEP responses during IHI testing. The incorporation of pre-trigger EMG activity measurements in MEP amplitude and area response measurements that comprise the IHI index may address this potential source of variability. Others have utilized similar strategies.^72,73 For example, Jayaram et al.⁷³ normalized MEP response area to pre-trigger EMG area during the construction of TMS recruitment curves. Stinear and Byblow⁷² calculated the root mean square value for pre-trigger level EMG to later examine differences in intracortical inhibition attributed to possible MEP modulation during pre-trigger EMG. Taken together, analyzing raw TMS data for outliers, implementing normalization procedures, and examining baseline EMG activity relative to MEP response are potential tactics to improve IHI index measurement reliability.

Conclusion

Repetitive TMS and other non-invasive brain stimulation approaches show promise as potential treatment strategies in several disease states. Capturing neurological correlates of behavioral improvement, such as changes in cortical excitability, using reliable TMS measures may strengthen the likelihood of implementation of these technologies to clinical practice. Low measurement reliability occurred from a combination of low between-subject variability in TMS responses during IHI testing and high within-subject variability from lenient inclusion criteria and experimental procedures. We identified several strategies to possibly improve measurement reliability. These strategies, along with preliminary standard error of measurement and MDD computations, should foster additional TMS IHI measurement reliability studies in clinical populations.