A novel model incorporating two variability sources for describing motor evoked potentials

Abstract

Objective

Motor evoked potentials (MEPs) play a pivotal role in transcranial magnetic stimulation (TMS), e.g., for determining the motor threshold and probing cortical excitability. Sampled across the range of stimulation strengths, MEPs outline an input–output (IO) curve, which is often used to characterize the corticospinal tract. More detailed understanding of the signal generation and variability of MEPs would provide insight into the underlying physiology and aid correct statistical treatment of MEP data.

Methods

A novel regression model is tested using measured IO data of twelve subjects. The model splits MEP variability into two independent contributions, acting on both sides of a strong sigmoidal nonlinearity that represents neural recruitment. Traditional sigmoidal regression with a single variability source after the nonlinearity is used for comparison.

Results

The distribution of MEP amplitudes varied across different stimulation strengths, violating statistical assumptions in traditional regression models. In contrast to the conventional regression model, the dual variability source model better described the IO characteristics including phenomena such as changing distribution spread and skewness along the IO curve.

Conclusions

MEP variability is best described by two sources that most likely separate variability in the initial excitation process from effects occurring later on. The new model enables more accurate and sensitive estimation of the IO curve characteristics, enhancing its power as a detection tool, and may apply to other brain stimulation modalities. Furthermore, it extracts new information from the IO data concerning the neural variability—information that has previously been treated as noise.

Introduction

One of the most important targets in brain stimulation—be it transcranial magnetic stimulation (TMS; (1)), transcranial electric stimulation (TES; (2,3)), or other stimulation modalities (4)—is the primary motor cortex. Activation of the upper motor neurons in that area results in easily detectable peripheral muscle twitches. These responses are detected using electromyography (EMG) as motor-evoked potentials (MEPs), and have the rare qualities in brain stimulation of being immediate and well quantifiable. Accordingly, they constitute a key effect for many brain stimulation techniques. Importantly, almost all TMS dosage decisions and safety guidelines are based on motor cortex stimulation (5,6).

TMS studies of motor cortex have tended to focus on certain stimulus strengths. One is the stimulation strength that evokes a median peak-to-peak EMG response of 50 μV, which is the most traditional measure used to establish TMS dosage (the resting motor threshold: MT) and which plays a role in almost every TMS intervention. Another involves the MEP response range of approximately 500–1000 μV induced by suprathreshold TMS, which is commonly used to probe changes caused by neuromodulatory interventions and anesthetics (7–10). However, these points are just a small subset of the total range of stimulus–response characteristics of MEPs elicited by varying the strength of TMS applied to motor cortex. The MEP amplitude (generally measured peak-to-peak) can be recorded across the full range from the lowest stimulator strength to the maximum stimulator output, and can be plotted as an input-output (IO) curve (also known as recruitment curve) (11). The IO curve represents the average increase in MEP amplitude, from a flat region of no detectable MEPs at low stimulation strength to an upper saturation level, beyond which a stronger stimulus can no longer increase the response. Special attention is commonly focused on the transition between the lower and upper IO plateaus which is typically quantified by its slope. The features of the curve are usually extracted by regression. Such a curve fit provides representative parameters and makes IO changes quantifiable.

The IO curve has served in many studies as a biomarker in analyzing the effect of interventions such as induced neuromodulation or medications (11–21). For example, preconditioning motor cortex with rTMS or transcranial direct current stimulation (tDCS) may affect the IO curve slope (18,22). Furthermore, different neuron populations contribute to the response at different points of the IO curve (23). Since simple probing at a single fixed stimulation strength cannot detect such complex changes, the IO curve as a whole may be a more sensitive tool for detecting neuromodulation.

While it is clear that the MEP response to varying TMS strengths is a useful measure, whether over the full device range as represented by the IO curve or over a small portion of that curve such as the motor threshold region, there is the major practical problem of pulse-to-pulse variability that affects generating and analyzing such data. Specifically, at any given TMS strength that generates MEPs, responses vary, often quite dramatically (24). This inter-pulse variation is on the same order as the response amplitude itself so that the scatter can cover the full range of response from microvolts to millivolts.

The source of this variation has been attributed to the natural fluctuations in excitability of cortical pyramidal cells and spinal motoneurons (25,26). In addition to microscopic physical mechanisms, such as thermal noise, deterministic neural sources may include, for instance, signals from remote sites as seen in motor facilitation (27,28). While some aspects of these fluctuations are known (11,14,29–39), such as their approximate strength and their highly skewed distribution (14,29), little is as yet known about their origin and the factors that influence them. For example, the focality of a TMS coil may influence MEP variability, as less focal circular coils were found to cause lower inter-stimulus response variability than figure-of-eight coils, although the reasons for this effect are unexplained (29). Furthermore, variability magnitude depends on the particular muscle being studied, a phenomenon that may be due to different levels of cortical inhibition (37). Other factors that are thought to influence MEP variability are muscle preactivation level (29), anesthetics (10), motor threshold, and age (14); however, a comprehensive theory of the mechanisms underlying MEP variability due to brain stimulation does not as yet exist. Further, knowledge about the corresponding statistical properties of the response distribution of MEPs is limited. Analyses of the responses at the MT suggest they may be modeled using a multiplicative variability (26,40), although others have assumed an additive variability (15,41) or argued for a more complex variability influence (29,42). We have previously used the assumption of multiplicative variability in IO analysis, since it provides a better fit for the data than additive variability (43); this approach, however, only partially improves the MEP distribution homogeneity along the IO curve.

Two reasons highlight the importance of a better characterization of MEP variability. First, from a practical perspective, users need exact information about the distribution of any kind of stimulation-evoked MEPs to perform a correct statistical analysis. The often skewed distributions of MEPs may violate normality assumptions in some statistical tests. In addition to such statistical issues with MEP data obtained at a single stimulation strength, the problem is intensified when analyzing values obtained from IO curves or comparing responses from different stimulation strength levels, where it is not clear how the distribution of responses varies over the range of stimulation strengths. This problem is usually undetected, due to the small number of samples typically obtained at each stimulation strength. The second reason is motivated epistemically. It need not be assumed that neural variability is just a source of noise in MEP measures, but instead it can be viewed as an intrinsic characteristic (44). That is, variations in MEPs may be due to the interaction of TMS with the state of the neural system being stimulated as known from brain circuits involved in cognitive processes (45). Accordingly, understanding the behavior of these variations and their sources may provide insight into neural information processing and how the brain reacts to stimulation from outside. Furthermore, the identification of multiple, distinct sources of variability could provide information about different parts of the motor system.

Previous analyses of TMS IO curves used regression in the form of a curve fit with a sigmoid function (11,43). This approach relies on the assumption that the distribution of the MEP scatter data (the MEP variability) has constant spread and shape at each stimulation strength (see model in Fig. 1 (a)) or that such homogeneous variability is obtained after a log-transform (see Fig. 1 (b)). However, we will show that this assumption is violated because the variability distribution shows characteristic properties such as spread and skewness that are dependent on the position along the curve. Traditional regression cannot describe this phenomenon and risks a violation of its statistical assumptions when used for parameter extraction. In this paper we derive a novel model that adds a second variability source at the input of the sigmoid nonlinearity, which, together with the traditional source at the output of the sigmoid function, can explain better the observed characteristics of the variability and which will be interpreted as being the result of neural excitability fluctuations. From a methodological perspective, the model can resolve the difficulties seen when using classical regression and act as a more sensitive means for extracting parameters from IO curves. In addition, it may provide a deeper insight into the mechanisms behind neural variability.

Figure 1.

Block diagram of input–output (IO) model structure. (a) Conventional model of IO curve assuming additive noise (single variability source model with additive noise, SVS-A). The stimulation strength x is transformed by a sigmoid S function which represents recruitment and synchronization of the motor pathways that are summed in the EMG. The sigmoid is followed by an exponential block reflecting the observed behavior of motor evoked potentials (MEPs). A variability source v_Vpp is assumed to add normal fluctuations to the output, yielding the peak-to-peak MEP amplitude V_pp. (b) Improved model adding the output noise source v_y before the exponential function block (single variability source model with multiplicative noise, SVS-M). This model more accurately reflects the multiplicative behavior of MEP noise. (c) Proposed IO model with two variability sources (dual variability source model, DVS). The model shares the output structure of the SVS-M model, but in addition the stimulation strength x is perturbed by an input-side variability source v_x before it is transformed by the sigmoid S.

Methods

IO Data Acquisition

In this study we reanalyze IO data that were part of a previous study (43). The IO data acquisition methods are summarized below; additional details can be found in (43).

Subjects

Twelve healthy subjects (7 female, 5 male, age range 19–49 years, 2 left-handed) were recruited and provided written informed consent for the study, which was approved by the NY State Psychiatric Institute IRB. All subjects were screened for psychiatric disorders, substance abuse/dependence, history of neurological disease, pregnancy, and seizure risk factors.

TMS Procedure

Subjects participated in a single TMS session, which included MT determination, IO curve measurement, and single-pulse stimulation of the dorsolateral prefrontal cortex, always administered in that order. Subjects were seated in a chair, with their heads supported by a head rest. Single pulse TMS was applied using a custom-built controllable pulse parameter TMS (cTMS) device that generated monophasic pulses with independent control of the pulse amplitude and pulse width (43, see 46 for more details). MTs were found at the scalp location over left motor cortex which produced MEPs of the greatest amplitude in the first dorsal interosseus muscle in the right hand (the optimum site), and were defined as the lowest pulse strength at that site that produced MEPs with amplitudes above 50 μV on at least 5 out of 10 trials (47). MEPs were recorded using a commercial EMG amplifier (BIOAMP 4, SA Instrumentation Co., San Diego, USA; bandwidth 30 Hz to 1 kHz, sampled at 4 kHz). The location of the optimum site was recorded using a Brainsight computerized frameless stereotaxic system (Rogue Research Inc., Montreal, Canada), and was used as the stimulation site for the IO procedure. The coil position relative to that site was found and monitored throughout the TMS session using Brainsight, and was adjusted if necessary to compensate for movements of the subject’s head relative to the coil. The coil was placed tangentially on the head approximately 45° to the median plane with the handle pointing backwards so that the initial pulse phase induced a posterior–anteriorly directed electric field.

IO curves were measured for each of three pulse widths (30, 60, and 120 μs). Responses from three to six TMS strengths were sampled below the MT, reaching the lower IO saturation region. Above the MT, MEP responses were sampled to the maximum amplitude allowable by the device output limit or by the subject’s discomfort. The transition region between the lower and upper IO plateaus was sampled with a step of 2% of the device maximum pulse amplitude (MA), while the saturation regions were sampled at 4–8% MA steps. Across all subjects and pulse width conditions, the number of distinct pulse strengths ranged from 8–24. We focused the data sampling predominantly in the IO transition and upper saturation regions since these regions contain the most physiologically relevant information whereas the lower extent of the curve is dominated by recording noise which can be estimated from the EMG recording baseline (see below). The MEP measurement was repeated 5 times for each pair of pulse width and amplitude. The order of the single stimulation strengths and pulse widths was pseudo-randomized across trials. The inter-stimulus interval ranged between 6 s and 8 s and was varied to counteract subject expectations.

While data from three pulse widths were recorded, the analyses presented in this study includes only responses for the 60 μs pulses, for the following reasons. The 60 μs data covered the widest range of stimulus amplitudes, included the most sample points in the saturation region of the IO curves, had fine spacing of the amplitude steps in the transition region between the plateaus of the IO curve, and matched most closely the duration of conventional sinusoidal monophasic pulses (46).

The MEP response to the TMS pulse was digitally detected, and defined as the difference between the maximum and minimum amplitudes in the window from 20 ms to 60 ms after the stimulus. The EMG recordings were monitored during the procedure for activity not related to the TMS pulse (motor facilitation). The operator rejected on-line samples that contained motor activation within 150 ms before the pulse. Thus, none of the EMG recordings contained any spikes or other activity higher than 50 μV (peak-to-peak) before the pulse. The average amplifier noise level was extracted from the recordings (7.3 μV) (43) and added as a regression sample at a negligible stimulation strength (0.001% MA) for stabilizing regression analyses.

IO Data Analysis

Outline

Previously, the distributions of MEPs at the various stimulation strengths and the shape of the IO curve have been analyzed separately (14,26,29,40,42). However, a more exacting modeling of IO curves should not rely on assumptions concerning sampling distributions but instead should test them, and our approach integrates analysis of distributions with modeling of the IO shape.

We began with a characterization of the statistical distribution and used the Box-Cox approach (48,49) to identify the variability distribution of the MEPs. In contrast to previous TMS studies (26,42), we applied this method separately at different stimulus strengths, i.e., at different points of the IO curve. This showed that the MEPs of the IO curve were not homoscedastic and that the distribution changed along the curve. Both were violations of necessary assumptions of traditional IO-curve regression methods: regression with a sigmoid function either in the linear or the logarithmic domain assume homoscedastic distributions with normal and lognormal residuals, respectively (see Fig. 1 (a) and (b)).

For modeling the shape of the IO curve, we used the hypothesis of excitability fluctuations (44) to set up a model that incorporates a second variability source (Fig. 1 (c)). This model was designed such that it could describe both the shape and the distribution of the IO scatter for the first time. Finally, we demonstrated with Akaike’s criterion and a cross validation that the novel model was more appropriate for modeling the IO curve data compared to the traditional linear and logarithmic regression methods as a reference.

Distribution Test of Raw Data

In order to study the distribution of the pulse-to-pulse variability at different locations of the IO curve, we performed a Brown-Forsythe heteroscedasticity and a Box-Cox power distribution test. We included six different stimulation strength levels relative to the subject’s MT (80%, 90%, 100%, 110%, 120%, and 150%) into the Brown-Forsyth procedure with subject and stimulation strength level as factors to test for an amplitude-dependent variability (50). The Box-Cox method selects the best fitting candidate from a predefined set of normal power distributions (exponentially transformed Gaussian density functions) which includes the normal and the lognormal distribution. In contrast to previous uses of this method in TMS (26,42), the current analysis applied the test at the given six different stimulation strength levels (80%, 90%, 100%, 110%, 120%, and 150% MT) and determined the corresponding response distribution with a maximum likelihood estimator for exponents in the interval [−2.5, 2.5] with an accuracy of 10⁻³. Since the IO data were acquired in discrete stimulus steps, we pooled MEPs evoked by pulse amplitudes within a ±1.5% interval around each of the specified stimulation strength levels relative to MT.

Formulation of New Response Model DVS

In our new model (dual variability source, additive; DVS), we split the excitation process into three basic elements as shown in Fig. 1 (c). A sigmoidal transformation function S models the empirically well-known neural recruitment characteristics with a low-side plateau formed by the recording noise floor, a monotonically growing transition section reflecting the increasing population of co-activated or co-recruited neurons, and an upper saturation level corresponding to the maximum achievable activation of motor units for the specific conditions.

In contrast to the models in Fig. 1 (a, b), the model in Fig. 1 (c) adds a source of stochastic variability, v_x, to the stimulus before the sigmoid nonlinearity. The v_x variability source acts along the x-axis of the IO curve, according to the following rationale. Some of the variability in responses to brain stimulation can be attributed to the short-term excitability fluctuations of the neurons directly activated by the electric field pulse. These excitability fluctuations will appear as modulations of the stimulation strength, i.e., the x-axis of the IO curve. For instance, if some or all of the directly activated neurons become temporarily more excitable, this excitability change will cause an interim shift of the IO curve along the x-axis so that weaker stimuli have the same effect as stronger stimuli during the prior lower excitability state of the neurons. Further, perturbations of the effective stimulation strength may be caused by variation of the coil position and the angular orientation, which have notable influence on the induced electric field at the target. Perturbations both in the distance between the TMS coil and the target neurons as well as in the excitability of these neurons modulate exclusively the effective strength of the stimulus, and therefore affect primarily the input of the motor system. All of these influences contribute to the variability at the system input that is largely unpredictable for an external observer. An invariant stochastic signal source assumed in conventional IO-curve regression cannot properly describe such input variability since both the deterministic stimulus and the stochastic signal at the input are refracted through the sigmoid recruitment nonlinearity, resulting in output variability that is strongly affected by the position on the IO curve.

In our model in Fig. 1 (c), a second source of variability, v_y, acts on the output of the sigmoidal nonlinearity. In conventional IO-curve regression, this is the only assumed site of stochastic fluctuations. In contrast to most analyses in the literature (see for instance (11,14,15) and Fig. 1 (a)), the output-side variability source v_y affects the responses in the logarithmic domain before the final exponentiation and accordingly has a multiplicative character as shown in the models in Fig. 1 (b) and (c) (51,52). This stochastic component may represent fluctuations in the spinal pathways, noise effects in the two synapses that motor signals have to pass on the way from the brain to a muscle, and especially short-term fluctuations of metabolic and signaling processes in the muscle cells (53). These processes occur after the main nonlinearity, which is represented by the sigmoid, because the interaction of the pulse with the neural system has already evoked a signal which is thenceforth likely just relayed and transmitted down the corticospinal tract to the muscle. The fact that these distinct motor pathways are separate and have not reached the final summation that forms the EMG signal does not matter. The key determinant of whether a fluctuation is represented by the output-side variability, v_y, is if this effect is independent of the stimulus amplitude. Fluctuations that are independent of the stimulus amplitude are represented by v_y. For instance, if a stronger stimulus cannot overcome or otherwise influence a synaptic insensitivity or the concentration of the muscular calcium release on a cellular level, then variability in these effects should be represented by v_y. However, this independence of an effect on the stimulation strength does not exclude any influence of other stimulus parameters, such as the pulse waveform and site of stimulation.

Taking into account the stochastic variability on the input-side, v_x,i, and output-side, v_y,i, in the model in Fig. 1 (c), the peak-to-peak MEP amplitude evoked by a stimulus i with strength x_i is

$$V_{\text{pp},i}=exp(y_i·ln(b))=exp(S(x_i+v_{x,i})+v_{y,i}),$$ (1)

where the base of the exponential function, b, can be chosen freely. We use b = 10 for easier interpretability of the parameters.

Estimation of Model Parameters

Calibration of the DVS model in Fig. 1 (c) requires only the measured stimulus–response pairs (x_i, y_i). No other data, such as additional information on the variability or how it is allocated to the two stochastic sources, is needed. The parameters to be estimated comprise two different types: those describing the shape of the sigmoid function S and those that determine the two independent sources of variability, v_x and v_y. In general, regression problems with multiple variability sources are not uniquely solvable because their entanglement in the measurement commonly renders it impossible to split the total variability into distinct contributions (54). However, the pronounced change of the slope along the IO curve as a consequence of the sigmoid nonlinearity enables separation of the contributions to the two stochastic sources. Specifically, in the regions of the IO curve with slope close to zero (the lower and upper saturation plateaus) the input-side variability does not affect the output, whereas it is strongly amplified in the transition region of the IO curve which has a steep slope.

The process leading from a stimulus with strength x_i to an MEP response with a peak-to-peak amplitude V_pp,i = exp(y_i ln b) is described as a forward model. This means that the model assigns to every x_i an outcome that is represented, in this case, by a conditional probability distribution f_y|x(y_i) of the log response y_i. With this distribution, the model yields a probability for every possible y_i value in dependence of the stimulation strength x_i and the specific properties of the sigmoid S as well as the two stochastic variability sources v_x and v_y. Accordingly, the probability distribution provides a quantitative measure of how likely a certain measured stimulus–response pair (x_i, y_i) would be for the chosen model properties.

Mathematically, the stimulus strength x is perturbed by the input variability v_x with a probability distribution g_vx. This leads to a probability distribution of the effective activation x̃,

$$f_{\stackrel{\sim }{x}}(\stackrel{\sim }{x})=g_{v_x}(\stackrel{\sim }{x}-x).$$ (2)

The distribution f_x̃(x̃) is in turn input of the sigmoid function S and is transformed into a corresponding distribution f_ỹ|x(ỹ) = f_ỹ|x(S(x̃)) after the sigmoid,

$$\mid f_{\stackrel{\sim }{y}\mid x}(S(\stackrel{\sim }{x}))\mathrm{d}S(\stackrel{\sim }{x})\mid =\mid g_{v_x}(\stackrel{\sim }{x}-x)\mathrm{d}\stackrel{\sim }{x}\mid .$$ (3)

The sigmoid output then interacts with the second variability source v_y, giving the full distribution of the output,

$$\begin{array}{l}\mid f_{y\mid x}(y)\mathrm{d}S(\stackrel{\sim }{x})\mid =\left|{\int }_{ℝ}{g}_{v_y}(\chi )·f_{\stackrel{\sim }{y}\mid x}(y-\chi \mid x)\mathrm{d}S\mathrm{d}\chi \right|\\ ={\int }_{ℝ}{g}_{v_y}(\chi )·g_{v_x}(S^{-1}(y-\chi )-x)\mid \mathrm{d}\stackrel{\sim }{x}\mid \mathrm{d}\chi \\ \mid f_{y\mid x}(y)\mid ={\int }_{ℝ}{g}_{v_y}(\chi )·g_{v_x}(S^{-1}(y-\chi )-x){\left|{\frac{\mathrm{d}S}{\mathrm{d}\stackrel{\sim }{x}}|}_{S^{-1}(y-\chi )}\right|}^{-1}\mathrm{d}\chi .\end{array}$$ (4)

We use closed parametric functions for the sigmoid S as well as the two variability sources v_x and v_y, and denote the combined parameter set with p. The distribution can then be incorporated into a Fisher maximum-likelihood estimator with likelihood L (55). The most likely parameter set of the model p_ml—and accordingly the most likely model within this framework—for a measured IO curve (x_i, y_i) is consequently estimated using

$${\mathbf{p}}_{\text{ml}}=arg{max}_{\mathbf{p}}L=arg{max}_{\mathbf{p}}{\prod }_i{f}_{y\mid x}(y_i\mid x_i,\mathbf{p})$$ (5)

under the assumption that the inter-pulse correlation is negligible.

For this analysis, the sigmoid S is implemented as a Hill-type function with five parameters according to

$$\stackrel{\sim }{y}(\stackrel{\sim }{x})={log}_{10}\left(\frac{{\stackrel{\sim }{V}}_{\text{pp}}}{1\mathrm{V}}\right)=p_1+\frac{p_2-p_1}{1+{\left(\frac{p_3}{\stackrel{\sim }{x}-p_5}\right)}^{p_4}}$$ (6)

where Ṽ_pp is the auxiliary variability-free peak-to-peak response, ỹ is the logarithmic response, and p₁, p₂, p₃, p₄, p₅ are shape parameters. Compared to other functions in the literature, e.g. Gaussian and Boltzmann sigmoids, the Hill-type function has the advantage that it is more flexible. However, such alternative sigmoid shapes can be used with the proposed approach as well.

Both variability sources v_x and v_y are implemented as independent stochastic elements with Gaussian distribution and respective spreads σ_x and σ_y. Accordingly, the total parameter vector of this implementation is p = (p₁, p₂, p₃, p₄, p₅, σ_y, σ_x). The likelihood maximization is achieved with the Nelder-Mead algorithm (56).

Standard Regression for Comparison

In the literature, IO curves are usually fitted in the linear domain with an additive error at the output side only (single variability source, additive; SVS-A), as shown in Fig. 1 (a) (11,12,14). For comparison, we applied such regression, although it neglects the multiplicative behavior of MEP variability (40). We used standard least-squares regression, which is commonly used in the literature and, because of its weaker assumptions, can be more robust compared to a likelihood estimator in case of an unknown, incorrectly estimated or assumed, or heteroscedastic error distribution, which did prove to be present in our IO data set.

Whereas the described standard regression is applied directly to the raw data with additive error, we furthermore implemented a regression with homogeneous multiplicative variability of V_pp (single variability source, multiplicative; SVS-M), as described in the model in Fig. 1 (b). This corresponds to an additive variability of the logarithmic peak-to-peak response y. This model takes the exponential characteristics of the IO curve variability into account and is equivalent to our proposed dual variability source model DVS for the case when the input variability approaches zero, v_x → 0. The curve fit was performed using maximum-likelihood regression.

Validation

To test the significance of the likelihood difference between the regression models, we performed a likelihood test ratio based on Wilk’s theorem (57,58). In addition, we carried out cross validation and statistical bootstrap analyses. The leave-one-out cross validation analysis fits the models separately to all possible different sample subsets of every curve that omit one sample (59,60). The likelihood of the missing samples, which did not contribute to the model fit, is used to estimate the predictive performance of the models. As a nonparametric estimator for the Kullback-Leibler information up to a constant (61,62), it supports the Akaike information criterion, which was designed as a parametric Kullback-Leibler estimator (63), to conclude which model has the highest performance in predicting further responses of a specific subject based on a training subset of samples. While the regression results from the resampled and reduced-by-one-sample datasets in the cross-validation could in principle serve as a so-called jackknife for deriving variances of the likelihood or parameters (64), we preferred a separate bootstrap with 600 repetitions to extract the entire distribution data for stability testing and nonparametric error estimates (65). Significance based on the bootstrap results was evaluated with Mann-Whitney’s U test.

Results

Individual MEP Distribution

Fig. 2 (a) shows the IO data for all of the subjects, normalized to their individual MT, displayed with both logarithmic and linear (inset) y-axis. A comparison of the individual IO scatters in Fig. 2 (a) reveals an important difference between the logarithmic and linear plots. In the linear plot the trends of the scatters for each subject have significantly different slopes. Because of the normalization of the stimulation strength, the different slopes cannot be explained by a difference in the individual MT. In the logarithmic plot, on the other hand, the slopes are similar and the individual data points lie within a more confined cluster. This phenomenon arises in the case of locally logistic or exponential growth when individual saturation levels are different, but the curves have the same growth rate.

Figure 2.

Combined input-output scatters of all subjects and corresponding response distribution. The right side shows the input-output scatters after normalization to each subject’s individual resting motor threshold (MT) in a semi-logarithmic plot for harmonizing the variability spread. Every symbol denotes a stimulus–response pair; the different subjects are represented by an individual symbol. The small inset presents the same data with linear y-axis. Blue vertical lines indicate at which stimulation levels the scatter was cut in order to generate the histograms on the left.

Fig. 2 (b) depicts histograms of the normalized MEP amplitudes across subjects at different stimulation strengths in ascending order (the selected stimulation strengths are indicated by blue stripes in Fig. 2 (a)). The spread of the distribution varies along the IO curve, from narrow spreads for weak stimulation in the noise-dominated IO-curve lower plateau, to responses around the threshold that cover the whole dynamic range for higher stimulation strengths, to a return to decreased variability in the upper saturation range. Indeed, the Brown-Forsythe test shows significant heteroscedasticity of the MEP amplitudes in response to the six stimulation levels for both assumed multiplicative variability (F(5, 354) = 10.38, p < 0.001) and assumed additive variability (F(5, 354) = 11.36, p < 0.001). Accordingly, a log-normalization cannot compensate the differences of the variability spread along the curve.

In addition to the spread, the skewness of the distribution of the MEP amplitudes in Fig. 2(b) changes along the IO curve as well. Specifically, the exponent of the most likely Box-Cox power distribution (49), λ, at the lowest two stimulation strengths is negative, indicating a right-tailed distribution after a log-transform, with λ = −1.51 ± 0.65 for a stimulation strength of 80% MT and λ = −1.01 ± 0.41 for 90% MT (the given error range denotes the 0.95 confidence interval). Near threshold (100% MT), the exponent approaches zero (λ = −0.13 ± 0.17), a value that would represent a lognormal distribution in the Box-Cox framework, and changes sign to positive values at higher stimulation levels (λ = 0.11 ± 0.14 at 110% MT, λ = 0.10 ± 0.22 at 120% MT, and λ = 0.18 ± 0.50 at 150% MT), reflecting a shift of the trend from a right-tailed toward a left-tailed distribution in Fig. 2 (b).

Model Regression Results

Dual Variability Source Regression

In contrast to the distribution tests above, the proposed dual variability source (DVS) regression analyzes every IO curve separately. For every set of stimulus–response pairs, (x_i, V_pp,i), the regression estimates a corresponding sigmoid curve and the expected variability distribution. Fig. 3 shows several representative examples of individual IO data and curve fits from the conventional regression assuming additive or multiplicative variability as well as from the novel dual variability source regression; the data for all subjects is available as Supplemental Material. In addition to the regression curve fits, Fig. 3 indicates the variability range of the stimulus–response pairs taken from the estimated distributions in the DVS model. The estimated variability is represented with a band within which the stimulus–response pairs are expected with 95% confidence; the measured stimulus–response pairs generally fall within this band. Within the two IO curve plateaus, the lower one at small stimulation strengths and the upper one at high stimulation strengths, the width of the variability band is almost exclusively determined by the output-side variability source, which acts along the y-axis. In the transition region between the lower and upper IO plateaus, the curve has a steep slope and therefore the influence of the input-side variability source, which acts along the x-axis, increases. Specifically, due to the steep slope in the transition region, the input-side variability source produces wide spreading along the y-axis that exceeds the influence of the output-side variability source and spans two orders of MEP magnitude. Thus, the depicted variability band not only suggests that an x-variability contribution, represented by the input-side variability source, may be required for an appropriate description of the IO data, but also indicates that in the IO transition region the input-side variability, v_x, contributes more to the total variability than the output-side source, v_y.

Figure 3.

Representative individual IO data and regression results. The x-axis is the TMS pulse amplitude representing the stimulation strength and expressed as percentage of the maximum amplitude (MA) for the device. The y-axis is the peak-to-peak amplitude of the MEPs. The plots show the regression results for five representative subjects including all regression methods (red: DVS; green: SVS-M; blue: SVS-A; see Fig. 1 for definition of the models). The measured MEP amplitudes are marked with ‘×’. The gray band surrounded with dotted lines represent the 0.95 variability ranges of the responses as predicted by DVS. The 0.95 variability bounds are extracted from the corresponding estimated local response distributions f_y|x. The insets show the corresponding data on a linear scale. The numbers prefixed with an “S” are the study subject identifiers. See the Supplemental Material for data from all subjects.

The quantitative variability and slope estimates are summarized in Table 1. Table 1 indicates that the x-variability is about 10% of the individual MT. Notably, σ_x did not vanish during the regression iterations as would be likely in case the assumption of a second variability source is superfluous. The σ_y value corresponds to a multiplicative y-variability of about ×/÷ 1.33, which is much lower than the apparent MEP variability in the transition region between the lower and upper IO plateaus, where the response amplitudes range from a few microvolts to a millivolt, representing more than two orders of magnitude.

Table 1.

Comparison of the estimated IO curve variability, slope, and goodness-of-fit for the dual variability source model and the standard regression with multiplicative variability. The spread, σ, and slope, S, values are given as mean ± standard deviation. For the y-variability source, the multiplicative impact on the MEP amplitude, exp(v_y), is reported. For the x-variability source, the spread is given both relative to the maximum pulse amplitude of the device (MA) and relative to the individual motor threshold (MT). The estimated slopes are significantly different (p < 0.017). Better model fit is indicated by higher log likelihood and smaller Akaike information. The standard deviations of both are estimated with a bootstrap analysis. The so-called quasi-log-likelihood of the leave-one-out cross validation provides a measure for the predictive performance of the models.

	Standard regression with multiplicative variability SVS-M	Dual variability source model DVS
y-variability source
• Spread σy (dB)	6.84 ± 1.04	2.46 ± 0.86
• Multiplicative impact on Vpp	×/÷ 2.2	×/÷ 1.3
x-variability source
• Spread σx (% MA)	n/a	5.0 ± 2.7
• Spread σx (% MT)	n/a	9.5 ± 3.9
Peak slope of sigmoids S (dB/%MA)	3.7 ± 1.9	4.5 ± 1.9
Model log likelihood ± bootstrap standard deviation	−594 ± 49	102 ± 35
Model Akaike information ± bootstrap standard deviation	1200 ± 97	−191 ± 71
Leave-one-out cross validation predictive quasi-log-likelihood ± standard error	−523 ± 55	−40 ± 37

The DVS model predicts the entire statistical distribution of the responses. Fig. 4 compares the model results and the measured response distributions for three subjects (see Supplemental Material for the data for all subjects). The plots show the intra-subject spread of the responses at fixed stimulation strength.

Figure 4.

Comparison of individual MEP amplitude distributions with regression predictions. The single plots depict the response histograms (black bars) and the predicted response distribution function f_y|x from DVS (red line). The curves are plotted for five stimulation strengths relative to an individual reference point, RP, corresponding to 200 μV on the regression curve. The model explains phenomena such as: widening of the distribution spread in the transition region of the IO curve compared to the lower and upper plateaus; skewness change with increasing amplitude from left-sided for low stimulation strength to right-sided for high stimulation strength; and bimodal distributions that can occur under certain circumstances (see, for instance, S136).

Comparison with Standard Regression

For comparison, the results of the standard regression models with assumed either additive (SVS-A, Fig. 1 (a)) or multiplicative (SVS-M, Fig. 1 (b)) y-variability are plotted in Fig. 3. Clearly, SVS-M regression fits the data better than SVS-A, which assumes additive output variability. For example, the correlation of the peak slope estimates of the SVS-A model and the DVS model is low (r = 0.2, p < 0.39), and the standard deviation of the former model (7.7 dB/%MA) is on the same order as the mean value of the slope (6.5 dB/%MA) indicating instability in the estimation. Therefore, the estimated variability and slope only for the SVS-M regression is summarized in Table 1 for comparison with the dual-source model. The SVS-M model assesses an overall y-variability of about ×/÷ 2.2. This factor is higher than the output-side variability of ×/÷ 1.33 estimated with the DVS model. Table 1 further shows that the slope of the estimated sigmoid function S before exponentiation, which represents the exponential growth of the IO curve, is lower for the SVS-M model (maximum exponential growth factor of (3.7 ± 1.9) dB/%MA) than for the DVS model ((4.5 ± 1.9) dB/%MA). This difference of approximately 20% is statistically significant (t(11) = −2.4, p < 0.017) and reflects a systematic bias in the slope estimation between the two methods.

In addition, Table 1 compares the goodness-of-fit for the standard regression with the SVS-M model to the DVS model using log-likelihood and Akaike information scores. The regression of the IO data with the DVS model leads to a significantly higher total log-likelihood than the standard regression (103 versus −594; likelihood ratio test: χ²(1) = 1392, p < 0.001), suggesting a higher descriptive quality. This comparison is appropriate because the likelihoods were evaluated equivalently with the same logarithmic reference and on the same data. The Akaike information criterion, which compensates the potential advantage of the new model due to its higher number of parameters and may therefore act as a fairer metric for assessment (63), leads to a score of −191 for the DVS model which outperforms the score of 1200 for SVS-M regression.

The Mann-Whitney U test of the bootstrap results shows that both the log-likelihoods (U(600) = 540300, p < 0.001) and the Akaike information scores (U(600) = 180300, p < 0.001) are significantly different between the models, and that the result is not a sampling artifact. The cross-validation results attribute a higher predictive performance to the DVS model. The predictive quasi-log-likelihood difference from the cross-validation analysis of 483 with a standard error of 66.5 furthermore shows a higher performance of the DVS model to predict a further sample that was not included in the training of the model.

Discussion

While IO curves could be useful biomarkers for measuring motor system properties as well as the effects of neuromodulation due to brain stimulation, conventional approaches to characterize them have limitations. In this analysis, we argued that basic variability assumptions used in standard regression models are violated when applied to MEP measures across the full range of TMS device stimulation strengths. We introduced a model incorporating a second source of variability, DVS, and demonstrated that its predictions provided superior fits to IO curve data compared to the standard models and could describe varying MEP distributions across the range of stimulus strengths.

Beyond providing a better estimate of IO curve parameters, the new model may be useful in understanding the mechanisms of the response to brain stimulation. The input variability source identified in the DVS model likely reflects both random and nonrandom physiological excitability fluctuations that interact with the stimulation strength. For instance, random noise at the membrane of the stimulated neurons (66–72) likely contributes to the variability at the model’s input side. The brain state can influence excitability in a more deterministic way, as exemplified by the facilitation of TMS MEPs associated with observation of another person’s movements (27, 28). On the other hand, effects that account for the variability at the output side of the IO model cannot be influenced or overcome by increasing the stimulation strength. Examples of such effects may include neural noise that interacts with the amplitude and the timing of a signal that is already propagating through a motor pathway, muscle conditions such as metabolic fatigue (53,73), as well as electrode impedance changes (74,75). These are not, to a first approximation, influenced by the stimulation strength and are therefore subsumed in the output-side variability. Finally, as in any measurement, recording noise (75,76) is added to the output variability, and may dominate at the low end of the IO curve. Instead of estimating the measurement noise from the IO data, it may be more practical to extract it from the baseline of the EMG signal (43).

It should also be noted that the DVS modeling approach could be applicable to other forms of neurostimulation where the transfer characteristic of the system has a strong nonlinearity preceded and followed by multiple variability sources. For example, potential applications include TMS-evoked electroencephalography potentials (TEPs), deep brain stimulation (DBS), and neuroprosthetics and brain-computer interfaces (BCI).

Performance of Dual Variability Source Model

The Brown-Forsythe heteroscedasticity test and the Box-Cox analysis provided a quantitative measure to conclude that the response distribution was most likely not constant along the IO curve (although Box-Cox analysis has limitations when applied to MEP distributions: see below). This change of the distribution affected not only the variance but the skewness as well. In consequence, a variability analysis at a single stimulation strength may not be informative and may be subject-specific. On the other hand, describing the distribution at various stimulation strength levels is also unsatisfactory without a systematic explanation.

IO curve regression provides unified models that can describe the full signal chain from the stimulus strength in the device to the MEP response amplitude. With a likelihood regression method, such models furthermore provide a statistical description of the response distribution as a function of the stimulation strength, f(V_pp|x), and enable a rigorous assessment of the model performance. The distributions derived with the novel dual variability source model, DVS, and shown in Fig. 4 reflect the basic skewness pattern. At low stimulation strength, the majority of responses are relatively small, although the tail indicates a certain rate of higher responses. At high strength, in contrast, the shape of the distribution curves is also skewed, but in the opposite direction, with an emphasis on high responses. In between, the distribution shape passes several intermediate forms having a wider spread than near the plateaus. In addition, the model explains some unusual distributions. When the DVS model indicates a strong contribution of variability on the input side that is dominating the output side, these intermediate distributions become bimodal (see S136 in Fig. 4) or, in less pronounced cases, feature a lower plateau or a bump in the tail on one side of the maximum before the curve decays to zero (see, e.g., in curve S123 in Fig. 4 or curves S127, S128, S134, and S137 in the Supplemental Material). The two maxima in the MEP distribution in the transition range are caused by the distinct sharp saturation levels on both sides of the IO curve which the input-side variability blurs along the x-axis and smears into the transition range. This behavior is not an artifact in the estimated distribution, as it is substantiated by the data which follow the estimated shape well. Accordingly, the graphs in Fig. 4 indicate that the estimated distributions do not simply play the role of a qualitative illustration; the agreement between the DVS model (red curves) and the measurements (black bars) reveals good predictive power, which is supported by the cross validation results.

The feature of multiple inflexion points and/or modes explains why Box-Cox power distributions, which can only contain one maximum, may not be an appropriate tool for describing the variability of IO curves. Although Box-Cox distribution estimation always suggests a potential fit, the underlying function class does not include and therefore cannot describe some of the observed distributions.

Comparison with Logarithmic Regression (SVS-M)

As shown in Table 1, the likelihood, the Akaike information criterion, and the prediction performance evaluated by cross validation favor the proposed DVS model over the standard single variability source models for an accurate description of the IO characteristics. However, a reasonable question in this context is if the simpler SVS-M regression model, although it may not represent the physical and physiological conditions well, could still be a reasonably accurate method for extracting parameters from IO curves. In general, the logarithmic sigmoids from the DVS and SVS-M models appear to be close to each other, as shown in Fig. 2. For example, the estimated saturation levels of both are almost identical, although this was expected, as MEP amplitudes deep into the upper saturation levels are not perturbed by the input-side variability. The level of the low-side plateau, in contrast, is systematically overestimated by the SVS-M model as illustrated by several cases in Fig. 3. This overestimation may be explained by the relatively sparse sampling at the low end of the curve: in that case, the ‘outliers’ of the skewed distribution drag the low-side plateau to higher values, whereas the DVS model anticipates the skewness correctly since it is included in the distributions. Such difficulty with the simpler regression could potentially be solved by a more extensive data collection at low amplitudes, although this is rarely done because of time limitations in TMS sessions.

However, an issue that cannot be compensated by appropriate sampling techniques is the discrepancy of the IO curve slope between the different models. The SVS-M model leads to a lower slope estimate than the DVS model. This difference can be observed in most of the standard regression curves in Fig. 3 and in the Supplemental Material, and is supported by the values in Table 1. The reason for the underestimation of the slope in the SVS-M regression is the changing skewness of the y-axis distribution of MEPs along the curve. The skewed MEP sample distributions include data points that lie far from the accumulation range. These data points are caused by the asymmetric and changing distribution, and resemble outliers, although they are not ‘outliers’ in the correct sense, but are a part of the signal with its intrinsic variability distribution. In the transition range of the sigmoid near the lower plateau, these ‘outliers’ are higher than the accumulation point and pull the curve up. On the other hand, near the upper plateau they are lower than the accumulation point and pull the curve down. Thus, the slope of the regression curve tends to be less steep in the SVS-M model than in the DVS model.

First, this dragging effect reduces the slope and ends in a systematic regression error. Second, the dependence on ‘outliers’ brings forth an instability of the estimated parameters such as the confidence interval. The random appearance of a far-lying sample point influences the estimated curve, due to the high regression weight in least-squares fitting or an underestimated likelihood in distribution estimators, more than a larger group of ‘appropriate’ data at the accumulation point.¹ Accordingly, this randomness is inherited by the respective estimated parameters. At the conventional motor threshold response level of 50 μV, for example, the difference along the x-axis between the advanced DVS model and the SVS-M regression curve reached up to 8% relative to their average. For lower response levels, this error is higher. Robust regression in combination with a sufficient number of samples may alleviate the problem, but this happens at the price of neglecting a portion of the data, which will be incorrectly classified as ‘outliers’ by the algorithms.

Comparison with Linear Regression, SVS-A

Regression with additive variability in y-direction, SVS-A, which is the dominant method of analyzing brain-stimulation IO curves in the literature, exhibits several major differences with the DVS model. The level of the low-side plateau is irrelevant in this regression approach and can reach almost any value close to zero without reasonably affecting the fitting metric, i.e., the sum of the squared residuals. The reason is that this level is far below the assumed additive variability, which is large in the linear, additive domain, especially in the saturation region as indicated in the IO plots in Figures 2 and 3 as well as in the Supplemental Material.

Whereas the exact value of the lower plateau is usually only of limited significance, other IO curve characteristics are not well described by standard regression with assumed additive variability either. In general, the slope of the regression curve deviates from the other two models, as demonstrated by the low correlation between the peak slopes of r = 0.2. Like the results from the SVS-M model, the slope is systematically misestimated with the SVS-A model. However, in contrast to the former, the deviation of the assumed Gaussian distribution from the actual one, especially regarding the skewness of the local variability distribution, is larger; consequently, the outlying tails of the distributions have a stronger influence on the slope. The only region in which SVS-A regression may provide appropriate values, agreeing well with the two alternative regressions, is the upper saturation level.

Conclusion Regarding Regression Modeling

In the comparison of regression models presented here, two particularly important results emerged. First, the high variability of MEP responses for fixed stimulation strength can be described by two variability sources with relatively small magnitudes. Specifically, the high spread in the transition region may result from two narrow distributions for the v_x- and v_y-variability source, coupled with the amplification effect the steep section of the sigmoid nonlinearity has on v_x. Second, the DVS model can use the same fixed distributions for the two sources along the entire curve. Despite the constant source distributions, the model describes the complex overall MEP distribution pattern, including phenomena such as the spread and skewness change as well as bimodality. In traditional models and explanations of MEP generation, in contrast, the variability has to be different at every location and therefore no longer explicable with a unified description.

Implications for TMS Methodology

In everyday TMS applications, the non-Gaussian distribution of MEP responses demonstrated in the present study shows that statistical analysis of any recordings has to be performed carefully. The data may have a Gaussian distribution only in few atypical cases. In addition, tests for normality are often not reliably doable because of the greater sampling needed. Several methods should be used with special caution, such as averaging of non-normally distributed data, parametric distribution tests that are limited to a small specific class of density functions without any goodness-of-fit test or independent validation, or least-squares regression of IO curves in the linear domain, i.e. with assumed additive error.

The study has important implications for IO curve regression. First, the DVS approach introduced here is more accurate than other existing methods in extracting IO parameters. Specifically, it reduces the vulnerability of the IO parameters to stochastic fluctuations caused by seeming outliers. For instance, the saturation level and the slope tend to be systematically misestimated with simpler models. Second, the dual variability source model, DVS, provides valuable additional information. An assignment of observed fluctuations to one of the two variability sources could provide insights into how a certain factor affects neuronal networks and answer questions that have been reserved for invasive techniques. For example, partitioning the effects of a pharmacologic intervention (10,36) to the two sources of variability in the model may shed light on neurophysiologic signaling-related mechanisms of neural variability and on biophysical noise sources as well as on localization of both in the signaling chain (33). As another example, study of the variability ‘fingerprint’ of different types of stimulation (such as paired pulses (30) or novel pulse shapes (77,78)) may provide important information about their different stimulation performance and potential activation selectivity (79,80). As a consequence, IO characterization could become a more sensitive tool in brain stimulation.