Three-Year Reliability of MEG Visual and Somatosensory Responses

Abstract

A major goal of many translational neuroimaging studies is the identification of biomarkers of disease. However, a prerequisite for any such biomarker is robust reliability, which for magnetoencephalography (MEG) and many other imaging modalities has not been established. In this study, we examined the reliability of visual (Experiment 1) and somatosensory gating (Experiment 2) responses in 19 healthy adults who repeated these experiments for three visits spaced 18 months apart. Visual oscillatory and somatosensory oscillatory and evoked responses were imaged, and intraclass correlation coefficients (ICC) were computed to examine the long-term reliability of these responses. In Experiment 1, ICCs showed good reliability for visual theta and alpha responses in occipital cortices, but poor reliability for gamma responses. In Experiment 2, the time series of somatosensory gamma and evoked responses in the contralateral somatosensory cortex showed good reliability. Finally, analyses of spontaneous baseline activity indicated excellent reliability for occipital alpha, moderate reliability for occipital theta, and poor reliability for visual/somatosensory gamma activity. Overall, MEG responses to visual and somatosensory stimuli show a high degree of reliability across 3 years and therefore may be stable indicators of sensory processing long term and thereby of potential interest as biomarkers of disease.

Introduction

Neural oscillatory activity is thought to underlie many cognitive and behavioral processes (Başar et al. 2001; Wang et al. 2011; Buzsáki and Watson 2012; Klimesch 2012). Magnetoencephalography (MEG) allows for the measurement of such activity due to its high temporal resolution, and source analyses of MEG data can provide good spatial precision for estimating the location of neural oscillatory responses. Aberrant oscillatory responses have been examined in a wide array of psychiatric and neurologic disorders, including schizophrenia and early-onset psychosis (Wilson et al. 2008; Uhlhaas and Singer 2010), epilepsy (Zijlmans et al. 2012), autism (Wilson et al. 2007; Simon and Wallace 2016), Alzheimer’s disease (Uhlhaas and Singer 2006), and Parkinson’s disease (Pollok et al. 2012; Heinrichs-Graham et al. 2014). Additionally, MEG and EEG studies have reported alterations in resting-state activity in developmental disorders such as autism (Cornew et al. 2012), schizophrenia (Fehr et al. 2001), and attention-deficit/hyperactivity disorder (Wilson et al. 2013), as well as neurodegenerative diseases, such as Alzheimer’s disease (Fernández et al. 2006) and mild cognitive impairment (Garcés et al. 2013). Many of these studies have discussed using neural oscillations as clinical biomarkers and/or have suggested that they could be used in the evaluation of behavioral and/or drug treatments.

As noted previously, reports relating aberrant oscillatory activity to clinical disorders are highly prevalent, but there is a shortage of studies assessing the reliability of oscillatory signatures in MEG. For such biomarkers of disease and/or treatment outcomes to be clinically useful, these neural responses must have high reliability. If measures vary significantly between sessions or visits, this would cast doubt on the validity of the biomarker because the presence of the disease is relatively stable (although severity can fluctuate). The intraclass correlation coefficient (ICC) is a common statistic used to assess test–retest reliability. The ICC is typically a ratio of the variance of interest over the sum of the variance of interest plus error (Ebel 1951; Bartko 1966; Shrout and Fleiss 1979). According to Koo and Li (2016), ICC values less than 0.5 indicate poor reliability, values between 0.5 and 0.75 indicate moderate reliability, values between 0.75 and 0.9 indicate good reliability, and values greater than 0.9 indicate excellent reliability. The coefficient can be used to assess either the absolute agreement or consistency of neural responses from session to session or visit to visit (McGraw and Wong 1996). Absolute agreement is more common than the degree of consistency definition in neuroimaging (Chen et al. 2017), but the correct selection of an ICC form depends on the experimental design, the types of measures being examined, and the aims of investigation.

Although far less common than fMRI investigations, MEG studies have also utilized the ICC to assess test–retest reliability. Martín-Buro et al. (2016) investigated the test–retest reliability of specific oscillations in healthy adults during resting-state MEG. With a 1-week interval between each of the three sessions, they found moderate-to-excellent reliability for resting-state theta-, alpha-, and low beta-bands at sensor and source space and poor-to-moderate reliability for delta, and gamma power, which aligns with the EEG test–retest literature (Gasser et al. 1985; Kondacs and Szabó 1999; McEvoy et al. 2000). Muthukumaraswamy et al. (2010) conducted a study to examine the test–retest reliability of occipital MEG responses and reported good-to-excellent reliability for visually elicited gamma-band responses in source space. Tan et al. (2016) also evaluated the test–retest reliability of visually induced gamma oscillations and found good-to-excellent reliability between two MEG sessions. Studies reporting MEG measures of functional connectivity have suggested poor-to-good ICCs (Deuker et al. 2009; Hinkley et al. 2011; Jin et al. 2011). In sum, the literature on the reliability of MEG is extremely sparse and has focused largely on visual gamma responses.

Among the wide array of neural processes studied using MEG, visual and somatosensory responses are among the most common. The oscillatory dynamics serving basic visual processing have been extensively examined, with visual theta, alpha, and gamma responses being the most prominent. Increased theta (3–7 Hz) activity during visual processing has also been found in the prefrontal and occipital cortices (Wiesman et al. 2017b; Proskovec et al. 2018; Wiesman and Wilson 2019), which may reflect the cognitive control of attention (Clayton et al. 2015; McDermott et al. 2017) and the indexing of early visual stimulus recognition and processing, respectively (Busch et al. 2009; Landau and Fries 2012; Landau et al. 2015). A decrease in alpha (7–13 Hz) activity in the occipital cortices and other visual regions has been strongly associated with inhibitory processing (van Dijk et al. 2008; Bonnefond and Jensen 2012; McCusker et al. 2020). Finally, studies have found increased gamma (>30 Hz) activity in superior parietal and primary visual areas (Wiesman et al. 2017a; Proskovec et al. 2018) and based on animal studies have inferred that gamma activity may serve fine-grain encoding of local visual stimulus features (Edden et al. 2009; Swettenham et al. 2009; Muthukumaraswamy and Singh 2013).

Neural activity related to somatosensory processing often reflects robust high-frequency oscillations (e.g., gamma activity). This activity arises briefly after the onset of somatosensory electrical stimulation (Cheng et al. 2016) and is generally very strong. This response is also known to exhibit gating, which is a robust neurophysiological phenomenon whereby neural responses to a second stimulus are significantly weaker than those to an identical first stimulus when they are paired in rapid temporal succession. This attenuation is thought to be an adaptive process for filtering out less salient stimulus features and preserving resources for relevant stimuli (Cromwell et al. 2008). Alterations in sensory gating for the auditory, somatosensory, and visual domains have been reported in schizophrenia (Arnfred and Chen 2004; Edgar et al. 2005; Thoma et al. 2007; Brockhaus-Dumke et al. 2008), cerebral palsy (Kurz et al. 2018), human immunodeficiency virus (Spooner et al. 2020a), migraine (Höffken et al. 2009), and aging (Lenz et al. 2012; Spooner et al. 2019), with many groups suggesting potential utility as a biomarker of disease.

Across all neuroimaging techniques, relatively few long-term reliability studies have been conducted. Within fMRI, long-term reliability has been examined in language (Nettekoven et al. 2018), classical learning (Aron et al. 2006), a continuous motor task (Kimberley et al. 2008), and resting state (Chou et al. 2012). EEG studies that have investigated long-term intra-individual reliability have focused on neural oscillations (Neuper et al. 2005), quantitative EEG (Kondacs and Szabó 1999), and sleep (Perkinson-Gloor et al. 2015) with moderate success. In contrast, long-term reliability studies using MEG have not been extensively performed, with the exception of limited resting state studies (e.g., Becker et al. 2012) that included 6-month follow-up visits. In this study, we investigated the long-term reliability of MEG-derived neural oscillations serving visuospatial processing (Experiment 1) and somatosensory gating (Experiment 2) in the same participants (N = 19) across three separate visits equally spaced across 36 months. Using this longitudinal design, we were primarily interested in the absolute agreement of measurements between the three visits in anatomical space, as MEG sensor space analyses can be biased by differences in head position between visits. Our primary hypotheses were that both somatosensory and visuospatial responses would have good to excellent reliability, especially during the peak response period, and that slower oscillations would generally have better reliability than faster responses. High ICC values (e.g., good-to-excellent) would provide substantial support to the growing interest in using MEG to derive biomarkers of disease and/or treatment response.

Methods

Participants

The two experiments (Experiment 1: visuospatial processing, Experiment 2: somatosensory gating) enrolled the same 19 healthy adults (11 male, 17 right handed, 25 White, 4 Asian/Pacific Islander, all non-Hispanic) who participated in three separate visits (Visit 1: M_age = 44 years, range = 23–61; Visit 2: M_age = 46 years, range = 25–63; Visit 3: M_age = 48 years, range = 27–65). The average time between Visit 1 and Visit 2 was 19 months and 6 days, while the average time between Visit 2 and Visit 3 was 18 months and 2 days. Exclusionary criteria included any medical illness affecting CNS function, neurological or psychiatric disorder, history of head trauma, current substance abuse, and non-removable metal implants that would adversely affect data acquisition. Each participant provided written informed consent and was compensated for their time and travel. The Institutional Review Board at the University of Nebraska Medical Center reviewed and approved this study, and all protocols were in accordance with the Declaration of Helsinki.

Experimental Paradigms

During the MEG recording, participants were seated in a custom-made nonmagnetic chair within a magnetically shielded room, with their heads positioned within the sensor array. In Experiment 1, participants were instructed to focus on a centrally located fixation crosshair, which was presented for a variable inter-stimulus interval of 1900–2100 ms. An 8 × 8 stimulus grid was then shown for 800 ms in one of four locations relative to the fixation point: above and to the left, above right, below left, or below right. The left/right orientations were defined as a lateral offset of 75% of the grid from the center of the fixation. A photodiode was used to quantify the stimulus delay, and this was accounted for in the analysis. Participants were instructed to respond as fast and accurately as possible using the index and middle digits of their right hand to indicate whether the grid was presented to the left or right of the fixation crosshair (Proskovec et al. 2018; Wiesman et al., 2017a; Fig. 1A). Each individual completed 240 trials of this task per visit, and only the correct trials were analyzed. Reaction times on the task were assessed, and to account for spurious reaction times, we performed standard data trimming procedures for each participant by excluding reaction times three median absolute deviations above or below each participant’s median prior to averaging.

Figure 1 .

Experimental tasks. (A) Experiment 1: Visuospatial attention task. Participants were instructed to attend to a central crosshair for 1900–2100 ms. An 8 × 8 grid then appeared in one of four quadrants, offset to the left or right, and top or bottom. Participants were instructed to respond to whether the stimulus was offset to the left or right. (B) Experiment 2: Somatosensory gating task. Electrical somatosensory paired-pulse stimulation applied to the participant’s right-hand median nerve for 80 trials. Stimuli were applied in pairs 500 ms apart, and stimulation pairs were applied 4500–4800 ms apart. ISI = inter-stimulus interval; IPI = inter-pair interval.

In Experiment 2, a paired-pulse electrical stimulation was applied to the median nerve of the right hand using external cutaneous stimulators connected to a Digitimer DS7A constant-current stimulator system (Digitimer Ltd, Garden City, UK). Each electrical pulse was comprised of a 0.2-ms constant-current square wave set to 10% above the motor threshold needed to elicit a subtle twitch of the thumb. Each participant received at least 80 paired-pulse trials with an inter-stimulus interval of 500 ms and an inter-pair interval that randomly varied between 4500 and 4800 ms (Spooner et al. 2018, 2019, 2020b).

MEG Data Acquisition

All MEG recordings took place in a one-layer magnetically shielded room with active shielding engaged for environmental noise compensation. A 306-sensor Elekta/MEGIN MEG system (Helsinki, Finland), equipped with 204 planar gradiometers and 102 magnetometers, was used to sample neuromagnetic responses continuously at 1 kHz with an acquisition bandwidth of 0.1–330 Hz. The same instrument was used across all recordings. Participants were monitored by a real-time audio-video feed from inside the shielded room throughout MEG data acquisition. Each MEG dataset was individually corrected for head motion and subjected to noise reduction using the signal space separation method with a temporal extension (tSSS; MaxFilter v2.2; correlation limit: 0.950; correlation window duration: 6 s; Taulu and Simola 2006). Only the gradiometer data was used in the primary analyses for both experiments.

MEG Processing, Time–Frequency Transformation, and Sensor-Level Statistics

Noise-reduced MEG data underwent standard data preprocessing procedures using the Brain Electrical Source Analysis (BESA) software. Cardiac and blink artifacts were identified in the raw MEG data and removed using an adaptive signal-space projection (SSP) approach, which was subsequently accounted for during source reconstruction (Uusitalo and Ilmoniemi 1997; Ille et al. 2002). Data were then divided into epochs tailored to each task. For Experiment 1, the data were divided into 2700-ms epochs (−500 to 2200 ms), with 0.0 s defined as stimulus grid onset, and the baseline defined as −400 to 0 ms. In Experiment 2, data were split into 3700-ms epochs, with the baseline extending from −700 to −300 ms prior to the onset of the first electrical somatosensory stimulation, which occurred at 0.0 ms. The baseline was shifted away from the period immediately preceding stimulus onset to avoid potential contamination by any anticipatory somatosensory responses, although there was no evidence of such anticipatory responses in the final analyses.

For both experiments, epochs containing artifacts were rejected using a fixed threshold method, supplemented with visual inspection. Briefly, in MEG, the raw signal amplitude is strongly affected by the distance between the brain and the MEG sensor array, as the magnetic field strength falls off sharply as the distance from the current source increases. To account for this source of variance across participants, as well as other sources of variance, we used an individually determined threshold based on the signal distribution for both amplitude and gradient to reject artifacts. Across all visits, an average of 200.46 (SD = 18.26) out of 240 trials in Experiment 1 and an average of 75.17 (SD = 3.80) out of 80 possible trials per participant in Experiment 2 were used for further analyses. Importantly, none of our comparisons were compromised by differences in the number of accepted trials per visit, which can affect the signal-to-noise ratio, as this metric did not significantly differ across visits.

Artifact-free epochs were transformed into the time–frequency domain using complex demodulation (Papp and Ktonas 1977), and the resulting spectral power estimations per sensor were averaged over trials for each experiment separately to generate time–frequency plots of mean spectral density. Time–frequency resolutions that were optimal for identifying each task’s respective responses were utilized. Specifically, for Experiment 1, the time–frequency analysis was performed with a frequency step of 2 Hz and a time step of 25 ms between 4 and 100 Hz, while for Experiment 2 a frequency step of 5 Hz and a time step of 10 ms between 10 and 100 Hz was used. These sensor-level data were then normalized by each respective bin’s baseline power for visualization purposes, calculated as the mean power during the baseline time period of each experiment. For oscillatory analyses, the specific time–frequency windows used for source reconstruction were determined by statistical analysis of the sensor-level spectrograms across the entire array of gradiometers and all participants, per experiment. Each data point in the spectrograms was initially evaluated using a mass univariate approach based on the general linear model. To reduce the risk of false-positive results while maintaining reasonable sensitivity, a two-stage procedure was followed to control for Type I error. In the first stage, paired sample t tests against baseline were conducted on each data point and the output spectrogram of t values was thresholded at P < 0.05 to define time–frequency bins containing potentially significant oscillatory deviations across all participants. In Stage 2, the time–frequency bins that survived the threshold were clustered with temporally and/or spectrally neighboring bins (per sensor) that were also above the threshold (P < 0.05), and a cluster value was derived by summing all of the t values of all data points in the cluster. Nonparametric permutation testing was then used to derive a distribution of cluster values, and the significance level of the observed clusters (from stage one) was tested directly using this distribution (Ernst 2004; Maris and Oostenveld 2007). For each comparison, 10 000 permutations were computed to build a distribution of cluster values. Based on these analyses, the time–frequency windows encompassing these significant clusters across all participants (per experiment) were subjected to the beamformer source reconstruction analysis.

Additionally, a time-domain analysis was performed for Experiment 2, using a time step of 5 ms. A similar nonparametric permutation testing approach (with 1000 permutations) was applied to the time-domain-averaged data across all participants, as performed previously (Wiesman and Wilson 2020). This resulted in temporal cluster maps, which were averaged over time following each somatosensory stimulation (i.e., 30–130 ms and 540–640 ms) across visits. Based on this analysis, the time-domain windows encompassing significant clusters across all participants were subjected to a different time-domain source analysis (see below).

Structural MRI Processing and MEG Coregistration

Prior to MEG acquisition, four coils were attached to the participants’ heads and localized, together with the three fiducial points and scalp surface, using a 3-D digitizer (Fastrak 3SF0002, Polhemus Navigator Sciences, Colchester, VT, USA). Once the participant was positioned for MEG recording, an electrical current with a unique frequency label (e.g., 322 Hz) was fed to each of the coils. This induced a measurable magnetic field and allowed each coil to be localized in reference to the sensors throughout the recording session. Since coil locations were also known in head coordinates, all MEG measurements could be transformed into a common coordinate system (Wilson et al. 2005). With this coordinate system (including the scalp surface points), each participant’s MEG data were co-registered with T1-weighted structural magnetic resonance images (sMRI) prior to source space analyses using BESA MRI (Version 2.0; BESA GmbH, Gräfelfing, Germany). All sMRI data were acquired with a Philips Achieva 3T X-series scanner using an 8-channel head coil (TR: 8.09 ms; TE: 3.7 ms; field of view: 240 mm; slice thickness: 1 mm; no gap; in-plane resolution: 1.0 × 1.0 mm). All sMRI data were aligned parallel to the anterior and posterior commissures and transformed into standardized space. Following source analysis, each participant’s functional images were also transformed into standardized space using the transform that was previously applied to the structural MRI volume and spatially resampled.

MEG Source Imaging and Statistics

Using a spherical head model, cortical oscillatory networks were imaged through an extension of the linearly constrained minimum variance vector beamformer known as dynamic imaging of coherent sources (DICS; Groß et al. 2001), which applies spatial filters to the time–frequency domain to calculate voxel-wise source power for the entire brain volume. Briefly, the DICS beamformer estimates source-level data using both cross-spectral density and lead field matrices. Imaging of oscillatory responses was performed per visit, experiment, and participant for the time–frequency bins identified using the previously described sensor-level statistical approach. The single images were derived from the cross-spectral densities of all combinations of MEG gradiometers averaged over the time–frequency range of interest, and the solution of the forward problem for each location on a grid specified by input voxel space. Following convention, we computed noise-normalized, source power per voxel using active (i.e., post-stimulus period of interest) and passive (i.e., baseline) periods of equal duration and bandwidth (Hillebrand and Barnes 2005). Such images are typically referred to as pseudo-t maps, with units (pseudo-t) that reflect noise-normalized power differences (i.e., active versus passive) per voxel. This generated participant-level functional maps for each time–frequency-specific response identified in the sensor-level cluster-based permutation analysis.

The resulting three-dimensional maps of brain activity were grand averaged across all participants and visits (and for Experiment 2, both stimulations), to assess the neuroanatomical basis of the significant oscillatory responses identified through the sensor-level analysis and to allow identification of the peak voxel coordinates for each time–frequency component. We then extracted pseudo-t amplitude values from the peak voxel(s) for display purposes and subsequent analyses. Of note, we used the grand averaged maps to select the peak voxel in order maximize the amount of data that was used to determine the spatial location of the responses, thus increasing external validity. As a sanity check, we also computed ICCs using the average peaks from Visit 1 alone, and this resulted in only negligible changes to the spatial locations and final outcome metrics. Next, voxel time series (i.e., “virtual sensors”) were extracted from each participant’s data individually per visit using the peak voxel coordinates identified from the grand-averaged beamformer images. Briefly, virtual sensor computation was performed by applying the sensor-weighting matrix derived through the forward computation to the preprocessed signal vector. These data were then decomposed back into time–frequency space and averaged across the previously identified time–frequency extents (i.e., used in the beamformer analysis) per time point to compute the envelope of each response per visit. Notably, voxel time series estimations do not utilize the cross spectral density matrix from the beamformer and allow for more rich temporal information. Once these virtual sensor time series were extracted, we computed the vector sum of the two orientations and then the relative (i.e., baseline-corrected) and absolute (i.e., not baseline-corrected) time series envelope of each participant per visit.

Additionally, for Experiment 2, time-domain source images were computed using standardized low-resolution brain electromagnetic tomography (sLORETA; regularization: Tikhonov 0.01%; Pascual-Marqui et al. 2002). The resulting whole-brain maps were four-dimensional estimates of current density per voxel, per time sample across the experimental epoch. These data were normalized to the sum of the noise covariance and theoretical signal covariance, and thus, the units are arbitrary. Using the temporal clusters identified in the sensor-level statistical analysis, these maps were averaged over time following each somatosensory stimulation (i.e., 30–130ms and 540–640 ms; see the Results section) across visits. The resulting maps were then grand averaged across the two stimulations to determine the peak voxel of the time-domain neural response to the stimuli across participants and visits. From this peak, the sLORETA time series was extracted per visit and participant to derive estimates of the time-domain response strength.

Statistical Analyses

To further examine the somatosensory paired-pulse stimulations in Experiment 2, a gating ratio was computed per participant for each visit to quantify the degree of sensory gating (i.e., response amplitude to stimulus 2 divided by that of stimulus 1). Gating ratios were computed on the pseudo-t amplitude peaks from the beamformer output, virtual sensor time series amplitudes, and sLORETA peak units. The resulting quotients were subjected to subsequent statistical analysis. Note that the gating ratio reflects an individual’s capacity to “gate” the second stimulus in an identical pair, with smaller quotients indicating better gating (i.e., greater suppression of the second response).

We calculated intraclass correlation coefficients (ICC) to measure the extent of reliability among our visual and somatosensory quantitative variables at the source level across the three visits. We only investigated the source level because previous MEG studies have found that the reliability of sensor and estimated source signals are comparable (Tan et al. 2015, 2016), with source space presenting more reliability due to several advantages (Martín-Buro et al. 2016; Tan et al. 2016). Specifically, we implemented a single-rater two-way mixed-effects model and absolute agreement definition, or ICC(A,1) defined by McGraw and Wong (1996) and based on Shrout and Fleiss (1979). This ICC definition is generally more conservative, and additionally, Koo and Li (2016) suggest using a two-way mixed-effects model and absolute agreement definition for test–retest reliability studies, which coincides with our long-term reliability design. ICC estimates and their 95% confidence intervals were calculated using the Matlab Central file-exchange ICC.m function (Salarian 2016) in Matlab (Version 2018b; Mathworks, Inc., Massachusetts, USA). Lastly, for display, we calculated the ICC at each time point in the relative and absolute voxel time series (both experiments) and in the time-domain (Experiment 2) across the entire epoch.

For a brief background, ICC estimates range from 0 to 1, with values closer to 1 indicating higher reliability. Low ICCs could be due to several reasons: 1) a low degree of rater (i.e., visit) or measurement agreement, 2) a lack of variability in sample participants, 3) a small number of participants, and 4) a small number of raters (i.e., visits) being tested (Koo and Li 2016; Chen et al. 2017). ICC estimate value interpretations vary (Cicchetti 1994; Portney and Watkins 2009; Koo and Li 2016), but we followed the most recent guidelines set by Koo and Li (2016), with values less than 0.5 indicating poor reliability, values between 0.5 and 0.75 indicating moderate reliability, values between 0.75 and 0.9 indicating good reliability, and values greater than 0.9 indicating excellent reliability. ICCs of 0.7 or higher are usually considered necessary to study individual psychometric or behavioral differences (Boutros et al. 1991; Kline 2000). Importantly, we evaluated the level of reliability based on the 95% confidence interval of the ICC estimate, not the estimate itself, since the interval reveals the chance that the true ICC value lands on any point between the bounds.

Finally, we conducted an exploratory analysis to examine whether variability in our MEG outcome metrics could be related to participant-specific age. Briefly, we computed coefficients of variation for each participant in each outcome metric, which reflect the variability across the three visits for each participant per outcome metric. We then correlated these coefficients of variation with age at Visit 1, with the hypothesis that older participants may have greater variability across the three years.

Results

Behavioral Results

All 19 participants completed the MEG and MRI protocols for all three visits. No participants reported a significant adverse health event across the three visits. For the visuospatial task (Experiment 1) specifically, all 57 datasets were available. When examining task performance, participants had a mean reaction time of 519.36 ms (SD = 74.52) for Visit 1, 523.29 ms (SD = 88.50) for Visit 2, and 508.85 ms (SD = 80.55) for Visit 3. Reaction times ultimately showed good-to-excellent reliability across the three visits, showing an ICC of 0.814 (95% CI = 0.653–0.916). For Experiment 2, while all 19 participants successfully completed each visit, there was no usable MEG data for the first visit in one of our participants. Thus, only 18 participants were included in our final analyses for the somatosensory gating experiment. Note that no behavioral response was required for the somatosensory gating experiment.

MEG Sensor-Level Results

For Experiment 1, examination of task-specific neural activity revealed oscillatory responses within the theta, alpha, and gamma range during visuospatial processing. We identified time–frequency windows identical to a previous study using this same paradigm (Wiesman et al. 2018) for each of the three visits. Specifically, a significant theta increase or synchronization (4–8 Hz), alpha decrease or desynchronization (8–16 Hz), and gamma synchronization (52–70 Hz) were identified post-stimulus at 100 to 500 ms, 300 to 700 ms, and 350 to 750 ms, respectively, in MEG sensors across the occipital cortices (P < 0.001, corrected; Fig. 2A). Beta (18–24 Hz) activity was also observed in sensors near the motor cortices, but such motor responses were beyond the scope of the current study.

Figure 2 .

Sensor-level spectrograms and topographic maps. (A) Experiment 1: Group-averaged representative spectrograms for Visit 1 (left), Visit 2 (middle), and Visit 3 (right) displaying the visual oscillatory activity elicited by the task. A significant theta response at 4–8 Hz, an alpha response at 8–16 Hz, and a gamma response at 52–70 Hz were identified post-stimulus at 100 to 500 ms, 300 to 700 ms, and 350 to 750 ms, respectively. Topographic maps for these time–frequency windows are shown below each spectrogram. Color bars showing percent change from baseline are shown to the right of the spectrograms and below the topographic maps, respectively. (B) Experiment 2: Group-averaged representative spectrograms of Visits 1, 2, and 3 show two significant synchronizations or increases in gamma activity (30–75 Hz), one at 0–50 ms and the other at 500–550 ms. Topographic maps for each stimulation time–frequency windows are displayed below each spectrogram. Note, the right frontal artifact for Visit 2 is most likely bleed through from the electrical stimulation on the right median nerve. Respective color bars showing percent change from baseline are shown to the right of the spectrograms and topographic maps. Across all spectrograms (both experiments), time (in ms) is denoted on the x-axis and frequency (in Hz) denoted on the y-axis.

For Experiment 2, statistical analysis of the sensor-level spectrograms revealed broadband (10–100 Hz) synchronizations across many sensors near the sensorimotor and parietal regions during approximately the first 50 ms of each stimulation (P < 0.001, corrected). There were two distinct significant gamma increases or synchronizations. We focused our beamformer analyses on the 30–75-Hz gamma frequency range in two discrete 50-ms time intervals following each stimulation (i.e., 0–50 and 500–550 ms; Fig. 2B). We limited analyses to 30 Hz on the low end because it was most consistent with the traditional definition of the gamma band and could be adequately resolved in a 50-ms window, and we limited analyses to 75 Hz on the high end because the relative power sharply decreased thereafter, especially for the second stimulation. We also statistically analyzed the time-domain signal at the sensor level, which revealed significant temporal clusters across many of the same sensors near the sensorimotor and parietal regions. Based on the temporal clusters identified at the sensor level, the latency of interest for stimulation 1 was 30–130 ms, and that for stimulation 2 was 540–640 ms.

MEG Source Mapping Results

Visual Responses

To identify the anatomical origin of these time–frequency responses, a frequency-resolved beamforming approach (see Methods) was utilized. The resulting functional maps were grand averaged across the entire sample and across the three visits to identify the specific brain areas generating the task-related neural oscillatory responses. The grand-averaged maps of the theta response (4–8 Hz) and the gamma response (52–70 Hz) revealed activity in bilateral clusters within the medial occipital cortices surrounding the primary visual cortex. In contrast, alpha oscillatory responses (8–16 Hz) were generated in more lateral visual association cortices. Using these grand-averaged bilateral peaks, beamformer pseudo-t values were extracted for each participant at each visit, and the ICC across the three visits was calculated. Maps of the theta response showed relatively moderate-to-good reliability across the three visits, with the left peak showing an ICC of 0.741 (95% CI = 0.537–0.881) and the right peak showing an ICC of 0.666 (95% CI = 0.433–0.839). Alpha responses, showed moderate-to-good 3-year reliability, with the left peak yielding an ICC of 0.747 (95% CI = 0.548–0.883) and the right peak yielding an ICC of 0.731 (95% CI = 0.525–0.874). Finally, visual gamma responses showed poor-to-good reliability, with the left peak displaying an ICC of 0.512 (95% CI = 0.241–0.748), and the right peak displaying an ICC of 0.528 (95% CI = 0.320–0.793; Fig. 3).

Figure 3 .

Visuospatial processing: grand average maps and ICCs. Grand-averaged source images (across all participants and visits) are displayed to the left. Theta (top) and gamma (bottom) activity localized to the primary visual cortices, with alpha activity (middle) being centered in more lateral occipital cortices. Respective color scale bars are displayed to the right, with warm colors representing an increase in noise-normalized power relative to baseline (pseudo-t), and cool colors representing a decrease relative to baseline. ICCs for the right and left peak voxels are shown to the right, displaying the reliability of the power at these peaks over 3 years. Error bars denote 95% confidence intervals. Overall, theta and alpha peaks showed moderate-to-good reliability, and gamma peaks showed poor-to-good reliability.

Somatosensory Responses

The two time–frequency windows identified in the sensor level analysis were imaged in each participant per visit using a DICS beamformer. First, the resulting functional maps, we averaged across all participants and visits per stimulation and then averaged across the stimulations (Fig. 4A). The two stimulation-specific averaged images and the grand-averaged image revealed peak responses in identical areas of the contralateral somatosensory hand region of the postcentral gyrus, with strongly diminished activity in response to the second stimulation. Next, we computed the gating ratio by dividing the pseudo-t value of the peak response to stimulation 2 by that of stimulation 1 in each participant for each visit. We observed robust sensory gating across all three visits, and there were no statistical differences in sensory gating between visits. To assess the absolute agreement among these pseudo-t peak amplitude values, ICCs were calculated for each stimulation and gating ratio. For stimulation 1 (30–75 Hz, 0–50 ms) there was poor-to-good reliability, with an ICC estimate of 0.561 (95% CI = 0.291–0.784). For stimulation 2 (500–550 ms), there was poor reliability, with an ICC estimate of 0.291 (95% CI = 0.006–0.601). Likewise, the gating ratio also showed poor-to-moderate reliability, with an ICC estimate of 0.282 (95% CI = −0.009–0.597). These are shown in Fig. 4A.

Figure 4 .

Somatosensory gating: grand average maps and ICCs. Grand-averaged source images (across all participants and visits) from the gamma-frequency beamformer analysis (A) and the time-domain sLORETA estimations (B) for Stimulation 1, Stimulation 2, and the average of the two. Corresponding color scale bars displaying respective beamformer (pseudo-t) and sLORETA statistics (arbitrary units) are shown. To the right, bar graphs displaying ICC estimates for Stimulation 1, Stimulation 2, and the respective gating ratio at the corresponding peak voxels are shown, displaying the reliability of the power at these peaks over three years. Error bars denote 95% confidence intervals. Overall, Stimulation 1 showed poor-to-good reliability in both time-domain and gamma-band source-level estimations, while Stimulation 2 and the gating ratio showed poor-to-good reliability in the time domain and poor-to-moderate reliability in the gamma band analysis.

Time-domain whole-brain maps were computed using sLORETA. Based on the temporal clusters identified at the sensor level, the latency of interest for stimulation 1 was 30–130 ms and that for stimulation 2 was 540–640 ms. Thus, we averaged the output volumes within these temporal windows per participant and visit, and then grand-averaged the images across all participants, visits, and stimulations to identify the peak coordinates of the time-domain neural response (Fig. 4B). To assess the reliability of the sLORETA time-domain estimates, ICCs were calculated for each stimulation and gating ratio. The ICC estimate for stimulation 1 was 0.654 (95% CI = 0.406–0.838) and that for stimulation 2 was 0.691 (95% CI = 0.460–0.857), both reflecting poor-to-good reliability. The corresponding gating ratio had poor-to-good reliability with an ICC estimate of 0.588 (95% CI = 0.326–0.799).

Neural Time Series Results

Visual Time Series Responses

Next, we extracted the time series from the peak voxels of each oscillatory response per participant and visit, averaged the time series across hemispheres (i.e., left/right occipital cortices), and then averaged over the respective time windows used to compute the beamformer image. Using these time series, we examined the reliability of the mean over the active window (after normalizing to baseline). Task responses from these estimations showed that theta activity had moderate-to-excellent reliability, with an estimated ICC of 0.824 (95% CI = 0.669–0.921), alpha activity showed good-to-excellent reliability with an ICC of 0.873 (95% CI = 0.753–0.944), and the gamma response showed poor-to-good reliability, with an ICC of 0.556 (95% CI = 0.292–0.776). For display, we also calculated the ICC for every data point in these relative time series across the entire epoch (Fig. 5, top row). Across all three oscillatory responses, baseline-corrected amplitude values increase in reliability after the stimulus and gradually decrease in reliability with time. These spectral responses also continue to show the pattern of relatively higher theta and alpha reliability and relatively lower gamma reliability.

Figure 5 .

Neural time series and corresponding ICC per experiment. For Experiment 1 (top row), time series were extracted from the right and left peak voxel per response (Fig. 3 and insets) in each participant per visit and these were averaged across hemispheres. In Experiment 2 (bottom row), the time series were extracted from the grand-averaged peak voxel (Fig. 4 and insets) in each participant. In all panels, the time series for Visits 1–3, averaged across participants, are plotted for each response; the legend appears in the middle of the lower row. To examine the reliability across the epoch for each response, time point-by-time point ICCs were calculated and these are plotted in bars behind the amplitude time series with a secondary y-axis on the right side. Time (in ms) is displayed on the x-axis. (top row) Visual theta and alpha band responses consistently showed good 3-year reliability throughout the entirety of their response periods (i.e., beamformer image windows), while gamma responses showed poor-to-moderate reliability post-stimulus. (bottom row) Somatosensory gamma band and time-domain responses showed good reliability following the electrical stimuli at 0 and 500 ms. Notably, the gamma responses, and respective reliabilities, were transient around the peak responses, while the time-domain responses and resulting reliabilities were of greater duration around the response, but were much lower during the period after the response.

Somatosensory Time Series Responses

For Experiment 2, we extracted virtual sensors (i.e., voxel time series) from the peak voxel identified in the prior analysis in each participant per visit and computed the baseline-normalized (i.e., relative) amplitude and absolute amplitude envelopes for the 30–75-Hz band. These virtual sensor time-series data were averaged over the 0–50-ms (Stimulation 1) and 500–550-ms (Stimulation 2) latency windows, and gating ratios were calculated. These averages and quotients were then subjected to ICC analysis. The relative amplitude virtual sensor averages showed good-to-excellent reliability, with an ICC estimate of 0.889 (95% CI = 0.779–0.953) for Stimulation 1 and an ICC estimate of 0.858 (95% CI = 0.721–0.939) for Stimulation 2. The corresponding gating ratio showed poor-to-moderate reliability, yielding an ICC estimate of 0.293 (95% CI = 0.008–0.603). For visualization purposes, we computed the ICC time series for relative amplitude (Fig. 5, bottom row) throughout the epoch, which expresses the time-varying gamma reliability. Additionally, in the time domain, we extracted values from the sLORETA volumes at the peak voxel identified earlier. These values and their respective time-varying ICCs are shown in Fig. 5, bottom row.

Spontaneous Oscillatory Activity

Finally, we examined spontaneous neural activity by computing the amplitude during the baseline period using the virtual sensor/voxel time series data from Experiment 1. Spontaneous theta activity showed poor-to-good reliability, with an ICC of 0.676 (95% CI = 0.443–0.846), spontaneous alpha activity showed good-to-excellent reliability, with an ICC of 0.919 (95% CI = 0.836–0.965), and spontaneous gamma activity showed poor reliability, with an ICC of 0.192 (95% CI = −0.029–0.483). As before, we calculated the ICC for every data point in the absolute time series across the entire epoch for visualization purposes. These plots generally show the robust reliability of spontaneous alpha activity, consistently remaining around excellent reliability. In contrast, absolute theta stays in the moderate range, while spontaneous gamma activity had consistently poor reliability (Fig. 6).

Figure 6 .

Neural absolute time series and corresponding ICC. Absolute (non-normalized to baseline) time series were computed to examine spontaneous activity during the baseline for Visits 1–3. To examine the reliability, time point-by-time point ICCs were calculated and plotted in bars behind the absolute amplitude time series, with a secondary y-axis on the right side. Time (in ms) is displayed on the x-axis, and source images are inset to illustrate the peak voxel. Absolute theta amplitude at the visual cortex (top left) showed good 3-year reliability throughout the baseline period, while alpha amplitude during the baseline showed excellent reliability across the three visits. Baseline gamma reliability was poor at both the visual and somatosensory peaks.

For Experiment 2, we used the absolute amplitude time series to estimate the average strength of spontaneous 30–75-Hz gamma activity during the baseline period per participant and visit. The ICC estimate of spontaneous gamma during the averaged baseline (−700 to −300 ms) was 0.416 (95% CI = 0.109–0.696), indicting poor-to-moderate reliability. Lastly, we computed the ICC for every data point in the absolute time series (Fig. 6) across the entire epoch for visualization purposes.

Coefficient of Variation with Age

Given the relatively long time period between visits and the wide age range of our sample, we conducted an exploratory analysis to examine whether some of the variability was due to participant age. To this end, we computed the coefficient of variation per person for each outcome MEG metric and then correlated this with participant age at visit one. We found no significant correlations between age and the coefficient of variation of any MEG measurement (all P > 0.05), suggesting within subject variance was not strongly affected by the participant’s age.

Discussion

In the current study, we examined the 3-year reliability of MEG visual and somatosensory neural responses. We found that the reliability of occipital visual responses ranged between moderate-to-excellent in the theta and alpha ranges, while visual gamma responses showed poor-to-moderate long-term reliability. During somatosensory processing, we found moderate-to-good degrees of reliability in both gamma and time-domain responses. When examining spontaneous activity during the baseline, we observed moderate-to-excellent long-term reliability in occipital alpha activity, poor-to-good reliability in occipital theta activity, and mostly poor reliability in occipital and somatosensory gamma activity. Ultimately, these findings are of critical importance in understanding the utility of MEG, especially in clinical contexts. That is, markers of neural activity must be stable and reliable if they are to be useful clinically (e.g., as biomarkers). Overall, our results show a general pattern of stability and reliability in MEG measured neural activity; however, certain neural responses were more reliable than others, and these nuances are discussed next.

Our study was uniquely optimized to examine the long-term reliability of MEG activity. That is, the vast majority of studies examine the reliability across two visits relatively close in time. In contrast, our study utilized three visits, equally spaced across a total of 3 years. This is important because the establishment of reliability long term is critical to understanding the stability of such activity, and the potential capacity of these responses to be used as biomarkers. Additionally, utilizing more than two measurements points is highly sought after in longitudinal studies due to the added confidence in the trajectory of the data over time. Herein we applied that same principle to test–retest reliability, which ultimately strengthened our conclusions. Relatively few neuroimaging studies to date have taken these approaches, with some examining long-term reliability in EEG (Kondacs and Szabó 1999; Neuper et al. 2005), fMRI (Aron et al. 2006; Wang et al. 2011; Liu et al. 2017; Nettekoven et al. 2018), and fNIRS (Schecklmann et al. 2008). In MEG studies, these approaches are rarely seen, with one study focusing on resting state in HIV (Becker et al. 2012), and another studying visual perception in the time domain using four participants (Sandberg et al. 2014). Therefore, our three-visit long-term study of 19 participants in two distinct experiments is seminal in establishing MEG estimates of neural activity as reliable and stable over 3 years.

Among all of the activity we studied, the activity which showed the highest overall reliability was visual alpha activity. Alpha responses showed good 3-year reliability, while spontaneous alpha during the baseline showed excellent reliability. This comes as no surprise since occipital alpha activity was one of the first electrophysiological signals detected from the brain and is arguably the most dominant rhythm in the brain. Visual alpha responses are thought to be associated with visual attention and functional disinhibition (Klimesch 2012), and in this paradigm, they are thought to index visuospatial attention (Wiesman et al. 2017b; Proskovec et al. 2018; Groff et al. 2020). Multiple previous test–retest studies have noted the short-term reliability of alpha activity as well (Gasser et al. 1985; Salinsky et al. 1991; McEvoy et al. 2000; Martín-Buro et al. 2016; Candelaria-Cook et al. 2020). Therefore, our results extend this literature by showing that both spontaneous and event-related visual alpha oscillations show high degrees of reliability across 3 years.

Similarly, our visual theta activity also showed reliability across the three years, although the ICC estimates for both the theta response and spontaneous theta during the baseline were of moderate reliability. This theta response contained at least some evoked activity and quickly followed stimulus presentation. This response is therefore thought to be associated with early basic visual stimulus processing, although theta activity has also been linked to temporal coding and long-distance communication of stimulus information (Başar et al. 2001; Landau et al. 2015; Wiesman et al. 2017b). Ultimately, our findings of reliability in theta activity are in agreement with previous literature has also shown short-term reliability in resting theta activity and theta responses (McEvoy et al. 2000; Martín-Buro et al. 2016).

In contrast, our visual gamma responses showed relatively poor reliability across the three visits. We speculate that this could be due to several reasons. First, the strength of the gamma signal, particularly in the visuospatial task, was relatively low. Time series estimates showed that gamma responses, although significant, were generally below 10% change from baseline, and spontaneous gamma activity during the baseline was on average around 2 nAm, a fraction of that of theta and alpha activity. This may be partially due to the fact that gamma activity, being a higher frequency, is naturally lower in power, resulting in lower signal to noise, especially during the baseline period. Indeed, other resting-state studies have noted poor reliability of resting gamma power (Martín-Buro et al. 2016). In regard to the oscillatory gamma response, we believe that our particular visuospatial processing task may not have been optimal for eliciting gamma activity. This might be reflected in the data, as the shape of the relative gamma time series shows a relatively broad response, which is contrary to the transient response one might expect with high-frequency activity. Therefore, our gamma response may be smeared in time, ultimately reflecting variability in the response to the grid stimulus. Indeed, other studies have seen good test–retest reliability of visual gamma activity using a visual grating stimulus (Fründ et al. 2007; Muthukumaraswamy et al. 2010; Tan et al. 2016). Therefore, our grid stimulus may not elicit gamma activity as strongly and reliably as a grating stimulus, and the poor reliability we saw may not be a reflection of gamma activity itself. Future studies are needed to clarify this.

In contrast to the visuospatial paradigm, electrical stimulation of the median nerve elicited gamma responses in the somatosensory cortex that did show good reliability across 3 years. Specifically, in Experiment 2, we observed robust responses in the gamma range following both stimulations, and the amplitude of the second stimulation was typically weakened, which reflects the sensory gating phenomenon (Hari and Forss 1999; Edgar et al. 2005; Thoma et al. 2007; Huttunen et al. 2008; Hsiao et al. 2013; Cheng et al. 2016; Wiesman et al. 2017a). The time series estimations support these results by demonstrating the gamma responses having good reliability although the respective gating ratio showed poor reliability. We did note, however, that the source beamformer pseudo-t values showed poorer reliability in gamma responses. We believe that this may be due to the influence of the broad time–frequency window, and the spatial specificity related to these differing estimations. Specifically, our source imaging approach takes into account the noise and baseline normalized power of the entire time–frequency window and utilizes spatial filters to minimize the influence of surrounding brain areas. In contrast, the time series estimation allowed for a more fine-grained view of the time series data, and although the source corresponded to the same peak voxel, the estimation did not employ spatial filters in the way that the beamformer does. Overall, this may indicate that, although there is good reliability in the somatosensory gamma response, obtaining a reliable measurement is more sensitive to the analysis approach.

Similar to the somatosensory gamma-frequency response, time-domain somatosensory responses and their respective gating showed moderate reliability. Given that localization of somatosensory evoked responses in MEG is utilized in pre-surgical mapping, it is not surprising that these responses show reliability (Sutherland and Tang 2006; Solomon et al. 2015). It is, however, a novel finding that the within-subject amplitudes of the responses show reliability, as opposed to simply the location, and that the gating of the response also shows reliability. This is in broad agreement with the auditory literature which has established good reliability of auditory gating (Smith et al. 1994; Fuerst et al. 2007; Rentzsch et al. 2008). However, unlike auditory paired-click paradigms, paired-pulse stimulation requires overcoming the visit-to-visit variability in the application of the external cutaneous stimulators (i.e., visit to visit they are in slightly different locations and the skin conductivity varies day-to-day in a given person). Therefore, it is especially encouraging that we were able to detect reliability despite this extra built-in variability in the protocol detail.

Notably, our study is limited in that it specifically examines visual and somatosensory MEG activity, and therefore our results cannot be generalized to other sensory modalities or cognitive processes. Specifically, beta motor activity is nearly as dominant as alpha activity, and further study should examine the long-term reliability of beta (and other oscillatory) motor dynamics. Note that short-term reliability measures already exist for motor-related beta oscillations (Wilson et al. 2014). Additionally, although our experiments were optimized to elicit strong visual and somatosensory MEG activity, such activity will differ by task design. For example, paradigms which utilize visual gratings may be better suited at examining visual gamma activity (Tan et al. 2016), and tasks which employ sensory entrainment (Tan et al. 2015; Legget et al. 2017; Wiesman et al. 2019; Wiesman and Wilson 2019) would be better tuned to analyze a specific entrained frequency. Further studies examining long-term reliability using other task designs such as these and those targeting other modalities are needed. Other study limitations include our relatively small sample size, which could have added variability to the neural response estimates and consequently impacted reliability values. Extraneous variability could also have arisen from any changes the participants may have experienced over the 3-year period, particularly in older participants. Notably, however, no significant adverse health events were reported by our participants, and participant age did not relate to participant-specific variability in any of the MEG measures. Additionally, our study specifically examined healthy adults, the majority of whom were White and non-Hispanic, and thus our findings may not be directly generalizable to children, the elderly, and other races and ethnicities. Finally, we note that absolute gamma power in both visual and somatosensory domains showed a pattern of higher power at Visit 1 than Visits 2 and 3. Whether this can be attributed to habituation or simply reflects increased variability is unknown and future studies should examine this with more time points.

In summary, our 3-year longitudinal study of MEG responses showed high degrees of reliability in theta and alpha visual responses, and gamma and time-domain somatosensory responses. Long-term reliability was also seen in spontaneous theta and alpha activity in visual occipital cortices, which is in agreement with previous resting-state studies. This validation establishes these responses as stable across an appreciable period of time, opening up the use of these responses in further longitudinal studies, and supporting their use for comparisons to patient populations in hopes of identifying potential biomarkers.

Funding

National Institutes of Health (R01-MH103220, R01-MH116782, R01-DA047828, R01-MH118013, and F30-DA048713); National Science Foundation (#1539067). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.