Excellent test-retest reliability of perturbation-evoked cortical responses supports feasibility of the balance N1 as a clinical biomarker

Abstract

There is a growing interest in measuring cortical activity during balance control for understanding mechanisms of impaired balance with aging and neurological dysfunction. The most well-characterized electrophysiological signal elicited by a balance disturbance is the perturbation-evoked N1 potential. We previously found associations between the N1 and balance ability, suggesting it may be a potential biomarker of balance health. However, a potential biomarker will be limited by its reliability and clinical feasibility, which has yet to be established. Here, we characterized the reliability of the balance N1 within and between sessions over a one-week interval in 10 younger and 14 older adults, and over a one-year interval in a subset of older adults (n=12). We extracted N1 amplitude and latency from the Cz electrode using an advanced, computationally-intensive approach (64 electrodes, many trials). Test-retest reliability was assessed using the intra-class correlation coefficient (ICC). Internal consistency was quantified by split-half reliability using the Spearman correlation coefficient. N1s varied across individuals, yet within individuals, showed excellent test-retest reliability (ICC>0.9) and internal reliability (r>0.9). N1 amplitude reliability generally plateaued within 6 trials, while more trials were needed to reliably measure latency. Similar results were obtained using a minimal approach (three electrodes, simple preprocessing) and at the component level (largest contributing N1 source). The N1’s stability, reliability, and feasibility make it well-suited for potential use as a clinical biomarker. Characterizing N1 reliability in different populations and contexts will be necessary to enhance our understanding, optimize experimental design, and determine its predictive validity (e.g., falls risk).

Graphical Abstract

1. Introduction

Individual differences in brain activity during balance recovery behavior may reveal brain mechanisms of balance impairment and fall risk with aging and neurological conditions. Electroencephalography (EEG) can quantify temporally-precise patterns of cortical activity on physiologically-relevant time scales during balance recovery behavior. The most well-characterized EEG signal elicited by a sudden balance disturbance is the perturbation-evoked N1 potential, a negative voltage deflection that typically peaks 100–200 msec post-perturbation (1, 2). The N1 is thought to reflect both sensorimotor (1–3) and cognitive processes (4–6)and can be elicited in different contexts (sitting (7), walking (8), lean-and-release (9), floor tilt, support-surface translations (10)) and in different populations (toddlers (11), younger adults, older adults (12, 13), Parkinson’s disease (14), stroke (15). Several studies have explored factors that modulate the N1 within individuals, including predictability (16, 17), task difficulty (3, 18), perceived threat (6), and attention (19). However, such within-subjects investigations cannot establish whether the N1 is suitable for distinguishing between individuals, which is necessary for most clinical applications. More recently, between-subjects investigations have linked the N1 to individual differences in balance (18) and cognitive (13) abilities, but have yet to assess the reliability of the N1 as a stable measure of individual differences in reactive balance control, a necessary step towards optimizing experimental designs in basic science and for advancing its potential clinical utility as an electrophysiological biomarker of brain function related to balance health.

Stability of the N1 over time, assessed using test-retest reliability, is a necessary precursor towards developing a monitoring or prognostic tool that could guide clinical decision making (21). Since the N1 reflects both sensorimotor (1, 2) and cognitive processes (4–6), changes in the N1 could provide mechanistic insight into detection of sensorimotor and/or cognitive problems. Further, since the N1 is altered with aging (22) and neuropathology (15) and correlates with behaviors that relate to balance impairments and fall risk (13–15), the N1 holds promise for quantifying age- or disease- related neurophysiological changes that cannot be captured with existing clinical tools. Altered cortical engagement is often inferred based on performance of behavioral assessments, such as dual-task interference (23, 24). However, many clinical assessments of balance are coarse, subjective, and have floor or ceiling effects (13, 25, 26), limiting our ability to stratify individuals for personalized rehabilitation, track progression over time, or evaluate treatment response. Given that changes in brain activity may precede behavioral declines (27), the N1 could represent an early marker of future balance impairments, fall risk, or cognitive decline that could guide preventative strategies. If the N1 is stable over short timescales (i.e., days to weeks), it could be useful for monitoring immediate responses to balance training or rehabilitation interventions, providing mechanistic insight into treatment efficacy. If it remains stable over longer periods (e.g., months to years), it could serve as a longitudinal biomarker for tracking neurophysiological decline or recovery, particularly in individuals at risk for falls, such as older adults (28), Parkinson’s disease(29), or after stroke (30).

One practical consideration for measuring an evoked brain response is the number of trials that are needed to obtain a reliable measurement. Evoked brain responses are typically assessed from averages across trials that are assumed to contain an identical evoked response contaminated by noise and non-time locked neural activity. Averaging across trials preserves the evoked response while reducing the noise that would otherwise confound the measurement of the N1. In theory, more trials will always result in less noise and therefore a more precise estimate of the N1 response. However, in practice, there are diminishing returns to additional trials (31–33), which limit the number of conditions that can be assessed, and can introduce additional confounds such as fatigue, changes in attention, or habituation of the evoked brain response (8, 9, 17). Further, from a clinical perspective, there are often time constraints and other considerations (e.g., limited strength, poor balance) that necessitate shorter testing sessions. Internal consistency reliability is a simple method of measuring the consistency of responses across trials to estimate the proportion of the final outcome measure that comes from the N1 relative to the noise that may remain after averaging, and can therefore be used to provide empirically supported practical guidelines on the minimum number of trials needed to obtain a reliable estimate of the N1.

The N1 is often quantified using mobile-brain body imaging (MoBI) approaches that rely on high-density EEG and advanced preprocessing (34), but for clinical applications, a biomarker should be accessible, quickly measured, and easily interpreted. While MoBI approaches were developed to handle the increased frequency and variety of artifacts present during whole-body movement, such as running, it is unclear whether a MoBI approach is necessary to reliably measure the N1. The time required for extensive electrode cap setup (e.g., ≥ 64 channels) and complexity of data cleaning approaches that have yet to be standardized or automated pose significant barriers to clinical translation. While some studies quantify the N1 with a minimal approach using a limited electrode set and simple preprocessing (12, 22, 35) that could be clinically feasible, no study to date has compared this to a full MoBI approach, which could involve channel- and component-based outcomes, and whether their reliabilities differ.

Here, we characterized the test-retest reliability and the internal consistency reliability of the N1 elicited by standing balance perturbations in younger and older adults over a one-week interval. A subset of older adults also completed an initial testing session approximately one year prior to enable comparison between short- versus long-term test-retest reliability of the N1. We also examined whether a minimum approach could reliably assess the N1 by comparing two approaches that differ in the number of electrodes and complexity of data preprocessing. These sessions also involved behavioral assessments that have previously been linked to N1 responses (14, 18). Therefore, we also examined the test-retest reliability of behavioral assessments and whether they are sensitive to practice effects.

2. Materials and Methods

2.1. Participants and Experimental Design

10 younger adults (YA) (24 ± 2 years [mean ± standard deviation], 6 identified as female) and 14 older adults (OA) (70 ± 5 years, 6 identified as female) participated in the main experiment involving two testing sessions separated by an interval of 1–7 days (YA: 3.9 ± 2.6 days; OA: 3.5 ± 2.5 days) at the same time of day to control for potential circadian and lifestyle influences. A subset of older adults (n=12, age: 71 ± 5 years, 5 identified as female) also completed one testing session that took place ~1 year prior to the main experiment (1.18 ± 0.51 years) (Figure 1A). One of these older adults withdrew after the first session of the main experiment due to fatigue and soreness. A second older adult had a longer delay between the two sessions (10 days) due to a fall that happened after the first session. Older adults were included if they were over age 60, had cognitive ability to provide informed consent, could stand without an assistive device for 10 minutes, had no recent history (< 6 months) of pain or injury to their back or lower extremities, and had no history of neurological conditions such as stroke or Parkinson’s disease. The experimental protocol was approved by the Emory University Institutional Review Board, and all participants gave written informed consent before entering the study.

Figure 1.

A. In the main experiment, younger adults (YA) and older adults (OA) completed two testing sessions within a one-week timespan. A subset of OA also had an initial session ~1 year prior to the main experiment. B. i) For each session, participants completed a two-alternative forced choice whole-body motion perception task while EEG was recorded. They were asked to discriminate the direction of a pair of perturbations. The first perturbation was directly backwards, while the second perturbation was backwards with a lateral component (∆θ) that was adapted online depending on their response. After their verbal response, the platform returned forward. EEG from the first perturbation in each pair was used for analysis (dashed rectangle). ii) Perturbation kinematics of the 1st perturbation C. N1s were compared between sessions using test-retest reliability (i) and within a session using internal consistency reliability (ii). Vertical dashed line denotes perturbation onset. D. Perceptual and balance ability were compared between sessions using test-retest reliability. i) Exemplar YA perceptual data for leftward and rightward platform directions. Threshold was defined as the angle at which participants correctly perceived as “different” on half of the trials. Acuity was defined as the range of angles with correct accuracy between 25–75%. ii) Balance ability was quantified as the normalized distance traversed across a narrowing beam, averaged across 6 trials.

For each session, participants first performed a beam walking task to assess balance ability (Figure 1Dii). They were then prepped with EEG and exposed to a series of balance perturbations that were used to elicit N1 brain responses (Figure 1C) and assess perceptual ability (Figure 1Di). Measurement reliabilities were then quantified in terms of how consistent outcome measures were across sessions (test-retest reliability) and across measurements within sessions (internal consistency) as detailed below.

2.2. Standing balance perturbations

Participants stood barefoot on a platform with their feet hip width apart and arms across their chest. EEG was continuously recorded during standing balance perturbations. They were exposed to translational support-surface balance perturbations and asked to recover their balance without taking a step. Participants first warmed up with a block of 16–22 perturbations (single ramp-and-hold, 5.1–7.5 cm displacement, 0.8–1 m/s²) evenly divided between forward and backward directions with their eyes open, followed by a seated rest break for 10 minutes. They then underwent a two-alternative forced choice perceptual task in which they discriminated whether a pair of perturbations (each 7.5 cm displacement, 15 cm/s peak velocity, 0.1 m/s² peak acceleration, ~650 ms duration) was in the same or different direction with their eyes closed (Figure 1B) (36–38). The perturbations were small enough to not elicit stepping responses.

Each trial of the perceptual task consisted of a pair of backward perturbations separated by 5 seconds, a verbal perceptual response (“same” or “different”) after the second perturbation ended, and a slow return of the support surface directly forward. The first perturbation occurred at unpredictable timing and was always straight backwards, and the second perturbation deviated slightly from backwards with a lateral component (Δθ). Δθ of the second perturbation was adapted online using the psi method implemented through the Palamedes toolbox in MATLAB (39), with leftward and rightward angles interleaved ranging from 0° to 40° in either direction. Up to 60 pairs of perturbations were delivered, with a 5-minute rest break every 15 trials or sooner if requested by the participant. Two older adult participants were unable to complete the full perturbation protocol due to fatigue and only completed 40 pairs of perturbations.

2.3. Behavioral assessments

Perceptual ability was assessed in each session using a psychometric function that fit the accuracy of their responses across the range of angles tested. Separate models were performed for leftward and rightward angles. Threshold was defined as the angle at which participants were equally likely to report same or different (Fig 1D, i. – vertical dashed line). Acuity was defined as the interquartile range, or range of angles that spanned 25%–75% accuracy (Fig 1D, i. – shaded rectangle) (37).

Balance ability was assessed using a modified version of the narrowing beam walking test (40). The beam was 9.15 m in length, divided into five fixed-width segments that were each 1.83 m in length. The beam started wide (18.6 cm) and became progressively narrower in discrete intervals (2 cm) (Figure 1D, ii.). At the beginning of the session prior to EEG setup, participants walked across the narrowing beam with their arms across their chest and one foot in front of the other with their eyes open. They started with one foot on the beam and one foot on the floor, and were told to walk as far as they could without losing their balance (i.e., uncrossing their arms or stepping off the beam). Distance traversed was measured as the position of the front of the toes of the last foot on the beam when they uncrossed their arms or stepped off the beam. Balance ability was computed as the average distance traveled normalized to the full beam length. Balance ability was assessed in all sessions for all older adults and for half of the younger adults, but only in one session for the remaining younger adults.

At the end of each session, participants rated their quality of sleep, attention, and fatigue on a Likert scale from 1–10 to determine any transient state-related changes that may influence test-retest reliability.

Participants completed Part A and B of the Trail Making Test as a pen and paper assessment of cognitive set-shifting ability (41, 42). Part A involves drawing lines to connect sequential numbers (1–2–3–4), while Part B involves alternating numbers and letters in sequential order (1-A-2-B) without lifting the pen. Set-shifting ability was computed as the difference in completion time between Part B and Part A. Given known practice effects on the Trail Making Test when retested in a short interval (43), this was only completed on the second visit of the main experiment. OAs also completed the Trail Making Test one year prior, a sufficient time frame from the main experiment to minimize any practice effects (44).

2.4. EEG acquisition and analyses

EEG was continuously recorded during standing balance perturbations (sampling frequency 1000Hz, impedance <25kOhm) using a 64-channel active electrode cap (actiCAP) and ActiCHamp amplifier with a 24-bit A/D converter and an online 20 kHz anti-aliasing low-pass filter (Brain Products, GmbH). Fz was the online reference electrode and Fpz was the ground electrode.

EEG analyses were performed in EEGLAB and based on the BeMoBIL Pipeline, a standardized toolbox that is optimized for mobile brain/body imaging data (45). Continuous data from the initial practice block and perception block were merged, then downsampled to 250 Hz. BeMoBIL preprocessing consisted of line-noise removal using the zapline plus plugin (46) and bad channel detection and removal using the cleanrawdata plugin. Since the cleanrawdata plugin uses a random initialization, it was run 20 times and for each iteration identified channels that had correlation with nearby channels < 0.7. A channel was removed if it was flagged as “bad” on more than 7/20 iterations. Bad channels were interpolated and then full-rank average referenced that recovered the online reference (Fz). Preprocessed data were then high-pass filtered, as required for Adaptive mixture component analysis algorithm (AMICA) used to identify maximally-independent components (ICs) (47). Given the discrete nature of the perturbations and variable rest breaks in between trials for participants, AMICA was run on epoched data from −2 to 2 seconds around each perturbation. Since AMICA performance improves with larger amounts of data, we included epochs from the practice blocks, and both the first and second perturbation of each trial pair in the main testing block. However, only perturbations from the first perturbation in each trial pair were used in statistical analyses. ICs were fit to a standard head-model using MNI template, with equivalent current dipoles estimated using the DIPFIT plugin (48). ICs were removed using the following criteria: any ICs with high residual variance (>20%), ICs that were identified by ICLabel as “eye”, “heart”, or “channel noise”, ICs in which the linear slope of the power spectrum between 2–40 Hz was > −0.33, as commonly seen in muscle artifacts (49) and any additional ICs labeled as “bad channels” using the TESA toolbox (50). Finally, powpowcat was used to visually inspect the remaining ICs, to further distinguish brain and muscle ICs (51–53). The resulting solution was applied to the BeMoBIL preprocessed but unfiltered, continuous data in channel space. In the final stage of preprocessing (Figure 2, “Shared preprocessing”), continuous data were referenced to the mastoids, high-pass filtered with a cutoff of 0.5 Hz, low pass filtered with a cutoff of 20 Hz, epoched −2 to 2 seconds, with baseline subtraction −150 to −50 msec (10).

Figure 2.

Schematic of different approaches for analyzing N1 amplitude and latency from Cz electrode. Despite differences in the number of electrodes and preprocessing complexity and time required, the mean ERP trace generated from each approach was largely overlapping.

Prior work has shown that N1s at the Cz electrode have excellent internal consistency using a reduced channel montage and a simple preprocessing pipeline (without AMICA) (12). In order to replicate these findings in two independent cohorts, we also characterized the N1 using a minimal approach that was performed with only 3 electrodes (Cz and two mastoids) and simple preprocessing, skipping steps involving the BeMoBIL pipeline and ICA (Figure 2, orange bypass). Preprocessing was identical to that of the final stage of the full approach (“Shared preprocessing”).

We identified the peak N1 amplitude (largest peak negative response) and latency (time of peak negative response) based on the most negative local minimum between 120–200 msec in YA, or 120–300 msec in OA. between 100–300 msec post-perturbation. While most N1s can be identified within 200 msec (18, 37), we extended the upper time bound to 300 msec for older adults since their N1s can be delayed (12, 15, 22). For ERPs without a local minimum, the most negative value in the time window was used. For ERPs with two negative peaks (e.g., Fig 5B), we chose the most negative local minimum. Outliers were identified and removed for any trials that exceeded a threshold above or below 3 standard deviations of the mean ERP for 10 consecutive data points between −150 to 300 msec. Across all participants, 3.5% of all trials were removed.

Figure 5.

Reliabilities as a function of trial number and approach. A. Exemplar ERPs for younger adults (YA) and older adults (OAs) obtained after averaging across the first 6 trials, 12 trials, or 24 trials for each session. Circles denote peak N1 response that was used to quantify N1 amplitude and latency. YA exemplars and one OA exemplar had one prominent negative peak that was consistent across trials. Another OA exemplar had two different negative peaks which gave rise to different latencies depending upon the number of trials used in the average ERP (e.g., first peak for T₂ and second peak for T₁ and T_{-1 year} with ERPs averaged from 6 trials compared to first negative peak at all time points with ERPs averaged from 24 trials). Vertical dashed lines denote perturbation onset. B. Test-retest reliability across one week and one year for N1 amplitude and latency with increasing trial numbers for both the minimal (dashed line) and full approaches (solid line). Black horizontal line denotes threshold for excellent test-retest reliability (ICC>0.9). C. Internal consistency obtained using split-half reliability with increasing trial numbers for N1 amplitude and latency comparing the full approach (solid line) versus minimal approach (dashed line) for each session. Black horizontal line denotes threshold for excellent internal consistency reliability (r>0.9).

2.5. Statistical analyses

N1s were first analyzed using the full approach (Figure 2, green arrows), where amplitude and latency were extracted from the average of all single-trial epochs for each participant and session. To test for group differences in the N1 across sessions, we ran a linear mixed effects model with factors of Group (YA, OA) and Session (T₁, T₂). Separate models were run for N1 amplitude and latency.

Test-retest reliability of N1 amplitude and latency were measured using the intraclass correlation coefficient (ICC) estimated with a 2-way mixed effects model with absolute agreement (54). Separate analyses were performed for each group and testing interval (one week – T₁ vs T₂; one year – T₁ vs T_{-1 year}).

Internal consistency was assessed with split-half reliability (12). Specifically, N1 amplitudes and latencies were measured from waveforms created by averaging across even and odd subsets of trials within each session for each individual. The consistency of these measures across the even and odd trial subsets was measured (within each group and session) using Pearson’s correlation coefficient corrected by the Spearman-Brown prophecy formula.

Because the full approach is time consuming, requires large amounts of data, and requires expertise that may not be available in a clinical setting (Figure 2, green arrows), we tested whether N1s can be reliably estimated using a minimal approach (Figure 2, orange arrows). We compared N1 amplitudes and latencies obtained using a full approach (64 channels, advanced preprocessing) compared to a minimal approach (3 channels, simple preprocessing) using Pearson correlations. Separate correlations were run for each group and session. All reliability measurements assessed with the full approach were repeated with the minimal approach.

We additionally assessed test-retest reliabilities and internal consistencies as a function of the number of trials included in analyses to provide empirically supported guidelines for determining the number of trials necessary to obtain a reliable measurement of the N1. Analyses were performed iteratively using limited subsets of initial trials (i.e., using only the first 2, 4, 6, etc. trials in the average ERP) to assess the minimum number of trials needed to obtain a reliable estimate of N1 between testing sessions (test-retest reliability) and within testing sessions (internal consistency reliability) (32, 55). These experiments were designed to collect 60 trials in all individuals, but some individuals were unable to complete the full series due to fatigue. Rather than excluding individuals from these analyses beyond the number of trials they were able to complete, which may overestimate reliabilities, we chose to carry the measurements from the largest number of trials available for those individuals through the analyses up to 60 trials for a more conservative estimate of reliabilities that may be obtained from experiments designed to collect up to 60 trials. These analyses can be used to provide precise guidelines on the minimum number of trials needed for a reliable estimate of the N1 within any experimental condition.

Given several research studies involving high-density EEG analyze a single source that contributes most to the N1, we also performed a parallel analysis on the source that contributes largest to the N1 potential, examining both its test-retest reliability and internal consistency. We further examined the stability of the source that was selected across sessions by computing a correlation of the scalp topography plots for each participant at T₁ compared to T₂, and between T₁ compared to T_{-1 year.}

Additionally, we assessed the test-retest reliability of balance ability (normalized beam walking distance) and perceptual ability (threshold and acuity). To examine their sensitivity to practice effects across testing sessions, we also ran paired t-tests comparing performance at T₁ compared to T₂, and between T₁ compared to T_{-1 year.}

We used commonly reported cutoffs for the reliability coefficients (56); values < 0.5 indicate poor reliability, values 0.5–0.75 indicate moderate reliability, values 0.75–0.9 indicate good reliability, and values > 0.9 indicate excellent reliability. Analyses were performed using MATLAB and R. Values are reported as mean ± SD unless otherwise stated.

3. Results

3.1. N1s are smaller and slower in older adults compared to younger adults

Regardless of session, N1s were smaller in older (26.4 ± 9.8 µV) compared to younger adults (44.2 ± 19.3 µV) (p<0.001). Further, N1s were later in older adults (206.6 ± 35.3 ms) compared to younger adults (159 ± 14.4 ms) (p<0.001) (Figure 3i – group ERPs). Each group had high variability between individuals, evidenced by wide shaded regions in group ERP waveforms, for N1 amplitude (younger: 17.5 – 82.0 µV; older: 12.0 – 44.0 µV) and latency (younger: 142 – 186 ms, older: 162 – 282 ms).

Figure 3.

Test-retest reliability across sessions. Perturbation-evoked potentials for YA (A) and OA (B) at the group level (i) and individual level (ii). T₁ and T₂ separated by ~one week. T₁ and T_{-1 year} separated by ~one year. Solid line represents the mean derived from all trials, and shaded region represents the standard deviation. Vertical dashed line denotes perturbation onset. Test-retest reliability for N1 amplitude (left column) and latency (right column) across a one-week (C) and one-year time span (D). Dashed diagonal denotes line of equality. Full ICC results are presented in Table 1.

3.2. N1 amplitude and latency have excellent test-retest reliability in YA and OA

Perturbation-evoked potentials varied across individuals but reflected stable cortical activity within an individual across a one-week timespan in younger and older adults (Figure 3 – overlapping blue versus pink traces) and across a one-year timespan in older adults (Figure 3B – overlapping gray traces). ICC analyses confirmed excellent test-retest reliability (ICC>0.9) for N1 amplitude and latency across a one-week (Figure 3C) and one-year time span (Figure 3D, YA – circles; OA – squares; dashed diagonal denotes line of equality (y=x)). Summary statistics including the ICC and corresponding 95% confidence interval (CI), which were computed separately for each group and retest interval, are summarized in Table 1.

Table 1.

Test-retest reliabilities and internal consistencies for N1 amplitude and latency (Cz electrode)

			Test-retest (ICC, 95% CI)		Internal consistency (r)
N1 Outcome	Group	Approach	T-1year vs T1 (one year)	T1 vs T2 (one week)	T-1 year	T1	T2
Amplitude	YA	full	–	0.980 (0.923, 0.995)	–	0.990	0.995
		minimal	–	0.989 (0.959, 0.997)	–	0.991	0.992
	OA	full	0.938 (0.784, 0.982)	0.976 (0.922, 0.993)	0.981	0.989	0.968
		minimal	0.875 (0.581, 0.964)	0.979 (0.933, 0.993)	0.963	0.943	0.965
Latency	YA	full	–	0.942 (0.775, 0.985)	–	0.973	0.976
		minimal	–	0.901 (0.619, 0.975)	–	0.981	0.876
	OA	full	0.984 (0.946, 0.995)	0.965 (0.888, 0.989)	0.996	0.952	0.989
		minimal	0.873 (0.573, 0.963)	0.948 (0.827, 0.984)	0.925	0.778	0.835

N1 reliabilities were obtained from the average of all trials at the Cz electrode in each session. Full approach – 64 electrodes, advanced preprocessing. Minimal approach – 3 electrodes, simple preprocessing. One-year retest only completed in older adults (OA). One-week retest completed in younger adults (YA) and OA. ICC - intraclass correlation coefficient.

3.3. N1 amplitude and latency have excellent internal consistency reliability in YA and OA

Perturbation-evoked potentials appear similar across individual trials (Figure 1C, exemplar YA and exemplar OA). Split-half reliability comparing even vs. odd trials was excellent (r>0.9) for both N1 amplitude and latency for both groups and all sessions (Table 1).

3.4. N1s obtained with a minimal approach are similar to those obtained with the full approach

N1 amplitude and latency extracted from the the full and minimal approaches were similar for each session and group (Figure 4A – exemplar participants with overlapping ERP waveforms for the minimal (orange) and full (green) approach. Figure 4B – all participants. YA-circles, OA- squares; gray- T_{-1 year}, blue – T₁, pink – T₂). Indiviudals with larger N1 amplitudes using the minimal approach had larger N1 amplitudes using the full approach (r= 0.98, p< 0.001; Figure 4B, left). Similarly, indiviudals with slower N1 responses using the minimal approach had slower N1 responses obtained with the full approach (r= 0.93, p<0.001; figure 4B, right). Correlations comparing N1 amplitude and latency between the different approaches for each session and group were all significant (p<0.002; Table 2).

Figure 4.

Comparison between N1s derived from the full approach (64 electrodes, advanced preprocessing) versus minimal approach (3 electrodes, simple preprocessing). N1s extracted from Cz electrode. A. ERPs from the minimal approach (orange) compared to the advanced approach (green) for exemplar YAs and exemplar OAs are highly overlapping. Each row within a group corresponds to ERPs from the same individual on different sessions. B. N1 amplitude (left) and latency (right) using the minimal approach were correlated with those from the advanced approach.

Table 2.

Pearson correlations comparing N1s obtained using the minimal versus full approach

	Younger Adults		Older Adults
	T1	T2	T-1 year	T1	T2
N1 amplitude	r=0.994, p<0.001	r=0.960, p<0.001	r=0.946, p<0.001	r=0.962, p<0.001	r=0.981, p<0.001
N1 latency	r=0.931, p<0.001	r=0.981, p<0.001	r=0.978, p<0.001	r=0.857, p<0.001	r=0.890, p<0.001

3.5. Excellent test-retest reliability and internal consistency were found for N1 amplitude in relatively few trials for both the minimal and full approach

Excellent test-retest reliability (ICC>0.9, Figure 5B left) and internal consistency (r>0.9, Figure 5C left) was generally achieved for N1 amplitude within 6 trials for YA and OA for both the minimal and full approaches. However, more trials were generally required to achieve between good-to-excellent reliabilities for N1 latencies (Figure 5B, C - right), particularly in older adults, as different peaks could be identified across trial subsets within individuals (Figure 5A, OA exemplar bottom row).

3.6. The source with the largest contribution to the N1 response has excellent test-retest and internal reliability

Similar results were obtained for component-based N1 measurements of amplitude and latency (Table 3, Figure 6). A stable component was identified within an individual across one week (YA: r = 0.98 ± 0.01, OA: r = 0.95 ± 0.06) and one year (r=0.93 ± 0.06) [mean correlation ± SD], with a typical mid-central scalp topography in nearly all participants. Interestingly, two older adults had distinct but stable scalp topographies (Supplemental Figure).

Table 3.

Test-retest reliabilities and internal consistencies for source-level N1 amplitude and latency

		Test-retest (ICC, 95% CI)		Internal consistency (r)
N1 Outcome	Group	T-1year vs T1 (one year)	T1 vs T2 (one week)	T-1 year	T1	T2
Amplitude	YA	–	0.984 (0.937, 0.996)	–	0.997	0.996
	OA	0.890 (0.328, 0.973)	0.964 (0.880, 0.989)	0.988	0.978	0.998
Latency	YA	–	0.969 (0.880, 0.992)	–	0.979	0.993
	OA	0.953 (0.839, 0.987)	0.973 (0.912, 0.992)	0.992	0.994	0.998

N1 reliabilities were obtained from the average of all trials in each session. Amplitudes and latencies extracted from the source that contributed largest to the N1 response. One-year retest only completed in older adults (OA). One-week retest completed in younger adults (YA) and OA. ICC – intraclass correlation coefficient.

Figure 6.

Test-retest reliability across sessions for component-based N1 measurements. Component determined based on the largest-contributing source to the N1 response. Perturbation-evoked potentials for YA (A) and OA (B) at the group level. T₁ and T₂ separated by ~one week. T₁ and T_{-1 year} separated by ~one year. Solid line represents the mean derived from all trials, and shaded region represents the standard deviation. Vertical dashed line denotes perturbation onset. Test-retest reliability for source N1 amplitude (left column) and latency (right column) across a one-week (C) and one-year time span (D). Dashed diagonal denotes line of equality. Full ICC results presented in Table 3.

3.7. Perceptual and balance ability have moderate to excellent test-retest reliability in YA and OA but may be sensitive to practice effects

Test-retest reliabilities for behavioral outcomes (Table 4) were generally lower than those of N1 amplitude and latency (Table 1). Since only half of the YA group had a retest for the beam walking test, we only calculated ICC in OAs. Balance ability had good test-retest reliability for both retest intervals (Figure 7, left column). As shown previously (37), perceptual thresholds and acuity varied across individuals but did not differ between leftward and rightward directions for either session (all p>0.38), and were therefore averaged prior to ICC analysis. Perceptual threshold in both groups and re-test intervals had good test-retest reliability (Figure 7, middle column). Perceptual acuity had moderate to excellent reliability depending upon group and re-test interval (Figure 7, right column). Regardless of group, beam score, perceptual threshold, and perceptual acuity were better at T₂ compared to T₁ (p=0.01, p= 0.04, p= 0.01, respectively). Beam score, perceptual threshold, and perceptual acuity were not different between T_-1year and T₁ (all p>0.13). Full summary statistics, including ICC and associated 95% CI for each group and re-test interval are shown in Table 4.

Table 4.

Descriptive results and test-retest reliabilities for balance ability and perceptual ability.

	Younger adults			Older adults
	T1	T2	ICC (95% CI)	T-1 year	T1	T2	ICC - one week (95% CI)	ICC - one year (95% CI)
Balance ability	0.75 ± 0.1	0.78 ± 0.1	N/A	0.45 ± 0.1	0.44 ± 0.1	0.48 ± 0.1	0.89 (0.707–0.958)	0.84 (0.552–0.941)
Perceptual threshold	6.4 ± 1.7	5.7 ± 2.6	0.80 (0.415–0.937)	9.9 ± 6.5	9.9 ± 5.6	8.5 ± 6.7	0.89 (0.706–0.959)	0.89 (0.702–0.960)
Perceptual acuity	4.5 ± 1.5	3.3 ± 1.6	0.67 (0.023–0.893)	6.5 ± 2.7	7.6 ± 3.5	6.7 ± 3.7	0.90 (0.746–0.964)	0.69 (0.190–0.886)

Reliabilities were calculated across a one-week timespan in younger (YAs) and older adults (OAs), and across a one-year time span in OAs. ICC was not calculated for balance ability in YAs since only five YAs had balance assessment at T₂. Balance – higher scores denote better balance. Threshold and Acuity – lower scores denote better perception. Values at T₁, T₂, and T_-1year reported as mean ± SD. ICC – intraclass correlation coefficient.

Figure 7.

Test-retest reliability for behavioral metrics across a one-week timespan (A) and a one-year timespan (B) (younger adults – blue circles, older adults – yellow squares). Dashed diagonal denotes line of equality. One OA had substantially worse perceptual threshold at T₁ relative to T₂, which may have reflected greater self-report of fatigue and soreness after T₁. Full ICC results are presented in Table 4.

DISCUSSION

Perturbation-evoked N1 potentials obtained during a behaviorally-relevant reactive balance task reflect stable individual differences in younger and older adults that can be reliably estimated in as few as six trials. This is the first study to assess test-retest reliability of the N1 response to a balance disturbance, finding individual differences are stable over intervals as long as one year, suggesting the N1 may be suitable for longitudinal investigations of an individual’s functional status. Results also provide precise empirical guidelines for determining the number of trials necessary to obtain a reliable estimate of the N1, and the extent to which that estimate improves with additional trials. We further demonstrate the clinical feasibility of a quick and simple measurement of the N1 by showing that a minimal approach (3 electrodes, simplified preprocessing) results in nearly identical outcome measures (r>0.93) to computationally intense and time-consuming analyses that are standard in MoBI. The excellent reliability for both channel (Cz) and component-based N1 measurements reinforces the robustness of the N1 for use in both clinical and research settings. The excellent stability of the N1 within and between sessions support its usage in individual differences research for determining its potential as a biomarker of balance health and for probing mechanisms underlying interventions.

Individual differences in N1 amplitude and latency were stable across time scales in younger and older adults, reflecting trait-specific cortical activity that could serve as a biomarker of balance health (57). The later and smaller N1s in older adults compared to younger adults are stable within an individual over a one-year time span, demonstrating its potential use in longitudinal studies tracking age-related or disease-related changes in behavior. Our findings strengthen the interpretation of prior brain-behavior associations that linked individual differences in the N1 to balance ability, balance confidence, and set-shifting ability (13, 14, 18), metrics that have been implicated in fall risk (41). Obtaining a stable marker of brain activity that can be mapped to behavior has important implications for guiding clinical decision making. For instance, if changes in brain activity were to precede behavioral declines, the N1 could be used as a monitoring tool to recommend earlier interventions to prevent behavioral decline. Further, since the N1 is stable over time in the absence of any intervention, any changes in the N1 after an intervention could indicate a reliable change in a targeted mechanism rather than random fluctuations or a noisy measurement.

The N1 is known to be modulated by state-related factors such as arousal and fear (6, 58), so at first glance the excellent test-retest reliability could be surprising. However, within an individual, the sessions were scheduled at the same time of day to minimize any potential effects of circadian rhythm (59), participants were asked to maintain their normal level of caffeine intake, and the experimental environment was kept as similar as possible across sessions. Although some adults reported large changes in quality of sleep (± 6 out of 10 Likert scale) and hours of sleep (± 4) between sessions, their N1s remained stable, suggesting that any factors that are outside of experimental control and could affect the N1 are likely small relative to the differences in N1 between individuals.

Unlike balance and perceptual ability, the N1 was less susceptible to practice effects across sessions, and therefore may represent a more precise and stable index of an individual’s balance health. Nearly all younger and older adults improved their balance and perceptual performance over a one-week timespan, changes that are likely a reflection of task exposure rather than a true change that meaningfully represents the current state of their balance health. Such practice effects are frequently reported on clinical metrics (25, 26, 43),and complicate interpretations related to the efficacy or effectiveness of an intervention. In contrast, since the N1 was highly consistent regardless of task exposure with repeated trials or between testing sessions, the N1 has the potential to be a more objective probe of their neurophysiological function for assessing an intervention’s effectiveness at a mechanistic level that could then serve to guide future clinical decision making. Besides practice effects, such behavioral metrics may be sensitive to state-related changes that could vary across sessions. For instance, one older adult reported pain and fatigue in one session but not the other, and presented with a substantial increase in perceptual threshold (nearly double) in that session despite nearly identical N1s. Lastly, some behavioral tests may have floor or ceiling effects and/or little variability between individuals, limiting their potential test-retest reliability. Here, perceptual acuity in the majority of younger adults clustered between 2–5 degrees, likely contributing to the lower ICC for acuity.

Our findings replicate and extend prior work that found excellent internal reliability for N1 amplitude elicited by support-surface translations in younger and older adults (12) by demonstrating that excellent reliability can be achieved in relatively fewer trials, for both a minimal approach, similar to prior work (12), and a full approach commonly used in mobile brain/body imaging (MoBI) research (34). Most MoBI research utilizes high-density EEG and advanced preprocessing pipelines to deal with the increased frequency and types of artifacts present during whole-body behaviors, such as head movement and muscle activity (45, 60, 61) Since the head doesn’t move until after the N1 in support-surface translations (10, 12), and the N1 is tightly linked to power in the theta band (3, 62), standard MoBI approaches may be less critical for quantifying the N1, and could hinder clinical translation. Based upon the correlation coefficients comparing N1s obtained from the full vs minimal approach (0.93–0.98), the largest negative impact of choosing the minimal approach would be a 2–7% reduction in the maximum possible correlation between the N1 and other behaviors of interest, which comes with several potential benefits. The minimal approach requires only 3 electrodes and is not dependent upon the number of trials, while the advanced approach requires a full-electrode cap setup and large amounts of data to increase the quality of blind source separation (e.g., AMICA). The ability to obtain a reliable estimate of the N1 with fewer trials offers flexibility in settings that have time constraints or involve populations who are vulnerable to fatigue, while also reducing potential confounds that may occur with longer data collections such as shifts in attention. Finally, the minimal approach can quantify N1s within minutes without extensive training using a fully automated pipeline, and the full approach requires several minutes to hours, including elements of subjective decisions (e.g., IC labeling of brain versus non-brain). However, there may be scenarios in which the full approach is needed, such as source-based analysis or for analyzing power in frequency bands that are known to overlap with muscle artifact, such as beta and gamma power. Further, it’s possible that the full approach is needed in paradigms that elicit larger head movements and/or head movements that occur before the N1. To what extent a minimal approach can be used for other brain metrics or other paradigms that elicit an N1 will require further investigation.

Though test-retest and internal reliability were excellent for N1 latency across all trials, reliabilities for N1 latency were generally lower and, unlike N1 amplitude, did not uniformly converge with increasing trial numbers, consistent with prior literature examining ERPs in the cognitive domains (e.g., the error related negativity) (63–65). The relatively lower, though still good-to-excellent reliability for N1 latency reflects how some individuals, particularly older adults, have multiple negative peaks during the N1 time window, that could result in different peaks being selected depending upon the subset of trials analyzed (e.g., Figure 5A, bottom row), producing large differences in latency despite similar amplitudes. Two consecutive negative peaks separated by < 100 msec has previously been reported in older adults (22), but there is no consensus on the best approach for analysis. Here we chose the most negative peak, even if there was a smaller preceding peak, but it is possible that another method of peak selection could result in more reliable latencies, such as taking the average of the two largest negative peaks. Another method is manual peak detection, but this comes at a tradeoff of inter-rater differences due to observer biases, time required for training and visual inspection, and reproducibility (66). In the context of developing a biomarker and enhancing replication, we believe that having a robust, automated pipeline outweighs potential benefits that could come with manual methods, even if that means collecting additional trials. Whatever approach is used, reliability will be influenced by how N1s are defined. Given large inter-individual variability in ERP shape, we recommend that future research clearly define their method of peak selection, including the time window and how multiple negative peaks in the time window are handled.

While N1 amplitude achieved excellent reliability within 6 trials, it’s important to consider that reliability is context-dependent and will need to be quantified on a study-by-study basis (67). Specifically, our results may not generalize to clinical populations, to N1s quantified during other types of tasks, or to N1s elicited by other perturbation methods that may result in more prominent artifacts or head movements. However, a recent study in children found excellent internal reliability for N1 responses elicited by lean-and-release perturbations that involved stepping responses (35), suggesting the potential generalization to other paradigms and populations. Unlike test-retest reliability which requires repeated experimental sessions, internal reliability can be quantified and reported in any EEG study. Because reliability limits statistical power, a metric with poor reliability could preclude or falsely magnify group- or task-related differences in brain activity or brain-behavior correlations (67–69). Routine reporting of internal consistency is needed to enhance reproducibility, interpretation of results, and optimize future experimental designs. The minimum number of trials needed for excellent reliability found here is not a hard rule, but is rather meant to serve as a reference for future studies and will need to be evaluated on a study-by-study basis. For instance, if N1 latency is a desired outcome, we suggest that researchers over-estimate the minimum number of trials, based on our current results and prior literature (65). However, if the primary outcome of a study is N1 amplitude, understanding the diminishing returns of additional trials could motivate the inclusion of more experimental conditions rather than excessive repetition of trials within any condition.

Another important consideration in determining reliabilities as a function of trial number is the role of practice and familiarization of the task. Some studies have suggested to remove the first trial (or first few trials) to control for transient emotional changes (e.g., anxiety, arousal) that occur with the novelty of a task (70, 71). One limitation of our study is that we had a practice block of perturbations that differed in several factors from perturbations in the main experiment, so it is unclear if and to what extent practice effects could have impacted N1 reliability. However, the practice block was intentional to allow participants to adapt to the perturbations, prior to adding complexity with a perception task used in the main experiment. For studies in which practice trials are similar to their main paradigm, it would be helpful to quantify any potential practice effects to justify whether initial trials are included or excluded to strengthen the interpretation of their main results of interest. Future research specifically designed to test the role of practice trials is needed to understand their impact on the N1.

Similar to our channel-based (Cz) findings, the amplitude and latency of the component that contributed largest to the N1 response had excellent test-retest reliability and internal reliability, further reinforcing the idea that N1 reflects a stable marker of individual differences. The consistent mid-central scalp topographies observed in nearly all participants aligns with prior studies that localize the N1 to the supplementary motor area (SMA) (3, 37, 72, 73). Interestingly, some older adults had a topography inconsistent with the SMA, which could indicate potential compensatory mechanisms from prefrontal or parietal areas (74, 75). The stability of the scalp topographies within an individual across sessions supports the possibility of using the N1 component to index intervention responses or in longitudinal studies. While analysis of other brain sources involved in reactive balance is beyond the scope of the current study, future work should examine whether multiple negative peaks at the Cz electrode reflect different contributing sources, and if so, how these may vary with aging or neuropathology. Together, our findings enhance the relevance of the N1 for both clinical applications with interest in quick yet precise assessments and for researchers who use high-density EEG with interest in isolating distinct sources in reactive balance.

Conclusion and future directions

The N1 is stable across a timespan of one week and one year, can be reliably estimated with relatively few trials using a minimal approach, and can be quickly quantified within minutes using entirely automated procedures, each of which is critical for translating any biomarker into clinical practice. The current results provide empirical guidance for reliably measuring the balance N1 elicited by support-surface translations. However, as reliability may differ across tasks, settings, and populations, we recommend reporting internal consistencies of N1s in all studies to inform the design of more powerful investigations of the N1. Since N1s are stable in the absence of interventions over a one-year time span in older adults, it may be possible to develop longitudinal trajectories of the N1 to detect changes in functional status prior to the onset of behavioral declines. As prior cross-sectional studies have linked individual differences in N1 to behaviors that predict falls (13, 14, 41), our findings demonstrating its excellent reliability lay the foundation to determine its predictive validity, or extent to which the N1 serves as a biomarker for behaviors of interest (e.g., fall risk).

Supplementary Material

Supplementary Data: Supplemental Fig S1: https://osf.io/c9sey/

New & Noteworthy:

Prior studies identified associations between the perturbation-evoked N1 potential and behaviors that predict falls, suggesting its potential as a biomarker of balance. We show excellent test-retest reliability of the N1 over a year in older adults, and that a reliable measurement can be obtained within 6 trials using simplified methods, demonstrating clinical feasibility. This study also guides the design of more powerful N1 experiments, necessary to identify underlying mechanisms and assess clinical utility.

Data and Code Availability:

Individual data and code (preprocessing, analysis, and plotting scripts) are publicly available through Open Science Framework: https://osf.io/c9sey/