Clinical utility of oculomotor and electrophysiological measures in identifying concussion history

Abstract

Objective

To examine if oculomotor and electrophysiological measures improve the clinical performance of the typical concussion protocol for classifying collegiate athletes with a history of concussion.

Design

Cross-sectional.

Setting

University Athletic Medicine and Research Facility.

Participants

Forty-five varsity collegiate athletes.

Independent Variables

Collegiate varsity athletes with or without a history of a diagnosed concussion.

Main Outcome Measures

Multivariate receiver operating curve and area under the curve (AUC) analyses tested the clinical performance of the typical concussion protocol (symptoms, postural control, neuropsychological abilities). We examined differences in clinical performance between this protocol and after adding reflexive saccade and event-related potential (ERP) indices. Hypotheses were formed after data collection.

Results

Significant AUCs were demonstrated for the typical concussion protocol (Model 1: AUC = 0.75, p = .007), after adding reflexive saccade eye excursion gain (Model 2: AUC = 0.80, p = .001), and ERPs (Model 3: AUC = 0.79, p = .002). The AUC for reflexive saccades and ERPs was significant (Model 4: AUC = 0.70, p = .030). Model 2’s increased clinical performance compared to Model 1 was non-significant, X²(2) = 1.871, p = .171.

Conclusion

All four models demonstrated adequate sensitivity and specificity for classifying athletes with a prior concussion. Adding reflexive saccades and ERPs did not significantly increase clinical performance of the typical concussion protocol. Future research should determine the clinical utility of saccades and ERPs for acute post-concussion assessments.

1.0. Introduction

Concussion and mild traumatic brain injury (mTBI) are associated with oculomotor challenges^1–6 and altered electrophysiological brain responses during cognitive tasks.^7–11 However, neither are included as standard measures in baseline clinical sports-concussion assessment protocols. The typical baseline concussion testing and return-to-play protocols include symptomatology, balance functioning, and neuropsychological performance.^12–14 For example, a battery including traditional neuropsychological tests, postural control, and symptomatology correctly identified approximately 96% of concussed college athletes within 24 hours.¹⁵ McCrea et al¹⁶ reported that college athletes returned to baseline levels of symptoms, postural control, and neuropsychological functioning within 7-days following concussion. Therefore, typical concussion protocols demonstrated successful clinical utility in diagnosing concussions in the acute phase and managing functional recovery on these measures. However, these measures are subjective, vulnerable to errors in self-reporting, and might not predict long-term outcomes following concussion. It is critical to identify objective measures that demonstrate excellent sensitivity and specificity for classifying athletes with a history concussion.

There is growing evidence that objective oculomotor and electrophysiological tests may improve the diagnostic accuracy of the typical concussion protocol. Athletes with a prior concussion demonstrated residual oculomotor impairments, which are unlikely to manifest on typical concussion protocols. For example, patients with mTBI generated impaired memory guided saccades (gain, errors) within 2-days following injury compared to controls¹ and impaired smooth pursuit eye movements from 3–16 days² through 12 months³ following injury. Near point fixation disparity greater than or equal to 15 cm demonstrated a significant cut-off value (AUC of 0.71) for identifying college ice-hockey athletes with a history of concussion.⁶

Concussions are also related to altered electrophysiological responses underlying cognitive processing. Event-related potential (ERP) components are segments of the ongoing electroencephalogram (EEG) that are time-locked to the onset of a stimulus, such as those displayed during cognitive tasks. During an oddball task, college football athletes an average of 4-years post-concussion generated larger P300 ERPs (reflecting increased memory-related attentional resources¹⁷) and delayed latencies (i.e., timing) than non-concussed teammates.¹⁰ However, the two groups of athletes did not perform differently on neuropsychological tests, a finding indicated in other similar research.¹¹

The current study investigated whether objective oculomotor measures and working-memory related ERPs improve the clinical performance of a typical concussion assessment protocol for classifying varsity collegiate athletes with a history of concussion. The measures within the typical concussion assessment protocol included those recommended for concussion management¹²: a symptom checklist, a test of postural control, and four traditional neuropsychological assessments. Two functional measures of oculomotor performance (reflexive saccades, smooth pursuit eye movements) were administered because of the relationship between concussion history and oculomotor deficits.^1–2 ERPs were recorded because they are sensitive to subtle changes in neurocognitive brain activity associated with prior concussion.¹⁰

2.0. Materials and Methods

2.1. Subjects

The current study was a cross-sectional investigation of 45 NCAA participating varsity men’s football and female soccer athletes (18–23 years) who completed testing prior to the their primary athletic season. Additional inclusion criteria included: 1) at least 18 years of age, 2) reported normal/corrected-normal vision (static visual acuity < 20/40), 3) capable of moving head left and right, 4) reported no consumption of alcohol in the past 24 hours, and 5) no reported orthopedic injuries.

The current study focused on the clinical performance of a novel protocol for examining athletes with a prior history of concussion (not an acute concussion). This included previously concussed athletes at least nine months post-concussion. Athletes’ concussion history was initially self-reported^7,8,10,18 and confirmed through athletic medical records. Athletes reported the number of prior concussion(s) diagnosed by a health care provider and the approximate date of those concussions. Athletes with a concussion history (n = 21) experienced their most recent concussion an average of 4.11 years prior to testing (0.81–12.48; SD = 3.87). Additional sample demographics are reported in Table 1.

Table 1.

Sample Demographics (Mean ± SD)

	Conc. History	No Conc. History
Participants	21	24
Age (years)	20.17 ± 1.55	20.03 ± 1.53
Gender (Males, Females)	(19, 2)	(19, 5)
Number of concussions	1.52 ± 0.87	---
Time since most recent concussion (years)	4.11 ± 3.87	---

Group means and standard deviations for demographic variables.

2.2. Measures

Athletes indicated the severity that they were currently experiencing 16-symptoms (e.g., headaches, photophobia, nausea) on a five-point Likert scale (0 = “not experienced at all”, 4 = “a severe problem”) (Rivermead Post Concussion Symptoms Questionnaire, RPSQ).¹⁹ Symptom values were summed to derive each athlete’s current total symptom score.¹⁵

The BESS included three standing balance conditions (feet together, non-dominant-leg only, and feet tandem with non-dominant leg in back). Each condition was performed on a firm surface and foam-pad. Instructions and scoring methods corresponded to recommended guidelines.²⁰ Composite scores reflected the total number of errors committed on the six tests.

A trained researcher administered three traditional paper-and-pencil neuropsychological tests: the Trail-Making Tests (TMT) A and B,²¹ the Wechsler Adult Intelligence Scale-IV Letter-Number Sequencing subtest (LNS),²² and the Color-word Interference-Inhibition (CWI) subtest of the Delis-Kaplan Executive Function System.²³ These tests were collectively chosen to assess task-switching and processing speed,²⁴ attention and working memory,²² and response-inhibition/self-control.²⁵ In depth-description of these assessments are provided elsewhere.¹⁰

Reflexive saccades and smooth pursuit eye movements were examined as objective, quantitative measures of oculomotor functioning.²⁶ Oculomotor tests, including saccades and smooth pursuit eye movements are stable measures both within and between sessions in healthy individuals.²⁷ During both tests, participants were first screened for normal ocular range of motion (congruence), restrictions, or palsies, and then seated 4 feet (+/− 2 inches) from a light-emitting diode (LED) bar in a dark room. Participants wore 2D video eye goggles (videonystagmography ICS Chartr 200; GN Otometrics, Schaumburg, IL, USA). The position of pupil changes over time was calculated (calibration) prior to beginning each task. Athletes were instructed to keep their head in the primary position (looking directly forward) as they followed a red target with their eyes.

Saccadic eye movements for each eye were recorded while participants followed the target presented in random locations on the LED bar. Sixty total target presentations were randomly positioned between subtended arcs of 5 and 30 degrees in the horizontal direction. Intervals between target presentations randomly varied between 1.5 and 2 seconds. The primary outcome measure included the average of right and left eye gain (eye velocity/target velocity) of accepted saccades. Saccade testing lasted 80 seconds and was repeated when necessary (i.e., abnormal performance and/or less than 50% of saccades accepted) to obtain best performance.²⁶

To record smooth pursuit eye movements, the target moved in a sinusoidal pattern from left and right of center with a maximum subtended arc of 30 degrees. The procedure included frequency sweeps from 0.2 to 0.7 Hz that were repeated up to three times to record participant’s best performance. Each frequency cycle lasted 50 seconds. Outcome measures included the average of the leftward and rightward congruency of eye movements with the target (i.e., “gain”). Catch-up saccades (i.e., corrective eye movements) were removed to calculate the gain value.²⁶

ERPs were recorded from a high-density Ag/AgCl 256-electrode channel net while participants performed a 2-back working memory task. Participants were seated 1 meter from a Dell 15.5″ computer screen in a darkened room. Participants viewed individual English letters with a visual angle of 1.16 x 1.16 degrees, which were presented one at a time for 1000 ms (E-prime 2.0; Psychology Software Tools, Inc, Pittsburgh, PA). Time between trials ranged from 1600–2200 ms. Participants pressed two different buttons (counterbalanced between participants) to indicate if the current letter matched or mismatched the letter presented two letters previously. Participants completed 100 total trials (50 Match randomly intermixed with 50 Mismatch). Behavioral accuracy (%) and response time (ms) were recorded for each trial and averaged within condition.

Net Station 4.4.2 software and a 250 Hz sampling rate were used to record the ERPs. Electrode impedances were maintained below 60 kΩ. EEG signals were bandpass filtered offline from 0.3–30Hz and segmented to 900 ms post-stimulus onset with a 200 ms baseline correction. Only correct trials were submitted to analyses. Eye blinks were classified as a voltage shift greater than 150 uV at any electrode during any trial. Trials with eye blinks were removed from analyses. Trials were averaged within each condition (Match, Mismatch) and re-referenced to an average reference.

The ERP measures of interest in the present study included the amplitude (μV) and latency (ms) of the P300 component recorded during Match trials. Research suggests that the individuals with a prior concussion or mild TBI demonstrated altered P300 amplitudes and latencies during attention¹⁰ and working memory tasks.^9,11 Using a temporal principal components analysis, the amplitude of the P300 was determined to be the mean positive deflection within the 284–680 ms time window recorded over a cluster of parietal scalp electrodes.¹⁰ The latency of the P300 was calculated as the temporal occurrence (ms) of the peak positive deflection during this temporal window.

2.3. Data Analysis

Statistical analyses were performed using SPSS version 23 (IBM, Chicago, IL, USA). Differences between concussion history and control groups on all outcome measures were first examined using independent samples t-tests (see Table 2). Second, we used receiver operating characteristics (ROC) curve and area under the curve (AUC) analyses to determine the clinical performance of each individual measure. As shown in Figure 1, the ROC curve is plotted with sensitivity (e.g., % Hit Rate) on the y-axis and false alarm rate (i.e., 1-specificity) on the x-axis. False alarm rate includes the percentage of athletes whose group membership was incorrectly classified. Specificity refers to the percentage of non-concussed athletes correctly classified. The goal of ROC analysis is to maximize the AUC, such that sensitivity is high but false alarm rates are low. Well-performing measures are those that are plotted within the upper-left space of the graph (i.e., high Hit Rate, low False Alarm Rate) and subsequently have a larger AUC.²⁸

Table 2.

Between-groups Comparison of Symptoms, Postural Control, Neuropsychological Performance, Oculomotor Measures, and ERP Responses (Mean ± SD)

	Conc. History	No Conc. History	t-value	p-value
RPSQ	1.71 ± 3.33	2.63 ± 5.01	0.70	.487
BESS	19.00 ± 6.33	21.08 ± 8.37	0.95	.349
TMT A	18.43 ± 5.14	18.86 ± 6.28	0.25	.803
TMT B	49.68 ± 21.19	43.57 ± 16.30	−1.09	.281
CWI Errors	11.05 ± 1.99	9.88 ± 2.85	−1.58	.122
WAIS-LNS	10.76 ± 2.82	10.29 ± 2.79	−0.56	.578
Saccades Gain1	84.61 ± 7.57	88.61 ± 7.45	1.68	.102
SP-0.7 Hz2	0.75 ± 0.12	0.72 ± 0.14	−0.70	.489
SP-0.6 Hz1	0.85 ± 0.10	0.79 ± 0.13	−1.60	.117
SP-0.5 Hz3	0.89 ± 0.08	0.89 ± 0.08	−0.01	.989
SP-0.4 Hz1	0.94 ± 0.04	0.90 ± 0.11	−1.38	.180
SP-0.3 Hz3	0.93 ± 0.06	0.94 ± 0.09	0.25	.801
SP-0.2 Hz4	0.93 ± 0.07	0.96 ± 0.10	0.90	.374
P300 Amplitude	3.79 ± 1.50	3.49 ± 2.03	−0.57	.569
P300 Latency	443.54 ± 73.04	438.47 ± 78.84	−0.22	.825

RPSQ = Rivermead Post Concussion Symptoms Questionnaire; BESS = Balance Error Scoring System; TMT A = Trail-Making Test A; TMT B = Trail-Making Test B, CWI Errors = D-KEFS Color Word Interference Subtest (number of errors scaled score); WAIS-LNS = WAIS-IV Letter Number Sequencing (scaled score); SP = Smooth Pursuit Eye Movements.

¹5 subjects missing because of unrecordable eye movements;

²4 subjects missing because of unrecordable eye movements;

³3 subjects missing because of unrecordable eye movements;

⁴2 subjects missing because of unrecordable eye movements. Eye movements deemed unrecordable may be due to excessive eye noise in the recording (e.g., eye makeup or eye lashes) or excessive eye blinks.

Figure 1.

The typical concussion assessment protocol (solid gray line) demonstrated a significant area under the curve (AUC) for classifying athletes with a history of concussion (AUC = 0.75, p = .007). Adding reflexive saccade eye excursion gain to the protocol demonstrated a larger AUC (solid black line; AUC = 0.80, p = .001).

Third, we examined the AUC for a typical concussion assessment protocol,¹⁵ including concussion-like symptoms (RPSQ), postural control (BESS errors), and neuropsychological performance (TMT A, TMT B, CWI, LNS). We first used binary logistic regression to derive probability estimates for each participant. Specifically, that the participant’s combined performance on these measures predicted their group membership (concussion history, no concussion history). These probability estimates were used to fit a multivariate ROC, which determined the AUC of the typical concussion protocol for correctly classifying varsity college athletes with a prior concussion (Model 1: Typical Concussion Protocol). We also examined the AUC after adding saccades gain (Model 2: Typical Concussion Protocol + Saccades) and P300 ERP amplitude and P300 ERP latency (Model 3: Typical Concussion Protocol + Saccades + ERP). Hierarchical binary logistic regression examined the extent to which these latter two models predicted group membership more accurately than the typical concussion protocol (Model 1). Fourth, the same multivariate ROC approach examined the combined AUC for the saccades and ERP outcomes (Model 4: Saccades + ERP).

3.0 Results

As reported in Table 2, individuals with and without a history of concussion did not significantly differ on all measures (p-values > .05). The results of the individual ROCs curves are reported in Table 3. No individual measure demonstrated an AUC significantly different from chance-levels (p-values > .05). We included reflexive saccades into the multivariate ROCs (but not smooth pursuit eye movements) because it was the only oculomotor measure that demonstrated adequate sensitivity. Table 4 illustrates the clinical performance of the four multivariate ROCs. The typical concussion protocol (Model 1) demonstrated a large AUC (AUC = 0.75, 95% CI = 0.60–0.91, p = .007). As shown in Figure 1, the inclusion of saccade gain (Model 2) resulted in a larger AUC (AUC = 0.80, 95% CI = 0.66–0.94, p = .001) with high sensitivity and specificity. Model 2 did not predict group membership significantly greater than Model 1, X²(2) = 1.871, p = .171. The inclusion of P300 amplitude and latency (Model 3) also resulted in a significant AUC (AUC = 0.79, 95% CI = 0.64–0.94, p = .002). However, including these ERP outcomes did not significantly improve Model 2’s predictability of group membership, X²(2) = 0.085, p = .959. The multivariate ROC including only the oculomotor and ERP outcomes (Model 4) was also significant (AUC = 0.70, 95% CI = 0.53–0.87, p = .030).

Table 3.

Results of Individual ROC Analyses

Measure	AUC (SE)	Sensitivity	Specificity	95% CI	p-value
RPSQ	0.47 (0.09)	0.43	0.54	(0.30, 0.64)	.759
BESS	0.44 (0.09)	0.57	0.46	(0.27, 0.61)	.488
TMT A	0.51 (0.09)	0.52	0.54	(0.34, 0.68)	.927
TMT B	0.57 (0.09)	0.57	0.58	(0.40, 0.74)	.400
CWI Errors	0.37 (0.08)	0.81	0.28	(0.21, 0.54)	.136
WAIS-LNS	0.44 (0.09)	0.48	0.26	(0.26, 0.61)	.460
Saccades Gain1	0.67 (0.09)	0.83	0.59	(0.50, 0.85)	.061
SP-0.7 Hz2	0.42 (0.09)	0.42	0.46	(0.24, 0.59)	.353
SP-0.6 Hz1	0.31 (0.09)	0.47	0.33	(0.14, 0.48)	.038^
SP-0.5 Hz3	0.47 (0.09)	0.57	0.48	(0.29, 0.65)	.753
SP-0.4 Hz1	0.39 (0.09)	0.47	0.43	(0.21, 0.57)	.288
SP-0.3 Hz3	0.52 (0.09)	0.58	0.39	(0.34, 0.70)	.820
SP-0.2 Hz4	0.55 (0.09)	0.65	0.48	(0.34, 0.72)	.592
P300-Amplitude	0.56 (0.09)	0.57	0.63	(0.39, 0.73)	.495
P300-Latency	0.52 (0.09)	0.38	0.50	(0.34, 0.69)	.856

AUC = Area under the curve; SE = Standard Error; CI = Confidence Interval.

¹Five subjects missing because of unrecordable eye movements;

²4 subjects missing because of unrecordable eye movements;

³3 subjects missing because of unrecordable eye movements;

⁴2 subjects missing because of unrecordable eye movements;

^{^}Significantly below the diagonal reference line. Eye movements deemed unrecordable may be due to excessive eye noise in the recording (e.g., eye makeup or eye lashes) or excessive eye blinks.

Table 4.

Results of Multivariate ROC analyses

Multivariate ROC	AUC (SE)	Sensitivity	Specificity	95% CI	p-value
Model 1: Typical Concussion Protocol	0.75 (0.08)	0.78	0.68	(0.60, 0.91)	.007**
Model 2: Typical Concussion Protocol + Saccades	0.80 (0.07)	0.78	0.77	(0.65, 0.94)	.001**
Model 3: Typical Concussion Protocol + Saccades + ERP	0.79 (0.07)	0.78	0.77	(0.64, 0.94)	.002**
Model 4: Saccades + ERP	0.70 (0.09)	0.78	0.68	(0.53, 0.87)	.030*

Typical Concussion Protocol= RPSQ + BESS + TMT A + TMT B + CWI Errors + WAIS-IV-LNS. Saccades from five subjects missing because of unrecordable eye excursions.

Listwise deletion was used to compare nested models (n = 40).

^*p < .05;

^**p < .01

We further explored if variability in post-concussion duration limited the clinical performance of the reflexive saccades and ERP measures. Time elapsed since concussion did not significantly correlate with reflexive saccade gain, P300 amplitude or P300 latency (ps > .05). In post hoc multivariate ROC analyses, the sample was limited to athletes whose most recent concussion occurred in the last four years (n = 16; M = 2.08 years, SD = 0.98). The AUCs for this less heterogeneous sample did not change for Model 2, (AUC = 0.80, 95% CI = 0.65–0.96, p = .003) or Model 3 (AUC = 0.79, 95% CI = 0.64–0.95, p = .004). The clinical performance of Model 4 declined and was no longer significant, (AUC = 0.69, 95% CI = 0.50, 0.88, p = .065).

4.0 Discussion

Our primary objective was to examine if oculomotor and ERP outcomes improve the clinical performance of the typical concussion protocol for classifying athletes with a history of concussion. The typical concussion protocol demonstrated a significant AUC with adequate sensitivity (0.78) and specificity (0.68). A combined model of reflexive saccades and P300 amplitude and latency also yielded a significant AUC but was not significantly different from the typical protocol.

It is recommended that oculomotor screening be incorporated into concussion assessment protocols.²⁹ However, there is currently limited research using objective oculomotor assessment with videonystagmography (VNG) equipment in concussed athletes. Altered antisaccades and memory-guided saccades gain were reported in acute concussed athletes compared to healthy subjects.⁴ During concurrent fMRI, these oculomotor deficits were accompanied by increased blood oxygen level dependent (BOLD) signal change in the cerebellum, primary motor cortex, visual cortex, and subregions of the frontal and temporal cortices.⁴ To our knowledge, this is the first study to examine the clinical performance of objective oculomotor measures for classifying athletes with a history of concussion.

The inclusion of reflexive saccade gain did not significantly improve the clinical performance of the typical concussion protocol in the long-term post-concussion period. Further, access to objective oculomotor assessment tools (i.e., VNG) is rare for clinicians assessing concussions; thus, screening measures such as the Vestibular/Ocular Motor Screening (VOMS)^5,29 may provide a more efficient alternative towards identifying college athletes with a history of concussion. The VOMS evaluates the extent to which five qualitative oculomotor tests provoke concussion-like symptoms. A multivariate ROC including three of these measures demonstrated a large AUC (0.89) for identifying those athletes with a concussion within 21 days.⁵ Importantly, the VOMS demonstrated high internal consistency, a low false-positive rate,^5,30 and is quick to administer. These are all particularly valuable criteria for athletic trainers and clinicians coordinating large-scale baseline concussion testing. Future research should examine the clinical performance of the VOMS for classifying athletes with a history of concussion. Given the clinical utility of the VOMS in the acute post-concussion phase,⁵ it may be used to refer athletes for further in-depth oculomotor testing,²⁹ such as that using VNG.

Research indicated that athletes with a history of mTBI generated aberrant neuroelectrical responses during cognitive tasks.^7,10,11 However, the ERP measures in the current study did not demonstrate significant clinical utility for classifying athletes with a history of concussion nor improve the clinical performance of the typical protocol. The discrepancy between prior findings and the current study may be due to task-related differences. For instance, we chose to focus on the P300 component elicited during a working memory task, given that memory deficits are often commonly reported following concussion.³¹ Even though fMRI research reported that athletes with a prior concussion generate altered functional brain activity underlying working memory,^32,33 the extent to which a concussion influences the P300 ERP component elicited during working memory tasks may be more tenuous. Gosselin et al³⁴ reported that individuals with a post-acute mTBI did not generate altered P300 amplitude or latencies during working memory but showed decreased brain activity in the dorsolateral prefrontal cortex. This suggests that fMRI may be a more sensitive index of altered neural correlates underlying working memory.

Research reported that college athletes return to baseline levels of performance on symptoms, postural control, and neuropsychological performance within 7–10 days following concussion.^16,31,35 To the authors’ knowledge, the current study is the first to report adequate clinical utility of this combined protocol in the post-acute period following concussion. The present study’s results may be limited to traditional neuropsychological tests, which may be more sensitive to a prior concussion than computerized neurocognitive batteries.¹⁵ The small sample size and heterogeneity of the concussion history group limits our results and could have lead to an inadequate classification rate. Although we established that duration post-concussion did not influence our findings, researchers should recruit large enough sample sizes to examine changes in oculomotor and electrophysiological measures based on number of prior concussions in gender-matched sports.

The concussion history and non-concussion history groups did not significantly differ on any single outcome measure. Although individual measures may provide limited clinical utility as single measures, they may be more useful for identifying college athletes with a history of concussion when included in a parallel protocol. Our findings suggest that such a multivariate clinical protocol successfully identified varsity collegiate athletes with long-term changes in functional outcomes associated with a history of concussion. Therefore, multivariate ROC analyses are advantageous to univariate between-groups comparisons because of their ability to identify the combined clinical utility of multiple measures for classifying previously concussed athletes.

A protocol with ERP and reflexive saccades also demonstrated significant clinical performance. Although not significant, reflexive saccade eye excursions increased the clinical performance of the typical concussion protocol. Taken together, these findings suggest that reflexive saccades and ERP outcomes demonstrate significant clinical performance for classifying varsity athletes with a prior concussion, but do not improve the clinical performance of the typical protocol. In the acute post-concussion phase, these measures may demonstrate greater sensitivity and specificity. However, the large amount of training and resources necessary to include VNG and/or electrophysiology testing may limit their clinical utility in the acute post-concussion period. Future research is warranted to investigate the clinical performance of quantitative, objective oculomotor and ERP indices in the acute post-concussion phase. This research is necessary for establishing the performance of objective oculomotor and electrophysiological testing for concussion assessment and management.