Visuo-oculomotor Function and Reaction Times in Athletes with and without Concussion

Abstract

Significance.

Oculomotor tests in concussion commonly show impairment in smooth pursuit and saccadic function. Honing in on the systems likely to be affected by concussion will streamline use of oculomotor function as a supplemental diagnostic and prognostic tool, as well as improve our understanding of the pathophysiology of concussion.

Purpose.

This study investigates oculomotor function between concussed and healthy collegiate athletes, as well as determines measurement test-retest reliability of those tools.

Methods.

Eighty-seven healthy athletes were recruited from a US Division 1 sports university and completed a 30-minute vestibular ocular testing battery in an enclosed rotary chair system equipped with 100Hz eye-tracking goggles. Forty-three individuals completed the battery twice. Twenty-eight individuals with a current diagnosis of concussion also completed the battery. All participants were ages 18–24. Bivariate statistical tests examined differences in scores across groups, and intraclass coefficients were computed to test reliability.

Results.

Concussed individuals had significantly longer saccadic, visual, and dual-task reaction times and reduced saccadic accuracy. There was no difference in optokinetic reflex gain, but few concussed individuals tolerated the task. Reaction time latencies and optokinetic gain show moderate test-retest reliability. Smooth pursuit tasks and saccadic accuracies showed poor test-retest reliability.

Conclusions.

Saccadic latency was the most sensitive oculomotor function to change following concussion and was reliable over time. Saccadic accuracy was significantly lower in the concussed group but had poor retest-reliability. Optokinetic gain may warrant more investigation due to its high test-retest reliability and symptom provocation in concussion, despite not showing a significant difference between groups.

Concussion diagnosis is typically based on a multidimensional assessment of cognition, balance function, and a review of systems via a self-reported symptom scale, e.g. the Sports Concussion Assessment.^1,2 Recent studies show that concussion may also lead to clinically impaired smooth pursuit and saccadic function, increased near point of convergence to receded near point of convergence, and symptom exacerbation with optokinetic stimulation.^3–7 More objective studies including eye movement recordings show that athletes with concussion have difficulty holding gaze stable and in performing saccadic and accommodative tasks, demonstrating the need for more research into the clinical utility of visuo-oculomotor function in concussion.^8-10

The pathophysiology of oculomotor dysfunction associated with concussion is not well understood.^11,12 However, the development of objective tests of oculomotor function may aid in monitoring of recovery, since these functions tend to improve as the concussion recovers. For those that do not recover, vision therapy has been shown to alleviate symptoms of concussion.^13–16 Therefore, better understanding of changes to the oculomotor systems associated with concussion could lead to more focused interventions, especially for those who are not recovering quickly.

Given that oculomotor dysfunction is a known issue in concussion, oculomotor testing offers an objective, quantifiable, and clinically feasible set of tools to improve data-informed decision making. To better understand the clinical utility of these tests, the oculomotor functions most sensitive to concussion should be identified. The purpose of this study was to complete a robust examination of visuo-oculomotor function through an eye-tracking equipped rotary chair system to determine which, if any, oculomotor functions are impaired following concussion and the test-retest reliability of those measures. In addition, we were interested in investigating visual and auditory gross reaction times.

METHODS

Healthy controls (n=87, average age: 20.6 +/− 1.8 years, sex: 39 female, 48 male) were recruited from a United States of America Division 1 university men’s football and women’s soccer teams as well as from the university’s undergraduate and graduate student body active in recreational sports. Besides football and soccer, these recreational sports included basketball, cross country, cheerleading, power lifting, and baseball. All participants were between 18 and 24 years of age. The healthy group consisted of n=45 white, n=34 black, and 8 other race individuals. Of the healthy sample, n=23 reported a previous concussion at some point in their life but were not currently diagnosed as concussed by their physician. To evaluate test-retest reliability, n=43 participants were asked to complete the protocol twice. n=25 (recreational athletes) repeated testing on the same day. The remaining 18 (varsity athletes) completed testing at the beginning of both the 2016 and 2017 fall seasons. Written informed consent was obtained from all participants and the protocol was approved by the university’s Institutional Review Board. The present project conformed to the tenets of the Declaration of Helsinki.

Participants with concussion (n=28, average age 20.7 +/− 1.9, sex: 11 female, 17 male) consisted of football and women’s soccer players (n=17) as well as athletes from the area (n=11). All participants were between 18 and 24 years of age. The concussed group consisted of n=12 white, n=8 black, and n=8 other race individuals. Athletes with concussion from the university were tested less than 72 hours after injury. All concussions were sports-related. Individuals from the community who were referred to the clinic had varying times from diagnosis, with a maximum of 2 weeks. For this study, all individuals with concussion were pooled.

All individuals (healthy controls and athletes with concussion) completed a 25–30 min vestibulo-ocular, visuo-ocular, and reaction time battery of tests using the clinical I-Portal Neuro Otologic Test Center (I-Portal-NOTC, Neurokinetics Inc., Pittsburgh, PA) chair system. Participants sat in the NOTC chair in a light-proof enclosure. Binocular infrared eye tracking cameras (100 Hz frame rate) embedded in a goggle set measured eye movements and participants used buttons on their armrests to respond to manual reaction time tasks. Stimulus presentation, timing, data acquisition and analyses were managed by the NOTC computer software (VEST version 7.4). All data were acquired with a time resolution of 10 ms. A trained tester communicated standardized task instructions to participants. The order of testing and protocols were the same for all participants. Due to symptom exacerbation, not all participants completed every task.

If any participant experienced a headache, nausea, or any other symptom, or significant worsening of existing symptoms while performing a task, the task was stopped, and the participant was given time to rest. We obtained baseline dizziness and headache scores on a scale 0 to 10 (0=none, 10=extremely severe) before the tests in both healthy and concussed individuals. These scores were only used to assess symptom exacerbation during our rotary chair protocol in order to refine our protocol for concussed individuals moving forward. Following each group of subtests, we asked for new dizziness and headache scores. If either score went up more than three points we allowed a rest period to see if they would return to baseline values. If not, we asked the participant if they would like to continue with other tasks. If the participant verbally stated that they did not want to continue, whether their symptoms scores increased or not, we stopped the entire protocol. No subject requested to stop entirely and at least put an effort into future subtests.

The saccadic and smooth pursuit target was a small red laser that appeared as a dot on the enclosure wall and its position was controlled by fast mirror galvanometers. The optokinetic stimulus, a random dot pattern projected by a rotating planetarium, was a set of high-density white dots covering the entire surface of the enclosure wall. If a participant stated that they were unable to complete any sub-test, that sub-test was not considered in the analysis.

The tests were administered to all participants in the same order as described below. When relevant, all horizontal directions were completed before vertical directions and in ascending frequency order (aka, 0.1Hz, 0.2Hz, 0.4Hz, 1.0Hz horizontal smooth pursuits were completed before any vertical).

Predictive Saccades

Participants followed a target as it moved horizontally in a fully predictable step-wise manner to determine the ability of the saccadic system to anticipate a repetitive motion pattern. The protocol parameters were as follows: 24 steps total, 5 target steps were random, then the target associated with the 6^th-24^th steps of the sequence periodically moved between 10° to the right and 10° to the left with a hold time of 0.65s. This results in 18 target steps that may be predicted by the participant. Saccades were counted as predictive if they occurred within 80 ms of stimulus movement. Data of interest were the percentage of predictive saccades (number of predicted saccades / total number of valid saccades (usually 18 but eye tracking problems may have removed 1–2 in some individuals)) and the first predictive saccade (Saccade #1–18) after the repetitive pattern began.

Random Horizontal and Vertical Saccades

Participants followed a target as it stepped randomly to the right and left along the horizontal meridian or up and down along the vertical meridian. Horizontal and vertical saccades were acquired in two separate subtasks. The protocol parameters were as follows: n=30 target steps; position range of the initial target from –22° to 24°; target step amplitudes were all the even values between −30° and 30° (n=30), one repetition for each target step and holding between target transitions 1.1 to 2.0 s. The data of interest were the mean of the primary saccade accuracy (i.e. the percent distance that the primary saccade covered towards the target), mean final accuracy (i.e. the percent total distance covered by the primary and the associated secondary saccades – if present – towards the target) and mean saccadic latency (i.e. the time in ms to initiate the movement of the eye following the target step).

Smooth Pursuit

Participants followed the target moving sinusoidally along the horizontal or the vertical meridian at 0.01 Hz, 0.02 Hz, 0.04 Hz, and 1.00 Hz (horizontal) or 0.75 Hz (vertical) +/−10° peak to peak (aka, furthest left location to furthest right location). Data of interest were average velocity gain (i.e. ratio of participant eye velocity to target velocity) and average position gain (i.e. ratio of participant slow phase eye location to target location). The saccadic contributions to the tracking response were clipped to limit the measures to the smooth pursuit component only.

Anti-Saccades

Participants were told to focus initially on the target at the center and, upon a random movement of the target to the left or right, to generate a saccade of the same size of the target step but in the opposite direction and to hold their eyes there, looking at nothing, until the target was turned off for a total of 16 trials. The parameter of interest was percent error, defined as the percent of trials with the initial saccade executed in the same (wrong) direction of the target step. Participants completed a brief training on this task to determine adequate understanding before collecting data.

Optokinetic Nystagmus

Participants watched the random dot pattern projected on the enclosure wall that moved counter-clockwise then clockwise at 20°/s or 60°/s for 20 s each and with 4 s between the inversion of direction. The data of interest included the gain (i.e. eye velocity/target velocity) of the optokinetic response after clipping of the resetting saccades of the nystagmus. Over the course of data collection, it became clear that the optokinetic stimulation commonly caused an exacerbation of symptoms. Although most individuals were able to tolerate the test, those who stopped (n=6 concussed) asked to stop during the second, clockwise direction. For this reason, only data collected during the first 20 seconds (counterclockwise rotation) will be used for all participants as we believe symptom exacerbation did not begin immediately.

Visual and Auditory Reaction Time

The participant used the dominant hand to push the button located on the corresponding armrest as soon as he/she saw a central target appear (visual reaction time task) or heard a beep (auditory reaction time task). The data of interest were mean latency for both tasks. The first trial of each test was considered training and not included in the analysis.

Combined Visual Reaction Time and Saccadic Task (Dual Task)

The participant looked at the target that moved in a random rightward or leftward direction, following the identical protocol of the horizontal random saccades task described earlier, and pushed the corresponding button (right or left) while also executing the oculomotor saccadic task. This test evaluated the ability of the participant to execute at the same time a direction-dependent manual reaction time task and a saccadic oculomotor task. Measure of interest for this study was button press latency.

Due to the number of tasks in this protocol, the results of the vestibular tests are reported in a separate manuscript. Saccadic dynamics, which required the development of new methodologies for its quantification, are also not included in this report.

Analysis

After extraction of the numerical results using the proprietary software, data analysis was completed with SPSS v23. Normality was assessed visually for all variables. To determine if the data of athletes who completed the test-retest protocol on the same day could be pooled with athletes who completed the tests over a 7–12 month period, we compared the difference of test 1 and test 2 between groups for each variable. Reliability was described with the Intraclass Correlation Coefficient (ICC) and 95% confidence intervals and described as poor (ICC ≤ 0.40), fair to good (ICC > 0.40 and < 0.75) and excellent (ICC ≥ 0.75).¹⁷ The magnitude of the Intraclass Correlation Coefficient depends on the between subject variability, meaning low subject variability can lead to a low Intraclass Correlation Coefficient.¹⁸ For this reason, we also calculated the smallest real difference to indicate the expected range of score on a second administration of the test. Using Intraclass Correlation Coefficient and the pooled variance of test 1 and test 2, a 95% confidence smallest real difference was calculated:

$$\mathrm{Smallest}\mathrm{Real}\mathrm{Difference}=\mathrm{Standard}\mathrm{Deviation}{}^{\ast }\surd (1-\mathrm{ICC}){}^{\ast }\surd 2{}^{\ast }1.96$$

The smallest real difference was compared to the mean score to calculate the “% mean,” the smallest real difference divided by the mean score (Test 1 + Test 2)/2. In a previous clinical test repeatability study a % mean (i.e. the % of the mean represented by the smallest real difference) of <30% was considered acceptable.¹⁹ While this will be utilized to give our data context, it is important to note that this is not a hard cut-off point and is not equivalent to a clinically meaningful difference. This measure is, put simply, a measurement that considers the Intraclass Correlation Coefficient in the context of the variation within the sample (standard deviation) to attempt to create a confidence interval of expected score ranges in the measured unit. It allows for a more clinically actionable measure of variability than the Intraclass Correlation Coefficient or standard deviation alone. As % means were calculated prior to trimming decimal places for table presentation, the % means presented here may be off by ~1% if calculations are done on the rounded means of test 1 and test 2. We do not feel like this small change should affect interpretation of the clinical reliability of these measures.

Healthy athletes (1^st session if tested twice) were compared to athletes with concussion using either Mann-Whitney U or t-test depending on data normality. Depending on the specific task and who was able to complete which task without symptom exacerbation, data from 80–87 healthy individuals were compared to 23–28 concussed individuals. For instance, even two healthy individuals reported motion sickness during the 60 deg/s optokinetic task and for that reason were removed from analysis. One-tailed tests were used for saccadic accuracy and reaction time tests as it was hypothesized from previous studies the concussed individuals would perform worse on these measures.^20–22 Two-tailed tests were used for all other variables.

Validity for individual saccades was determined by the NKI program. Validity for continuous tasks (smooth pursuit, for example) was determined by spectral purity (eye tracking data time / total test time). If the spectral purity was below 70%, the test was not used. During testing it is possible to view spectral purity of a task; if data collectors saw a completed task was below 70% it was run again (unless it caused symptom exacerbation) to try to record a valid trial.

Some of our healthy controls came to our lab in the middle of the work/class day wearing mascara. In the case of concussion we always asked the mascara to be removed due to our smaller sample size, but we did not want to inconvenience the ~4 healthy subjects. For that reason, spectral purity was lower for some healthy participants (and their test removed) for smooth pursuit tasks which is reflected in the tables which could not be fixed through a second administration of the task. Healthy individuals’ data was never removed due to symptom exacerbation besides the OKN task.

RESULTS

See Tables 1 (saccadic tasks), 2 (smooth pursuit tasks), and 3 (optokinetic, reaction time tasks) for reliability results. There was no significant difference between healthy athletes who completed the retest on the same day and the athletes who completed the retest 7–12 months later (P range: 0.15–0.90), therefore, reliability data were pooled. Visual inspection revealed that all data from healthy controls were normally distributed, and skewness values support the use of non-parametric analysis (range: −2.23 – 2.40, Kolmogorov-Smirnov tests all insignificant (P>0.05). Depending on the subtest validity, 40–43 healthy individuals were used for reliability calculations.

Table 1.

Test-retest reliability for predicted saccades, horizontal and vertical random saccades and anti-saccades (n=40-43).

Saccadic Parameter	Test 1 Mean ± SD	Test 2 Mean ± SD	ICC (95% CI)	SRD	% Mean
Predictive Task
Percent Predicted (%)	51.7 ± 29.3	45.5 ± 26.6	*n/a	*n/a	*n/a
First Predicted(#)	5.3 ± 3.4	5.4 ± 3.4	*n/a	*n/a	*n/a
Horizontal Random Task
Primary Accuracy (%)	93.4 ± 6.7	93.0 ± 5.5	0.12 (0.01 - 0.72)	16.2	17.5
Final Accuracy (%)	97.1 ± 5.5	97.5 ± 6.0	0.20 (0.04 - 0.60)	13.5	14.1
Latency (ms)	180 ± 30	180 ± 30	0.74 (0.59 - 0.85)	40	22.0
Vertical Random Task
Primary Accuracy (%)	97.5 ± 10.5	98.1 ± 16.7	0.48 (0.26 - 0.70)	21.5	22.2
Final Accuracy (%)	97.9 ± 8.6	97.5 ± 15.2	0.03 (0.00 - 1.00)	27.9	28.7
Latency (ms)	200 ± 20	200 ± 20	0.68 (0.50 - 0.82)	40	21.6
Anti-saccadic Task
Errors (%)	52.3 ± 24.0	44.9 ± 24.7	*n/a	*n/a	*n/a

SD = Standard Deviation, ICC = Intraclass Correlation Coefficient, CI = Confidence Interval SRD = Smallest Real Difference, % Mean = SRD / Average Measure.

See Tables 4 (saccadic tasks), 5 (smooth pursuit tasks), and 6 (optokinetic and reaction time tasks) for comparisons between athletes with and without concussion. Not all data sets for athletes with concussion were normally distributed (Kolmogorov-Smirnov tests significant (P<0.05). Therefore, Tables 4–6 indicate medians and ranges. Depending on individual subtest validity, the number of healthy athletes’ data on each subtest ranged from n=80–87 while athletes with concussion ranged from n=23–28. Due to poor tolerance of the optokinetic task by athletes with concussion, which caused symptom provocation, only the first 20 seconds (counterclockwise stimulation) for both 20°/s and 60°/s were used for analysis.

Table 4.

Comparison of athletes with concussion to healthy athletes on saccadic parameters. P-values represent Mann-Whitney U results. Sample sizes below each variable represent the number of non-concussed and number of concussed, respectively, individuals in each analysis, representing the number of individuals from each group who could complete the task.

Saccadic Parameter	Non-concussed Median	Non-concussed Range	Concussed Median	Concussed Range	P-value
Predictive Task
Percent Predicted (%) (n=85 vs n=26)	48.5	0 - 94.1	47.1	0 - 100	.50
First Predicted (#) (n=85 vs n=26)	4	2 - 17	5	1 - 16	.34
Horizontal Random Task
Primary Accuracy (%) (n=85 vs n=26)	91.3	81.6 - 119.6	89.2	67.0 - 108.0	.04*
Final Accuracy (%) (n=85 vs n=26)	95.3	72.3 - 120.1	94.6	72.9 - 104.5	.04*
Latency (ms) (n=85 vs n=26)	180	140 - 280	200	160 - 370	.02*
Vertical Random Task
Primary Accuracy (%) (n=85 vs n=26)	95.4	77.7 - 130.7	91.3	56.0 - 118.4	.03*
Final Accuracy (%) (n=85 vs n=26)	95.4	76.1 - 151.0	92.9	70.6 - 116.1	.04*
Latency (ms) (n=85 vs n=26)	190	150 - 310	210	170 - 470	.01*
Anti-saccadic Task
Errors (%) (n=80 vs n=25)	50.0	6.3 - 100.0	62.5	12.5 - 81.25	.20

% = Percentage (Accuracies can be >100 if participant overshot laser target stimulus)

Bolded = Statistically significant at alpha < .05

^*= One-tailed Mann-Whitney U P-value due to hypothesis that concussed individuals would perform worse

Table 6.

Comparison of athletes with concussion to healthy athletes on optokinetic and reaction time parameters. Optokinetic gain is defined as fast phase eye velocity / optokinetic stimulus velocity. Sample sizes below each variable represent the number of non-concussed and number of concussed, respectively, individuals in each analysis, representing the number of individuals from each group who could complete the task.

Optokinetic Gain	Non-concussed Median	Non-concussed Range	Concussed Median	Concussed Range	P-value
20°/s ccw (n=85 vs. n=24)	0.89	0.39 - 1.20	0.84	0.36 - 1.06	.14
60°/s ccw (n=85 vs. n=23)	0.55	0.13 - 0.92	0.44	0.24 - 0.81	.06
Reaction Times
Auditory (ms) (n=87 vs n=29)	246	143 - 552	241	138 - 775	.30*
Visual (ms) (n=87 vs n=29)	273	169 - 507	293	182 - 992	.02*
Dual Task (ms) (n=85 vs n=23)	482	261 - 880	522	318 - 1276	.03*

Bolded = statistically significant at alpha < .05

^*= One-tailed Mann-Whitney U results due to hypothesis that concussed individuals would perform worse

Random Horizontal and Vertical Saccades

Random horizontal and vertical saccade latencies had fair/good reliability while accuracies showed poor reliability. However, both the % mean for latency and accuracy variables were <30% and therefore unacceptable by smallest real difference standards (Table 1 and Table 4). The <30% cutoff is not an established cutoff and is only loosely used as a guideline for this manuscript.

Individuals with concussion had significantly worse scores on all random saccadic variables compared to healthy controls, based on one-tailed Mann-Whitney U tests (P<0.05), with significantly lower accuracy and increased saccadic latencies as hypothesized.

Predictive Saccades and Anti-Saccades

There was no significant difference between athletes with and without concussion on either subtest. For the anti-saccades test, healthy athletes had a median prosaccade error rate of 50.0% versus 62.5% in athletes with concussion (P=0.20) (Table 1 and Table 4). On the predictive saccades test, healthy athletes had a median of 48.5% predictive saccades vs 47.1% in athletes with concussion (P=0.50). Due to the nature of the “first predicted” and “percent predicted” variables (both are non-continuous variables), the mean is not a valid measure of central tendency. For that reason, smallest real difference cannot be calculated and is not calculated for these variables.

Smooth Pursuit

Smooth pursuit Intraclass Correlation Coefficients ranged from poor to fair/good and the Intraclass Correlation Coefficient 95% CI includes zero for five out of the 16 variables (Table 2 and Table 5). The % mean, however, was ≤ 22.8% for all horizontal measures and ≤ 31.4% for 6 of the 8 vertical measures indicating acceptable agreement for these variables. The fastest horizontal frequency, 1.0 Hz, had the best reliability (ICC ≥ 0.70). See Figures 1 and 2 for comparison between a low Intraclass Correlation Coefficient (0.1Hz Horizontal Smooth Pursuit) and a high Intraclass Correlation Coefficient task (1.0Hz Horizontal Smooth Pursuit) that have equal smallest real differences. These figures help to underscore the importance of variance in calculation of the Intraclass Correlation Coefficient.

Table 2.

Test-retest reliability for horizontal and vertical smooth pursuit (n=40-43). Velocity gain= eye/target velocity. Position gain=eye/target position.

Smooth Pursuit Parameter	Test 1 Mean ± SD	Test 2 Mean ± SD	ICC (95% CI)	SRD	% Mean
Horizontal
0.1 Hz Velocity Gain	0.97 ± 0.07	0.94 ± 0.09	0.09 (0.00 - 0.78)	0.19	19.9
0.1 Hz Position Gain	1.02 ± 0.10	1.02 ± 0.04	0.12 (0.01 - 0.69)	0.20	19.2
0.2 Hz Velocity Gain	1.01 ± 0.07	1.00 ± 0.06	0.27 (0.08 - 0.61)	0.15	14.8
0.2 Hz Position Gain	1.03 ± 0.05	1.02 ± 0.04	0.08 (0.00 - 0.82)	0.12	12.2
0.4 Hz Velocity Gain	1.02 ± 0.05	0.99 ± 0.07	0.13 (0.01 - 0.67)	0.17	17.1
0.4 Hz Position Gain	1.03 ± 0.05	1.02 ± 0.04	0.01 (0.00 - 1.00)	0.13	12.7
1.0 Hz Velocity Gain	0.85 ± 0.14	0.87 ± 0.12	0.70 (0.51 - 0.83)	0.19	22.8
1.0 Hz Position Gain	0.86 ± 0.11	0.87 ± 0.12	0.71 (0.53 - 0.84)	0.17	19.3
Vertical
0.1 Hz Velocity Gain	0.93 ± 0.12	0.90 ± 0.14	0.56 (0.35 - 0.75)	0.27	30.3
0.1 Hz Position Gain	1.02 ± 0.11	1.00 ± 0.10	0.08 (0.00 - 0.85)	0.32	30.6
0.2 Hz Velocity Gain	0.93 ± 0.16	0.94 ± 0.14	0.67 (0.49 - 0.81)	0.23	24.2
0.2 Hz Position Gain	1.01 ± 0.11	0.99 ± 0.10	0.37 (0.16 - 0.64)	0.24	23.7
0.4 Hz Velocity Gain	0.90 ± 0.17	0.91 ± 0.18	0.52 (0.31 - 0.72)	0.27	29.5
0.4 Hz Position Gain	0.98 ± 0.11	0.97 ± 0.10	0.02 (0.00 - 1.00)	0.36	35.5
0.75 Hz Velocity Gain	0.77 ± 0.22	0.86 ± 0.14	0.51 (0.29 - 0.73)	0.39	47.9
0.75 Hz Position Gain	0.92 ± 0.15	0.91 ± 0.10	0.37 (0.17 - 0.64)	0.29	31.4

SD = Standard Deviation, ICC = Intraclass Correlation Coefficient, CI = Confidence Interval, SRD = Smallest Real Difference, % Mean = SRD / Average Measure.

Table 5.

Comparison of athletes with concussion to healthy athletes on smooth pursuit parameters. Velocity gain= eye/target velocity. Position gain=eye/target position. Sample sizes below each variable represent the number of non-concussed and number of concussed, respectively, individuals in each analysis, representing the number of individuals from each group who could complete the task.

Smooth Pursuit	Non-concussed Median	Non-concussed Range	Concussed Median	Concussed Range	P-value
Horizontal
0.1 Hz Velocity Gain (n=82 vs n=25)	0.96	0.75 - 1.16	0.96	0.72 - 1.04	.75
0.1 Hz Position Gain (n=82 vs n=25)	1.01	0.61 - 1.31	1.01	0.92 - 1.06	.61
0.2 Hz Velocity Gain (n=82 vs n=25)	1.01	0.70 - 1.30	1.00	0.58 - 1.05	.34
0.2 Hz Position Gain (n=82 vs n=25)	1.02	0.92 - 1.29	1.01	0.75 - 1.05	.46
0.4 Hz Velocity Gain (n=82 vs n=24)	1.00	0.73 - 1.25	1.00	0.43 - 1.06	.20
0.4 Hz Position Gain (n=82 vs n=24)	1.01	0.89 - 1.26	1.01	0.09 - 1.06	.25
1.0 Hz Velocity Gain (n=82 vs n=26)	0.86	0.38 - 1.11	0.85	0.16 - 1.08	.24
1.0 Hz Position Gain (n=82 vs n=26)	0.86	0.53 - 1.09	0.85	0.15 - 1.07	.43
Vertical
0.1 Hz Velocity Gain (n=82 vs n=25)	0.93	0.43 - 1.19	0.90	0.67 - 1.22	.48
0.1 Hz Position Gain (n=82 vs n=25)	1.02	0.66 - 1.54	0.99	0.78 - 1.22	.22
0.2 Hz Velocity Gain (n=82 vs n=25)	0.95	0.43 - 1.30	0.95	0.60 - 1.14	.34
0.2 Hz Position Gain (n=82 vs n=25)	1.01	0.62 - 1.31	0.95	0.78 - 1.25	.048
0.4 Hz Velocity Gain (n=83 vs n=24)	0.96	0.25 - 1.26	0.92	0.35 - 1.21	.21
0.4 Hz Position Gain (n=83 vs n=24)	1.00	0.66 - 1.68	0.97	0.28 - 1.25	.03
0.75 Hz Velocity Gain (n=82 vs n=25)	0.87	0.27 - 1.13	0.80	0.16 - 1.10	.12
0.75 Hz Position Gain (n=82 vs n=25)	0.95	0.39 - 1.28	0.85	0.16 - 1.11	.003

Bolded = statistically significant at alpha < .05

Figure 1.

Correlation between Position Gain on first and second administration of 0.1Hz Horizontal Smooth Pursuit task in 40 healthy individuals. Intraclass Correlation Coefficient (ICC) = 0.09 (range=0.00-0.78), Smallest real difference = 19.0%. Note the very tight spread of data (with 3 outliers) vs Figure 2.

Figure 2.

Correlation between Position Gain on first and second administration of 1.0Hz Horizontal Smooth Pursuit task in 40 healthy individuals. Intraclass Correlation Coefficient (ICC) = 0.71 (range=0.53-0.84), Smallest real difference = 19.3%. Note the wide spread of data vs Figure 1 which results in a much stronger ICC but essentially identical smallest real differences between tasks.

There was no significant difference in horizontal smooth pursuit between individuals with and without concussion. For vertical smooth pursuit tasks, individuals with concussion had significantly lower position gains for all frequencies except the slowest. As the frequency increased, the difference between group distributions got progressively larger (P = 0.048 for 0.2Hz, 0.03 for 0.4Hz, and 0.003 for 0.75Hz).

Optokinetic Nystagmus

Optokinetic gain Intraclass Correlation Coefficient’s ranged from fair/good (20°/s) to excellent (60°/s). However, the % means were all >30%, indicating that gain may be too variable to use clinically as a singular measure of optokinetic reflex change (Table 3 and Table 6).

Table 3.

Test-retest reliability for Optokinetic Reflex and Reaction Times (n=40-43). Optokinetic gain is defined as fast phase eye velocity / optokinetic stimulus velocity.

*Optokinetic Gain	Test 1 Mean ± SD	Test 2 Mean ± SD	ICC (95% CI)	SRD	% Mean
20°/s ccw	0.87 ± 0.15	0.80 ± 0.16	0.57 (0.36 - 0.76)	0.28	32.1
60°/s ccw	0.54 ± 0.21	0.50 ± 0.18	0.75 (0.59 - 0.86)	0.29	54.4
Reaction Times
Auditory (ms)	256 ± 69	247 ± 76	0.76 (0.61 - 0.86)	99	38.4
Visual (ms)	266 ± 46	276 ± 56	0.33 (0.13 - 0.62)	121	43.5
Dual Task (ms)	457 ± 107	421 ± 118	0.78 (0.65 - 0.88)	158	32.6

SD = Standard Deviation, ICC = Intraclass Correlation Coefficient, CI = Confidence Interval, SRD = Smallest Real Difference, % Mean = SRD / Average Measure. CCW = Counterclockwise Optokinetic Stimulus, ms = milliseconds.

There was no significant difference in counter-clockwise optokinetic reflex gain (P>0.05) between participants with and without concussion. However, concussed individuals had significantly more difficulty tolerating the task (n=6 (20%) concussed athletes vs n=2 (2.3%) healthy athletes). Despite the lack of difference in gain, it may be possible had we continued the task past the point of their symptom exacerbation there would have been notable differences in gain.

Visual, Auditory and Dual Task Reaction Times

Auditory reaction time and dual task reaction time showed excellent agreement while simple visual reaction time showed poor agreement (Table 3). The % mean ranged from 32.6% to 43.5%, indicating that these measures are hovering above the acceptable test-retest variability. A larger dataset may help distinguish if these measures are too variable or not.

Visual and dual-task reaction time latencies were significantly increased in the concussed group as hypothesized (P=0.02 and P=0.03, respectively), but not in the auditory reaction task (P=0.30).

DISCUSSION

Reliability of scores varied greatly among the variables of interest associated with the visuo-oculomotor and reaction time tasks. Athletes with concussion scored significantly worse than healthy athletes on the following tests: random saccade latency and accuracy, vertical smooth pursuit position gain, and visual and dual task reaction times.

Healthy Control Test-Retest Reliability

We found low Intraclass Correlation Coefficient values for saccadic accuracy and smooth pursuit except for horizontal smooth pursuit at 1.0 Hz (ICC ≥ 0.70). One possible explanation for poor reliability is that participants who completed both batteries on the same day were tired; however, this is unlikely since there was no difference between test 1 and test 2 scores in those who completed the tests on the same day or one year apart. Upon examination of the random saccade and smooth pursuit data, test 1 and test 2 were almost identical with low between subject variance (Tables 1 and 2). As stated before, the Intraclass Correlation Coefficient is affected poorly by low between subject variability; the resulting smallest real difference % mean for each measure was <30% representing an acceptable test-retest score difference based on previous literature on this measure.¹⁹ Saccade latencies and optokinetic gains showed much greater agreement across sessions. Interestingly, visual reaction time showed poor agreement (ICC=0.33) whereas the dual-task and auditory reaction times were much more reliable (ICC = 0.78 and 0.76, respectively). It is possible that the visual reaction time test, which occurred first, may require more trials. This task came immediately after our OKN task and represented a drastic shift from eye movement testing to gross motor testing. Together, these may represent a need to give even healthy individuals more of a break following OKN stimulus (they may have been still recovering from feelings of movement and were not paying as much attention) and help ease them into pressing the chair buttons (giving them a dummy test to get used to how the buttons feel). Clinicians who are using these tests and systems should be aware of reliability issues. Future research should test new protocols to improve reliability.

Concussed vs. Non-concussed Performance

Participants with concussion showed decreased performance as hypothesized during both horizontal and vertical saccade tasks (i.e. poor accuracy, longer response latency) when compared to healthy controls. There was no significant difference in performance during predictive or anti-saccades tasks. These results differ substantially from those of Balaban et al. 2016, who found that both predictive and anti-saccade tasks were strong predictors of concussion status.²³ Our anti-saccades error rate for healthy controls (median of 50%) markedly differed from Balaban et al.’s healthy controls (mean=12.8%). Similarly, in our healthy controls only 48.5% of saccades were identified as predictive, compared to 80% in their sample. We have no obvious explanation for these large discrepancies. Balaban et al. recruited their concussed participants from emergency departments, suggesting they may have had more severe injuries than as our athletes rarely were sent to the hospital. However, that is in contrast with their results because their concussed cohort performed better than our own (anti-saccade error rate ~30% in Balaban et al vs ~50% in our cohort). In addition, this does not explain our markedly different healthy cohorts. It is possible our population of athletes just performs differently than a general population sample as in Balaban et al, but we would subjectively expect improved responses due to athletes showing visuo-motor reaction times when compared to nonathletes.²⁴ The worse responses in our group may represent long-term sub-concussive effects even in our healthy athletes, but these effects would be dramatic. We were very careful to provide the same amount of coaching and training to participants with and without concussion in all tests not to bias the results. Our participants without concussion were young and healthy athletes who should not have difficulty with these tasks. Further research is needed to determine the clinical utility and best methodology for the anti-saccades and predictive saccades tasks.

Athletes with concussion had difficulty with vertical smooth pursuits, which became more obvious at higher frequencies. Horizontal smooth pursuit tasks were completed first in the protocol, so it could be that this finding may reflect fatigue or decreased focus and not in fact a deficient vertical smooth pursuit system. More work needs to be done looking at the smooth pursuit system in concussion. One limitation of using sinusoidal stimuli and focusing on the later predictive component of the response is that the open-loop gain of the system cannot be determined. This could be achieved using a Rashbass step-ramp stimulus and limiting the gain measures to the open-loop section of the smooth pursuit response.²⁵

Optokinetic reflex gain was not significantly different between groups. It is worth noting that this task provoked the most symptoms out of the battery of tests, and many participants with concussion could not tolerate it. The optokinetic test simulates the environment rotating around a stationary observer. In nature, retinal slip is typically generated by our own motion. Therefore, this stimulus causes sensory conflict, with the brain expecting corresponding vestibular signals to indicate head movement. Most healthy participants (98%), while experiencing sensation of movement, seemed to tolerate this conflict, while participants with concussion experienced symptom provocation and requested to stop (20%). Although the optokinetic system appears to be intact in athletes with concussion, the enhanced effect of this sensory conflict in these participants is worth exploring further. It may be possible to minimize this symptom exacerbation by using brief shifts of the pattern instead of prolonged stimuli. Clinicians testing OKN should be careful and constantly assess symptoms.

Athletes with concussion had slower simple visual reaction times (P=0.02) and dual-task visual reaction times (P=0.03) compared to athletes without concussion. Auditory reaction times were not significantly different (P=0.30). Although another study showed changes in auditory processing with concussion, those protocols used more complex tasks.²⁰ A possible explanation for this discrepancy is that visual reaction, especially when adding a saccadic task and decision to push the right or left button, is more sensitive to change than simple auditory reaction due to higher cognitive load. It has been shown frequently that dual-task performance is more sensitive to change than single task in patients with previous concussion.^26,27 This long term, specifically dual-task deficit may suggest that impaired inhibitory control and/or attentional resources last longer than changes in simple sensory processing, which may explain our acutely concussed sample’s differences in both simple visual and dual-task visual reaction time.

Strengths and Limitations

To our knowledge, this is the first study to describe test-retest reliability, smallest real difference and expected range of scores for healthy athletes aged 18–24 on typical visuo-oculomotor and reaction time tests. We were mindful of exacerbating symptoms during each testing session and stopped tasks if necessary. Therefore, we are confident that any differences seen in these tasks are not due to symptom provocation. It is possible, however, that fatigue, attention problems, or unwillingness to discuss symptoms may have had an impact.

A limitation of this study is that the concussed individuals were not all tested at the same time from injury. All were diagnosed as “concussed”, but due to the transient nature of concussion different times from diagnosis may present distinct oculomotor findings. In addition, while all acute concussions (<72 hours) in this study were diagnosed by the UAB team physician guided by their own physician opinion through Sport-Concussion Assessment Tool (SCAT-5)¹ and Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT)²⁸, the patients beyond 72 hours were not diagnosed by the same physician. It is possible these different physicians had different standards for diagnosis of clinically-significant concussion. However, as there is no uniform objective diagnosis, this is a limitation of many studies of similar design.²

Conclusions and Future Directions

These results suggest that athletes with concussion have impairment in their ability to follow a visual target, most prominently during random saccade tasks and vertical smooth pursuit. Although the Intraclass Correlation Coefficient results imply poor test-retest reliability, the small smallest real differences suggest that baseline testing may be of value.

Like previous studies, reaction time latencies were increased in concussion.^21,22 Novel to this study is evidence that not only gross motor reaction times are impaired, but also saccadic latencies. It is possible these changes in saccadic reaction time reflect the same changes in sensory processing that are responsible for changes in other reaction times.

We believe our decision to utilize both Intraclass Correlation Coefficient and smallest real difference in our discussion on reliability of these measures leads to a much more robust understanding on their clinical utility. It would be interesting to see how relevant the smallest real difference is for the tests we found significantly changed in concussion; in other words, does acute concussion lead to a greater change in dual-task reaction time from baseline than we would expect from our hypothesized smallest real difference? This would require baseline testing many athletes; however, this is already frequently done to some degree in colleges and adding some simple reaction time measures would not be difficult. More data on acute concussions who also had baseline data would help shed light on how relevant Intraclass Correlation Coefficients and smallest real differences are for clinical tests in concussion.

Future research will include more in-depth analysis of the oculomotor data to determine if there are changes in smooth pursuit and saccade signatures. If a real change in the oculomotor system is found, it might shed light on specific neural mechanisms affected by concussion to assist with targeted diagnosis, prognosis, and rehabilitation.