Reconsidering Vestibular/Ocular Motor Screening Cutoff Scores for Concussion

ABSTRACT

Introduction

Vestibular/Ocular Motor Screening (VOMS) is often part of a comprehensive evaluation to identify acute mild traumatic brain injury. Most of the reports describe the use of the VOMS in adolescents/young adults and not in older adults or military service members. The purpose of this study was to describe VOMS findings in healthy civilians and active duty military service members up to the age of 50 years.

Materials and Methods

Seventy-seven healthy civilians between 18 and 50 years of age (22 males, age 31.8 [9.0] years) participated across three sites in addition to 40 healthy active duty service members (25 males, age 27.5 [4.9] years) from one site. Demographics, Neurobehavioral Symptom Inventory scores, mean near point convergence (NPC) distance, and Total Symptom Change (TSS) scores from the VOMS were evaluated.

Results

For civilians, the group mean NPC distance was 4.98 (3.8) cm. For military service members, the group mean NPC distance was 6.17 (4.57) cm. For civilians, the mean TSS was 1.2 (2.3) with 53.2% reporting 0 TSS, 27.3% reporting one TSS, and 19.5% reporting two or more TSS. For military service members, the mean TSS was 0.20 (0.72) with 92.5% reporting 0 TSS, 0% reporting one TSS, and 7.5% reporting two or more TSS. Age did not correlate with the mean NPC distance and TSS in healthy civilians and active duty military service members.

Conclusions

Reconsideration of the Military Acute Concussion Evaluation, Version 2 cutoff value for abnormal mean NPC distance may be warranted to improve diagnostic accuracy in both civilian and military adult populations. Similarly, re-evaluating criteria for interpreting the TSS results of the VOMS, specifically in civilians, may be warranted.

INTRODUCTION

Vestibular/Ocular Motor Screening (VOMS) is often part of a comprehensive evaluation to identify acute mild traumatic brain injury (mTBI) and related vestibular and oculomotor issues in adolescent/young adult athletes.¹ In 2018, the Military Acute Concussion Evaluation was revised to include the VOMS. This Military Acute Concussion Evaluation, Version 2 (MACE2) is the required clinical tool for the assessment of mTBI/concussion for the Defense Health Agency.²

Few reports describe the use of the VOMS in individuals older than collegiate athletes or in military populations.^3–5 Original VOMS cutoff scores for identifying abnormal findings were set as an increase in symptoms of ≥2 points from baseline and a near point convergence (NPC) distance of >5 cm.¹ However, the MACE2 identifies any score above baseline (an increase ≥1 point) and a NPC distance of >5 cm as an abnormal finding.² There are anecdotal reports of extensive clinical use of the VOMS for adult non-athlete civilians with non-resolving symptoms from varied mechanisms of injury; however, the validity and diagnostic utility of the VOMS in these populations are understudied.

As with other self-report measures such as the Neurobehavioral Symptom Inventory (NSI),⁶ one important consideration when using the VOMS is that individuals with no history of mTBI can report symptoms similar to those who have sustained a concussion.^7–9 Factors like sleep deprivation^10,11 and non-brain-related injuries such as lower extremity orthopedic injuries¹² and generalized anxiety or stress¹³ can cause individuals with no mTBI history to report symptoms that would meet the International Classification of Diseases-10 symptom criteria for post-concussion syndrome. Given these reported findings, we anticipated that a one-point change in symptoms may erroneously identify a healthy individual as concussed.

Recognizing that the 5-cm cutoff proposed by Mucha and colleagues¹ was based on adolescent and college-aged athletes, a NPC distance in older individuals that is greater than 5 cm may need to be considered a normal variant. Normal NPC distance has been described as age dependent^14–16 and dependent on the font size and test method,¹⁷ with normal values reported to be 6-10 cm or more.^14–17

The purpose of this study was to describe VOMS findings in healthy civilian adults and active duty military service members up to age 50 years. Data for healthy civilian adults and military service members may improve interpretation of findings when the VOMS is used for adults with post-concussion symptoms across the age spectrum and outside of adolescent/collegiate athletic settings.

MATERIALS AND METHODS

Participants

Seventy-seven healthy civilian adults between 18 and 50 years of age (22 males, age 31.8 [9.0] years) from three sites (Oregon Health & Science University, Portland, Oregon; Courage Kenny Research Center, Minneapolis, Minnesota; and The University of Utah, Salt Lake City, Utah) and 40 healthy active duty service members (25 males, age 27.5 [4.9] years) from Joint Base San Antonio—Fort Sam Houston, Texas, participated. Data were collected between June 2019 and March 2022. Fifty-seven of the civilian subjects were recruited as part of a larger multisite study (ClinicalTrials.gov identifier: NCT03892291).¹⁸ An additional 20 subjects were recruited from the facility staff, family members, and visitors of the Courage Kenny Research Center for testing on the VOMS and NSI only. Inclusion criteria for all healthy participants allowed for a remote (>7 years) history of concussion with no lasting symptoms. Subjects for the larger study were excluded if they (1) had a history of any other injury or major psychological or neurological illness that could potentially impair their balance (i.e., stroke, lower extremity injury, recent surgery, and psychosis), (2) were pregnant, or (3) were unable to communicate in English. The study was approved by the Institutional Review Boards at each of the sites. Informed consent was obtained before participation. Information related to the Military Occupational Specialty (MOS) was collected for the military participants, whereas employment status was collected for the civilian participants.

Procedures

All participants were tested on a single occasion in a clinical research laboratory for evaluation of NSI, VOMS, and recording of demographic information. The NSI is a 22-item self-report questionnaire of neurobehavioral symptoms that is used for traumatic brain injury research and clinical assessment by the Department of Defense and Department of Veterans Affairs.⁶

The VOMS test battery includes smooth pursuit, saccades (horizontal and vertical), vergence, Vestibular Ocular Reflex (horizontal and vertical), and visual motion sensitivity.¹ Individuals rate symptoms (headache, dizziness, nausea, and fogginess) on a scale of 0 (none) to 10 (worst experienced) at baseline and after each test item.¹ The VOMS has high internal consistency (Cronbach’s alpha = 0.97).¹⁹ The NPC distance portion of the VOMS was calculated as the mean of three trials measured using a NPC ruler with a 14-point target held at the tip of the nose (Good-lite, Inc., Elgin, IL). The examiner slid the target toward the participant until they reported double vision/blurring of the target or the examiner observed one eye deviating from the target.

Statistical Analysis

To describe VOMS findings in healthy civilian adults and active duty military service members up to age 50 years, descriptive statistics (mean, standard deviation [SD], median, interquartile range [IQR] as appropriate) were calculated for primary outcomes of VOMS Total Symptom Change (TSS) scores and NPC distance, as well as the NSI Total Score. TSS was calculated as the sum of the change from baseline after each VOMS item, with any negative changes (symptom improvement from baseline) corrected to 0 before summing item change scores to reflect only symptom provocation. To determine if NSI, VOMS TSS, and mean NPC differed between military service members and civilians, non-parametric Mann–Whitney U-tests were implemented to test for group differences. Non-parametric tests were confirmed using Shapiro–Wilk tests to assess normality. To assess the association between VOMS outcomes and overall self-reported symptom burden, Spearman’s rank-order correlation coefficients assessed the relationship between NSI total score, VOMS TSS, and mean NPC within both civilian and military groups. Finally, recognizing potential age-related changes for NPC distance, we grouped the civilian participants into younger (18–40 years) and middle-aged (41–50 years old) groups for reporting of NPC distance findings. The military participants were kept as one group as only one military participant was over 40 years of age. IBM SPSS Statistics 27 (IBM Corporation, Armonk, New York, USA) were used for all statistical analyses using a significance level set at 0.05.

RESULTS

The data for TSS, NSI, and mean NPC distance were not normally distributed, with the Shapiro–Wilk test of normality significant at P ≤ 0.003. As there were 40 participants in the military and 77 participants in the civilian groups, we reported both mean (SD) and median (IQR) to fully characterize the data. For civilians, the mean TSS was 1.2 (2.3) points, the median was 0.0 (IQR = 12) points, with 53.2% of individuals reporting 0 TSS, 27.3% reporting one TSS, and 19.5% reporting a TSS of 2 or more. This was statistically higher symptoms than for the military participants (P < 0.001) where the mean TSS was 0.20 (0.72) points, the median was again 0.0 (IQR = 0) points with 92.5% of service members reporting 0 TSS, 0% reporting 1 TSS, and 3 individuals (7.5%) reporting a TSS of 2 or more. TSS on the VOMS by participant type (civilian and military) is shown in Fig. 1A.

FIGURE 1.

(A) Total Symptom Change Score by participant type. (B) Mean near point convergence distance by participant type.

The mean (SD) NSI total score for civilians was 4.8 (4.1) (range 0–22; median 4, IQR = 5), which was statistically greater (P = 0.026) than that for military participants who had a mean of 3.58 (4.61) (range 0–20; median 2, IQR = 5) (Table I). Using Spearman’s rho, the NSI total score did not correlate with the VOMS TSS in either civilian (r = 0.131, P = 0.255) or military (r = 0.034, P = 0.833) participants. Five of the 77 civilian participants had a NSI total score greater than 10 (range from 11 to 22), while 3 of 40 military participants had a NSI total score greater than 10 (range from 11 to 20). The MOS categories of the military participants reflected testing of individuals in non-combat specialties (Table I).

TABLE I.

Participant Characteristics

Characteristic	Civilian (n = 77)	Military (n= 40)
Mean age in years (SD)	31.8 (9.0)	27.5 (4.9)
Gender
Male	22	25
Female	55	15
Race/ethnicity
Asian/Pacific Islander	0	4
Black/African American	2	1
Hispanic/Latino	1	2
White/Caucasian	71	33
Biracial	3	0
Highest education level
High school graduate/GED	6	0
Some college/associates	11	2
Bachelor’s degree	18	10
Some graduate school	5	17
Master’s degree	22	6
Doctoral degree	15	5
Work status
<20 h/week	3	0
20–34 h/week	17	0
Working full time	47	40
Student	10	0
Military occupational specialty
Physical therapist	n/a	24
Human resources	n/a	7
Health services administration	n/a	2
Physical medicine	n/a	2
Cyber defense operations	n/a	1
Maritime Space Officer	n/a	1
Medical-surgical nurse	n/a	1
Physical therapy specialist	n/a	1
Physician assistant	n/a	1
Mean BMI (SD)	24.5 (4.1)	24.8 (3.3)
NSI total, mean (SD)	4.8 (4.1)	3.6 (4.5)
History of mTBI > 7 years	5 (6.5%)	5 (12.5%)

Abbreviations: BMI, body mass index; GED, General Educational Development test; NSI, Neurobehavioral Symptom Inventory; SD, standard deviation; mTBI, mild traumatic brain injury; n/a, not applicable.

For civilians, the group mean NPC distance was 5.0 (3.8) cm (Fig. 1B), with a median of 4.3 cm (IQR = 3.6). The mean NPC distance for younger civilians [18–40 years; (n = 62)] was 4.7 (3.3) cm, with 22 participants (35.5%) at >5 cm. The mean NPC distance for older civilians [41–50 years; (n = 15)] was 6.3 (5.4) cm, with seven participants (46.7%) at >5 cm. For military service members, the group mean NPC distance was 6.2 (4.6) cm, with a median of 5.6 cm (IQR = 5.5). These military service members were 22 to 43 years old. A scatterplot of mean NPC distances by participant type (civilian and military) and age (Fig. 2B) indicates that neither attribute contributed to the NPC distances found. The mean NPC distance was not statistically different between civilian and military groups (P = 0.209).

FIGURE 2.

(A) Total Symptom Change Score by age. (B) Mean near point convergence distance by age.

Spearman’s correlation coefficients between the TSS and age (Fig. 2A) showed little to no association in either civilians (r = −0.191, P = 0.097) or military service members (r = 0.205, P = 0.204). Similarly, Spearman’s correlation coefficients between the mean NPC distance and age showed little to no association in either civilians (r = 0.154, P  = 0.181) or military service members (r = 0.092, P = 0.574).

Given our sample size, our study was not sufficiently powered to determine if a remote concussion history influenced symptom reporting. Five service members in our cohort reported a remote history (>7 years) of concussion with only one of these five reporting a TSS change of 2 on the VOMS; the remaining four reported no symptom provocation on any VOMS component. Similarly, five of the 77 civilians reported a remote history of concussion with two of these five reporting a one-point increase in symptoms reflected in the TSS and the remaining three reporting no symptom provocation. The total NSI scores for the five remotely concussed military participants varied between 0 and 6, and for the civilians between 2 and 10.

DISCUSSION

The use of VOMS, both as part of the MACE2 for informing earlier diagnosis and treatment for mTBI in military personnel and for civilian use in identifying and following prolonged recovery from mTBI, is becoming increasingly common. VOMS scoring is symptom based; hence it is important to characterize responses for healthy individuals older than adolescent and collegiate athletes. In this study of military service members and civilians without a recent history of concussion, the mean and range of NPC and TSS scores suggest that a reconsideration of cut-off values is warranted to reduce the potential for false-positive results.

False positives may interfere with military force readiness or civilian clearance for full work/sport or other activity participation. Kontos et al.⁵ explored false-positive rates for identification of concussion in 416 healthy United States Army Special Operations Command personnel (28.0 ± 5.4 years of age) using scoring cutoffs as outlined in the original VOMS article (≥2-point increase on VOMS symptom items and >5 cm for NPC distance).¹ This group identified VOMS item false-positive rates from 10.6% (smooth pursuits) to 17.5% (NPC), and also described an association between the false-positives with age and with a history of mTBI. Since the MACE2 specifies that any score above baseline (1 or more point increase) as an abnormal finding, these findings⁵ may reflect an underreporting of VOMS false-positives in the United States Army Special Operations Command cohort of healthy military personnel. We found that 46.8% of healthy civilians reported symptom provocation on at least 1 VOMS item, while 3 of 40 (7.5%) of healthy service members reported symptom provocation of at least 2 on the VOMS items. The remaining 37 military participants reported no symptom provocation on any VOMS item. Although limited by sample size, our findings did not identify a relationship between age (Fig. 2A) and prior history of concussion with symptom provocation on the VOMS.

It is not uncommon for military medical providers to evaluate and treat civilian victims of trauma in garrison and in deployed settings, where they follow Department of Defense policy guidance. When using the MACE2 criteria of any increase in score above baseline (one or more point change), almost one-half of the civilian sample would be incorrectly identified as abnormal. This would result in a large number of healthy civilian adults being restricted from work and recreational activity and increase the burden on healthcare systems for unwarranted follow-up care. Using the originally published criteria of a two or more point change as identifying concussion, only one-fifth of the civilian sample would be incorrectly classified. Using either the original cutoff¹ or the MACE2² cutoff of one point, only three healthy service members would have been incorrectly classified. This less frequent finding of a false-positive base on TSS in service members may be limited by our sample size and the MOS categories of military participants. Future studies should validate our recommendation that applying the originally published criteria of a two or more point change (instead of the current policy of a one or more point change) would result in fewer false positives in civilian and military participants evaluated using the MACE2.

For healthy civilians, the group mean NPC was <5 cm. However, using >5 cm as a cutoff for an abnormal finding, 35.5% of healthy civilians aged 18–40 years would have incorrectly been identified as concussed. Similarly, 46.7% of healthy civilians aged 41–50 years would have incorrectly been identified as concussed. Again, this would result in many healthy civilian adults being restricted from work and recreational activity and increase the burden on healthcare systems for unwarranted follow-up care. Given the large percentage of our civilian cohort that would have been incorrectly identified as having a concussion based on an abnormal NPC during VOMS testing, the >5 cm cutoff should be reevaluated.

Although the VOMS sets the cutoff at >5 cm for an abnormal finding on NPC distance, other literature describes different cutoff distances both higher and lower than that proposed in the VOMS. Based on the testing in adolescents, Elbin et al. suggest an abnormal cutoff at ≥3 cm.²⁰ Scheiman and colleagues tested 175 optometry students between 22 and 37 years of age and agrees with a 5-cm cutoff for normal NPC distance.²¹ Population-based studies in Iran with more than 3800 participants reported NPC distances that varied with age; mean NPC distance in those under 5 years of age were 5.6 (±1.9) cm and increased to 10.4 (±2.8) cm for those over 70 years of age.^14,15 Abraham and colleagues¹⁶ reported age-dependent findings as well; average NPC distance was 6.3 (2.8) cm for 10- to 18-year olds and 9.5 (3.0) cm for 28-to 35-year olds with normal corrected vision. Findings for U.S. school children from kindergarten to sixth grade identified a NPC distance of ≤6 cm as normal,²² further highlighting the heterogeneous values within the literature. While our findings did not support an age-related increase in NPC distance (Fig. 2B), they provide valuable normative data that adds to the available literature on NPC distance in healthy non-concussed adults.

The large variance in reported NPC distance across healthy populations makes establishing any single cutoff value for clinical diagnosis challenging, but our findings and others suggest that the >5 cm cutoff is too low. A study specifically designed to evaluate methods for testing NPC distance as described in the VOMS, using a 14-point font target, found an average 6.2 cm (95% confidence interval 5.6–6.9 cm) NPC distance in a group of 75 recreational athletes between 18 and 50 years of age from a university setting.¹⁷ Our findings agree with this study; healthy military members exhibited a mean NPC distance >6 cm. Using >5 cm as a cutoff for an abnormal finding, 52.5% of healthy military service members in our cohort would have incorrectly been identified as “concussed.” Applied across a typical platoon (about 40 service members), approximately 21 soldiers per platoon could be misclassified with a false-positive concussion and removed from duty, thereby creating devastating effects on combat power and force readiness. These service members identified as concussed would enter a 7-day (minimum) progressive return to activity protocol, resulting in lost duty days.²³ Further, following the recommendation for follow-up with a provider every 3 days may quickly overwhelm military medical resources. This highlights a need, especially in large-scale combat operations, to train every medical provider in properly administering and interpreting the MACE2. A misdiagnosis of concussion could potentially allow a healthy service member to incorrectly qualify for disability benefits. Given the large percentages of our healthy military cohort that would have been incorrectly referred for follow-up evaluation based on an abnormal NPC during VOMS testing, the >5 cm cutoff should be reevaluated.

A consideration of the VOMS as a diagnostic tool for concussion is its reliance on self-reported symptoms. Reports of symptoms similar to those described after a concussion are not specific to brain injury and have been reported in healthy adolescent athletes,⁹ adults with non-brain-related injuries^10–12 and active duty service members.^7,8 A symptom profile similar to post-concussion complaints occurs more often in individuals with attention-deficit hyperactivity disorder, depression or anxiety, and other related comorbidities.¹³ When constructed on the total symptom change score, the VOMS is based on symptom provocation with baseline symptom report subtracted from post-test symptom report and may therefore be expected to be less affected by baseline symptoms related to comorbidities.

The relatively small number of participants with limited diversity (Table I) in our study restricts the generalizability of our study findings. A potential limitation of this research study is that the military participants were 22–43 years of age with only 1 participant over age 40. Additionally, only active duty military service members participated in this research with a limited range of MOS categories. Future studies should explore VOMS findings in diverse populations, older military and civilians, active duty military including those in a full range of MOS categories, and National Guard and Reserve components. Despite these limitations, these preliminary findings begin to formulate a picture of VOMS characteristics in healthy adults and are relevant to clinical management of both civilian and military individuals recovering from mTBI.

CONCLUSION

The reconsideration of the MACE2 cutoff value for abnormal mean NPC distance may be warranted to improve diagnostic accuracy in both civilian and military adult populations. Similarly, reevaluating criteria for interpreting the TSS results of the VOMS, specifically in civilians, may be warranted to avoid unnecessary removal from work, sport, or duty following a potentially concussive event.

FUNDING

This work was supported by the Assistant Secretary of Defense for Health Affairs endorsed by the Department of Defense, through the Congressionally Directed Medical Research Program under award no. W81XWH1820049.

CONFLICT OF INTEREST STATEMENT

None declared.

DATA AVAILABILITY

The data that support the findings of this study are available on reasonable request to the corresponding author.

CLINICAL TRIAL REGISTRATION

ClinicalTrials.gov identifier: NCT03892291

INSTITUTIONAL REVIEW BOARD (HUMAN SUBJECTS)

The study was approved by the Institutional Review Boards at each of the sites: Oregon Health and Science University (#18749), University of Utah (#114616), Courage Kenny-Allina Health (#1311177), and Joint Base San Antonio-Fort Sam Houston (#935315).

INSTITUTIONAL ANIMAL CARE AND USE COMMITTEE

Not applicable.

INDIVIDUAL AUTHOR CONTRIBUTION STATEMENT

L.A.K., M.M.W., M.E.L., L.E.D., and P.C.F. designed the procedures and obtained funding for the larger study from which these data originate. M.M.W., L.E.D., L.P., R.P., P.G.M., H.R., C.W.H., M.E.L., and P.C.F. coordinated and assisted with the data collection procedures for the data reported in this study. C.W.H., M.M.W., and L.A.K. conceptualized the question of interest for this paper. All authors contributed to refining the objective of the current study. M.M.W., C.W.H., and L.A.K. conducted all statistical analysis. M.M.W., C.W.H., P.G.M., and L.A.K. created figures and tables and wrote the first draft. All authors contributed to the interpretation and revising of subsequent drafts and approved the final manuscript.

INSTITUTIONAL CLEARANCE

Institutional clearance from the U.S. Army Medical Center of Excellence is pending.