Concussion-Management Practice Patterns of National Collegiate Athletic Association Division II and III Athletic Trainers: How the Other Half Lives

Abstract

Context

The National Collegiate Athletic Association (NCAA) has published concussion-management practice guidelines consistent with recent position and consensus statements. Whereas NCAA Division I athletic trainers appear highly compliant, little is known about the concussion-management practice patterns of athletic trainers at smaller institutions where staffing and resources may be limited.

Objective

To descriptively define the concussion-management practice patterns of NCAA Division II and III athletic trainers.

Design

Cross-sectional study.

Setting

Web-based questionnaire.

Patients or Other Participants

A total of 755 respondents (response rate = 40.2%) from NCAA Division II and Division III institutions.

Main Outcome Measure(s)

The primary outcome measures were the rate of multifaceted concussion-assessment techniques, defined as 3 or more assessments; the specific practice patterns of each assessment battery; and tests used during a clinical examination.

Results

Most respondents indicated using a multifaceted assessment during acute assessment (Division II = 76.9%, n = 473; Division III = 76.0%, n = 467) and determination of recovery (Division II = 65.0%, n = 194; Division III = 63.1%, n = 288) but not at baseline (Division II = 43.1%, n = 122; Division III = 41.0%, n = 176). Typically, when a postconcussion assessment was initiated, testing occurred daily until baseline values were achieved, and most respondents (80.6% [244/278]) reported using a graded exercise protocol before return to participation.

Conclusions

We found limited use of the multifaceted assessment battery at baseline but higher rates at both acute assessment and return-to-participation time points. A primary reason cited for not using test-battery components was a lack of staffing or funding for the assessments. We observed limited use of neuropsychologists to interpret neuropsychological testing. Otherwise, most respondents reported concussion-management protocols consistent with recommendations, including a high level of use of objective measures and incorporation of a progressive return-to-participation protocol.

Key Points

• Use of a baseline multifaceted concussion protocol by National Collegiate Athletic Association Divisions II and III athletic trainers (ATs) was limited.

• Overall, concussion-management practice patterns of these ATs were largely consistent with recommendations, including the use of a multifaceted assessment battery at acute assessment and a return-to-participation protocol.

• Use of objective assessments was less among Division II and III ATs than among Division I ATs, possibly reflecting staffing and funding differences.

• Interpretation of neuropsychological testing by neuropsychologists was limited.

Concussions have recently moved to the forefront for the general public, legislators, and the sports medicine community, as all 50 states have passed concussion-management legislation and scientific bodies have produced numerous position or consensus statements.^1–3 In 2010, the National Collegiate Athletic Association (NCAA) released a revised concussion-management policy based primarily on the consensus statement from the 3rd International Conference on Concussion in Sport (CIS), which both provided institutional requirements and recommended best practices for concussion management.^4,5 The required aspects of the management plan were a concussion-management plan; a process for student-athlete acknowledgment of symptom reporting; immediate removal from participation; no same-day return to participation for athletes suspected of having concussions; and an institutional policy requiring medical clearance for return to participation after a supervised, stepwise exertional program. In addition, the policy recommended multifaceted baseline testing, including symptoms, balance, cognitive, and neuropsychological testing, the last of which should be interpreted by a neuropsychologist. Noncompliance with the requirements would result in an institutional violation; however, no institutions to date have been penalized publicly despite multiple high-profile potential incidents.^6–8

During the past decade, substantial changes have occurred in concussion-management practice patterns as the treatment paradigms have transitioned from subjective grading-scale–based guidelines to multifaceted objective-testing protocols.^3,9 Indeed, early studies of the practice patterns of athletic trainers (ATs) indicated heavy use of physician recommendations and return-to-participation guidelines as principal evaluative tools for concussion management, with only 3% using a multifaceted objective-assessment battery.^10,11 It is encouraging to note that a decade later, almost all NCAA Division I (DI) ATs performed baseline testing, and more than two-thirds used multifaceted assessments at baseline, at acute injury evaluation, and when making return-to-participation decisions.⁹ Perhaps because of these improved assessment procedures and increased concussion awareness, the rate of diagnosed concussions doubled in recent years in a small sample of DI programs.¹² Unfortunately, most investigations of the concussion-management practice patterns of ATs have been across employment settings, and ATs in corporate health, academic, high school, collegiate, and professional sports were grouped despite differences associated with these settings, which restrict the applicability of these studies.^{10,11,13–15} Authors⁹ of only 1 study have examined concussion management in a specific practice setting, NCAA DI ATs, so little is known about the practice patterns in NCAA Division II (DII) or Division III (DIII).

Considerable recent media attention has focused on concussion-related concerns in NCAA DI athletics; however, the concussion rate may actually be higher in DII and DIII.¹⁶ Furthermore, these schools are not immune to tragic outcomes or legal liability associated with sport-related concussion.¹⁷ However, a substantial difference exists in the annual program budgets of athletic departments between DI Football Bowl Subdivision ($56 million) and DII ($4.0–$5.3 million) and DIII ($1.4–$3.0 million), which may affect the funding for available ATs and equipment to appropriately manage concussions.^18–20 Therefore, the purpose of our study was to assess the concussion-management practice patterns among NCAA DII and DIII ATs. We aimed to provide a thorough description of the practice patterns among NCAA DII and DIII ATs to allow a descriptive comparison with DI ATs and to evaluate compliance with existing consensus statements and NCAA policies.

METHODS

Participants

To identify all NCAA DII and DIII ATs, we viewed the intercollegiate athletics Web sites of each of the 720 institutions classified as DII (N = 296) and DIII (N = 424) in the 2012 to 2013 academic year. Athletic trainers were included if they were classified as full-time certified or licensed ATs who provided athletic training services to the institution's collegiate athletics program and had publicly available e-mail addresses. Potential participants were excluded if they were classified as part-time ATs, interns, or graduate-assistant ATs or if they were not employed by the host institution. We made every effort to identify only clinically practicing ATs and to exclude academic or research faculty members unless the intercollegiate athletics Web site indicated that they also provided clinical athletic training services (eg, they were listed as being responsible for athletic training services for a specific team or supervised an athletic training facility). From these criteria, we identified 851 DII and 1125 DIII e-mail addresses. All participants implied consent by selecting the link within the e-mail to begin the questionnaire, and the study was approved by the Georgia Southern University Institutional Review Board (H11191).

Procedures

We e-mailed 1976 ATs in May 2013 and invited them to participate in the study, sending follow-up reminder e-mails 7 and 14 days after the initial e-mail. The e-mail included an overview of the study and appropriate informed consent documentation and provided a hyperlink to the questionnaire that the volunteers could select if they opted to participate in the study. If they agreed to participate, they were taken to a Web site (Surveymonkey.com; Palo Alto, CA) to complete the questionnaire. Participants were not required to answer all questions and could exit the questionnaire at any time; however, they were not provided with the option of returning to an earlier page and changing an answer.

The 65-item questionnaire has been used in a study of NCAA DI ATs.⁹ We originally based the questionnaire on earlier investigations of the concussion-management patterns of ATs and then expanded those questionnaires to include detailed follow-up questions.^10,11,14 At the time the questionnaire was developed, the 3rd CIS was the current international consensus statement, so the questionnaire referred to it.⁵ The questionnaire consisted of 3 sections: (1) a demographic section that consisted of 13 questions related to the respondent's demographics, professional experience, number of concussions assessed, and awareness of concussion consensus statements but did not include personal or institutional identifying information; (2) a concussion-assessment section with 31 questions on the acute evaluation of a suspected concussion; and (3) a concussion-management section containing 21 questions specific to recovery and return-to-participation procedures after a concussion. Sections 2 and 3 began with questions about the inclusion of general testing techniques (eg, balance testing), and the sections clearly delineated between acute assessment and identifying recovery. Follow-up questions then were designed to ascertain the specific practice patterns of the respondent if he or she indicated using a particular test (eg, which test was used, when the test was conducted, how often it was performed, in what environments it occurred). These questions used the logic feature of the software and, therefore, not all respondents were presented with all questions in sections 2 and 3. For example, if respondents indicated that they did not use balance testing, they were not presented with follow-up questions on that test administration. Instead, they were presented with a set of mutually exclusive questions on why the testing was not performed. Many follow-up questions used “check-all-that-apply” responses, as multiple responses were reasonably expected and, therefore, responses could exceed 100%. Respondents were allowed to skip questions, which resulted in uneven response numbers per question. Finally, respondents were given the opportunity to write in responses to many questions, and these responses were categorized appropriately.

Data Analysis

The responses were exported into Excel (version 2010; Microsoft Corporation, Redmond, WA) for analysis. Demographic data and questionnaire responses were reported with frequencies, means, and standard deviations. To maintain consistency with the NCAA DI ATs' concussion-management practice-pattern study, a concussion-assessment protocol was operationally defined as multifaceted if it included at least 3 of the recommended tests from the balance, cognitive, neuropsychological, and symptoms checklists.⁹ The phrase cognitive testing, consistent with the 3rd CIS, refers to Standard Assessment of Concussion (SAC) testing rather than formal neuropsychological testing, as the various terminologies (cognitive, neurocognitive, neuropsychological) have been used interchangeably in the literature.⁵ The clinical examination was presented as a frequency distribution of the 22 items in the questionnaire. Evaluation procedures, which were written in by respondents, were classified by the research team and, if appropriate, added to the selected options, grouped with similar write-in options, or reported independently.

RESULTS

Participant Characteristics

We initially sent 1976 e-mails; however, 96 (4.9%; DII = 65, DIII = 31) e-mails were returned as undeliverable or invalid. Of the remaining 1880 potential participants, 755 ATs (DII = 299, DIII = 456) responded, for a 40.2% response rate. The highest level of education for most respondents was a master's degree for 82.8% (246/297) of DII and 85.6% (391/457) of DIII respondents, and all 10 National Athletic Trainers' Association (NATA) districts were represented. Participant demographics are provided in Table 1. Nearly all respondents were aware of the 2004 NATA²¹ position statement (DII = 94.7% [232/245], DIII = 95.7% [355/371]), and a high percentage were also aware of the 3rd CIS (DII = 83.7% [242/289], DIII = 81.8% [356/435]). The most common methods for learning of the statements were through journal articles (DII = 33.8% [77/228], DIII = 36.3% [121/333]) and at conferences (DII = 26.8% [61/228], DIII = 26.7% [89/333]). Most respondents (86.1% [522/606]) reported receiving 7.3 ± 4.0 continuing education units (CEUs) on concussion-related topics over the previous 3 years. The most commonly reported methods for learning about concussions were research articles (80.3% [484/603]), position statements (71.3%, n = 430), and conferences (70.8%, n = 427).

Table 1.

Respondent Characteristics

	National Collegiate Athletic Association Division
Characteristic	II	III
Sex, % (n/total)
Men	52.5 (157/299)	48.5 (221/456)
Women	47.5 (142/299)	51.5 (235/456)
Athletic training experience, y
Mean ± SD	11.5 ± 8.2	11.4 ± 8.1
Mode	3	6
Range	1–54	1–60
Concussions evaluated annually, No.
Mean ± SD	9.2 ± 5.8	8.0 ± 5.3
Range	0–26	0–45
Certification route, % (n/total)
Commission on Accreditation of Athletic Training Education or  Commission on Accreditation of Allied Health Education program	57.2 (171/299)	63.0 (288/457)
Internship program	30.4 (91/299)	23.6 (106/457)
Pre-Commission on Accreditation of Athletic Training Education  or pre-Commission on Accreditation of Allied Health Education  curriculum program	10.7 (32/299)	13.3 (61/457)
Job title, % (n/total)a
Head athletic trainer	41.8 (123/294)	43.2 (192/445)
Assistant athletic trainer	39.1 (115/294)	38.4 (171/445)
Split responsibilities	12.2 (36/294)	15.3 (68/445)
Director of sports medicine	6.5 (19/299)	NA
Associate athletic trainer	NA	4.5 (20/445)
Other	5.8 (17/299)	5.8 (26/445)
Athletic training sport responsibilities, % (n/total)a,b
Baseball	35.4 (104/294)	39.8 (177/445)
Basketball	61.6 (181/294)	66.1 (294/445)
Bowling	2.4 (7/294)	NA
Cheerleading	10.9 (32/294)	9.7 (43/445)
Cross-country	37.4 (110/294)	40.4 (180/445)
Fencing	NA	1.8 (8/445)
Field hockey	7.8 (23/294)	18.9 (84/445)
Football	33.0 (97/294)	32.4 (144/445)
Golf	21.1 (62/294)	19.8 (88/445)
Gymnastics	NA	1.8 (8/445)
Ice hockey	6.8 (20/294)	11.2 (50/445)
Lacrosse	19.4 (57/294)	38.0 (169/445)
Rifle	1.4 (4/294)	NA
Rowing	1.4 (4/294)	6.3 (28/445)
Skiing	1.4 (4/294)	NA
Soccer	49.3 (145/294)	55.3 (246/445)
Softball	38.4 (113/294)	45.2 (201/445)
Swimming and diving	11.2 (33/294)	25.6 (114/445)
Tennis	31.3 (92/294)	42.2 (188/445)
Track and field	27.6 (81/294)	28.1 (125/445)
Volleyball	36.4 (107/294)	41.6 (185/445)
Water polo	2.7 (8/294)	2.9 (13/445)
Wrestling	7.5 (22/294)	9.2 (41/445)
Other	3.4 (10/294)	5.8 (26/445)

Abbreviation: NA, not applicable.

^aRespondents were instructed to choose all that applied and, therefore, the total may exceed 100%.

^bRespondents were instructed to choose all sports for which they directly provided athletic training services but not to include sports for which they supervised graduate assistants, interns, or other staff members.

Baseline Testing

Most respondents did not use a multifaceted concussion assessment at baseline (DII = 43.1%, n = 122; DIII = 40.1%, n = 176; Figure 1). Among the individual assessment tools at baseline, computerized neuropsychological testing (CNT) was most commonly used (90.8% [445/490]), followed by self-reported symptoms (69.7% [428/614]) and balance testing (77.6% [422/544]); the least commonly used assessment technique was cognitive testing (60.4% [299/495]; Table 2, Figure 2). Most often, baseline testing occurred on 1 occasion when the student-athlete entered the institution and was performed by any available AT, and the results were stored in the student-athlete's medical file (Table 2). Furthermore, more than one-quarter of 712 respondents (27.5%, n = 196) reported performing either 1 (19.8%, n = 141) or 0 (7.7%, n = 55) baseline concussion assessments, whereas 12.4% (n = 88) reported using all 4 baseline assessments. Of the 141 respondents who used only 1 assessment, the most common tool was CNT (48.9%, n = 69) and the least, cognitive testing (8.5%, n = 12).

Figure 1.

Multifaceted concussion-assessment protocols performed by athletic trainers in National Collegiate Athletic Association Divisions II and III combined. Three or more objective assessment tools were used by 41.9% (298/712) of respondents at baseline, 76.4% (550/720) during the acute concussion assessment, and 63.9% (419/656) for the return-to-participation evaluation.

Table 2.

Concussion-Assessment Practice Patterns^a

	Assessment
Characteristic	Balance	Cognitive/Mental Status	Neuropsychological	Symptoms
Overall use, % (n/total)
Division II	79.7 (228/286)	71.9 (197/274)	74.6 (200/268)	94.0 (250/266)
Division III	81.3 (351/432)	73.7 (300/407)	75.1 (295/393)	93.8 (365/389)
Baseline testing, % (n/total)
Division II	79.0 (168/213)	64.3 (126/196)	88.8 (174/196)	72.4 (181/250)
Division III	76.7 (254/331)	57.9 (173/299)	92.2 (271/294)	67.9 (247/364)
When is baseline testing performed?, % (n/total)
	One time only as student-athlete enters the program: 69.3 (248/358)	One time only as student-athlete enters the program: 71.1 (207/291)	One time only as student-athlete enters the program: 77.8 (339/436)	One time only as student-athlete enters the program: 70.3 (284/404)
	Annually: 24.0 (86/358)	Annually: 19.9 (58/291)	Biennially: 11.9 (52/436)	Annually: 19.8 (80/404)
Who performs the baseline test?, % (n/total)
Any available athletic trainer	43.1 (156/362)	42.5 (121/285)	NA	NA
Team-specific athletic trainer	22.7 (82/362)	19.6 (56/285)	NA	NA
Where are the test results stored?, % (n/total)
Student file	71.1 (261/367)	64.5 (185/287)	NA	NA
Online	18.8 (69/367)	32.4 (93/287)	NA	NA
Where is the test performed postinjury?, % (n/total)
Athletic training room	63.8 (241/378)	51.4 (242/471)	NA	NA
Sidelines	57.7 (218/378)	67.1 (316/471)	NA	NA
When is the test first readministered postinjury?, % (n/total)
	When symptom free: 65.9 (270/410)	Day 1 postinjury: 52.3 (174/333)	When symptom free: 53.9 (241/447)	Day 1 postinjury: 91.6 (500/546)
	Day 1 postinjury: 28.8 (118/410)	When symptom free: 39.9 (133/333)	Day 1 postinjury: 30.6 (137/447)	Day 2 postinjury: 4.9 (20/401)
How often is the test performed during recovery?, % (n/total)
	Daily: 58.0 (215/371)	Daily: 57.2 (171/299)	Weekly: 32.0 (108/338)	Daily: 85.2 (472/554)
	Every other day: 24.8 (92/371)	Every other day: 24.1 (72/299)	Daily: 22.5 (76/338)	Every other day: 6.9 (38/554)
Time until the test result reaches baseline, d
Mean ± SD	5.0 ± 2.8	5.0 ± 2.6	6.4 ± 3.2	5.6 ± 2.9
Mode	5	4	7	5
Range	1–25	0–14	0–25	0–25

Abbreviation: NA, not applicable.

^aSome questions allowed respondents to select multiple responses; therefore, responses may exceed 100%.

Figure 2.

Respondents' use of each component of the multifaceted concussion protocol at baseline, at acute assessment, and during return-to-participation decisions.

Acute Assessment

We observed near universal (97.1% [597/615]) incorporation of a concussion-management plan by the respondents. The use of a multifaceted concussion battery was highest during the acute assessment of a suspected concussion (DII = 76.9%, n = 473; DIII = 76.0%, n = 467).

Most respondents reported using balance testing as part of their concussion-evaluation and -management programs (80.6% [579/718]). The most commonly used balance test was the Balance Error Scoring System (BESS) (77.6% [422/544]) followed by the Romberg test (38.8%, n = 211). Technologically based assessments, including “exergamming” tests (1.7%, n = 9) and the Sensory Organization Test (1.1%, n = 6), were rarely used. Balance testing was most often performed 5 to 15 minutes postinjury (40.7% [153/376]), with the person either wearing socks (33.7% [138/409]) or barefoot (33.3%, n = 136). The original BESS (3 stances on 2 surfaces; 47.5% [187/394]) and the Sideline Concussion Assessment Tool-2 BESS (3 stances on firm surface only, no foam; 48.7% [192/394]) were frequently used. The BESS scoring was split between the revised version, with multiple errors occurring simultaneously being counted as 1 error (56.9% [228/401]), and the original version, with each error being counted independently (41.1%, n = 165). The 2 primary reasons respondents indicated for not performing balance testing were “other tools allowed us to recognize and assess the presence of a concussion without having to employ balance testing” (58.7% [74/126]) and lack of staffing or funding to perform the tests (56.3%, n = 71).

Both cognitive tests (73.0% [497/681]) and CNT (74.9% [495/661]) were extensively endorsed by the respondents (Table 2). The primary reasons the SAC test was not used by respondents were “other tools allowed us to recognize and assess the presence of a concussion without having to employ SAC testing” (66.9% [117/175]), using a different cognitive test (46.9%, n = 82), and lack of staffing or funding to perform the test (13.1%, n = 23). The most commonly used CNT was ImPACT (ImPACT Applications, Inc, Pittsburgh, PA; 90.9% [422/464]), with the remaining 9.1% (n = 42) split between pen-and-paper testing (5.2%, n = 24) and other commercially available computerized programs (3.9%, n = 18). The CNT results were most often interpreted by ATs (85.6% [403/471]) and physicians (72.0%, n = 339); less than one-fourth were interpreted by a neuropsychologist (22.7%, n = 107). The primary reason CNTs was not used by respondents were lack of staffing or funding to perform the test (78.4% [120/153]), use of tools other than CNT to manage a concussion (40.5%, n = 62), and their physicians' advising against using CNT (12.4%, n = 19).

The use of self-reported symptom checklists was overwhelmingly endorsed by the respondents (93.9% [615/655]; Table 2). A minority of respondents indicated using concussion-grading scales within their concussion-management plan (27.7% [183/661]). Most respondents (71.7% [450/628]) reported using a clinical examination as part of the concussion assessment, and the specific components are presented in Figure 3. Finally, the use of imaging techniques (21.3% [134/628]) and neuropsychologist referral (13.9%, n = 87) was limited among respondents.

Figure 3.

Components of the clinical concussion assessment that exceeded 50% of respondents' endorsement.

Return-to-Participation Assessment

The respondents generally endorsed a multifaceted assessment approach to identifying concussion recovery and making return-to-participation decisions (DII = 65.0%, n = 194; DIII = 63.1%, n = 288); however, a small percentage (10.8% [71/656]) reported using no or 1 assessment tool. The most frequently used assessments were symptoms (92.0% [575/625]) and CNT (73.7% [463/628]), followed by balance testing (64.7% [422/652]) and SAC (54.4% [347/638]). In addition, most respondents used a clinical examination (86.2% [244/283]), physician recommendation (72.4%, n = 205), or return-to-participation guidelines (70.3%, n = 199) in assessing concussion recovery. After a concussion was diagnosed, follow-up testing was performed either the first day postinjury (SAC and symptoms) or when the patient was symptom free (balance and CNT). After testing was initiated, CNT was repeated weekly, whereas the remaining assessment tools were administered daily (Table 2). Most respondents (80.6% [224/278]) reported using a graded exercise protocol that lasted an average of 4.8 ± 1.4 days. Respondents indicated that the average time from injury to full unrestricted participation was 8.8 ± 3.3 days.

DISCUSSION

The NCAA's best practices for concussion-management guidelines, which are consistent with the recommendations of most consensus and position statements, indicate that baseline measurements of cognitive, neuropsychological, and balance tests and self-reported symptoms should be performed; these measurements can then be repeated to both acutely assess a concussion and help make return-to-participation determinations.^2,3,7 Our main finding was a lack of multifaceted baseline assessments by NCAA DII and DIII ATs; however, the multifaceted approach was more common for acute assessment and return to participation. The primary reason provided for not performing some of the components, particularly CNT, was insufficient staffing and funding to perform the assessments. At all 3 time points, the use of the multifaceted assessment battery was less than previously reported among NCAA DI ATs (baseline = 28.1%–31.9%, acute evaluation = 2.3%–3.2%, return to participation = 1.9%–3.8%).⁹

Baseline testing has been endorsed by the NCAA Sports Medicine Handbook best practices and consensus statements.^2,3,7 However, multifaceted baseline testing was endorsed by fewer than half (40.1%–43.1%) of the respondents. Unfortunately, more respondents reported using 1 or no assessment (27.5%) than all 4 assessments (12.3%) during baseline testing and, most concerning, 7.7% reported performing 0 baseline assessments. This response rate is far less than that reported for NCAA DI ATs, 71.2% of whom used multifaceted baseline testing, with few respondents indicating 1 (9.0%) or 0 (1.1%) baseline tests.⁹ Yet in a recent study across practice settings, Rigby et al¹⁵ reported that 24.3% of respondents did not perform baseline testing, which suggests that collegiate ATs may exceed their professional colleagues in use of baseline testing. The primary reason our respondents provided for not performing baseline testing was a lack of funding or staffing to perform the tests. Specifically, most respondents who did not use CNT (78.4%) or balance testing (56.3%), respectively, indicated a lack of funding or staffing to perform the tests. Unfortunately, the questionnaire did not further investigate this, but it is likely that the lack of funding was associated with purchasing computerized testing, whereas the lack of staffing may have limited the ability to perform time-consuming balance or cognitive testing. This observation is consistent with reports from a recent NCAA concussion-management study in which clinicians across all divisions identified better preseason testing and more sports medicine staff as 2 primary areas to improve concussion-management plans.²² The NCAA DII and DIII athletic budgets are, on average, substantially smaller than DI budgets (<10% of the latter), which may directly influence concussion-management practice patterns because these limited resources may restrict staff hiring and the ability to purchase expensive computerized testing programs annually. Indeed, based on publically available Web-site information, the mean staff size was 2.7 ATs for DII and DIII institutions and 5.6 ATs for DI institutions, which may further restrict the staff's ability to perform time-consuming testing procedures. Thus, creative solutions to financing student-athlete health care at DII and DIII levels are required.

Concussion underreporting, either due to a lack of awareness or intentionally hiding the injury, is highly prevalent among athletes.²³ Therefore, to accurately identify concussions, ATs should use both a thorough clinical examination and an objective multifaceted assessment battery and should not rely solely on athlete self-report of symptoms.^2,3,7 The multifaceted approach is especially critical because individual components of the assessment battery have poor sensitivity and specificity.^24,25 Whereas baseline testing is recommended to maximize the sensitivity of the assessment, comparison with normative data is a viable alternative.²⁶ It is encouraging that the acute assessment time point had the highest use of the multifaceted assessment at 76.0% to 76.9% of respondents. This percentage of respondents represents a substantial increase from a decade earlier, when only about 3% used multifaceted objective assessments, but it is still slightly less than that of NCAA DI ATs (79.2%).^9,11 Rigby et al¹⁵ recently postulated that ATs who perceived that they lacked the authority to implement recommended guidelines were less likely to use multifaceted concussion assessments. The 2013–2014 NCAA Sports Medicine Handbook indicated that athletic health care providers should have “unchallengeable authority” to make concussion-related decisions; however, numerous anecdotal reports suggest that this is not uniformly practiced.^7,27 Resource control, which includes adequate staffing and funding, was also a barrier to ATs establishing a multifaceted concussion-management protocol, and our respondents indicated that these were primary reasons that tests were not performed.¹⁵ This response is similar to a recent report on concussion-management practice patterns of secondary school ATs, who reported lack of money and time as the primary factors for not performing baseline testing.²⁸ Thus, a real or perceived lack of funding, staffing, and control at the DII and DIII levels may adversely affect student-athlete health care, specifically as it relates to concussion management, and has been identified as an area for improvement in concussion management.²²

Beyond objective assessment techniques, the clinical examination remains a key component and was used widely by respondents.⁷ Authors^{10,11,13–15,29} of most concussion-management practice-pattern studies have simply reported clinical examination without operationally defining the components. Our respondents defined a clinical examination similarly to the way DI ATs defined it, with memory and orientation questions being the most common components (Figure 3).⁹ A minority of respondents (27.7%) indicated using grading scales in their concussion management; this which is consistent with the current NATA position statement and 4th CIS, which emphasize the individualistic nature and management of each concussion.^2,3 This represents a substantial decrease from the previous decade ^9–11 but is higher than the 11% reported in a recent study of physician practice patterns.²⁹ It is encouraging that nearly all respondents indicated the presence of a concussion-management plan (97.1%), which exceeded the rate in a recent study among secondary school ATs (82.4%).²⁸

Identifying concussion recovery remains a clinical challenge because many of the acute assessment protocols have practice effects, which reduce their sensitivity and clinical utility.^30,31 Testing limitations notwithstanding, most people with concussions are considered recovered within 1 to 2 weeks postinjury.^2,3 About two-thirds of our respondents used a multifaceted assessment program in which self-reported symptoms (92.0%) and CNT (73.7%) were the most used components and the SAC test (54.4%) was the least used. The recent NATA concussion-management position statement, published after this study was conducted, recommended using the SAC test only within the first 24 hours, so it would not be used in making return-to-participation decisions.² With the exception of the SAC test, postinjury testing began when a student-athlete self-reported being asymptomatic and typically was performed daily until a baseline value was achieved. One area in which DII and DIII ATs differed substantially from DI ATs was the administration of CNT. Specifically, DI ATs most commonly performed CNT daily, whereas the DII and DIII ATs most often performed the testing weekly (32.0%); however, the second most frequent response was daily (22.5%) despite the manufacturer's recommendation of 2 test sessions per week.³² After the athlete has achieved baseline values on the multifaceted assessment and the clinical examination is normal, the CIS statements and the NATA position statement recommend a stepwise progression for return to participation.^2,3,5 Most of our respondents (80.6%) indicated that they used a stepwise progression; however, the protocol (4.8 ± 1.4 days) was several days shorter than recommended by the 3rd and 4th CIS.^3,5 The mean time to return to participation was 8.8 ± 3.3 days, which is still within the 1-week to 2-week window of suspected increased vulnerability.^2,3

Beyond the limited use of a multifaceted assessment protocol, DII and DIII ATs could increase compliance with contemporary concussion research findings in several additional areas. Most baseline tests were performed by any available AT. The BESS test has poor interrater reliability, and having a different AT perform the baseline test and postinjury assessment may reduce the sensitivity of the overall assessment.³³ However, small staff sizes may lessen the likelihood of this limitation. Furthermore, a surprisingly large number of respondents (n = 211) reported using the Romberg test, but no evidence supports its use in acute concussion assessment.³⁴ In addition, and similar to DI ATs, interpretation of CNT was performed most commonly by ATs (85.6%) and physicians (72.0%) and least commonly by neuropsychologists (22.7%).⁹ The ATs need to be aware of state licensure acts regarding CNT interpretation, and both the NATA position statement and NCAA best practices indicate that a neuropsychologist or physician should interpret these results.^2,7 At a minimum and assuming compliance with state practice acts, ATs should attend workshops to improve their CNT interpretation skills (eg, validity checks).³⁵ In addition, ATs need to consider compliance with state practice acts, as 27.6% of respondents did not endorse physician recommendations in making return-to-participation decisions; however, this value is substantially more than a recent finding²² that only 2.7% of ATs reported having final responsibility for return-to-participation decisions. Most respondents indicated that baseline test results from the BESS and SAC were stored in student-athlete files, which may not be accessible during a sideline evaluation. It is promising that the use of electronic records for storing BESS (18.8%) and SAC (32.4%) test results was greater for our respondents than for DI ATs.⁹ Most respondents (86.1% [522/606]) had earned concussion-related CEUs within the previous 3 years (mean = 7.3 ± 4.0 years), this exceeded the rate in a recent physician study²⁹ in which just over half had received continuing medical education credits. Another encouraging finding was the high rate of awareness of the 3rd CIS (DII = 83.7%, DIII = 81.8%), which greatly exceeded the 33.6% awareness of the 2nd CIS reported by Covassin et al¹⁴; however, simply being aware of the CIS does not mean the statement has been read, interpreted, or implemented. Whereas our study was limited to clinically practicing ATs, researchers should investigate the practice patterns of team physicians and other athletic department consultants (eg, neuropsychologists, neurologists, research faculty).

The questionnaire was administered online in May 2013, shortly after the 4th CIS was published and before the release of the most recent NATA position statement and the NCAA best-practice guidelines.^2,3 Procedural changes secondary to the release of the 4th CIS may not have occurred in the 2-month window between the consensus statement's release and the administration of this questionnaire; therefore, many comparisons are made with the 3rd CIS, which was released 4 years earlier. The response rate in this study (N = 755 [40.2%]) exceeded that of most previous studies^11,14,35 on concussion-management practice patterns, but both response bias and the potential for multiple responders from institutions must be considered when extrapolating the results of this study. Furthermore, this Web-based questionnaire has inherent Web-based survey limitations, including anonymous responses, incomplete data, question misinterpretation, and societal response bias (eg, answering how I should respond rather than how I actually practice). It is interesting that the DII and DIII respondents, a relatively homogeneous group, were very similar in most of their responses, which further emphasizes the gap between DI institutions and DII and DIII institutions. Finally, the questionnaire design, we realized in retrospect, was biased toward CNT procedures, and the questions were formatted based on these assumptions.

CONCLUSIONS

The concussion-management practice patterns of NCAA DII and DIII ATs, with the notable exception of limited baseline testing, were largely compliant with the recommendations put forth by the NCAA best-practice guidelines and with recent position and consensus statements.^2,3,7 However, the use of objective assessments was consistently less than that by DI ATs. This may reflect staffing and funding differences, which our respondents frequently cited as limitations.⁹ We are encouraged that our respondents reported receiving numerous concussion-related CEUs and were highly aware of recent position statements. These results demonstrate continued improvement over the past 15 years in the concussion-management practice patterns of ATs.