Calling Injury Timeouts for the Medical Evaluation of Concussion: Determinants of Collegiate Football Officials' Behavior

Abstract

Context: 

Sports officials can play an important role in concussion safety by calling injury timeouts so that athletic trainers can evaluate athletes with possible concussions. Understanding the determinants of whether officials call an injury timeout when they suspect a concussion has important implications for the design of interventions to better support officials in this role.

Objective: 

To assess the knowledge of US collegiate football officials about concussion symptoms and to determine the associations between knowledge, perceived injunctive norms, and self-efficacy in calling injury timeouts for suspected concussions.

Design: 

Cross-sectional study.

Setting: 

Electronic survey.

Patients or Other Participants: 

Of the 3074 US collegiate football officials contacted, 1324 (43% response rate) participated.

Main Outcome Measure(s): 

Concussion knowledge, injunctive norms (belief about what others would want them to do), and behavioral self-efficacy (confidence in their ability to call injury timeouts for suspected concussions in athletes during challenging game-day conditions).

Results: 

Officials reported calling approximately 1 injury timeout for a suspected concussion every 4 games during the 2015 season. Structural equation modeling indicated that officials with more concussion-symptom knowledge had greater self-efficacy. Independent of an official's symptom knowledge, injunctive norms that were more supportive of calling an injury timeout were associated with greater self-efficacy.

Conclusions: 

Concussion education for officials is important because when officials are aware of concussion symptoms, they are more confident in calling injury timeouts. Beyond increasing symptom knowledge, fostering sports environments that encourage concussion safety can support officials in calling injury timeouts. Athletic trainers can help by educating stakeholders, including officials, about the importance of concussion safety. When officials believe that other stakeholders support concussion safety, they are more likely to call injury timeouts if they suspect a concussion has occurred.

Key Points

• Ensuring that all officials know the symptoms of concussion can promote their confidence in calling injury timeouts.

• The sport environment influences the behavior of officials.

• Athletic trainers can help create a culture that encourages concussion safety, which can affect the behavior of officials.

Sports have rules that are designed to keep players safe, and sports officials (eg, referees) are key agents in enforcing these rules.^1,2 In football, concussion is a notable health risk. One way that officials can enforce rules intended to limit the health burden of concussion is to recognize potentially concussed athletes and call injury timeouts appropriately so that medical personnel can conduct evaluations. The immediate removal from play of athletes who have sustained concussions is important because, during the symptomatic postinjury period, the concussed brain is in a metabolically vulnerable state, and additional impacts can magnify neurologic consequences.³

Many sports injuries limit mobility and, thus, are easily identifiable. However, as symptoms of concussions can be cognitive, somatic, or emotional,⁴ the injury is more challenging to identify immediately. Ideally, symptoms of suspected concussions will be reported by athletes themselves. However, accumulating evidence⁵ suggests that athletes often try to play through the symptoms of a possible concussion. Alternatively, concussions can be identified by an athletic trainer (AT), physician, or independent medical observer. Such medical personnel require the official to stop play so they can conduct an evaluation. Stopping play represents an access point for care that is otherwise denied while the game continues. However, these personnel are not present at all levels of play. Only 70% of high schools have an AT on staff in any capacity, and this individual is not necessarily present at all games.⁶ Even when present, ATs are typically not as proximate to the injured athletes as officials are, and given the complexity of the game of football and number of players on the field, they may be attending to another player.

In many competitions sanctioned by the National Collegiate Athletic Association (NCAA), strict rules dictate when and how frequently teams can call discretionary timeouts,⁷ and these timeouts are typically used for strategic competitive purposes. However, in most sports, game officials have the discretion to call injury timeouts that are not charged to a specific team and can be used to evaluate an athlete with a possible injury.⁷ This can be an effective tool for ensuring timely injury evaluation and preventing repeated injury. Recently, to promote their willingness and ability to call such timeouts, the NCAA released a video for collegiate football officials, providing a primer about concussions and encouraging them to make appropriate use of the injury-timeout rule to allow for the medical evaluation of players who they suspect may have sustained a concussion.

The role of officials in supporting concussion safety has been relatively understudied. For officials to call an injury timeout for a possible concussion, they must be able to recognize symptoms that may indicate a concussion. Therefore, the first purpose of our study was to describe officials' knowledge about concussion symptoms. However, factors other than knowledge can influence behavior; social cognitive theory posits that a triadic reciprocal relationship exists among individual characteristics and cognitions, environmental influences, and behavior.⁸ Self-efficacy is a central component of social cognitive theory and tends to be a strong predictor of behavior, reflecting the individual's confidence in his or her ability to perform the behavior in real-world conditions.^8–10 In our study, these conditions reflected injury-timeout decisions in challenging game-day scenarios. Having the self-efficacy to call injury timeouts may be influenced by interactions with others in the sport environment; officials interact with and obtain feedback on their decisions from a variety of groups in the sport environment. Given the role that officials play in determining the flow of games and potentially influencing the competitive outcomes, these interactions may be antagonistic. Information can be communicated to officials through formal channels (eg, decisions related to their employment, written critiques of their job performance in the media) or informal channels (eg, fans booing, coaches yelling on the sidelines, athletes reacting emotionally to decisions on the field of play). Therefore, the second goal of our study was to build a more comprehensive model of how officials decide whether to call an injury timeout for suspected concussions, focusing on the role of their knowledge about concussion and the influence of others in their environments.

METHODS

Procedure and Sample

The NCAA's Sport Science Institute distributed an electronic survey to 3074 collegiate football officials using an e-mail distribution list. A reminder e-mail was sent 2 weeks after the initial invitation. A total of 1324 officials (mean age = 30.41 ± 9.77 years) completed the survey (43% response rate). Data were collected in October 2015. Officials provided informed consent before completing the survey, and the study was approved by the Seattle Children's Research Institute's Institutional Review Board.

Measures

Concussion Symptom Knowledge.

Participants completed a 12-item assessment of concussion-symptom knowledge,¹¹ responding to the following prompt: “An athlete is experiencing the signs and symptoms listed below after a bump, blow, or jolt to the head or body. Which are sufficient reasons to call an injury timeout?” Symptoms were included in the list if they could potentially be observed by an on-field official during the game. Response options were Definitely call an injury timeout, Depends on the severity of the symptom, Definitely do not call an injury timeout, and I don't know. The number of symptoms for which the officials chose Definitely call an injury timeout was summed to create an index score with a possible range of 0 to 12.

Perceived Injunctive Norms.

Participants indicated how frequently they believed that 7 reference groups (coaches, athletes, fans, media, athletic administrators, other officials, and parents) would want them to call an injury timeout if they suspected that a concussion may have occurred. Responses were rated on a 5-point scale, with 1 indicating never; 2, rarely; 3, sometimes; 4, most of the time; and 5, always.

Behavioral Self-Efficacy.

Four challenging game-day situations were generated with input from experts in the clinical management of concussion who work closely with football officials: “Score of game is very close,” “It is a very important game,” “Not 100% certain that the athlete has sustained a concussion,” and “An injury timeout for a suspected concussion has already been called during that game.” Participants indicated how confident they were that they could call an injury timeout in each situation using a 100-point scale, ranging from 0 (not at all confident) to 100 (extremely confident).

Injury Timeouts for Suspected Concussion.

Participants indicated the number of collegiate football games at which they had officiated so far during the 2015 season and the number of times they had called an injury timeout because they suspected an athlete had sustained a concussion. From these 2 values, the ratio of injury timeouts to games during the 2015 season was calculated.

Statistical Analysis

Descriptive statistics are reported for all measured variables. We calculated pairwise correlations between knowledge-norms items and self-efficacy items. Measurement models were fit for norms and self-efficacy to ensure good local fit before we combined them into the larger structural equation models. Fit statistics for all models and χ² difference tests comparing nested models are presented. Using the criteria specified by Hu and Bentler,¹² we set thresholds for a close model fit as a comparative fit index greater than 0.95, Tucker-Lewis index greater than 0.95, root mean square error of approximation greater than 0.05, and P value for the χ² test greater than .05. Two structural equation models were tested. The first model tested the association between norms and self-efficacy. The second model added knowledge as an independent predictor of self-efficacy to the first model. Maximum likelihood estimation was used to account for missing values in all models, meaning that estimates were unbiased under the assumption of missing at random. “Missingness” ranged from a low of 8.3% for self-efficacy items to a high of 10.3% for symptom-knowledge items. Analyses were conducted in Stata (version 13.1; StataCorp LLC, College Station, TX), and the α level was set at .05.

RESULTS

Sample Characteristics and Past Behavior

On average during the 2015 collegiate football season, officials called 1.98 ± 2.34 injury timeouts and officiated 8.21 ± 2.34 games. This means that officials called approximately 1 injury timeout in every 4 games (0.26 ± 0.26 injury timeouts per game).

Symptom Knowledge

For 4 of the 12 listed symptoms (balance problems, confusion, loss of consciousness, and neck pain), nearly all officials indicated that an injury timeout should definitely be called (Table 1). Fewer than 75% of respondents indicated that an injury timeout should definitely be called for athletes who are experiencing nausea or vomiting (72.93% [n = 870]), headaches or “pressure” in the head (72.06% [n = 859]), difficulty recognizing people or places (70.03% [n = 834]), or double or blurry vision (66.89% [n = 796]). The mean number of symptoms that officials endorsed was 10.00 ± 2.37 out of a possible 12.

Table 1. .

Descriptive Statistics for Symptoms That Officials Believe Indicate an Injury Timeout Should be Called for a Suspected Concussion

Symptom (No. of Respondents)	Response
	Definitely Call an Injury Timeout, % (n)	Depends on the Severity of the Symptom, % (n)	Definitely Do Not Call an Injury Timeout, % (n)	I Don't Know, % (n)
Balance problems (1193)	99.25 (1184)	0.34 (4)	0.08 (1)	0.34 (4)
Confusion (1192)	96.14 (1146)	3.44 (41)	0.08 (1)	0.34 (4)
Loss of consciousness (1193)	95.98 (1145)	2.60 (31)	0.25 (3)	1.17 (14)
Neck pain (1193)	95.22 (1136)	4.02 (48)	0.25 (3)	0.50 (6)
Feeling sluggish or drowsy (1192)	91.19 (1087)	7.47 (89)	0.17 (2)	1.17 (14)
Dizziness (1191)	90.34 (1076)	5.71 (68)	0.25 (3)	3.69 (44)
Sensitivity to light (1189)	86.80 (1032)	11.52 (137)	0.50 (6)	1.18 (14)
Nausea or vomiting (1193)	72.93 (870)	18.61 (222)	1.42 (17)	7.04 (84)
Headaches or “pressure” in head (1192)	72.06 (859)	20.47 (244)	0.76 (9)	6.71 (80)
Difficulty recognizing people or places (1191)	70.03 (834)	22.67 (270)	1.85 (22)	5.46 (65)
Double or blurry vision (1190)	66.89 (796)	23.28 (277)	1.85 (22)	7.98 (95)
Sensitivity to noise (1192)	62.08 (740)	31.46 (375)	1.17 (14)	5.29 (63)

Injunctive Norms

The groups that officials believed wanted them to call an injury timeout the least frequently were athletes (2.87 ± 1.09) and fans (2.84 ± 1.13; Table 2). The groups that officials believed wanted them to call an injury timeout the most frequently were parents (4.52 ± 0.85), other officials (4.42 ± 0.81), and athletic administrators (4.29 ± 0.94). To create a measurement model that included all the injunctive-norms information for use in subsequent structural equation modeling, we built a confirmatory factor model. The goal of this modeling process was to ensure that the norms items had good local fit before being combined in the larger structural equation model. Fit statistics and the P values for difference tests comparing nested models are presented in Table 3. For norms, the initial model with no covariances between items was a poor fit to the data, so we examined modification indices and explored theory-driven modifications (eg, residual correlations between fans and media were expected, as fans often obtain information from media and media tailor content to fan preferences). These analyses indicated that a bifactor model was the best fit to the data, with 1 general factor measured by all items and 1 method factor pointing toward coaches, athletes, and fans. We hypothesized that this method factor, field norms, would reflect normative information communicated from these referent groups to officials in a distinct setting: the field of play. In this bifactor model, the correlation between norms and field norms was constrained to zero, meaning that these variables were interpreted as conceptually distinct.

Table 2. .

Descriptive Statistics for Perceived Injunctive Norms Related to Calling an Injury Timeout for a Suspected Concussion (N = 1324)

Referent Group	Mean ± SDa
Coaches	3.49 ± 1.15
Athletes	2.87 ± 1.09
Fans	2.84 ± 1.13
Media	3.41 ± 1.15
Athletic administrators	4.29 ± 0.94
Other officials	4.42 ± 0.81
Parents	4.52 ± 0.85

^aResponses were rated on a 5-point scale, with 1 indicating never; 2, rarely; 3, sometimes; 4, most of the time; and 5, always.

Table 3. .

Fit Statistics for All Models Tested^a

Model	χ2	Degrees of Freedom	P Value	Comparative Fit Index	Tucker-Lewis Index	Root Mean Square Error of Approximation	Difference Test P Value
Norms
Base model	455.86	14	<.001	0.878	0.817	0.161
Fans and mediab	253.75	13	<.001	0.933	0.893	0.124	<.001
Athletes and coachesc	168.12	12	<.001	0.957	0.924	0.104	<.001
Athletes and fansd	71.39	11	<.001	0.983	0.968	0.067	<.001
Bifactor modele	41.32	10	<.001
Self-efficacy
Base	120.54	2	<.001	0.972	0.915	0.222
Score and importantf	0.00	1	>.99	1.000	1.000	0.000	<.001
Final models
Model 1g	26.45	37	.90	1.000	1.000	0.000
Model 2h	298.66	47	<.001	0.969	0.957	0.066	<.001
Model 3i	253.29	45	<.001	0.974	0.962	0.062	<.001

^aThresholds for a close model fit were set as a comparative fit index >0.95, Tucker-Lewis index >0.95, root mean square error of approximation <0.05, and P value for the χ² test >.05, using the criteria specified by Hu and Bentler.¹²

^bThe fans and media model adds a correlation between the fans and media items to the base model.

^cThe athletes and coaches model adds a correlation between the athletes and coaches items to the fans and media model.

^dThe athletes and fans model adds correlation between the athletes and fans items to the athletes and coaches model.

^eBifactor model includes 1 factor with loadings from all 7 normative referents and a correlation between the fans and media items and 1 factor with loadings from athletes, coaches, and fans.

^fThe score and important model adds to the base model a correlation between self-efficacy items referring to the score of the game being close or the game being important.

^gModel 1 is presented in Figure 1.

^hModel 2 is model 3 without correlations between knowledge and norms.

ⁱModel 3 is presented in Figure 2.

Behavioral Self-Efficacy

The most challenging situation for calling an injury timeout was when participants were “Not 100% certain that the athlete has sustained a concussion” (70.47 ± 30.06). Higher levels of self-reported confidence were observed for the 3 other situations: “Score of game is very close” (84.80 ± 23.73), “It is a very important game” (85.52 ± 23.20), and “An injury timeout for a suspected concussion has already been called during that game” (83.79 ± 24.92). A confirmatory factor model was built to ensure that the 4 self-efficacy items had good local fit before being combined in the larger structural equation model. After including a residual covariance between the items “Score of game is very close” and “It is a very important game,” the fit for the self-efficacy measure was excellent.

Final Models

Pairwise correlations between all model components are provided as Supplemental Material. We first examined a model in which norms predicted self-efficacy (model 1). Standardized path coefficients for this model are presented in Figure 1. Fit of the model was good. Both norms factors predicted self-efficacy, with a larger standardized association between self-efficacy and the inclusive-norms measure (β = 0.40, P < .001) than the field-norms measure (β = 0.08, P = .04).

Figure 1. .

Structural equation model describing the association among perceived injunctive norms and the self-efficacy of collegiate football officials related to calling an injury timeout if they suspect that a concussion has occurred. ^a P < .001. ^b P < .05.

As a final step, we added concussion knowledge to the previously tested model as an independent predictor of self-efficacy (model 2). Fit was improved when covariances between knowledge and both norms items were added to the model (model 3). Standardized path coefficients for this model are shown in Figure 2. The final model was a good fit to the data. In this model, knowledge (β = 0.18, P < .001) and norms (β = 0.36, P < .001)—but not field norms—were associated with self-efficacy. Fit statistics for all tested models are provided in Table 3.

Figure 2. .

Structural equation model describing the association among perceived injunctive norms, knowledge, and the self-efficacy of collegiate football officials related to calling an injury timeout if they suspect that a concussion has occurred. ^a P < .001.

DISCUSSION

In US collegiate football, 1 concussion is diagnosed for every 2.23 games played, based on an average of 60 athletes dressed per game and a rate of 3.74 concussions sustained per 1000 athlete-exposures to a game.^13,14 Athletic health care providers have a primary responsibility to identify and diagnose athletes with concussions, and officials can play an important role in supporting this process. In our study, officials reported calling an injury timeout for a suspected concussion about 1 time in every 4 games. Some of the gap between concussions sustained and injury timeouts called may be due to concussion symptoms presenting after the individual left the field of play or the individual choosing to self-report symptoms or being identified by team medical personnel during normal transitions off the field. However, officials may be able to reduce some of the delay in concussion identification by calling more frequent injury timeouts, allowing team ATs and physicians to access the field of play to evaluate a potentially concussed athlete.

Overall, we observed a high level of knowledge about concussion symptoms that suggest an injury timeout should be called for evaluation of the athlete. However, some variability in responses pointed to opportunities for improved education of officials about this topic. Officials indicated that they were most likely to call an injury timeout if the athlete was experiencing highly visible symptoms, such as loss of consciousness or balance problems. These symptoms are likely also observable to personnel on the sidelines and difficult for any stakeholder in the sport environment to ignore. Other symptoms, such as nausea or vomiting, headaches or “pressure” in the head, difficulty recognizing people and places, or double or blurry vision, were less frequently endorsed as symptoms for which an injury timeout should be called. The reason may be a perceived overlap between these symptoms and the normal symptoms of extreme athletic exertion. Determining whether athletes are experiencing these symptoms may be challenging for officials and may present an opportunity for augmented concussion education. It may also be useful to offer NCAA guidance for officials that directly addresses the potential for apparent overlap between concussion symptoms and symptoms of extreme athletic exertion, emphasizing that when in doubt, a medical professional should determine the cause of symptoms. Importantly, education efforts should not strive to turn officials into on-field medical providers. Messaging to officials should emphasize that identifying athletes with symptoms may not be possible while they are fulfilling their primary duties. It is unrealistic to believe that officials should become 100% confident in their ability to identify a concussion. However, officials do not need to be 100% certain of the injury diagnosis to call an injury timeout. Instead, when they have a reasonable degree of suspicion about the presence of a concussion, they should be trained to call an injury timeout.

In US collegiate football, 7 officials are on the field of play. To have the best chance of stopping play when medically indicated, all of these officials must be aware of all potentially observable concussion symptoms. Encouraging officials to communicate as a team about calling injury timeouts for the medical evaluation of athletes with potential concussions may be useful. For example, if 1 official observes an athlete with visible symptoms that may be interpreted as being due to a concussion or extreme exertion, he or she could ask another official who viewed the recently completed play from a different vantage point whether a potentially concerning impact occurred.

Officials with greater knowledge scores had greater confidence in their ability to call an injury timeout. However, our observations suggest that symptom knowledge alone is not a sufficient explanation for whether officials call injury timeouts. Perceptions about what others in the sport environment thought they should do, independent of concussion knowledge, were important predictors of behavioral self-efficacy in game-day conditions. The emergence of a bifactor-norms measurement model and the pattern of association between norms and self-efficacy offer the possibility that normative information is obtained through different channels, even for the same referent populations. The field-norms factor was hypothesized to represent information obtained from referents with whom the official interacts on the field of play. Information acquired in these contexts may be driven by heat-of-the-moment communications and potentially emotional cognitions¹⁵ by the athletes, coaches, or fans. The information communicated in the midgame setting may be different from what these referents might believe and what might be expressed in calmer settings that support more reasoned or deliberative thought processes. Supporting the hypothesized interpretation of the bifactor-norms model was the finding that symptom knowledge was correlated with the full-norms measure and not the field-norms measure. Officials who know more about concussions and are more confident about identifying concussion symptoms may be less influenced by heat-of-the-moment on-field communication. We noted that the item-total correlations on the field-norms measure were highest for athletes and lowest for coaches and fans. One interpretation is that athletes likely have the most direct interaction with officials during games.

Our observations underscore the importance of continued efforts by the NCAA and other sports leagues to ensure that officials are aware of the symptoms of concussions and are encouraged to follow league directives for calling injury timeouts for suspected concussions. However, our findings suggest that factors beyond symptom knowledge must also be addressed if officials are to call injury timeouts when concussions are suspected. The officials appeared to be influenced by perceptions of what they thought others would want them to do about calling injury timeouts for concussion evaluation. Importantly, these perceptions do not necessarily reflect what the referent groups actually want the officials to do. To the extent that a gap exists between perception and reality, correcting misperceptions in the direction of safety may help shift injunctive norms and consequently shift behavior. However, it is possible that officials may be correctly perceiving injunctive norms. If this is the case, it further supports continued efforts to educate all stakeholders about concussions in order to create sport environments that support safe behavior by fostering safety-oriented norms.

Athletic trainers can play an important role in educating officials about concussion and in creating an on-field environment that supports concussion safety. One useful strategy could be to include officials in a pregame huddle of all personnel tasked with ensuring athlete safety (eg, ATs, physicians, emergency medical services). Including officials in this pregame process would help reinforce their role in concussion safety. In addition, ATs can support officials indirectly by their continued efforts to educate other stakeholders (eg, athletes, coaches) about concussions. Lastly, ATs and team physicians can try to function as an important antidote to the potentially toxic effects of field norms on an official's willingness to call timeout. Reassurances by medical personnel about the importance and appropriateness of the injury-timeout decision, as well as the repeated use of the reminder that “when in doubt, sit them out,” could help empower officials to make an otherwise difficult decision when unsure if the injury is a concussion.

Our study had limitations. A primary limitation was that it was cross-sectional; consequently, we cannot make causal inferences about the associations among knowledge, norms, and self-efficacy. Furthermore, the primary outcome studied was self-efficacy and not behavior. The number of injury timeouts recalled was possibly inaccurate. Prospective research is encouraged to determine the association between preseason cognitions and in-season behavior. Future investigation in this area is also recommended to collect more information about the sporting environments in which the officials work (eg, their athletic conferences) and their personal attributes (eg, years officiating), as these factors may allow the development of more comprehensive models of behavior. The study was also limited by the sampling strategy and response rate. We invited 3074 coaches to participate in the study via the NCAA Sport Science Institute's e-mail distribution list of collegiate football officials. It is possible that the list did not include all current collegiate football officials. Furthermore, only 43% of officials responded. We do not know how respondents compare with nonrespondents, so we suggest using caution in generalizing these findings to all collegiate football officials.

CONCLUSIONS

Promptly identifying athletes with concussions is one important strategy for reducing concussion-related morbidity. Although officials are not responsible for identifying and removing symptomatic athletes from play, they are nonetheless important stakeholders in supporting this process. Ensuring that all officials are aware of the symptoms that may indicate a concussion is a necessary step in helping them call injury timeouts. However, the most important implication of our study was that the sport environment influences the behavior of officials. Athletic trainers can help create a culture that encourages concussion safety, as communicated on and off the field of play and by a variety of stakeholders. This cultural change can have many benefits, including supporting officials in calling injury timeouts when they believe they are warranted.