Reach and Knowledge Change Among Coaches and Other Participants of the Online Course: “Concussion in Sports: What You Need to Know”

Abstract

Objectives

To describe the reach of the Heads Up “Concussion in Sports: What You Need to Know,” online course and to assess knowledge change.

Setting

Online.

Participants

Individuals who have taken the free online course since its inception in May 2010 to July 2013.

Design

Descriptive, uncontrolled, before and after study design.

Main Measures

Reach is measured by the number of unique participants and the number of times the course was completed by state and sport coached and the rate of participation per 100 000 population by state. Knowledge change is measured by the distribution and mean of pre- and posttest scores by sex, primary role (eg, coach, student, and parent), and sport coached.

Results

Between May 2010 and July 2013, the online concussion course was completed 819 223 times, reaching 666 026 unique participants, including residents from all US states and the District of Columbia. The distribution of overall scores improved from pre- to posttests, with 21% answering all questions correctly on the pretest and 60% answering all questions correctly on the posttest.

Conclusion

Online training can be effective in reaching large audiences and improving knowledge about emerging health and safety issues such as concussion awareness.

Sports and recreation activities are an important part of child and adolescent growth and development. These activities encourage physical activity and teach important skills, such as sportsmanship, discipline, and teamwork. However, participation in sports and recreation activities can result in unintentional injuries such as concussion, which carries the potential for adverse short- and long-term health outcomes.^1–4

The Centers for Disease Control and Prevention (CDC) defines a concussion as a mild traumatic brain injury (TBI), occurring when an impact or forceful motion of the head or body results in a brief alteration of mental status, such as confusion or disorientation, loss of memory for events immediately before or after the injury, or brief loss of consciousness.⁵ Traumatic brain injuries, including concussions, are commonly reported as a result of sport or recreational activities. About two-thirds of these injuries treated in emergency departments occur among children and adolescents ages 5 to 17 years.⁶ In 2009, there were an estimated 248 000 visits to an emergency department among children and adolescents aged 19 years or younger for a sports- or recreation-related TBI.⁷ Although children and adolescents may be more at risk for concussion and take longer to recover than adults,⁸ research indicates that the consequences of concussions can be reduced by responding appropriately to these injuries when they occur.⁹

Sports coaches set the tone for safety among their athletes and are uniquely positioned to recognize a potentially concussed athlete and respond in a way that reduces the risk of developing adverse health outcomes, for example, by removing him or her from play.¹⁰ Thus, it is imperative that coaches are knowledgeable about how to recognize, respond, and prevent sport-related concussions. Despite this, previous studies suggest that coaches’ knowledge of concussion risk factors and their ability to recognize a potentially concussed athlete are limited. A 2007 study reported that only 61% of youth coaches correctly recognized the signs and symptoms of a concussion.¹¹ Researchers also found inconsistency in how coaches handled concussion management and return-to-play policies relative to published guidelines.¹²

To reduce the risk of concussion consequences among children and adolescents, most states now have concussion in sports policies, which commonly focus on school sports programs. Specific policies vary by state but typically require that coaches and others involved in youth sports receive educational training on recognizing and responding to concussions among their student athletes.¹³ Other common components of state laws include requiring removal from play and requiring clearance by a health professional to return to play for athletes with suspected concussions.

To improve knowledge on how to recognize, respond to, and prevent concussions, the CDC’s Heads Up education campaign launched a series of educational initiatives aimed at educating coaches and others on:

• What a concussion is and the potential effects of this injury;

• Concussion signs and symptoms;

• What to do if a concussion occurs; and

• Prevention strategies.

Heads Up materials are currently available through many formats, including online courses, fact sheets, posters, videos, and public service announcements.

One educational product titled, “Concussion in Sports: What You Need to Know,” is an online training developed by the CDC in collaboration with the National Federation of State High School Associations (NFHS). The course is available online as part of the NFHS’s Learning Center and Coach Education Program (available at http://www.nfhslearn.com/electiveDetail.aspx?courseID=38000). Although trainings in the Coach Education Program are targeted at coaches and some have a fee to register, the 20-minute “Concussion in Sports” online training is free and promoted by NFHS as relevant for the broader interscholastic athletic community, including officials, parents, and students, as well as coaches. The course is designed to educate participants on (1) the importance of sports-related concussions, (2) how to recognize a concussion, (3) how to respond to a suspected concussion, and (4) the steps to helping players safely return to play after recovering from a concussion. Notably, this course is not designed to enable coaches to evaluate an injured athlete, which is the role of a trained healthcare provider. A recent report from the Institute of Medicine has identified a need for “changing the culture” of youth sports as it relates to reporting and responding to concussion symptoms.¹⁴ By teaching coaches and others involved in youth sports how to recognize, respond to, and prevent concussions, “Concussion in Sports: What You Need to Know” has the potential to contribute to that culture change.

The purpose of this study was to describe the reach of the NFHS/CDC “Concussion in Sports: What You Need to Know” online course since its inception in May of 2010 and to assess the change in knowledge among course participants through analysis of pre- and posttest results. Results can guide the development of further educational efforts targeting those involved in youth and high school sports.

METHODS

The NFHS maintains a registry of all persons who register and take the NFHS/CDC “Concussion in Sports: What You Need to Know” online course. The NFHS also conducts pre- and posttest knowledge assessments on participants. Data for this paper are from the NFHS registry of course participants.

Reach

Data for the first portion of the study include all participants who completed the NFHS/CDC online course since its inception (May 2010 to July 2013). Participant counts are presented by state and sports coached. To allow for easier comparison between states with different population sizes, we present state participation rates per 100 000 population using 2010 Census data. State population-based participation rates were calculated instead of subgroup-specific rates—such as coach or athletic trainer participation rates—because subgroup population sizes are unknown. For data presented by sport coached, we allow individuals to be counted multiple times, as participants could indicate that they coached more than 1 sport. Because many states and organizations require that persons affiliated with a high school sport complete a qualifying concussion course on a regular basis (eg, annually), some participants have completed the NFHS/CDC concussion course multiple times. Thus, we present results for both unique participants as well as total course completions (a single participant may be counted multiple times). Unique participants are identified on the basis of the e-mail address provided at registration.

Pre- and posttest comparison

The second portion of the study examines a subset of course participants with pre- and posttest results. Course participants take a 5-question pretest before starting the course and are required to pass a posttest (identical to the pretest) to receive a certificate of completion. Although these tests have been required since the course inception, responses were not recorded until fall 2012. Thus, this analysis includes only pre- and posttest results from November 2012 to July 2013. To examine knowledge gained from this course, we limit the pre- and posttest data set to registrants in the NFHS database that took the course for the first time during this timeframe.

The 5 test questions are included in Table 1, with the correct response indicated. To determine which questions are most discriminating in terms of knowledge change before and after the course, we calculate the percentage of participants who responded correctly to each question in the pre- and posttest.

TABLE 1.

Pre- and posttest questions and response options from NFHS/CDC’s “Concussion in Sports: What You Need to Know”

Pre- and posttest questions
Question 1: Which one of the following statements regarding concussion is true? It is a minor brain injury that usually does not interfere with normal brain function. It is a bump or blow to the head that can be prevented by wearing a helmet. It is only possible when the individual is knocked unconscious. It is a traumatic brain injury that interferes with normal brain function. They are less prevalent in adolescents as their brain is able to heal more quickly than adults.
Question 2: Which of the following statements is true? Only CAT scans or MRIs can identify a concussion. Most athletic events have athletic trainers or medical professionals present to take responsibility when a serious injury occurs. Even what appears as only a mild jolt or blow to the head or body may cause the brain to shift or rotate suddenly possibly causing a brain injury. Studies suggest that it usually takes brain cells 24 hours to regain normal function following a concussion. If a player is knocked unconscious it permissible to return him or her to play as long as the player’s pupils are not dilated.
Question 3: A player that returns to activity before they have fully healed from the concussion is at increased risk for a repeat concussion.
TRUE/FALSE
Question 4: We know that concussion can affect a person’s brain function. Which of these areas of brain function can be affected by a concussion? The way a person feels How a person thinks A change in emotions How well they perform in their sport All of the above
Question 5: Forty (40)% of players who died or have serious long-term problems were still having symptoms from a previous concussion. Which of the following will least likely decrease the risks of death or serious long-term problems from concussion? Adequate recovery time Follow appropriate “Return to Play” protocols Being educated on the dangers of playing with symptoms of concussion Having an MRI of the brain Delaying the return to school

Abbreviations: CAT, computed tomography; MRI, magnetic resonance imaging.

Note: correct responses are indicated with bold.

For each participant, we calculate an overall score ranging from 0 to 100 for the pre- and posttest, on the basis of the percentage of correct responses (eg, a participant who answers all questions correctly receives a score of 100). To compare pre- and posttest results, we present the distribution of pre- and posttest scores overall. We also present the percentage of participants answering all questions correctly and the mean score by sex, self-identified primary role (eg, coach, student, and parent), and sports coached. We do not present tests for statistical significance because the large sample size (n = 133 764) would result in statistical significance of small differences, which do not necessarily indicate meaningful differences.¹⁵

RESULTS

Reach

Between its inception in May 2010 and July 2013, the NFHS/CDC online concussion course has been completed 819 223 times, reaching 666 026 unique participants, including residents from all 50 states and the District of Columbia (see Table 2). The largest number of participants is from Ohio (16%) and Massachusetts (12%). Massachusetts had the highest rates of participation, with 1238 participants per 100 000 population, followed by North Dakota, Ohio, Vermont, and South Dakota (see Figure 1). The lowest rates of participation were in New York and Illinois, with fewer than 25 participants per 100 000 population. Football was the most frequently reported sport coached among course participants, involving 146 616 participants (22%), followed by 87 262 from baseball (13%), 76 739 from boys basketball (12%), and 63 759 from girls basketball (10%) (see Table 3).

TABLE 2.

Reach of NFHS/CDC’s “Concussion in Sports: What You Need to Know” online training by state, May 2010 to July 2013

	No. participants	%	No. completionsa	%
Overall	666 026	100	819 223	100
State
Alabama	17 535	3	18 338	2
Alaska	1 884	0	1 987	0
Arizona	28 423	4	29 996	4
Arkansas	4 371	1	4 736	1
California	42 820	6	45 103	6
Colorado	26 059	4	37 342	5
Connecticut	1 817	0	2 522	0
Delaware	2 798	0	3 158	0
District of  Columbia	574	0	680	0
Florida	34 931	5	49 012	6
Georgia	12 547	2	14 717	2
Hawaii	4 532	1	5 296	1
Idaho	7 981	1	10 063	1
Illinois	2 955	0	3 114	0
Indiana	5 478	1	5 745	1
Iowa	4 396	1	4 901	1
Kansas	2 744	0	3 288	0
Kentucky	2 925	0	3 052	0
Louisiana	2 827	0	3 349	0
Maine	4 174	1	4 499	1
Maryland	19 616	3	22 627	3
Massachusetts	81 053	12	105 977	13
Michigan	8 828	1	8 980	1
Minnesota	12 720	2	13 384	2
Mississippi	2 534	0	2 625	0
Missouri	9 639	1	13 088	2
Montana	3 228	0	3 278	0
Nebraska	6 721	1	8 138	1
Nevada	1 030	0	1 171	0
New Hampshire	5 699	1	6 806	1
New Jersey	30 386	5	48 669	6
New Mexico	6 255	1	9 513	1
New York	4 681	1	4 957	1
North Carolina	4 374	1	4 755	1
North Dakota	6 865	1	8 506	1
Ohio	103 795	16	106 154	13
Oklahoma	9 516	1	14 686	2
Oregon	17 737	3	25 362	3
Pennsylvania	37 414	6	46 942	6
Rhode Island	4 307	1	7 224	1
South Carolina	12 017	2	18 855	2
South Dakota	4 322	1	7 757	1
Tennessee	2 272	0	2 371	0
Texas	7 336	1	8 123	1
Utah	5 997	1	6 689	1
Vermont	4 321	1	5 271	1
Virginia	24 377	4	35 631	4
Washington	3 253	0	4 097	1
West Virginia	5 071	1	6 588	1
Wisconsin	4 402	1	4 732	1
Wyoming	1 697	0	2 421	0
Other/unknown	2 792	0	2 948	0

^aParticipants may have completed the course more than once during the time period, so each unique participant may contribute multiple completions.

Figure 1.

NFHS/CDC’s “Concussion in Sports: What You Need to Know” online training participation rates per 100 000 population, by state, May 2010 to July 2013.

TABLE 3.

Reach of NFHS/CDC’s “Concussion in Sports: What You Need to Know” by sport coached, May 2010 to July 2013

	No. participants	%	No. completionsa	%
Overall	666 026	100	819 223	100
Sport coachedb
Baseball	87 262	13	103 619	13
Basketball—boys	76 739	12	95 252	12
Basketball—girls	63 759	10	79 935	10
Bowling	2 535	0	3 205	0
Cross Country—boys	22 063	3	29 154	4
Cross Country—girls	21 329	3	28 079	3
Field hockey	8 983	1	12 247	1
Flag football	4 630	1	4 971	1
Football	146 616	22	183 854	22
Golf—boys	12 415	2	16 251	2
Golf—girls	7 929	1	10 357	1
Gymnastics—boys	804	0	901	0
Gymnastics—girls	3 915	1	4 801	1
Ice hockey	10 273	2	12 528	2
Lacrosse—boys	13 468	2	16 896	2
Lacrosse—girls	8 359	1	10 847	1
Rugby	4 067	1	4 263	1
Soccer—boys	54 498	8	65 237	8
Soccer—girls	52 033	8	62 428	8
Softball	52 421	8	64 024	8
Spirit	13 009	2	15 513	2
Swim, dive, water polo—boys	12 448	2	15 690	2
Swim, dive, water polo—girls	12 979	2	16 333	2
Tennis—boys	9 882	1	12 458	2
Tennis—girls	11 066	2	13 985	2
Track and field—boys	42 676	6	55 982	7
Track and field—girls	40 106	6	52 346	6
Volleyball	50 932	8	63 800	8
Wrestling	25 912	4	32 925	4
Other	56 659	9	75 703	9
No sport coached	54 854	8	65 668	8

^aParticipants may have completed the course more than once during the time period, so each unique participant may contribute multiple completions.

^bParticipants select sport(s) coached from a list provided at registration. Participants may indicate that they coach more than 1 sport, thus column percentages add up to more than 100%.

Pre- and posttest comparison

Among the 133 764 participants who completed the course for the first time during the data collection period, 2 of the 5 questions were answered correctly by more than 98% of participants in both the pre- and posttests (see Table 4). About 91% of respondents correctly indicated during the pretest that even a mild jolt can cause a brain injury, and this increased to 98% at posttest. A total of 86% of respondents correctly responded during the pretest that a concussion is a “traumatic brain injury that interferes with normal brain function,” which improved to 98% at the posttest. Finally, while only 24% of participants correctly identified the option that was least likely to decrease the risks of death or serious long-term problems from concussion in the pretest (correct answer: having magnetic resonance imaging), 62% responded correctly in the posttest.

TABLE 4.

Percentage correct by pre- and posttest question, NFHS/CDC’s “Concussion in Sports: What You Need to Know,” November 2012 to July 2013

	Pretest		Posttest
Question	No. correct	%	No. correct	%
Q1. Interferes with brain function	115 479	86.3	130 822	97.8
Q2. Mild blow can cause injury	121 835	91.1	130 393	97.5
Q3. Returning too soon can increase risk	132 312	98.9	133 477	99.8
Q4. Symptoms can affect all brain function	132 494	99.1	133 054	99.5
Q5. How to decrease long-term problems	32 367	24.2	82 348	61.6

Note: n = 133 764.

The distribution of scores improved from pre- to posttests, with 21% of respondents answering all 5 questions correctly on the pretest and 60% answering all questions correctly on the posttest (see Figure 2). There were minimal differences in scores between men and women, but some differences by primary role, which was self-identified by the participant (see Table 5). For example, before taking the course, participants who identified as students exhibited the lowest levels of knowledge, with only 17% answering all 5 questions correctly on the pretest. Athletic trainers showed the most initial knowledge, with 41% answering all questions correctly on the pretest. Although participants from all roles showed improvement between pre- and posttests, students also had the lowest percentage of all questions answered correctly at posttest and athletic trainers had the highest percentage (48% and 69%, respectively). Among sports coached, spirit coaches (eg, cheerleading) had the lowest levels of initial knowledge, with only 17% answering all questions correctly in the pretest, whereas field hockey (34%) and ice hockey (33%) coaches had the highest percentage answering all questions correctly in the pretest. Differences by sport coached were smaller at posttest, with 55% to 66% answering all questions correctly.

Figure 2.

Distribution of pre- and posttest scores, NFHS/CDC’s “Concussion in Sports: What You Need to Know,” November 2012 to July 2013.

TABLE 5.

Percentage of participants answering all questions correctly and mean score on pre- and posttest by sex, self-identified primary role, and sport coached, NFHS/CDC’s “Concussion in Sports: What You Need to Know,” November 2012 to July 2013

		% Answering all questions correctly			Mean score
	n	Pretest	Posttest	Difference	Pretest	Posttest	Difference
Overall	133 764	21%	60%	39%	80	91	11
Sex
Female	31 147	21%	59%	38%	78	91	13
Male	90 770	20%	60%	40%	80	91	11
Unknown	11 847	27%	63%	36%	83	92	9
Primary rolea
Assistant Principal	151	25%	67%	42%	81	93	12
Athletic Director	1 233	27%	62%	35%	83	92	9
Athletic trainer	427	41%	69%	28%	85	93	9
Coach	83 179	21%	60%	40%	80	91	11
Counselor	790	18%	59%	41%	77	91	14
Official	16 748	19%	57%	38%	80	91	11
Other school personnel	772	25%	60%	35%	80	91	10
Parent	6 696	25%	65%	40%	81	92	12
Principal	200	25%	65%	40%	83	92	9
Student	6 403	17%	48%	32%	75	88	13
Teacher	6 864	24%	65%	41%	82	93	11
Other	7 361	20%	58%	38%	79	91	12
Un reported	2 940	24%	60%	36%	82	91	9
Sport coachedb
Baseball	32 815	20%	60%	40%	80	91	11
Basketball—boys	12 734	21%	60%	39%	81	91	11
Basketball—girls	9 716	21%	60%	39%	80	91	11
Bowling	450	23%	57%	35%	80	90	10
Cross Country—boys	2 907	26%	66%	40%	82	92	10
Cross Country—girls	2 839	27%	66%	40%	82	93	11
Field hockey	762	34%	66%	33%	84	93	9
Flag football	1 636	21%	62%	41%	81	92	11
Football	23 254	20%	58%	37%	81	91	10
Golf—boys	1 667	26%	64%	38%	82	92	10
Golf—girls	1 141	25%	64%	39%	82	92	10
Gymnastics—boys	333	27%	59%	32%	81	91	11
Gymnastics—girls	1 067	24%	63%	40%	79	92	13
Ice hockey	1 298	33%	66%	33%	84	93	8
Lacrosse—boys	3 371	27%	65%	38%	82	92	10
Lacrosse—girls	1 796	30%	65%	35%	83	92	10
Rugby	1 181	21%	64%	43%	81	92	11
Soccer—boys	17 108	22%	63%	41%	80	92	12
Soccer—girls	15 984	23%	64%	41%	81	92	12
Softball	16 581	20%	59%	39%	79	91	12
Spirit	1 490	17%	55%	38%	77	90	13
Swim/dive	2 805	23%	63%	41%	80	92	12
Tennis—boys	1 641	25%	63%	38%	82	92	10
Tennis—girls	1 768	25%	62%	37%	81	92	10
Track and field—boys	7 247	23%	62%	39%	81	92	11
Track and field—girls	6 865	24%	64%	40%	81	92	11
Volleyball	7 428	20%	59%	39%	80	91	12
Water polo—boys	2 381	23%	64%	41%	81	92	12
Water polo—girls	2 384	23%	64%	41%	81	92	12
Wrestling	3 008	22%	60%	38%	81	91	10
Other	6 715	21%	58%	37%	79	91	12
No sport coached	8 955	22%	59%	37%	79	91	12

^aPrimary role is self-identified by participant.

^bParticipants select sport(s) coached from a list provided at registration. Participants may indicate that they coach more than 1 sport.

DISCUSSION

This study demonstrates that the NFHS/CDC Heads Up “Concussion in Sports: What You Need to Know” online course can improve knowledge of concussion recognition and response among coaches, officials, and others involved in high school sports. In the current analysis, all groups saw improvements in scores between pre- and posttests, suggesting that the training may provide a basic level of knowledge across all participants. Differences between groups parallel differences in training and educational backgrounds, with students performing worst and athletic trainers performing best. Findings are similar to previous examinations of efforts to improve concussion awareness and response among those involved in high school and youth sports, which examined the use of Heads Up information provided by toolkits including written materials.^16,17 In these prior evaluations, coaches self-reported improved knowledge about concussion, improved recognition of athletes at risk for concussion, and changes in behavior in preventing and responding to concussion.^16,17

The course has had a wide reach confirming the usefulness of online delivery of public health trainings to reach large targeted audiences. At the time of reporting, this training had reached more than 650 000 participants and more than 800 000 completions, with participants including coaches and others from a range of sports and roles. These findings understate the true reach of www.headtraumarehab.com online concussion training, since other online trainings are available, including CDC’s “Heads Up Concussion in Youth Sports” (available at http://www.cdc.gov/concussion/HeadsUp/Training/index.html), which reaches approximately 25 000 to 35 000 users each month.¹⁸ Consequently, differences in reach between states do not necessarily suggest a lack of training in locations with low participation rates, as some states may rely on other educational resources. For example, the state of Washington has developed its own concussion training video based on the state’s policies for the management of concussion in youth sports.¹⁹

Limitations

This study is subject to a number of limitations. First, while these data allow us to assess knowledge gained immediately after participating in an online educational training, we are unable to assess knowledge retained over time, nor whether improvements in knowledge resulted in behavior change (eg, pulling athletes from play for medical evaluation). Second, knowledge is based on 5 questions, some of which most respondents answered correctly before taking the training. Future examinations with more rigorous questions may reveal important knowledge gains and gaps. To this end, the NFHS has added and altered questions for the pre- and posttest beginning in August 2013. Third, for many sports, coaches are requested or required to take the training prior to the start of their season. However, our examination of pre- and posttest scores did not include an entire year of data collection. Thus, our results may not be representative of sports with seasons starting between August and October (eg, football). Finally, if a participant registered and completed the course using multiple e-mail accounts, that person would have been incorrectly counted as multiple individuals.

This study had a number of strengths. It involved a large number of participants. It also documented the timely and broad reach of this online training, the representation of many high school activities, and the nationwide distribution of participants.

CONCLUSIONS

Concussion risks have come to the forefront recently through scientific publications and media coverage. Online training can be an effective way to reach large audiences and improve knowledge about emerging health and safety issues, such as concussion awareness. The NFHS/CDC’s “Concussion in Sports: What You Need to Know” and similar trainings are designed to meet the needs of coaches and others involved in high school sports, so they may recognize and respond appropriately to potential concussions among their student athletes. Results from this analysis have been used to improve and expand pre- and posttest questions. Findings also will inform dissemination and targeting of Heads Up training materials to ensure that we reach coaches of all sports.