Skip to main content
NCBI home page
HHS Author Manuscripts logoLink to HHS Author Manuscripts
. Author manuscript; available in PMC: 2022 Sep 1.
Published in final edited form as: J Safety Res. 2021 May 7;78:221–228. doi: 10.1016/j.jsr.2021.04.004

Effectiveness of the CDC HEADS UP online training on healthcare providers’ mTBI knowledge and self-efficacy

Kelly Sarmiento 1,*, Jill Daugherty 1, Dana Waltzman 1
PMCID: PMC8375598  NIHMSID: NIHMS1717249  PMID: 34399918

Abstract

Background:

Many healthcare providers do not consistently implement recommendations contained in clinical guidelines on mild traumatic brain injury (mTBI). As such, the Centers for Disease Control and Prevention (CDC) created the HEADS UP to Healthcare Providers online training to promote uptake of five key recommendations in the CDC Pediatric mTBI Guideline.

Methods:

Using data from modules in the CDC HEADS UP to Healthcare Providers online training, healthcare providers’ self-reported knowledge and self-efficacy prior to and immediately following completion of the training was analyzed.

Results:

Improvements for 8 out of the 10 knowledge questions had a high level of practical significance. The knowledge question with the highest level of practical significance pre- to post-test improvement was for the key guideline recommendation on neuroimaging (pre-test correct: 70.2%; post-test correct: 87.8%; (p < 0.0001, Cohen’s g = 0.39). Four out of the six questions had a self-efficacy level increase of a high level of practical significance (r > 0.50) between the pre- and post-tests. The self-efficacy question with pre- to post-test improvement with the highest level of practical significance was “I am confident in my ability to manage the return to sports progression for my patients” (p < 0.001; r = 0.54).

Conclusions:

The HEADS UP to Healthcare Providers online training led to significant improvements in knowledge and self-efficacy related to mTBI diagnosis and management. Expanded use of this training among healthcare providers who commonly provide care for pediatric patients with mTBI may be beneficial.

Practical Applications:

This study highlights several factors guideline developers may take into consideration when creating an implementation tool, such as using health behavior theories, working with partners and key stakeholders, and focusing on digital-based tools.

Keywords: Concussion, Guideline, Physician, Nurse, Education, Training

1. Introduction

To provide comprehensive guidance to healthcare providers who care for pediatric patients with mild traumatic brain injury (mTBI), the Centers for Disease Control and Prevention (CDC) published an evidence-based guideline on mTBI diagnosis, prognosis, management, and treatment in 2018 (Lumba-Brown et al., 2018). As demonstrated in the guideline, the science and clinical recommendations regarding the diagnosis and management of pediatric patients with mTBI have evolved substantially over the last two decades. Despite this progress, previous studies have found that many healthcare providers do not consistently implement recommendations contained in clinical guidelines on mTBI (Carl & Kinsella, 2014; Greene, Kernic, Vavilala, & Rivara, 2014; Melnick et al., 2012; Stache, Howell, & Meehan, 2016).

Challenges with implementation of evidence-based guidelines into clinical practice are not unique to mTBI. While a plethora of clinical guidelines for a variety of health topics are available to healthcare providers, many patients receive treatment that is not based on scientific evidence (Institute of Medicine Committee on Standards for Developing Trustworthy Clinical Practice, 2011). Several barriers to guideline implementation by healthcare providers have been identified (Fischer, Lange, Klose, Greiner, & Kraemer, 2016; Institute of Medicine Committee on Standards for Developing Trustworthy Clinical Practice, 2011). Many of these barriers are believed to stem from an interaction between individual characteristics of the guideline (e.g., clarity, specificity, strength of the evidence), perceptions of healthcare providers (e.g., self-efficacy, perceived importance of the recommendations, relevance to practice) and practice environment or context-related characteristics (e.g., inpatient, ambulatory, long-term care setting) (Institute of Medicine Committee on Standards for Developing Trustworthy Clinical Practice, 2011).

Reaching healthcare providers with information on mTBI is an ongoing, yet critical challenge. CDC created a set of implementation tools (e.g., a checklist for healthcare providers and discharge instructions for patients and their families) to promote uptake of its recent clinical practice guideline on pediatric mTBI. An example of one such implementation tool is the HEADS UP to Healthcare Providers online training. Developed in partnership with the American Academy of Pediatrics (AAP), the training provides an overview of the evidence-based recommendations in the CDC Pediatric mTBI Guideline, as well as practical strategies to integrate these recommendations into clinical practice. The training provides a special emphasis on five key recommendations in the CDC guideline: when to use neuroimaging for pediatric patients with mTBI; use of validated, age-appropriate symptom scales to diagnose mTBI; the role of certain risk factors on prolonged recovery; instructing patients on return to activity customized to their symptoms; and how best to manage a patient’s return to non-sports activities soon after the injury (Table 1).

Table 1.

Knowledge questions contained in the Centers for Disease Control and Prevention’s HEADS UP to Healthcare Providers online training pre- and post-test modules and their alignment with the five key recommendations in the CDC Pediatric Mild Traumatic Brain Injury (mTBI) Guideline.

Five Key Recommendations in the CDC Pediatric mTBI Guideline Knowledge questions contained in the HEADS UP to Healthcare Providers pre- and post-test modules
1. Do not routinely image patients to diagnose mTBI.
  • Which indications should prompt admission to a hospital for a patient with mTBI?
    1. Any signs of intracranial injury that require monitoring and repeat neurological exams.
    2. Fluctuating or deteriorating neurological or cognitive symptoms.
    3. Patient complains of trouble concentrating and feeling fatigued.
    4. Both A and B.
    5. All of the above.
  • A healthcare provider should order a head CT scan for patients with suspected mTBI:
    1. In all cases.
    2. If indicated by a validated decision tool.
    3. When requested by the patient’s parents.
    4. Both B and C.
    5. Never.
2. Use validated, age-appropriate symptom scales to diagnose mTBI.
  • A 14-year old patient hit her head while playing soccer. She presents with a headache and says she “just doesn’t feel right.” What is the appropriate course of action?
    1. Conduct a symptom assessment, and if she receives an acceptable score, provide approval for return to sports.
    2. Order a CT scan to evaluate for intracranial injury.
    3. Assess her symptoms and concussion history and require that she be observed at home for 24 to 48 hours for signs of deteriorating neurological function.
    4. Set a date no sooner than 14 days from today for her to return to play if her symptoms have lasted more than 30 minutes; 7 days if they lasted less than 30 minutes.
  • Which of the following is TRUE regarding validated mTBI symptom rating scales:
    1. They can be used as the sole diagnostic criteria.
    2. Only computer-based assessment scales are validated.
    3. They should assess changes from a patient’s usual or baseline symptom presentation.
    4. Both B and C are true.
    5. All of the above are true.
3. Assess evidence-based risk factors for prolonged recovery.
  • Which of the following factors are associated with a prolonged recovery from an mTBI?
    1. Neurological or psychiatric disorder.
    2. Higher cognitive ability.
    3. Older age (older children/adolescents).
    4. Both A and C.
    5. All of the above.
  • A healthcare provider should refer a patient for further evaluation by a specialist when:
    1. Problems with attention, memory and learning, response speed, and other cognitive impairments interfere with school.
    2. The patient experiences ongoing headaches 2 to 3 days after the injury.
    3. Sleep problems emerge or continue despite the patient engaging in appropriate sleep hygiene measures.
    4. Both A and C.
    5. All the above.
4. Provide patients with instructions on return to activity customized to their symptoms.
  • What happens when an athlete’s symptoms return after they’ve initiated a step-wise return to play progression?
    1. The athlete should progress to the next level so long as they do not lose consciousness, vomit or have problems with balance.
    2. The athlete should drop back to the previous level at which they were asymptomatic and try to move forward only after a 24-hour period of rest has passed and they are again asymptomatic.
    3. If they are on Step 1–3, the athlete should repeat the step-wise process, starting at Step 1. If they are on Step 4 – 5, they should stay on the same step another day.
    4. The athlete should rest for 10 to 14 days.
  • A 6-year-old boy diagnosed with mTBI complains of continuing headaches one week after the injury, but no other neurological symptoms. What is the appropriate action to manage his headaches?
    1. Refer him to neurology for baseline neurocognitive testing.
    2. Recommend over-the-counter analgesics.
    3. Recommend complete (“strict”) physical and cognitive rest until he is asymptomatic.
    4. Both B and C.
    5. None of the above.
5. Counsel patients to return gradually to non-sports activities after no more than 2–3 days of rest.
  • Prior to discharging a patient from the ED or your office, it is important to counsel patients and their parents that:
    1. Children are allowed to return to sports activities before school if they feel well enough.
    2. Most children will experience a prolonged recovery.
    3. Within a few weeks, the patient can begin non-strenuous activities that do not exacerbate symptoms.
    4. They should monitor for signs of deteriorating neurological function.
    5. All of the above.
  • When discharging a patient with mTBI, you should counsel patients and their parents to:
    1. Rest cognitively and physically for one to two weeks until they no longer experience symptoms.
    2. Give them approval to return to sports if their symptoms aren’t severe.
    3. Following one to two days of rest, gradually begin return to non-sports activity as long as symptoms do not worsen.
    4. None of the above.
Open in a new tab

Development of the HEADS UP to Healthcare Providers training was guided by constructs of the Health Belief Model (HBM) (Becker, 1974). First developed in the 1950s, the main assumption of the HBM is that an individual’s beliefs (e.g., perceived risk and susceptibility to a particular condition), coupled with their perception of the benefits of a specific action, serve as drivers for behavior adoption or change. In the 1970s, the HBM was expanded to include a construct on self-efficacy, which is a person’s perceived ability (efficacy expectations) to perform an action or task (Bandura, 1977). The use of self-efficacy as a valuable predictor of behavior change in the HBM, and other health theories, is well-documented in the literature (Leventhal, Meyer, & Nerenz, 1980; So, 2013). Moreover, self-efficacy is linked to and considered to be a precursor for behavior change related to guideline implementation (Fischer et al., 2016).

Thus, the purpose of this study is to assess the effectiveness of the HEADS UP to Healthcare Providers online training on healthcare providers’ mTBI knowledge and self-efficacy related to the five key recommendations in the CDC Pediatric mTBI Guideline. Findings can be used to inform the development of additional implementation tools for the CDC Pediatric mTBI Guideline, as well as tools for guidelines that cover other health topics.

2. Methods

This study analyzed data obtained from pre- and post-test modules contained in the HEADS UP to Healthcare Providers training. First released in 2011, the training was revised and re-launched in 2018 to include a specific focus on the recommendations in the CDC Pediatric mTBI Guideline. The primary audience of the training is healthcare providers who care for pediatric patients with mTBI (e.g., pediatricians, family practice providers); however, the training is accessible and available to anyone at no cost from the CDC website. The pre-test module contains 11 questions on participant demographics and experience (e.g., provider type, practice location [based on zip code], and use of mTBI assessment tools), 10 knowledge questions, and 6 questions focused on self-efficacy related to diagnosis and management of pediatric mTBI. In the one-year time period that we analyzed, 19,208 individuals took the training. However, 5,493 people were dropped from the analysis due missing provider type, which was a key part of our inclusion criteria (i.e., desire to ensure that these were actual healthcare providers) and an additional 3,815 people were dropped due to failure to take any part of the post-test. This left us with 9,900 individuals with complete data. Individuals who did not self-identify in the pre-test module as a physician, nurse practitioner, or physician assistant were then excluded from the analysis (n = 7,795). The final sample included 2,105 healthcare providers: 781 physicians, 1,078 nurse practitioners, and 246 physician assistants. These groups were selected for the analysis as they are the key audiences for the CDC Pediatric mTBI Guideline.

Questions in the post-test module were identical to those in the pre-test; the sole exception being that questions on demographics were not included in the post-test module. The knowledge questions in the test modules were derived from and aligned to the five key recommendations in the CDC Pediatric mTBI Guideline (Table 1). Responses to the knowledge questions were re-coded for analysis such that correct responses = 1 and incorrect responses = 0. The self-efficacy questions were created to assess healthcare providers’ confidence related to diagnosis and management of pediatric mTBI. The self-efficacy questions were each measured on a 5-point Likert scale ranging from “strongly disagree” to “strongly agree.” Respondents received 1 to 5 points for each item, with higher scores indicating a greater level of agreement with each statement. Responses were condensed for presentation into three categories: disagree/strongly disagree, neither agree nor disagree, and strongly agree/agree. In addition, healthcare providers’ knowledge and self-efficacy were analyzed using a scale from 10 pre-test knowledge questions and 6 pre-test self-efficacy questions. CDC determined that data collection was not subject to Institutional Review as the data were collected as part of the regular function of the training and designed for training improvement and evaluation.

2.1. Data analysis

Descriptive statistics for study variables were computed using the sample of healthcare providers who completed both the pre- and post-test between January 1 and December 31, 2019. McNemar’s tests were computed to detect statistically significant differences between responses to the pre- and post-test knowledge questions, while Wilcoxon signed rank tests for paired observations were used to compare pre- and post-test self-efficacy questions. Medians were reported for the self-efficacy questions given the ordinal nature of the data. SAS version 9.4 (http://www.sas.com) and IBM SPSS were used to compute all statistics.

Effect sizes were computed for each McNemar’s and Wilcoxon signed rank test. Cohen’s g was computed for each McNemar’s test; a g of less than 0.15 is considered to have a small effect size, a g between 0.15 and 0.25 has a medium effect size, and a g of 0.25 or greater has a large effect size (Cohen, 1988). Effect sizes (r) were also computed for each Wilcoxon signed rank test using the Z-score and interpreted in accordance with Cohen (1988). An r of 0.1 represents a small effect size, an r of 0.3 represents a medium effect size and an r of 0.5 represents a large effect size (Cohen, 1988). Medium and large effects were considered to indicate a practical or substantive change (“practical significance”) between the pre-test and post-test. The number of missing data was negligible for the knowledge questions; between 0 and 16 (0.0–0.8%). The number of missing data was higher for the self-efficacy questions, ranging from 0 to 165 (0.0–7.8%). An internal analysis demonstrated that those with missing data were not significantly different than those with complete data.

3. Results

Most healthcare providers (65.4%) had worked 5 years or fewer in their profession, and roughly half (55.6%), evaluated a pediatric patient for a suspected mTBI in the 12 months preceding administration of the survey (Table 2). When asked how often they adhere to current evidence-based recommendations on mTBI related to decision and assessment tools and discharge instructions, 41.8% reported using decisions tools, 42.3% reported using standardized assessments, and 62.4% reported providing written discharge instructions “often or very often.” To learn about clinical practice recommendations, most healthcare providers preferred seeking information from websites (e.g., UpToDate and Medscape) (78.6%), viewing presentations from experts (51.5%), and/or attending medical conferences (50.7%).

Table 2.

Background characteristics of respondents (n = 2,105) who completed the Centers for Disease Control and Prevention HEADS UP to Healthcare Providers online training, 2019.

Frequency Percent
Healthcare provider type
Physician 781 37.1
Nurse practitioner 1,078 51.2
Physician assistant 246 11.7
Total 2,105 100.0
Number of years in practice
0–5 1,377 65.4
6–10 207 9.8
11–20 284 13.5
21–30 166 7.9
31+ 71 3.4
Total 2,105 100.0
Percentage of practice that is pediatric
0–25% 1,145 54.5
26–50% 407 19.4
51–75% 38 1.8
76+% 511 24.3
Total 2,101 100.0
Have you evaluated a patient for a suspected mild traumatic brain injury (mTBI) in the previous 12 months
Yes 1,166 55.6
No 848 40.5
Unsure 82 3.9
Total 2,096 100.0
Healthcare provider uses decision tools to evaluate for mTBI in their practice
Very often 418 19.9
Often 458 21.9
Sometimes 613 29.3
Never 607 29.0
Total 2,096 100.0
Healthcare provider uses standardized assessments of concussion in their practice
Very often 466 22.2
Often 422 20.1
Sometimes 607 29.0
Never 601 28.7
Total 2,096 100.0
Healthcare provider provides written discharge instructions for patients with mTBI
Very often 865 41.3
Often 443 21.1
Sometimes 354 16.9
Never 434 20.7
Total 2,096 100.0
How healthcare provider prefers to learn about clinical practice recommendations a
Websites (like UpToDate and Medscape) 1,654 78.6
Presentations from experts (such as Grand Rounds) 1,085 51.5
Medical conferences 1,067 50.7
Scientific publications 998 47.4
Medical organizations 888 42.2
Blogs and social media 275 13.1
Open in a new tab
a

Respondents were permitted to select multiple responses to this question, therefore the total adds up to over 100%

The percentage of correct responses for each of the knowledge questions increased significantly between the pre- and post-tests (Table 3). Improvements for 8 out of the 10 knowledge questions had a high level of practical significance. Of these, the questions with pre- to post-test improvement with the highest level of practical significance was for the key guideline recommendation on neuroimaging (“A healthcare provider should order a head computerized tomography (CT) scan for patients with suspected mTBI”) (pre-test correct: 70.2%; post-test correct: 87.8%; p < 0.0001, Cohen’s g = 0.39). Other knowledge questions that demonstrated improvements with a high level of practical significance included: “A 14-year old patient hit her head while playing soccer. She presents with a headache and says she ‘just doesn’t feel right.’ What is the appropriate course of action?” (pre-test correct: 84.0%; post-test correct: 94.8%; p < 0.0001, Cohen’s g = 0.36); “What happens when an athlete’s symptoms return after they’ve initiated a step-wise return to play progression?” (pre-test correct: 83.5%; post-test correct: 95.3%; p < 0.0001, Cohen’s g = 0.36); and, “A 6-year-old boy diagnosed with mTBI complains of continuing headaches one week after the injury, but no other neurological symptoms. What is the appropriate action to manage his headaches?” (pre-test correct: 30.5%; post-test correct: 50.0%; p < 0.0001, Cohen’s g = 0.34). The two questions with improvements in knowledge with low and medium levels of significance included: “Which of the following is TRUE regarding validated mTBI symptom rating scales?”; pre-test correct: 74.9%; post-test correct: 77.8% (p < 0.01, Cohen’s g = 0.07) and “Which indications should prompt admission to a hospital for a patient with mTBI?”; pre-test correct: 76.0%; post-test correct: 83.4% (p < 0.0001, Cohen’s g = 0.19).

Table 3.

Comparison of pre- and post-test concussion-related knowledge questions contained in the Centers for Disease Control and Prevention HEADS UP to Healthcare Providers online training, 2019 (n = 2,105).

Pre-test Post-test Difference
Percent Correctb Percent Correct s p-value Cohen’s g
Guideline recommendation 1: Do not routinely image patients to diagnose mTBI
Which indications should prompt admission to a hospital for a patient with mTBI? 59.00 <0.0001 0.19
Correct 76.0 83.4
Incorrect 24.0 16.6
A healthcare provider should order a head CT scan for patients with suspected mTBI: 288.14 <0.0001 0.39
Correct 70.2 87.8
Incorrect 29.8 12.2
Guideline recommendation 2: Use validated, age-appropriate symptom scales to diagnose mTBI
A 14-year old patient hit her head while playing soccer. She presents with a headache and says she “just doesn’t feel right.” What is the appropriate course of action? 163.71 <0.0001 0.36
Correct 84.0 94.8
Incorrect 16.0 5.2
Which of the following is TRUE regarding validated mTBI symptom rating scales? 7.47 0.0063 0.07
Correct 74.9 77.8
Incorrect 25.1 22.2
Guideline recommendation 3: Assess evidence-based risk factors for prolonged recovery
Which of the following factors are associated with a prolonged recovery from an mTBI? 159.05 <0.0001 0.25
Correct 59.7 75.2
Incorrect 40.3 24.9
A healthcare provider should refer a patient for further evaluation by a specialist when: 281.03 <0.0001 0.31
Correct 48.5 69.8
Incorrect 51.5 30.2
Guideline recommendation 4: Provide patients with instructions on return to activity customized to their symptoms.
What happens when an athlete’s symptoms return after they’ve initiated a step-wise return to play progression? 179.97 <0.0001 0.36
Correct 83.5 95.3
Incorrect 16.6 4.8
A 6-year-old boy diagnosed with mTBI complains of continuing headaches one week after the injury, but no other neurological symptoms. What is the appropriate action to manage his headaches? 279.34 <0.0001 0.34
Correct 30.5 50.0
Incorrect 69.5 50.0
Guideline recommendation 5: Counsel patients to return gradually to non-sports activities after no more than 2–3 days of rest.
Prior to discharging a patient from the ED or your office, it is important to counsel patients and their parents that: 103.35 <0.0001 0.27
Correct 80.4 89.4
Incorrect 19.6 10.6
When discharging a patient with mTBI, you should counsel patients and their parents to: 189.72 <0.0001 0.29
Correct 68.5 84.3
Incorrect 31.5 15.7
Open in a new tab

The questions are displayed as the percentage of respondents who answered the question correctly (i.e., answered a true question as true or a false question as false or selected the correct response for the multiple-choice item).

The level of self-efficacy measured for each of the six questions in the post-test demonstrated statistically significant improvements as compared to the pre-test (Table 4). All six questions had a self-efficacy level increase of a high level of practical significance (r > 0.50) between the pre- and post-tests. These questions were: “I am confident in my ability to diagnose an mTBI” (p < 0.001; r = 0.62), “I am confident in my ability to treat mTBI symptoms” (p < 0.001; r = 0.63), “I am confident in my ability to manage the return to sports progression for my patients” (p < 0.001; r = 0.63), “I am confident in my ability to manage return to school for my patients” (p < 0.001; r = 0.63).“I am confident in my ability to identify patients who should be referred for evaluation by an mTBI specialist” (p < 0.001; r = 0.59), and “I am confident in my ability to communicate with patients about mTBI prevention strategies” (p < 0.001; r = 0.54).

Table 4.

Pre- and post-test concussion self-efficacy questions contained in the Centers for Disease Control and Prevention HEADS UP to Healthcare Providers online training, 2019 (n = 2,105).

Pre-test Post-test
Frequency Percent Median Median IQR Frequency Percent Median Median IQR Z-score P-value R
I am confident in my ability to diagnose an mTBI
Strongly agree/agree 1,259 64.5 4.0 3–4 I am confident in my ability to diagnose an 95.8 4.0 4–5 −27.3 <0.001 −0.62
Neither agree nor disagree 446 22.9 80 3.8
Disagree/strongly disagree 247 12.7 8 0.4
I am confident in my ability to treat mTBI symptoms
Strongly agree/agree 1,178 60.5 4.0 3–4 1,975 93.8 4.0 4–5 −27.6 <0.001 −0.63
Neither agree nor disagree 528 27.1 122 5.8
Disagree/strongly disagree 240 12.3 8 0.4
I am confident in my ability to manage the return to sports progression for my patients
Strongly agree/agree 1,155 59.2 4.0 3–4 1,981 94.1 4.0 4–5 −27.8 <0.001 −0.63
Neither agree nor disagree 506 25.9 112 5.3
Disagree/strongly disagree 291 14.9 12 0.6
I am confident in my ability to manage return to school for my patients
Strongly agree/agree 1,256 64.5 4.0 3–4 2,012 95.6 4.0 4–5 −27.7 <0.001 −0.63
Neither agree nor disagree 454 23.3 88 4.2
Disagree/strongly disagree 236 12.1 5 0.2
I am confident in my ability to identify patients who should be referred for evaluation by an mTBI specialist
Strongly agree/agree 1,360 69.7 4.0 3–4 2,033 96.6 4.0 4–5 −26.2 <0.001 −0.59
Neither agree nor disagree 408 20.9 68 3.2
Disagree/strongly disagree 184 9.4 4 0.2
I am confident in my ability to communicate with patients about mTBI prevention strategies
Strongly agree/agree 1,514 78.0 4.0 4–5 2,045 97.2 5.0 4–5 −23.8 <0.001 −0.54
Neither agree nor disagree 309 15.9 56 2.7
Disagree/strongly disagree 117 6.0 4 0.2
Open in a new tab

For all attitude items, “strongly agree”=5, “agree”=4, “neither agree nor disagree”=3, “disagree”=2, and “strongly disagree”=1.

IQR = Inter-quartile range.

4. Discussion

This study examined the effectiveness of the HEADS UP to Healthcare Provider online training on improving healthcare providers’ mTBI knowledge and self-efficacy related to the five key recommendations in the CDC Pediatric mTBI Guideline. Healthcare providers who completed the HEADS UP to Healthcare Providers online training not only demonstrated significant improvements in knowledge but also reported improved self-efficacy related to mTBI diagnosis and management. These findings suggest that the HEADS UP to Healthcare Providers online training may be an effective tool to support implementation of the CDC Pediatric mTBI Guideline. Expanded use among healthcare providers who care for pediatric patients with mTBI may be beneficial.

More than 19,000 people accessed the HEADS UP to Healthcare Providers online training during the study period. High use of this online training may be attributed to mTBI and concussion training requirements instituted by some health organizations, schools, and states, as well as the inclusion of continuing education credits available through AAP upon its completion (Fischer et al., 2016). Prior research suggests that continuing education opportunities and online training for healthcare providers on concussion is associated with improvements in clinical practice (Babul, Turcotte, Lambert, Hadly, & Sadler, 2020; Broshek, Samples, Beard, & Goodkin, 2014). An online training approach was used for the HEADS UP to Healthcare Providers training as it is a cost-effective approach that allowed organizations that require the training to disseminate it widely and provide flexibility for where (e.g., home or place of work) and when (during or outside of practice hours) a healthcare provider could complete their training requirement. As the use of online trainings to educate healthcare providers by CDC and other organizations have increased in popularity, evaluation of the effectiveness of this educational approach is critical. At least two systematic reviews concluded that training healthcare providers through online trainings in place of non-computer-based trainings (e.g., in-person presentations, lectures, and workshops) is equally effective in improving healthcare providers’ knowledge and clinical behaviors (Cook et al., 2008; Richmond, Copsey, Hall, Davies, & Lamb, 2017). These findings suggest that health educators may consider the development of an online training as one component of a comprehensive approach for guideline implementation.

Healthcare providers with knowledge of and a high self-efficacy related to clinical recommendations may be more likely to adopt and adhere to guidelines (Fischer et al., 2016). Consistent with increases in self-efficacy related to mTBI diagnosis and management, some of the largest improvements in knowledge between pre- and post-tests were observed for the questions aligned with the CDC Pediatric mTBI Guideline recommendations on diagnostic use of neuroimaging and managing a patient’s return to activity. Changing healthcare provider behaviors around CT scans for mTBI is an ongoing challenge (Halaweish, Riebe-Rodgers, Randall, & Ehrlich, 2018). Decreasing routine use of CT scans for patients with mTBI may help reduce adverse health outcomes related to radiation exposure (Mannix, Meehan, Monuteaux, & Bachur, 2012; Stanley et al., 2014); up to 35% of CT scans conducted in the emergency department for patients with mTBI may not be warranted based on clinical guidance (Melnick et al., 2012). While some clinical recommendations have been consistent for numerous years (such as that on neuroimaging), the CDC Pediatric mTBI Guideline recommendation of a gradual return to non-sports activities represents a shift in clinical care (Lumba-Brown et al., 2018). Previous guidance recommended a longer rest period; however, healthcare providers are now advised to instruct pediatric patients with mTBI to return to their regular non-sports activities within 2–3 days. As compared to prescribing “strict rest,” this change in guidance is associated with a shorter recovery and a lower symptom burden (Thomas, 2015). Findings from this study indicate that training may show promise in furthering adoption of neuroimaging and return to activity recommendations that can improve patient health outcomes. However, additional studies are needed to assess the sustainability of these improvements and their translation into clinical practice.

Previous studies suggest that medical students and residents may not receive adequate training on mTBI diagnosis and management and that more educational opportunities on this topic are needed (Donaworth, Grandhi, Logan, Gubanich, & Myer, 2016; Haider et al., 2017). Donaworth and colleagues (2016) found most U.S. medical school curriculums do not include lectures on concussion, and that the majority of medical students do not gain clinical experience with diagnosis and management of concussion during their medical school training. Interestingly, approximately two-thirds of healthcare providers who completed the CDC training reported working five years or fewer in their profession. A desire to learn about concussion, widespread use of digital or mobile-based tools, and use of social media to promote the training, may be some of reasons why the training was accessed more frequently by newer healthcare providers (Donaworth et al., 2016; Ventola, 2014). Taken together, this points to the potential of the HEADS UP to Healthcare Provider training to help fill a current information gap for healthcare providers new to their profession.

This study only measured changes in knowledge and self-efficacy based on constructs of the HBM prior to and immediately following the HEADS UP to Healthcare Providers training. As such, environmental and individual provider characteristics (a limitation of the HBM) were not taken into consideration (Janz & Becker, 1984). Moreover, as noted above, the long-term effectiveness on healthcare providers’ knowledge and self-efficacy was not measured. Ensuring successful and sustained improvements among healthcare providers following use of the training may benefit further from a multi-pronged approach that is inclusive of system-based changes (e.g., use of electronic health records (EHR)) and support from decision-makers (Campanella et al., 2016). Previous studies suggest that integrating an online training into a comprehensive implementation effort may lead to improved patient-healthcare provider communication and symptom-based assessments (Arbogast et al., 2017). Arbogast and colleagues found that implementation of a concussion-specific EHR-based decision support tool, along with use of the HEADS UP to Healthcare Provider online training, substantially increased documentation of healthcare provider-patient discussions about recovery (e.g., return to school and sports; Arbogast et al., 2017). This is consistent with other studies that found that EHR-based systems may strengthen guideline adherence among healthcare providers (Campanella et al., 2016).

4.1. Practical applications

Guideline efforts may be inclusive of development, dissemination, and implementation planning (Fischer et al., 2016). Yet, approximately one-third of guidelines published between 2010 and 2017 did not offer guideline implementation tools (Liang, Abi Safi, & Gagliardi, 2017). Including health educators’ and other public health professionals’ participation in guideline development may help to ensure that implementation strategies are considered while recommendations are drafted. This may include ensuring recommendations are written with patient-centered language and practical strategies (Fischer et al., 2016).

Guideline implementation tools may improve healthcare providers’ adherence to guideline recommendations (Liang, Abi Safi, et al., 2017). This study highlights several factors guideline developers may take into consideration when creating such tools. First, guideline implementation tools developed using a theoretical framework, such as the online training examined in this paper, are considered to be most effective (Liang, Bernhardsson, et al., 2017). Guideline developers can ensure that relevant implementation tools (customized for both patients and healthcare providers) are designed using health behavior theories and tested to assess their ability to support evidence-based patient care (Fischer et al., 2016; Liang, Abi Safi, et al., 2017). The HEADS UP to Healthcare Providers training took advantage of constructs of adult learning theory and components of the HBM by integrating interactive knowledge checks and content aimed at building healthcare provider’s self-efficacy and promoting positive learning outcomes (Cook, Levinson, & Garside, 2010). Second, working with partners and key stakeholders to promote and disseminate online trainings, such as this one, can help improve training use among the target audiences and has benefitted other aspects of the HEAD UP campaign. Finally, the widespread usage of digital based tools and preferences for seeking out clinical information from websites suggests that a shift from print materials to online, tablet, and mobile-based formats may improve uptake and adherence to guidelines (Gagliardi & Alhabib, 2015; Ventola, 2014). Online training, which can be a cost-effective approach to reach a large audience, may be one important way to achieve this.

4.2. Limitations

There are several limitations to this study. First, data from the pre- and post-test modules were obtained from a convenience sample of healthcare providers who completed the HEADS UP to Healthcare Providers training. Thus, the findings are not intended to be generalizable to a wider population. Second, this study did not have a control group of healthcare providers who did not complete the training with which the authors could compare the results. Thus, it is unclear if other sources of information identified as commonly used by healthcare providers in the study (e.g., websites and presentations from experts) are similarly effective. Future studies may explore this. Third, the pre- and post-test knowledge questions were composed of multiple choice and true/false questions. This may have led to an overestimate in the level of knowledge of the respondents, as respondents had a 25–50% chance of randomly guessing the correct response. Fourth, social desirability may play a role in the respondents’ answers to questions related to self-efficacy. It is likely that the respondents know what the “correct” response is or what “should” be the answer, particularly with the self-efficacy items. This may inflate both the pre- and post-test agreement with these items; however, it may not impact changes in responses observed between the pre- and post-tests. Finally, most healthcare providers completed the post-test immediately after taking the training. Thus, it is not possible to determine whether changes in knowledge and self-efficacy will persist over time. Further, this study did not evaluate actual changes in patient care. Thus, it is not possible to determine whether an individual’s gains in knowledge and self-efficacy will translate into changes in their clinical care practices. Future research that examines whether the training led to actual improvements in diagnosis and management decisions may be beneficial.

5. Conclusion

This study examined the effectiveness of the HEADS UP to Healthcare Providers online training on healthcare providers’ mTBI knowledge and self-efficacy related to the five key recommendations in the CDC Pediatric mTBI Guideline. Findings suggest that upon completion of the training, healthcare providers demonstrated significant improvements in knowledge related to the five key recommendations in the guideline, as well as improvements in self-efficacy related to mTBI diagnosis and management. Expanding use of this training may be an effective way to reach a large number of healthcare providers and improve use of recommendations in the CDC Pediatric mTBI Guideline.

6. Financial disclosure

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Biography

Kelly Sarmiento, MPH is a public health advisor at the Centers for Disease Control and Prevention’s (CDC) Injury Center in Atlanta, Georgia. Over the last 15 year at CDC’s Injury Center, Kelly has worked on multiple national and award-winning educational campaigns and initiatives that focus on raising awareness and improving prevention, recognition, and management of traumatic brain injuries, including concussion. Kelly has a Master of Public Health from Yale University School of Epidemiology and Public Health, and a BA in Anthropology and a BA in Spanish from the University of California, Santa Barbara.

Jill Daugherty, PhD, MPH is an epidemiologist at the Centers for Disease Control and Prevention, National Center for Injury Prevention and Control, Division of Injury Prevention. She earned her doctorate in sociology from Emory University in Atlanta where she focused on social inequality. Her work at the CDC focuses on disparities in traumatic brain injury and concussion knowledge and prevention. Previously she was employed at CDC’s National Center for Health Statistics in Maryland.

Dana Waltzman, PhD, is a behavioral scientist on the Traumatic Brain Injury (TBI) Team in the Division of Injury Prevention (DIP) at the Injury Center. Her work focuses on understanding the public health burden, prevention, and long-term effects of TBI. Dr. Waltzman received a bachelor of science degree in psychology, a bachelor of arts degree in cognitive science from the University of Georgia, and a doctoral degree in cognitive neuroscience at the University of California, Los Angeles. She completed two post-doctoral fellowships at the Memory and Aging Center at the University of California, San Francisco, and a joint fellowship at Stanford University and the Department of Veterans Affairs, Palo Alto.

Footnotes

8.

Publisher's Disclaimer: Disclaimer

Publisher's Disclaimer: The findings and conclusions in this report are those of the author(s) and do not necessarily represent the official position of the Centers for Disease Control and Prevention.

Conflict of interest

The authors have indicated they have no potential conflicts of interest to disclose.

References

  1. Arbogast KB, Curry AE, Metzger KB, Kessler RS, Bell JM, Haarbauer-Krupa J, … Master CL (2017). Improving primary care provider practices in youth concussion management. Clinical Pediatrics (Phila), 56(9), 854–865. 10.1177/0009922817709555. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Babul S, Turcotte K, Lambert M, Hadly G, Sadler K, & Swaine IL (2020). Delivering evidence-based online concussion education to medical and healthcare professionals: The Concussion Awareness Training Tool (CATT). Journal of Sports Medicine, 2020, 1–10. 10.1155/2020/8896601. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Bandura A (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84(2), 191–215. 10.1037//0033-295x.84.2.191. [DOI] [PubMed] [Google Scholar]
  4. Becker MH (1974). The health belief model and personal health behavior. Health Education Monographs, 2, 324–473. [Google Scholar]
  5. Broshek DK, Samples H, Beard J, & Goodkin HP (2014). Current practices of the child neurologist in managing sports concussion. Journal of Child Neurology, 29(1), 17–22. 10.1177/0883073812464525. [DOI] [PubMed] [Google Scholar]
  6. Campanella P, Lovato E, Marone C, Fallacara L, Mancuso A, Ricciardi W, & Specchia ML (2016). The impact of electronic health records on healthcare quality: A systematic review and meta-analysis. Eur J Public Health, 26(1), 60–64. 10.1093/eurpub/ckv122. [DOI] [PubMed] [Google Scholar]
  7. Carl RL, & Kinsella SB (2014). Pediatricians’ knowledge of current sports concussion legislation and guidelines and comfort with sports concussion management: A cross-sectional study. Clinical Pediatrics (Phila), 53(7), 689–697. 10.1177/0009922814526979. [DOI] [PubMed] [Google Scholar]
  8. Cohen J (1988). Statistical power analysis for the behavioral sciences. New York: NY Routledge. [Google Scholar]
  9. Cook DA, Levinson AJ, & Garside S (2010). Time and learning efficiency in Internet-based learning: A systematic review and meta-analysis. Advances in Health Sciences Education, 15(5), 755–770. 10.1007/s10459-010-9231-x. [DOI] [PubMed] [Google Scholar]
  10. Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, & Montori VM (2008). Internet-based learning in the health professions: A meta-analysis. JAMA, 300(10), 1181–1196. 10.1001/jama.300.10.1181. [DOI] [PubMed] [Google Scholar]
  11. Donaworth MA, Grandhi RK, Logan K, Gubanich PJ, & Myer GD (2016). Is current medical education adequately preparing future physicians to manage concussion: An initial evaluation. The Physician and Sportsmedicine, 44(1), 1–7. 10.1080/00913847.2016.1135039. [DOI] [PubMed] [Google Scholar]
  12. Fischer F, Lange K, Klose K, Greiner W, & Kraemer A (2016). Barriers and strategies in guideline implementation-a scoping review. Healthcare (Basel), 4(3), 36. 10.3390/healthcare4030036. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Gagliardi AR, & Alhabib S (2015). Trends in guideline implementation: A scoping systematic review. Implementation Science, 10, 54. 10.1186/s13012-015-0247-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Greene NH, Kernic MA, Vavilala MS, & Rivara FP (2014). Variation in pediatric traumatic brain injury outcomes in the United States. Archives of Physical Medicine & Rehabilitation, 95(6), 1148–1155. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Haider MN, Leddy JJ, Baker JG, Kiel JM, Tiso M, Ziermann KA, & Willer BS (2017). Concussion management knowledge among residents and students and how to improve it. Concussion (London, England), 2(3), CNC40–CNC40. 10.2217/cnc-2017-0001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Halaweish I, Riebe-Rodgers J, Randall A, & Ehrlich PF (2018). Compliance with evidence-based guidelines for computed tomography of children with head and abdominal trauma. Journal of Pediatric Surgery, 53(4), 748–751. 10.1016/j.jpedsurg.2017.07.008. [DOI] [PubMed] [Google Scholar]
  17. Institute of Medicine Committee on Standards for Developing Trustworthy Clinical Practice, G. (2011). In Graham R, Mancher M, Miller Wolman D, Greenfield S, & Steinberg E (Eds.), Clinical Practice Guidelines We Can Trust. Washington (DC): National Academies Press (US). Copyright 2011 by the National Academy of Sciences. All rights reserved. [PubMed] [Google Scholar]
  18. Janz NK, & Becker MH (1984). The health belief model: A decade later. Health Education Quarterly, 11(1), 1–47. 10.1177/109019818401100101. [DOI] [PubMed] [Google Scholar]
  19. Leventhal H, Meyer D, & Nerenz D (1980). The common sense representation of illness danger. Contributions to Medical Psychology, 2, 7–30. [Google Scholar]
  20. Liang L, Abi Safi J, & Gagliardi AR (2017a). Number and type of guideline implementation tools varies by guideline, clinical condition, country of origin, and type of developer organization: Content analysis of guidelines. Implementation Science, 12(1), 136. 10.1186/s13012-017-0668-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Liang L, Bernhardsson S, Vernooij RW, Armstrong MJ, Bussieres A, Brouwers MC, & Gagliardi AR (2017b). Use of theory to plan or evaluate guideline implementation among physicians: A scoping review. Implementation Science, 12(1), 26. 10.1186/s13012-017-0557-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Lumba-Brown A, Yeates KO, Sarmiento K, Breiding MJ, Haegerich TM, Gioia GA, … Timmons SD (2018). Centers for Disease Control and Prevention guideline on the diagnosis and management of mild traumatic brain injury among children. JAMA Pediatrics, 172(11), e182853. 10.1001/jamapediatrics.2018.2853. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Mannix R, Meehan WP, Monuteaux MC, & Bachur RG (2012). Computed tomography for minor head injury: Variation and trends in major United States pediatric emergency departments. Journal of Pediatrics, 160(1), 136–139.e131. 10.1016/j.jpeds.2011.06.024. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Melnick ER, Szlezak CM, Bentley SK, Dziura JD, Kotlyar S, & Post LA (2012). CT overuse for mild traumatic brain injury. The Joint Commission Journal on Quality and Patient Safety, 38(11), 483–489. 10.1016/s1553-7250(12)38064-1. [DOI] [PubMed] [Google Scholar]
  25. Richmond H, Copsey B, Hall AM, Davies D, & Lamb SE (2017). A systematic review and meta-analysis of online versus alternative methods for training licensed health care professionals to deliver clinical interventions. BMC Medical Education, 17(1), 227–227. 10.1186/s12909-017-1047-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. So J (2013). A further extension of the Extended Parallel Process Model (E-EPPM): Implications of cognitive appraisal theory of emotion and dispositional coping style. Health Communication, 28(1), 72–83. 10.1080/10410236.2012.708633. [DOI] [PubMed] [Google Scholar]
  27. Stache S, Howell D, & Meehan WP 3rd., (2016). Concussion management practice patterns among sports medicine physicians. Clinical Journal of Sport Medicine, 26(5), 381–385. 10.1097/jsm.0000000000000270. [DOI] [PubMed] [Google Scholar]
  28. Stanley RM, Hoyle JD, Dayan PS, Atabaki S, Lee L, Lillis K, … Kuppermann N (2014). Emergency department practice variation in computed tomography use for children with minor blunt head trauma. Journal of Pediatrics, 165(6), 1201–1206.e2. 10.1016/j.jpeds.2014.08.008. [DOI] [PubMed] [Google Scholar]
  29. Thomas DG, Apps JN, Hoffmann RG, McCrea M, & Hammeke T (2015). Benefits of strict rest after acute concussion: a randomized controlled trial. Pediatrics, 135(2), 213–223. 10.1542/peds.2014-0966 [DOI] [PubMed] [Google Scholar]
  30. Ventola CL (2014). Mobile devices and apps for health care professionals: Uses and benefits. P&T, 39(5), 356–364. [PMC free article] [PubMed] [Google Scholar]

RESOURCES