Skip to main content
BMJ Open logoLink to BMJ Open
. 2020 Aug 23;10(8):e038166. doi: 10.1136/bmjopen-2020-038166

Association between concussion education and concussion knowledge, beliefs and behaviours among youth ice hockey parents and coaches: a cross-sectional study

Amanda M Black 1,2,3,4,, Keith Owen Yeates 2,3,5, Shelina Babul 6,7, Alberto Nettel-Aguirre 1,2,4,8, Carolyn A Emery 1,2,3,4,8
PMCID: PMC7445332  PMID: 32830117

Abstract

Objectives

To examine the association between self-reported exposure to concussion education and knowledge, beliefs and self-reported behaviour among parents and coaches of youth ice hockey players.

Design

Cross-sectional.

Setting

Community ice hockey teams from Calgary and Edmonton, Alberta, Canada.

Participants

Parents and coaches of ice hockey players (ages 11–17, all divisions of play).

Primary and secondary outcome measures

Participants completed a questionnaire developed and validated to measure concussion knowledge, beliefs and concussion management behaviour (ie, coaches removing athletes from play; parents taking children with suspected concussions to physicians) consistent with the Health Action Process Approach (HAPA). The questionnaire examined specific HAPA constructs (ie, risk perception, outcome expectancies, action self-efficacy, intention, action planning, maintenance self-efficacy, recovery self-efficacy) relevant to concussion management behaviour.

Results

Participants included 786 parents (31.8% with coaching experience) and 10 non-parent coaches. Of the participants, 649 (82.6%) previously received concussion education. Based on a multivariable regression analysis adjusting for coaching experience, previous history of a child sustaining one or more concussions, first aid experience and cluster by team, exposure to concussion education was associated with a mean score difference of 1.36 (95% CI 0.68 to 2.03), p<0.0001, in the knowledge score. Exposure to concussion education was not significantly associated with any of the HAPA constructs based on Wilcoxon rank-sum tests.

Conclusion

Exposure to concussion education may be associated with small overall differences in concussion knowledge but may not be associated with significant differences in beliefs or intended behaviours related to concussion management among youth hockey parents and coaches.

When providing education or recommendations for concussion education sources to coaches and parents, educational strategies grounded in behavioural change theory that specifically target the motivators of behavioural change should be considered.

Keywords: sports medicine, public health, medical education & training


Strengths and limitations of this study.

  • This study examines concussion knowledge, beliefs and behaviours using a behavioural change model that assesses factors beyond intention; models used in previous studies have not addressed the intention-to-behaviour gap.

  • Concussion management behaviours are examined in a large sample of hockey parents, including those with and without coaching experience.

  • Because the study is cross-sectional, we cannot establish temporality between concussion education and outcomes.

  • Because of the small number of coaches who had not received previous concussion education, we were limited to a non-parametric model to examine differences between study groups.

Introduction

Concussions in youth sport are a major health concern that can negatively impact an adolescent’s daily functioning, including the ability to focus in school.1 Symptoms of concussion can be somatic, cognitive and/or emotional.1 Approximately 60%–80% of children who sustain a concussion recover within a month without secondary complications.1–3 However, concussions that are not managed appropriately can lead to prolonged recovery, long-term complications and in rare cases death.4–7

Primary prevention of concussion is ideal, but is not always possible. Therefore, secondary prevention aimed at the recognition of concussion and appropriate management to prevent concussion recurrence and consequences of concussion has become a policy focus and concussion education is an essential tool.8 Appropriate sideline management includes recognising that a suspected concussion has occurred and removing the athlete from play.1 In youth sport, however, healthcare providers are typically not on the sidelines. Parents, athletes and coaches are crucial in concussion recognition and management.

Most research evaluating concussion education focuses on the player’s willingness to report symptoms to a coach or parent.9 10 However, sometimes athletes do report their symptoms or recognise a concussion has occurred and the responsibility lies with the coach to recognise a suspected concussion, remove the athlete from play and inform the parents to seek medical attention.11 Parents are responsible for taking their child to a physician for assessment and clearance to return to school and play. Occasionally, concussion symptoms are delayed and may go undetected by the coach.7 In this case, the parent needs to recognise their child may have sustained a concussion and seek care. Experts postulate that learning about concussion signs and symptoms can inform appropriate management.7 However, parent and coach beliefs can affect their response.12 Parent beliefs can be understood within the context of behavioural change models and can include risk perceptions of negative outcomes associated with a given behavioural response, their expectation of what will happen if they perform the desired behaviour and their perceived confidence in their ability to perform the behaviour under different circumstances.13

Providing concussion education can increase coach and parent knowledge.11 14–19 However, evidence regarding the effect of concussion education on beliefs or behaviours of parents and coaches is sparse.15 16 The Health Action Process Approach (HAPA) is a theoretical framework to understand the processes involved in health behavioural change.20 HAPA has been used to inform how interventions can affect multiple health behaviours (eg, neuromuscular training programmes, physical exercise, seat belt use and dietary behaviours).21 HAPA proposes that changing health behaviour consists of two phases: forming an intention to do the behaviour (motivational phase), followed by a volitional stage of self-efficacy and planning to do the behaviour. In the motivational phase, risk perception, positive and negative outcome expectancies, and action self-efficacy influence an individual’s motivation or intention to do a behaviour.21 The volitional phase involves postintentional factors including planning and self-efficacy. Importantly, this phase addresses a key limitation of previous social cognitive models that are characterised by a gap between intention and behaviour.22 Educational resources can target constructs for individuals in either phase. Using a theory that examines postintentional factors is important given that parents and coaches may be subjected to environmental and situational factors that influence a parent’s intention to take their child to a physician for assessment and clearance (eg, scheduling conflicts, resistance from the child, playoff season) or a coach’s intention to remove a player from play (eg, playoff season, parent expectations, player’s insistence on playing).

Currently in Canada, most provinces do not have legislation mandating concussion education. Individual sporting associations can decide whether to provide coaches or parents with concussion education and when recommended, the content is not specified. Recommendations regarding future changes to concussion education policies require a clear understanding of the effect of providing concussion education on both knowledge of concussion and beliefs related to concussion management behaviour. This issue is especially important in Canadian youth ice hockey, where there are high participation and concussion rates.23 24 Therefore, the primary objective of this study, designed in concert with the HAPA, was to examine the association between self-reported exposure to concussion education and knowledge, beliefs and self-reported concussion management behaviour among the parents and coaches of youth ice hockey players (ages 11–17). A secondary objective was to describe from where hockey parents and coaches obtain concussion education, how helpful they find the education, and where they would like to receive further concussion education. Most available educational materials on concussion were designed to translate knowledge but not with the tenets of behavioural change theory in mind. Therefore, the research team hypothesised that parents and coaches who received concussion education, when compared with those who did not, will demonstrate increased knowledge about concussion, but may not display differences in beliefs or behaviours related to removing an athlete from play (coach) or taking their child into a physician for assessment and clearance (parent).

Methods

Participants and procedure

Parents and coaches of youth ice hockey players (ages 11–17) were identified through their child’s or team’s participation in at least one of two prospective cohort studies designed to assess risk factors for concussion in youth ice hockey and the effect of body checking rule changes. The prospective cohort studies both began in 2013, with recruitment annually around the start of the hockey season. The questionnaires for the current study (Concussion Knowledge, Beliefs and Behaviour Questionnaire (CKBBQ: parent and coaches, CKBBQ: coaches [non-parent]) were completed at the start of the 2015–2016 season. There were two versions of the questionnaire: one for parents of youth ice hockey players, and one for youth hockey coaches who did not have children participating in the sport. The parent questionnaire was delivered in a package of forms to be completed at home before their child’s baseline assessment. Parents who were also coaches of Pee Wee (11–12 years old), Bantam (13–14 years old) or Midget (15–17 years old) teams that season were asked to complete an additional set of questions related to coach beliefs and behaviours attached to the parent questionnaire. Coach forms were delivered to coaches by team designates who helped organise the team’s participation. Each parent or coach on a team individually consented to the study. Only one parent per player was requested to complete the questionnaire. Parents and coaches who completed this questionnaire received no financial benefits. Participants were not required to answer every question or complete all the forms provided.

Patient and public involvement

Integrated knowledge translation was a priority for this research programme in youth ice hockey. This study was designed in consultation with association members from Hockey Calgary and Hockey Canada. Knowledge brokers supported the research process for the large cohort study. Results were presented to the association.

Questionnaire

The CKBBQ was developed based on a review of the literature, the fourth International Consensus on Concussion in sport and the resources provided by Schwarzer and Luszczynska25 26 on developing HAPA scales. Knowledge questions were adapted from previous questionnaires by Rosenbaum and Arnett,27 White et al,28 Krol et al,29 and Guilmette et al.30 Three additional items were generated to assess participants’ understanding of different responses to potential red flag symptoms, defined as more serious symptoms that require an immediate visit to the emergency room. The parent and coach versions were designed to assess beliefs related to specific behaviours for management (coach removing the athlete from play, parent taking the child to a physician). Outcome expectancies specific to the youth ice hockey parent and coach population were determined based on qualitative work by Echlin31 and extended and modified using qualitative interviews (n=24). A panel of experts, consisting of certified athletic therapists, physicians, physiotherapists and concussion researchers, reviewed the questionnaires for face validity and clarity. The questionnaires were pilot tested with a group of parents and coaches in Vancouver and Calgary for clarity and option response verification. Cognitive interviews with three parents were also conducted where they were asked to talk aloud while filling out the questionnaire to assess if they were interpreting items as the questionnaire intended and to comment if any additional response options should be considered. The full questionnaire took approximately 25 min for parent coaches, 20 min for coaches and 15 min for parents with no coaching experience to complete (online supplementary file S1 includes the full questionnaire).

Supplementary data

bmjopen-2020-038166supp001.pdf (5.4MB, pdf)

Study measures

Participant characteristics (eg, coaching experience, age, hockey playing experience, hockey concussion history of their child, and personal as well as child hockey experience) were collected. The primary exposure variable for this study was whether the parent or coach reported having ever received concussion education. Parents and coaches were also asked whether the education was provided during the current 2015–2016 season. Concussion knowledge was assessed using 32 multiple-choice items assessing knowledge of the concussion definition (one item), awareness of risks (three items), misconceptions (two items), recognition (three items), signs and symptoms (eight real symptoms, eight distractor symptoms), management response to concussion (three times) and recovery/return to play (four items) (see online supplementary file S1). The internal consistency of the knowledge measure in this sample was ∝=0.71. To identify participants not paying attention to the questionnaire, a validity item asked participants to respond to the following statement: ‘Ice hockey is a game typically played with a puck.’ This item was not scored but was used to help identify participants for removal from the analysis. Parent and coach versions contained the same questions assessing concussion knowledge. Specific HAPA constructs (eg, risk perception, outcome expectancies, action self-efficacy, intention, action planning, maintenance self-efficacy, recovery self-efficacy) were calculated using 2–8 items, all measured on a seven-point Likert Scale (ie, 1-strongly disagree—7-strongly agree, 1-not at all likely—7-extremely likely or 1-not at all bad—7-extremely bad). The coach sections contained items related to removing the athlete from play, and the parent sections contained items related to taking a child with a suspected concussion to a physician for assessment and clearance. The internal consistency of the constructs in this sample ranged from ∝=0.60 to ∝=0.98. The last section of the CKBBQ asked the participant how often they performed the targeted behaviours over the last season. A dichotomous yes/no question assessed the possibility that the behaviour could have occurred last season. The ratio of participants who did the behaviour to the number of possible opportunities was calculated. The primary behaviours are related to a coach removing an athlete from play or to a parent seeking medical attention for his or her child’s concussion for assessment and clearance.

Analysis

Data were analysed using STATA V.14. The significance level was set a priori at 0.05 but adjusted for multiple comparisons based on the number of independent tests for parents (p=0.05/17=0.002) and for coaches. Participant characteristics were compared descriptively (frequencies and proportion) between parents and coaches who were exposed to concussion education and those that had not received concussion education. The frequency and proportion of participants reporting specific sources of concussion education, as well as their rating of perceived helpfulness, are reported. To examine the characteristics of the coaches who completed the section of the questionnaire on coach beliefs and behaviours, we report a separate table describing respondent characteristics.

The proportion of individuals with correct responses is reported for all knowledge items, stratified by exposure to education. The association between previous education and total coach and parent knowledge was examined using multivariable linear regression adjusting for cluster by team, coaching experience, first aid experience and a history of managing a child’s concussion. Previous exposure to managing a child’s concussion was dichotomised due to the low number of parents who managed more than one concussion. Statistical modelling was performed using backward elimination examining pairwise interactions between concussion education and each covariate (ie, coaching experience x concussion education, first aid experience x concussion education, completion of safety programme x concussion education and a history of managing a child’s concussion x concussion education). Due to incomplete data, age was not included as a covariate in the full model. The assumptions of the final multivariable linear regression model were assessed graphically using Normal Q-Q plots of the residuals and examining the distribution of residuals against the fitted values and against covariates.

Medians and IQRs for all constructs in the HAPA are presented by exposure to education. Because of concerns regarding the distribution of the HAPA constructs relative to exposure, a non-parametric Wilcoxon rank-sum test was used to examine the association between education and the HAPA constructs. The association between parent and coach behaviour over the last season and education was assessed using X2 tests.

Results

Of the 1847 potential players from the cohort studies (n=130 teams), 809 parents and 11 non-parent coaches completed the CKBBQ. One non-parent coach and 17 parents did not answer the validity question correctly and were excluded. Three forms were completed incorrectly (both parents replied on same form) and were excluded. Three parents were excluded because they did not complete the question about previous concussion education. The final sample included 786 parents, 250 (31.8%) with coaching experience and 10 non-parent coaches (1.3%). In total, 649 participants (421/536 [78.5%] non-coach parents) had received concussion education; of those, 195 (30.1%) participants obtained the education during the current season. A total of 134 coaches and parent coaches completed the questions on coach beliefs and behaviour. Because not all participants answered all of the questions, the denominators for each outcome are listed. Table 1 describes the participant characteristics stratified by whether or not they received previous education. Most coaches were also parents and more mothers than fathers completed the questionnaires. Participants ranged from 21 to 61 years of age. All six non-parent assistant coaches and three of the head coaches were under the age of 29. Overall, the majority of participants (73.4%) had been hockey parents for greater than 7 years and a higher percentage of those with greater than 7 years of experience with being a hockey parent had received concussion education.

Table 1.

Characteristics of all study participants

No concussion education (n=147), % Received concussion education (n=649), %
Participant type
 Mother 90 (61.2%) 353 (54.4%)
 Father 52 (35.4%) 279 (43.0%)
 Non-parent head coach 0 (0.0%) 4 (0.6%)
 Non-parent assistant coach 1 (0.7%) 5 (0.8%)
 Missing 4 (2.7%) 8 (1.2%)
Coaching experience
 No coaching experience 115 (78.2%) 421 (64.9%)
 Coaching experience 32 (21.8%) 228 (35.1%)
Sex of child in study
 Male 136 (92.5%) 621 (95.7%)
 Female 11 (7.5%) 28 (4.3%)
Hockey playing experience
 No experience 99 (67.3%) 356 (54.9%)
 Currently plays hockey 21 (14.3%) 128 (19.7%)
 Used to play hockey 27 (18.4%) 164 (25.3%)
 Missing 0 (0.0%) 1 (0.2%)
City
 Calgary 85 (57.8%) 436 (67.2%)
 Edmonton 58 (39.5%) 209 (32.2%)
 Vancouver 4 (2.7%) 4 (0.6%)
Age, mean (SD) 45.22 (5.64) 45.81 (5.39)
 Missing 23 (15.6%) 107 (16.5%)
Age group of either parent’s child or coach’s team
 Pee Wee (ages 11–12) 2 (1.4%) 36 (5.5%)
 Bantam (ages 13–14) 113 (76.9%) 457 (70.4%)
 Midget (ages 15–17) 32 (21.8%) 156 (24.0%)
Participant's highest level of education
 Some grade school 2 (1.4%) 9 (1.4%)
 High school 16 (10.9%) 76 (11.7%)
 Bachelor’s degree 40 (31.0%) 213 (36.0%)
 Graduate degree 18 (12.2%) 96 (14.8%)
 College 53 (41.1%) 197 (33.3%)
 Other 9 (6.1%) 28 (4.3%)
 Missing 9 (6.1%) 30 (4.6%)
Years as a hockey parent
 First season 1 (0.7%) 2 (0.3%)
 1 year 0 (0.0%) 1 (0.2%)
 2 years 0 (0.0%) 3 (0.5%)
 3 years 5 (3.4%) 2 (0.3)
 4 years 5 (3.4%) 17 (2.6%)
 5 years 6 (4.1%) 26 (4.0%)
 6 years 11 (7.5%) 35 (5.4%)
 7 years 24 (16.3%) 58 (8.9%)
 >7 years 92 (62.6%) 492 (75.8%)
 Missing/coach 3 (2.0%) 13 (2.0%)
No of children in hockey
 1 86 (58.5%) 339 (52.2%)
 2 46 (31.3%) 235 (36.2%)
 3 11 (7.5%) 48 (7.4%)
 4 0 (0.0%) 6 (0.9%)
 Missing/coach 4 (2.7%) 21 (3.2%)
Total no of previous concussions among children
 0 115 (78.2%) 336 (51.8%)
 1 18 (12.2%) 157 (24.2%)
 2 7 (4.8%) 88 (13.6%)
 3 3 (2.0%) 32 (4.9%)
 4 1 (0.7%) 14 (2.2%)
 5 1 (0.7%) 7 (1.1%)
 6 0 (0.0%) 1 (0.2%)
 >6 0 (0.0%) 2 (0.3%)
 Missing/coach 2 (1.4%) 12 (1.8%)
Medical or first aid certification
 No 105 (71.4%) 376 (57.9%)
 Yes 41 (27.9%) 263 (40.5%)
 Missing 1 (0.7%) 10 (1.5%)
Completed the hockey Canada safety programme
 No 91 (61.9%) 301 (46.4%)
 Yes 30 (20.4%) 237 (36.5%)
 I don't know 17 (11.6%) 87 (13.4%)
 Missing 9 (6.1%) 24 (3.7%)

Sources of concussion education

Table 2 describes where hockey parents and coaches obtain education about concussion and how helpful they found the education provided. The top sources of concussion education reported by parents and coaches included the internet (63.8%), healthcare professionals (67.1%) and the hockey association (56.8%). Other sources described by participants included the associations of different sports, personal experience with concussion, the concussion study in which their child was participating, professional associations, radio, the university, smart phone applications (ie, apps) and work. The most helpful resources included healthcare professionals, workshops/coaching clinics and research articles. Parents and coaches who had received concussion education in the last year reported a median of 4 sources, ranging between 1 and 10 sources on the list. Out of 727 respondents, 583 (80.1%) reported an interest in being educated about concussion. Parents and coaches preferred sources for education were healthcare professionals, medical doctors and the hockey association (online supplementary file S1 includes full description).

Table 2.

The source and helpfulness of concussion education obtained by parents and coaches of youth ice hockey player

Source of education Do you get education from the following places? Degree of helpfulness
n No Yes Not helpful A little Somewhat Very Extremely helpful Missing
Hockey association 639 276 (43.2%) 363 (56.8%) 2 (0.6%) 31 (8.5%) 145 (39.9%) 134 (36.9%) 44 (12.1%) 7 (1.9%)
Workshops/coaching clinics 633 446 (70.5%) 187 (29.5%) 1 (0.5%) 14 (7.5%) 61 (32.6%) 72 (38.5%) 34 (18.2%) 5 (2.7%)
Research articles 635 331 (52.1%) 304 (47.9%) 0 (0.0%) 15 (4.9%) 105 (34.5%) 138 (45.4%) 40 (13.2%) 6 (2.0%)
TV 633 394 (62.2%) 239 (37.8%) 3 (1.3%) 53 (22.2%) 125 (52.3%) 44 (18.4%) 12 (5.0%) 2 (0.8%)
Newspapers 636 439 (69.0%) 197 (31.0%) 4 (2.0%) 35 (17.8%) 104 (52.8%) 42 (21.3%) 10 (5.1%) 2 (1.0%)
Social (Facebook, Twitter) 630 509 (80.8%) 121 (19.2%) 4 (3.3%) 17 (14.0%) 55 (45.5%) 29 (24.0%) 10 (8.3%) 6 (5.0%)
Magazines 632 498 (78.8%) 134 (21.2%) 1 (0.7%) 17 (12.7%) 70 (52.2%) 35 (26.1%) 10 (7.5%) 1 (0.7%)
Internet 635 230 (36.2%) 405 (63.8%) 0 (0.0%) 23 (5.7%) 172 (42.5%) 154 (38.0%) 48 (11.9%) 8 (2.0%)
Healthcare professionals 639 210 (32.9%) 429 (67.1%) 0 (0.0%) 9 (2.1%) 65 (15.2%) 197 (45.9%) 149 (34.7%) 9 (2.1%)
Other parents 636 368 (57.9%) 268 (42.1%) 4 (1.5%) 45 (16.8%) 127 (47.4%) 67 (25.0%) 20 (7.5%) 5 (1.9%)
Other 618 539 (87.2%) 79 (12.8%) 0 (0.0%) 2 (2.5%) 16 (20.3%) 24 (30.4%) 30 (38.0%) 7 (8.9%)

Parent and coach concussion knowledge

Table 3 summarises the proportion of parents and coaches who correctly responded to knowledge questions, stratified by whether or not they had received concussion education. The mean knowledge score for all participants was 25.85/32. Overall, participants scored lowest on items related to management and return to play, as well as the distinction between concussion symptoms and specific distractors (eg, fever, panic attacks, weight gain, reduced breathing rate). Compared with parents and coaches who did not receive concussion education, a larger proportion of parents and coaches who did receive education were able to correctly identify distractor symptoms including arthritis (85.7% vs 78.9%) and excessive studying (82.0% vs 69.4%). A higher proportion of participants who received concussion education also had heard of the terms ‘graduated return to play protocol’ or ‘stepwise return to play’ (58.6% vs 27.2%), recognised an athlete should not return to play while experiencing symptoms (84.0% vs 72.6%), and identified that postconcussion symptoms can be delayed for hours or days (90.3% vs 81.3%). The average knowledge score for participants with coaching experience was 26.45/32 whereas the average knowledge score for participants without coaching experience was 25.57/32. Based on the multivariable regression analysis adjusting for coaching experience, previous history of a child sustaining one or more concussions, first aid experience and cluster by team, exposure to concussion education was associated with a mean score knowledge that was 1.36 points higher (95% CI 0.68 to 2.03), p<0.0001. Coaching experience (coefficient=0.65 [95% CI 0.17 to 1.13], p=0.008) and first aid experience (coefficient=0.69 [95% CI 0.26 to 1.12], p=0.002) were also associated with a significantly higher mean knowledge score.

Table 3.

Parent and coach knowledge by exposure to previous education

No concussion education Received concussion education Total sample
n n (% correct) n n (% correct) n n (% correct)
Definition
 A concussion is a brain injury (T)  145 142 (97.9)  640 622 (97.2)  785 764 (97.3)
Understanding the risk of concussion
 There is a higher risk of long term problems if someone has a second concussion before recovering from the first one (T)  147 139 (94.6)  648 625 (96.5)  795 764 (96.1)
 People who have had one concussion are more likely to have another concussion (T)  146 124 (84.9)  643 522 (81.2)  789 646 (81.9)
 There are few risks to long-term health and well-being from multiple concussions (F)  147 116 (78.9)  646 537 (83.1)  793 653 (82.3)
Misconceptions
 Sometimes a second concussion can help a person remember things that were forgotten after the first concussion (F)  147 123 (83.7)  647 564 (87.2)  794 687 (86.5)
 There are helmets that prevent all concussions (F)  147 139 (94.6)  649 639 (98.5)  796 778 (97.7)
Recognition
 Concussions can sometimes lead to emotional problems (T)  147 124 (84.4)  648 596 (92.0)  795 720 (90.6)
 In order to be diagnosed with a concussion, you have to be knocked out (F)  147 144 (98.0)  648 645 (99.5)  795 789 (99.2)
 A concussion can only occur if there is a direct hit to the head (F)  147 124 (84.4)  648 578 (89.2)  795 702 (88.3)
Signs and symptoms
 Hives (N)  146 119 (81.5)  645 582 (90.2)  791 701 (88.6)
 Headache (Y)  147 146 (99.3)  646 642 (99.4)  793 788 (99.4)
 Fever (N)  147 75 (51.0)  641 334 (52.1)  788 409 (51.9)
 Arthritis (N)  147 116 (78.9)  645 553 (85.7)  792 669 (84.5)
 Sensitivity to light (Y)  147 142 (96.6)  646 642 (99.4)  793 784 (98.9)
 Difficulty remembering (Y)  146 144 (98.6)  646 644 (99.7)  792 788 (99.5)
 Panic attacks (N)  147 33 (22.4)  643 149 (23.2)  790 182 (23.0)
 Feeling tired (Y)  147 138 (93.9)  643 619 (96.3)  790 757 (95.8)
 Feeling in a fog (Y)  147 139 (94.6)  643 638 (99.2)  790 777 (98.4)
 Weight gain (N)  147 95 (64.6)  643 455 (70.8)  790 550 (69.6)
 Feeling slowed down (Y)  147 134 (91.2)  643 595 (92.5)  790 729 (92.3)
 Reduced breathing rate (N)  147 47 (32.0)  641 218 (34.0)  788 265 (33.6)
 Excessive studying (N)  147 102 (69.4)  645 529 (82.0)  792 631 (79.7)
 Difficulty concentrating (Y)  147 144 (98.0)  645 641 (99.4)  792 785 (99.1)
 Dizziness (Y)  147 144 (98.0)  646 646 (100.0)  793 790 (99.6)
 Hair loss (N)  147 115 (78.2)  644 547 (84.9)  791 662 (83.7)
Response
 After a collision, body check, or fall your child/player starts vomiting repeatedly (ED)  146 135 (92.5)  647 627 (96.9)  793 762 (96.1)
 After a collision, body check, or fall your child/player has a headache that continues to get worse (ED)  146 76 (52.1)  645 361 (56.0)  791 437 (55.2)
 Immediately after a collision, body check, or fall your child/player immediately complains of a headache and dizziness but starts to feel better by the end of the game (NU)  145 104 (71.7)  646 500 (77.4)  791 604 (76.4)
Management and return to play
 Younger players (under the age of 18) typically take longer to recover from a concussion than adults (T)  147 29 (19.7)  646 132 (20.4)  793 161 (20.3)
 Post concussion symptoms can be delayed for hours or days (T)  147 120 (81.6)  648 585 (90.3)  795 705 (88.7)
 An athlete can return to play while experiencing symptoms of a concussion if directed to do so by an athletic trainer or a physician (F)  146 106 (72.6)  648 544 (84.0)  794 650 (81.9)
 Have you heard of the term ‘graduated return to play protocol’ or ‘stepwise return to play’ for concussion? (Y)  147 40 (27.2)  645 378 (58.6)  792 418 (52.8)
Total score, mean (SD)  139 24.57 (3.89)  602 26.15 (2.95)  741 25.85 (3.20)

Items that are in bold had greater than a 5% difference in the point estimate proportion of correct responses between those who had received concussion education and those that had not received concussion education.

DN, do nothing; ED, go to the emergency department; F, false; N, no; NU, non-urgent, book an appointment with a physician; T, true; Y, yes.

Parent and coach beliefs under the scope of the health action process approach

Only 14/134 coaches who responded to the belief and behaviour sections of the questionnaire reported no exposure to concussion education. The majority of these participants (12/14) were assistant coaches. Table 4 summarises the characteristics of this sample. Table 5 presents summary statistics for the HAPA constructs, stratified by exposure to previous education. The HAPA constructs were not normally distributed, and the number of participants in each group was limited. Therefore, groups were compared using a Wilcoxon rank sum test. No significant differences were found between those exposed to concussion education in the last year versus those who were not exposed (table 5).

Table 4.

Characteristics of coaches who completed the questionnaire section on coach beliefs

No concussion education
(n=14)
Received concussion education
(n=120)
Coach type
 Head coach 2 (14.3%) 36 (30.0%)
 Assistant coach 12 (85.7%) 79 (65.8%)
 Missing 0 (0.0%) 5 (4.2%)
Age, mean (SD) 44.30 (8.59) 45.38 (7.87)
 Missing 4 (28.6%) 16 (13.3%)
Coaching age group*
 Tyke 3 (21.4%) 20 (16.7%)
 Timbit 7 (50.0%) 70 (58.3%)
 Novice 11 (78.6%) 93 (77.5%)
 Atom 11 (78.6%) 99 (82.5%)
 Pee wee 12 (85.7%) 102 (85.0%)
 Bantam 12 (85.7%) 100 (83.3%)
 Midget 3 (21.4%) 43 (35.8%)
 Junior 0 (0.0%) 6 (5.0%)
 Varsity 0 (0.0%) 2 (1.7%)
Years of coaching experience
 0 to <5 years 3 (21.4%) 22 (18.3%)
 5 to <10 years 6 (42.9%) 53 (44.2%)
 ≥10 years 4 (28.6%) 38 (31.7%)
 Missing 1 (7.1%) 7 (5.8%)
Access to medical professionals on the bench*
 No access 7 (50.0%) 65 (54.2%)
 Athletic therapist 1 (7.1%) 14 (11.7%)
 Physiotherapist 1 (7.1%) 1 (0.8%)
 Physician 0 (0.0%) 7 (5.8%)
 Paramedic 2 (14.3%) 11 (9.2%)
 Other† 3 (21.4%) 23 (19.2%)

*Participants could select more than one response.

†Other participants listed included: chiropractor, fireman, first aid person, nurse, respiratory therapist.

Table 5.

Summary statistics of the HAPA constructs related to parent and coach concussion behaviour management

Outcome No concussion education Received concussion education Wilcoxon rank-sum test
No of items Internal consistency n Median (IQR) n Median (IQR) P value
HAPA constructs related to parents taking their child in for physician assessment and clearance
 Risk perception 6 0.68 143 5.50 (3.33–7) 629 5.67 (3.17–7) 0.5542
 Outcome expectancies 8 0.68 140 6.19 (4.00–7) 624 6.13 (2.50–7) 0.4756
 Action self-efficacy 2 0.6 144 6.25 (3.50–7) 637 6.50 (1.00–7) 0.1966
 Intention 2 0.91 146 6.50 (4.00–7) 637 7.00 (1.00–7) 0.9979
 Maintenance self-efficacy 3 0.95 145 7.00 (4.00–7) 638 6.67 (1.00–7) 0.8311
 Recovery self-efficacy 2 0.84 143 7.00 (2.00–7) 613 7.00 (1.00–7) 0.5733
 Planning 2 0.79 138 5.50 (1.00–7) 628 6.00 (1.00–7) 0.1135
HAPA constructs related to coaches removing a player from a game or practice due to a suspected concussion
 Risk perception 7 0.75 14 5.57 (5.29–5.86) 118 5.71 (4.29–6.71) 0.5908
 Outcome expectancies 6 0.6 14 6.08 (5.50–6.67) 117 6.00 (4.00–7) 0.5085
 Action self-efficacy 3 0.8 14 6.00 (5.33–6.33) 120 6.33 (4.00–7) 0.2541
 Intention 2 0.98 14 6.00 (6.00–7) 118 7.00 (4.00–7) 0.3432
 Maintenance self-efficacy 3 0.92 14 6.67 (6.00–7) 120 7.00 (4.00–7) 0.4192
 Recovery self-efficacy 2 0.78 13 7.00 (6.00–7) 120 6.50 (4.50–7) 0.4783
 Planning 2 0.87 13 4.00 (1.00–7) 118 4.00 (1.00–7) 0.6556

HAPA, Health Action Process Approach.

Self-reported parent and coach behaviour

Of 786 parents, 93 reported having a child who sustained at least one suspected concussion over the last hockey season (88 participants reported one suspected concussion and 5 reported two suspected concussions). Of the parents who reported a child with a suspected concussion, 75/93 (80.6%) reported seeing a physician for diagnosis and 71/93 (76.3%) reported seeing a physician for clearance to return to play. A higher proportion of individuals who reported receiving concussion education saw a physician for assessment (73/85 [85.9%]) and clearance (69/85 [81.2%]) when compared with those who reported not having received concussion education (2/8, 25% for both assessment and clearance). A higher proportion of parents who received concussion education reported that they educated their child about concussion (422/610, 69.2%) than those who had not received education (48/139, 34.5%). Participants who believed they had the resources to educate their child about concussion (458/732, 62.7%) indicated they would educate their child more often than those who felt they did not have sufficient resources (340/458 vs 118/274, X2=80.23, p<0.0001). Of the 132 coaches who completed the section on coach behaviour, 71 (53.8%) reported having suspected a concussion in the last season (65/71 also reported being exposed to concussion education). Only 6/70 (8.6%, 1 missing response) coaches reported not removing an athlete when a concussion was suspected and all reported receiving concussion education. However, 12/68 (17.6%) coaches reported that at least one player continued to play for a short time following a potential concussion mechanism (eg, being struck in the head) before being removed (11/12 reported being exposed to concussion education).

Discussion

This is the first published study to use the HAPA to examine the association between concussion education and knowledge and beliefs related to concussion management. The finding that previous education is associated with small differences in knowledge is consistent with previous studies in parents and coaches.14–18 32 For example, Hecimovicha et al19 reported that exposure to previous concussion training was associated with Australian Rules Football parent knowledge of signs and symptoms of concussion, management and return to play. Kroshus et al14 reported similar associations between coach knowledge and previous concussion education in a sample of collegiate sport coaches. Overall, gaps in youth parents’ and coaches’ knowledge of concussion signs and symptoms appear to be improving. For example, approximately 90% of participants in this sample correctly reported that emotional symptoms can occur after concussion. Conversely, a previous study in youth football parents in 2011 reported only 50% of participants correctly answered that emotional symptoms could be related to concussion.32 Given that answers to this question in this study were not found to be different between those with and without education, the increased awareness of emotional symptoms being associated with concussion may be the result of stories in the media highlighting the association between concussions and depression.33 Parents and coaches primarily completed the questionnaire at home. We had no control on whether parents may or may not have sought outside resources to complete the questionnaire, thus potentially increasing knowledge scores in both groups; however, they were instructed to use their own knowledge. Regardless, there are still gaps in parent and coach knowledge that can be better addressed with improved concussion education programmes. For example, the participants in this study scored poorly on knowledge items related to response and management. This may have important implications given that this study also identified that 19% of parents still choose not to see a physician regarding management and these participants may therefore not be aware of the graduated return to play protocol.

The finding that exposure to concussion education was not associated with differences in the HAPA constructs is expected given the majority of concussion education programmes do not target these constructs. Notably, the HAPA construct related to planning had the lowest median scores for parents (median: 5.5–6.0) and coaches (median: 4.0) irrespective of whether they were provided concussion education or not. According to HAPA, action planning is a post-intentional factor that helps bridge the intention-to-behaviour gap. Concussion education that includes a specific concussion protocol provided to the coaches, parents and athletes (eg, at a team meeting) could be developed to improve parents’ and coaches’ action and coping planning. When the protocol is presented to parents, the importance of seeing a physician for assessment and clearance could be discussed. Resources could include information about nearby clinics and physicians trained in concussion management where parents could take their children if they suspected a concussion. Practice in scenarios involving difficult removal from play decisions for coaches, and testimonials of successful concussion management experiences, could increase both action self-efficacy and outcome expectancies.

Two previous studies examined the effect of concussion education using a behavioural change framework in parents16 and coaches.15 Both sets of authors reported significant differences in behavioural intention and self-efficacy.15 16 However, the educational intervention used in these previous studies was designed and informed by a behavioural change theory (health belief model) whereas the education in the present study could have taken any form. This is a key difference between the studies that may provide some evidence regarding the potential benefits of incorporating behavioural change theory in both the design of the educational programme and the evaluation of its effects.

In this study, approximately 15%–20% of parents did not consult a physician for assessment and clearance to return to play following all of their child’s suspected concussions and 17.6% of coaches reported at least one player continuing to play with a suspected concussion before removal. The percentage of coaches is similar to the findings by Bramley et al34 who reported that when provided with scenarios describing concussions, 80% of coaches or more would never allow a player back into a game. They found that the severity of the concussion described in the scenario affected the percentage of coaches that would never allow a player to return.34 Due to the low number of participants who suspected a concussion in either their child or their players in the last year, we could not examine how concussion education affected behaviours directly.

Limitations

Because this is the first published study to try to measure HAPA constructs to understand concussion management behaviours, no previous validated questionnaires were available. While the questionnaires were validated using experts in concussion, pilot testing and qualitative interviews, a risk of measurement bias remains. Future steps in questionnaire development, such as structural equation modelling, may be needed to improve the internal reliability of the HAPA constructs. Furthermore, some participants rated the highest value for all items on the maintenance self-efficacy, intention, action self-efficacy and outcome expectancy scales. While this may indeed represent higher ratings for these beliefs, it may also represent a lack of exposure or recognition of the experience outlined in the construct. The limited sample of coaches without concussion education is a positive finding in this paper that suggests most associations are providing education but, it also may have resulted in an inflated type II error when examining differences. Future research using a larger sample of coaches is needed to address this limitation.

This study did not aim to adjust for type, description, or timing of concussion education received because quantifying the exposure as ‘educated’ or ‘not’ reflects the nature of blanket educational policies that allow individual associations to select their education format. However, prospective research is needed to better control both the short-term and long-term effects of education using different behavioural change approaches. This is especially important given that the type of education can affect behaviour outcomes.

Conclusions

The results from this study suggest that most youth coaches are provided with concussion education and that exposure to education may be associated with small differences in parent and coach knowledge, but is not associated with parent or coach beliefs. While most parents and coaches report appropriate behaviour related to concussion management, exposure to concussion education does not guarantee that the appropriate action will be taken. Parents who reported exposure to concussion education also reported being more likely to try and educate their children about concussion than those who do not receive concussion education. Further research is needed to examine the accuracy of the education provided by parents to youth and how different ways of providing it may affect player beliefs. It is also possible that concussion policies that do not guide the type, quality or content of education recommended for parents and coaches may not have the intended effects. Parents and coaches identified healthcare professionals as the most common and preferred source of concussion education. However, this research suggests that the education currently being provided may not influence constructs that are related to concussion management behaviours including immediate removal from play (coaches) or physician assessment and clearance (parents). Consideration of behavioural change techniques when providing concussion education, rather than focusing on knowledge alone warrants further consideration in follow-up studies.

Supplementary Material

Reviewer comments
Author's manuscript

Acknowledgments

The Sport Injury Prevention Research Centre is one of the International Research Centres for Prevention of Injury and Protection of Athlete Health supported by the International Olympic Committee. We acknowledge funding from the Canadian Institutes of Health Research, Alberta Innovates Health Solutions, Hotchkiss Brain Institute and Alberta Children’s Hospital Foundation. We acknowledge Hockey Canada, Hockey Calgary, Hockey Alberta, BC Hockey and all the coaches and parents involved for their time and support in completing this research project. The authors thank all of the research coordinators and research assistants who assisted with this project.

Footnotes

Twitter: @aacademic

Contributors: AMB designed the study and led the data collection, data entry, data cleaning, data analysis, interpretation of study results and drafted the manuscript. KOY, SB, AN-A and CAE contributed to design, data analysis and interpretation of results. All authors critically reviewed and edited the manuscript before submission.

Funding: This study was funded by the Canadian Institutes of Health Research (CIHR), Hotchkiss Brain Institute, Alberta Children’s Hospital Foundation, and Alberta Innovates Health Solutions. AMB was supported by the Hotchkiss Brain Institute Dr T Chen Fong Doctoral Scholarship in Neuroscience. CAE is supported by a Canada Research Chair (Tier 1) Concussion. KOY is supported by the Ronald and Irene Ward Chair in Pediatric Brain Injury from the Alberta Children’s Hospital Foundation. The University of Calgary Sport Injury Prevention Research Centre is one of the International Research Centres for Prevention of Injury and Protection of Athlete Health supported by the International Olympic Committee. The role of the sponsors and funding bodies was to provide financial support for this research and the translation of the findings to all community partners.

Patient and public involvement: Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.

Patient consent for publication: Not required.

Ethics approval: The research ethics boards at the University of Calgary (Ethics ID: REB14-0348 and Ethics ID: 14–2209), University of Alberta (Ethics ID: REB Pro00024093_AME3) and the University of British Columbia (Ethics ID: CW14-0304/H14-01894) approved the study.

Provenance and peer review: Not commissioned; externally peer reviewed.

Data availability statement: Data are available on reasonable request. Deidentified participant data are held by the Sport Injury Prevention Research Centre, Faculty of Kinesiology, University of Calgary.

References

  • 1.McCrory P, Meeuwisse WH, Dvořák J, et al. 5th international conference on concussion in sport (Berlin). Br J Sports Med 2017;51:837. 10.1136/bjsports-2017-097878 [DOI] [PubMed] [Google Scholar]
  • 2.Barlow KM, Crawford S, Stevenson A, et al. Epidemiology of postconcussion syndrome in pediatric mild traumatic brain injury. Pediatrics 2010;126:e374–81. 10.1542/peds.2009-0925 [DOI] [PubMed] [Google Scholar]
  • 3.Kerr ZY, Zuckerman SL, Wasserman EB, et al. Concussion symptoms and return to play time in youth, high school, and college American football athletes. JAMA Pediatr 2016;170:647–53. 10.1001/jamapediatrics.2016.0073 [DOI] [PubMed] [Google Scholar]
  • 4.Wetjen NM, Pichelmann MA, Atkinson JLD. Second impact syndrome: concussion and second injury brain complications. J Am Coll Surg 2010;211:553–7. 10.1016/j.jamcollsurg.2010.05.020 [DOI] [PubMed] [Google Scholar]
  • 5.Elbin RJ, Sufrinko A, Schatz P, et al. Removal from play after concussion and recovery time. Pediatrics 2016;138:e20160910. 10.1542/peds.2016-0910 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Asken BM, McCrea MA, Clugston JR, et al. "Playing through it": delayed reporting and removal from athletic activity after concussion predicts prolonged recovery. J Athl Train 2016;51:329–35. 10.4085/1062-6050-51.5.02 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.McCrory P, Meeuwisse W, Aubry M, et al. Consensus statement on concussion in sport: the 4th international conference on concussion in sport held in Zurich, November 2012. Br J Sports Med 2013;23:89–117. [DOI] [PubMed] [Google Scholar]
  • 8.Concannon LG. Effects of legislation on sports-related concussion. Phys Med Rehabil Clin N Am 2016;27:513–27. 10.1016/j.pmr.2015.12.006 [DOI] [PubMed] [Google Scholar]
  • 9.Caron JG, Bloom GA, Falcão WR, et al. An examination of concussion education programmes: a scoping review methodology. Inj Prev 2015;21:301–8. 10.1136/injuryprev-2014-041479 [DOI] [PubMed] [Google Scholar]
  • 10.Kerr ZY, Register-Mihalik JK, Marshall SW, et al. Disclosure and non-disclosure of concussion and concussion symptoms in athletes: review and application of the socio-ecological framework. Brain Inj 2014;28:1009–21. 10.3109/02699052.2014.904049 [DOI] [PubMed] [Google Scholar]
  • 11.Rivara FP, Schiff MA, Chrisman SP, et al. The effect of coach education on reporting of concussions among high school athletes after passage of a concussion law. Am J Sports Med 2014;42:1197–203. 10.1177/0363546514521774 [DOI] [PubMed] [Google Scholar]
  • 12.Glanz K, Bishop DB. The role of behavioral science theory in development and implementation of public health interventions. Annu Rev Public Health 2010;31:399–418. 10.1146/annurev.publhealth.012809.103604 [DOI] [PubMed] [Google Scholar]
  • 13.Murgraff V, McDermott MR, Walsh J, Self-Efficacy WJ. Self-efficacy and behavioral enactment: the application of Schwarzer's health action process approach to the prediction of low-risk, single-occasion drinking. J Appl Soc Psychol 2003;33:339–61. 10.1111/j.1559-1816.2003.tb01900.x [DOI] [Google Scholar]
  • 14.Kroshus E, Baugh CM, Daneshvar DH, Content DDH. Content, delivery, and effectiveness of concussion education for us college coaches. Clin J Sport Med 2016;26:391–7. 10.1097/JSM.0000000000000272 [DOI] [PubMed] [Google Scholar]
  • 15.Glang A, Koester MC, Beaver SV, et al. Online training in sports concussion for youth sports coaches. Int J Sports Sci Coach 2010;5:1–11. 10.1260/1747-9541.5.1.1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Glang AE, Koester MC, Chesnutt JC, et al. The effectiveness of a web-based resource in improving postconcussion management in high schools. J Adolesc Health 2015;56:91–7. 10.1016/j.jadohealth.2014.08.011 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Parker EM, Gilchrist J, Schuster D, et al. Reach and knowledge change among coaches and other participants of the online course: "concussion in sports: what you need to know". J Head Trauma Rehabil 2015;30:198–206. 10.1097/HTR.0000000000000097 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Valovich McLeod TC, Schwartz C, Bay RC. Sport-related concussion misunderstandings among youth coaches. Clin J Sport Med 2007;17:140–2. 10.1097/JSM.0b013e31803212ae [DOI] [PubMed] [Google Scholar]
  • 19.Hecimovicha M, Kingb D, Maraisc I. Player and parent concussion knowledge and awareness in youth Australian rules football. Sport J 2016:1–19 http://thesportjournal.org/article/player-and-parent-concussion-knowledge-and-awareness-in-youth-australian-rules-football/ [Google Scholar]
  • 20.Schwarzer R. Self-efficacy in the adoption and maintenance of health behaviors: theoretical approaches and a new model : Schwarzer R, Self-Efficacy: thought control of action. New York: Taylor & Francis, 1992: 217–44. [Google Scholar]
  • 21.Schwarzer R. Modeling health behavior change: how to predict and modify the adoption and maintenance of health behaviors. Appl Psychol 2008;57:1–29. 10.1111/j.1464-0597.2007.00325.x [DOI] [Google Scholar]
  • 22.Schwarzer R. Some burning issues in research on health behavior change. Appl Psychol 2008;57:84–93. 10.1111/j.1464-0597.2007.00324.x [DOI] [Google Scholar]
  • 23.Pfister T, Pfister K, Hagel B, et al. The incidence of concussion in youth sports: a systematic review and meta-analysis. Br J Sports Med 2016;50:292–7. 10.1136/bjsports-2015-094978 [DOI] [PubMed] [Google Scholar]
  • 24.Canada H. Annual report 2015-2016, 2016. Available: www.hockeycanada.ca
  • 25.Renner B, Schwarzer R. Risk and health behaviours: documentation of the scales of the research project: risk appraisal consequences in Korea (RACK), 2005. Available: http://userpage.fu-berlin.de/gesund/RACK-English.pdf
  • 26.Schwarzer R, Luszczynska A. How to overcome health-compromising behaviors. Eur Psychol 2008;13:141–51. 10.1027/1016-9040.13.2.141 [DOI] [Google Scholar]
  • 27.Rosenbaum AM, Arnett PA. The development of a survey to examine knowledge about and attitudes toward concussion in high-school students. J Clin Exp Neuropsychol 2010;32:44–55. 10.1080/13803390902806535 [DOI] [PubMed] [Google Scholar]
  • 28.White PE, Newton JD, Makdissi M, et al. Knowledge about sports-related concussion: is the message getting through to coaches and trainers? Br J Sports Med 2014;48:119–24. 10.1136/bjsports-2013-092785 [DOI] [PubMed] [Google Scholar]
  • 29.Krol AL, Mrazik M, Naidu D, et al. Assessment of symptoms in a concussion management programme: method influences outcome. Brain Inj 2011;25:1300–5. 10.3109/02699052.2011.624571 [DOI] [PubMed] [Google Scholar]
  • 30.Guilmette TJ, Malia LA, McQuiggan MD. Concussion understanding and management among new England high school football coaches. Brain Inj 2007;21:1039–47. 10.1080/02699050701633080 [DOI] [PubMed] [Google Scholar]
  • 31.Echlin PS. Concussion education, identification, and treatment within a prospective study of physician-observed junior ice hockey concussions: social context of this scientific intervention. Neurosurg Focus 2010;29:E7. 10.3171/2010.10.FOCUS10222 [DOI] [PubMed] [Google Scholar]
  • 32.Mannings C, Kalynych C, Joseph MM, et al. Knowledge assessment of sports-related concussion among parents of children aged 5 years to 15 years enrolled in recreational tackle football. J Trauma Acute Care Surg 2014;77:S18–22. 10.1097/TA.0000000000000371 [DOI] [PubMed] [Google Scholar]
  • 33.Sullivan SJ, Schneiders AG, Cheang C-W, et al. "What's happening?" A content analysis of concussion-related traffic on Twitter. Br J Sports Med 2011;46:258–63. [DOI] [PubMed] [Google Scholar]
  • 34.Bramley H, Kroft C, Polk D, et al. Do youth hockey coaches allow players with a known concussion to participate in a game? Clin Pediatr 2012;51:283–7. 10.1177/0009922811422434 [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary data

bmjopen-2020-038166supp001.pdf (5.4MB, pdf)

Reviewer comments
Author's manuscript

Articles from BMJ Open are provided here courtesy of BMJ Publishing Group

RESOURCES