Abstract
Objectives
The primary objectives of this work were to (1) identify the risk and/or protective factors associated with playground injuries among children less than 18 years of age and (2) identify interventions/programs/policies aimed at preventing playground-related injuries among children less than 18 years of age. Secondary objectives include a summary of the data that reflect alignment with current playground standards.
Methods
For the primary outcome, 4 databases and 12 online resources were systematically searched to include observational and experimental studies examining risk and/or protective factors or interventions associated with playground injury or outcomes related to playground injury (e.g., risky playground behaviour). Data extraction included study design, participants, outcome, exposure or intervention, data analysis, and effect estimates. The quality of evidence for all studies was assessed using the Downs and Black criteria.
Results
This review included studies of moderate quality, revealing a number of risk factors and effective interventions for playground injury. Risk factors included absence of handrails and guardrails on playground equipment, non-impact-absorbing surfacing, and critical fall heights. Effective interventions included modifying playground surfacing and reducing equipment height to less than 1.5 m.
Conclusion
Equipment- and structure-based playgrounds should adhere to and maintain playground standards in order to reduce the risk of serious injury. Organizations responsible for installing and maintaining playgrounds should consider alternative play spaces that allow children to play outdoors, in a natural environment that supports healthy child development and promotes physical activity.
Keywords: Interventions, Playgrounds, Systematic review, Play and playthings, Wounds and injuries, Risk factors
Résumé
Objectifs
Les principaux objectifs de l’étude consistaient : 1) à dresser la liste des facteurs de risque et/ou de protection associés aux blessures sur les terrains de jeux chez les enfants de moins de 18 ans et 2) à dresser la liste des interventions, des programmes ou des politiques visant à prévenir les blessures sur les terrains de jeux chez les enfants de moins de 18 ans. Le résumé des données en phase avec les normes en vigueur pour les terrains de jeux était un objectif secondaire.
Méthode
Pour les objectifs principaux, nous avons systématiquement interrogé 4 bases de données et 12 ressources en ligne pour repérer les études observationnelles et expérimentales portant sur les facteurs de risque et/ou de protection associés aux blessures sur les terrains de jeux (p. ex., les comportements dangereux sur les terrains de jeux), sur les interventions ou sur les résultats des blessures sur les terrains de jeux. Nous avons extrait les données sur le protocole d’étude, les participants, le résultat, l’exposition ou l’intervention, l’analyse de données et les estimations des effets. La qualité des données probantes de chaque étude a été évaluée selon les critères de Downs et Black.
Résultats
Les études de cette revue étaient de qualité moyenne; elles ont mis au jour des facteurs de risque de blessures sur les terrains de jeux et des interventions efficaces. Les facteurs de risque étaient l’absence de mains courantes et de garde-corps sur les équipements de jeu, les surfaces n’absorbant pas les impacts, ainsi que les hauteurs de chute critiques. Les interventions efficaces consistaient à modifier la surface des terrains de jeux et de réduire la hauteur des équipements à moins de 1,5 m.
Conclusion
Les terrains de jeux qui comportent des équipements et des structures devraient être conformes aux normes pour les terrains de jeux afin de réduire les risques de blessures graves. Les organismes chargés de l’installation et de l’entretien des terrains de jeux devraient envisager des aires de jeu différentes qui permettent aux enfants de jouer à l’extérieur, dans un milieu naturel propice à leur développement sain et à l’activité physique.
Mots-clés: Interventions, Terrains de jeux, Revue systématique, Jeu et accessoires de jeu, Plaies et blessures, Facteurs de risque
Introduction
Playgrounds are outdoor spaces that provide children the opportunity to explore, be creative and imaginative, and participate in forms of physical activity that create both social and health benefits. Despite these benefits, however, children playing in playgrounds and on play equipment may be at an increased risk for injury. Approximately 29,000 child injuries from playground equipment present to Canadian emergency departments each year (Fuselli and Yanchar 2012). These injuries most frequently occur among younger children, ages 5–9 years old (Norton et al. 2004; MacKay 2003), and are most often associated with children falling from heights (Vollman et al. 2009). Playground falls often result in upper extremity injuries, with the predominant injury type being an arm fracture (Fuselli and Yanchar 2012).
Since the 1980s, playgrounds, particularly in Canada and the USA, have transformed into equipment- and structure-based play spaces (Herrington and Nicholls 2007). In response to this transformation, as well as from a 1979 report documenting injuries in playgrounds in Canada (Herrington and Nicholls 2007), the Canadian Standards Association (CSA) as well as significant collaborative work from other organizations in Canada and the USA, including the American Public Health Association and the American Academy of Pediatrics, went into investigating the design of playgrounds and how design features might mitigate injury (American Academy of Pediatrics, American Public Health Association, National Resource Center for Health and Safety in Child Care and Early Education 2011). Since dissemination of this work in 1992, and the release of the CSA standards in 1998, there has not been an updated review of the research literature and its alignment with these guidelines. A synthesis of the existing data on the risk and protective factors associated with playground injury, and the effective interventions that can reduce this risk, can be used to support evidence-based decisions in playground design. This can include decision-making between equipment-based playground designs over nature-based designs. Either of these decisions must result in age-appropriate, hazard-free play spaces. The primary objectives of this work, therefore, were to (1) identify the risk and/or protective factors associated with playground injuries among children less than 18 years of age and (2) identify interventions/programs/policies aimed at preventing playground-related injuries among children less than 18 years of age. Secondary objectives include a summary of the data to reflect alignment with current playground standards.
Methods
Data sources and selection of eligible studies
To identify observational studies examining risk and/or protective factors for playground injuries, search terms and strategies were developed a priori to the literature search (Table 1). To identify experimental studies examining interventions aimed to reduce playground injuries, outcomes related to playground injuries (e.g., risky playground behaviour) or outcomes that examined changes in the risk and/or protective factors targeted by the intervention, search terms and strategies were developed specific to this objective (Table 1). The following inclusion/exclusion criteria were determined a priori for both observational and experimental studies: (1) original data, (2) literature published in English, (3) exposure/interventions and outcomes specific to the search objectives, and (4) an analytic component to the study design, by objective (i.e., the study examined the relationship between risk factor and outcomes, and the study examined the effectiveness of the intervention on injury or injury pathway outcomes).
Table 1.
Search terms and strategies
| Concept A (outcome) | Concept B (exposure) | Concept C (context) |
|---|---|---|
| Risk/protective factors | ||
| 1. Injur* (keyword) | 4. Risk factor | 6. Playground (keyword) |
| 2. Wounds and injuries | 5. Protective | |
| 3. Fall | ||
| Search strategy | ||
| A. 1 OR 2 OR 3 | ||
| B. 6 | ||
| C. 4 OR 5 | ||
| D. A AND B | ||
| E. A AND B AND C | ||
| Interventions | ||
| 1. Injur* (keyword) | 4. Intervention | 8. Playground (keyword) |
| 2. Wounds and injuries | 5. Program | |
| 3. Fall | 6. Policy | |
| 7. Prevent* (keyword) | ||
| Search strategy | ||
| A. 1 OR 2 OR 3 | ||
| B. 8 | ||
| C. 4 OR 5 OR 6 OR 7 | ||
| D. A AND B | ||
| E. A AND B AND C | ||
Between September 2016 and December 2016, one reviewer (SR) conducted a comprehensive search of the following 4 databases and 12 online resources: Medline (1946–present), CINAHL (1982–present), Physical Education Index (1978–present), and SportDiscus (1975–present), Parachute, Canadian Pediatric Society, Canada Safety Council, Canadian Playground Safety Institute, Canadian Playground Practitioners, Centers for Disease Control and Prevention (CDC), SafeKids Worldwide, National Safety Council, National Program for Playground Safety, Injury Free Coalition for Kids, Children’s Safety Network, and Google Scholar. Identification of studies to be included in this review involved a three-stage process. In the first stage, a title screen of all articles obtained from the literature search was completed. Due to the significant number of hits lost from setting limits in both CINAHL and the Physical Education Index, no limits were set for these two databases. To reduce publication bias, we included a search of dissertations and studies conducted but not published (e.g., abstracts) in our grey literature search. In the event that a study was found, authors would be contacted to retrieve unpublished data. Titles were screened for relevance, as per the established inclusion criteria. The second stage included a review of full texts, applying the same inclusion criteria. Articles that did not meet inclusion criteria were removed. Finally, relevant playground resources and reference lists of all included studies were reviewed for any missing articles. A second round of relevancy assessment was performed by two additional reviewers (TC and AM) with an open discussion between the three reviewers (SR, TC, AM) to form the final list of relevant articles.
Quality assessment
Articles included in this systematic review were critically appraised using the Downs and Black Quality Assessment Tool (Downs and Black 1998). This tool, although designed to evaluate experimental studies, has demonstrated applicability and validity for use in observational study design (Downs and Black 1998). For the observational studies addressing risk factors for playground injuries, six questions on the Downs and Black checklist (items 4, 8, 14, 23, 24, and 27) were not applicable, and were therefore excluded from the final score. Two reviewers were involved in the quality assessment process (SR, TC) and each article was independently appraised. Quality assessment ratings were compared and discussed between these reviewers, and any differences in ratings were resolved through open discussion and consensus. If discrepancies in ratings were unresolvable, the third reviewer (AM) was consulted.
Data extraction
One reviewer (SR) extracted data for all articles that met final inclusion using a structured data extraction form. The following data were extracted: title, author, year, study (design, duration, country), participants (e.g., age, sex, sample size), outcome (definition), exposure or intervention (definition), data analysis (statistical tests used), effect estimates [incidence rate ratios (IRR), odds ratio (OR), etc., with 95% confidence intervals (CI) or the association between exposure/intervention and outcome, reported or calculated based on data provided], conclusions, and quality assessment rating (Tables 2 and 3).
Table 2.
Data extraction for included risk factor studies aimed to examine the association between an exposure and playground injury, or injury-related outcomes
| Title (author, year of publication) | Study (design, duration, year(s), country) | Participants (age, sample size, sex) (SD = standard deviation) | Exposure | Outcome | Data analysis | Effect estimates [presented as odds ratios (OR), relative risk (RR) with 95% confidence intervals (CI)] | QA* score |
|---|---|---|---|---|---|---|---|
| Height and surfacing as risk factors for injury in falls from playground equipment: a case control study (Chalmers et al. 1996) | Case control design, 1989–1992, New Zealand | Cases: children (< 14 years old) injured from a fall from playground equipment at an early childhood education centre or school and received medical attention (n = 110). Controls: children (< 14 years old) with no medically attended injury from falling from playground equipment (n = 190), total n = 300 | Non-compliant playgrounds to NZS 5828 (New Zealand) standards (including playground surfacing, equipment height | Injuries from falls from playground equipment | Multivariable logistic regression |
Adjusted† results: non-impact (concrete, asphalt, grass, earth) vs. impact surfacing (loose + rubber), OR = 2.28 (95% CI 1.04–4.96), non-impact vs. loose fill surfacing (bark chips, pea gravel, sand), OR = 2.27 (95% CI 1.04–4.97) Adjusted‡ results: in impact-absorbing surfaces, fall height over 1.5 m increased risk of injury, OR = 3.80 (95% CI 2.01–7.17) and in non-impact-absorbing surfaces by 14 times, OR = 14.89 (95% CI 3.33–66.54) |
17 |
| Severity of playground fractures: play equipment vs. standing-height falls (Fiissel et al. 2005) | Case control design, 1995–2002, Canada | Cases and controls: children (≤ 4 ≥ 15 years of age) were age matched, 54.7% male. Cases were children with a fracture from standing height (n = 351, 32.8%) and controls were children with a fracture from falls from playground equipment (n = 719, 67.2%) | Playground equipment height | Severity of the fracturesustained (severity classification determined by treatment received). Major fracture defined as those that required emergency or operating room reduction, minor, those that did not | A logistic regression was used to determine the odds of a major fracture comparing playground equipment height falls to standing-height falls | The odds of sustaining a major fracture was 3.91 (95% CI 2.76–5.54) times greater when the fall was from a piece of playground equipment, compared to standing-height falls | 15 |
| Playground equipment injuries at home vs. those in public settings: differences in severity (Keays and Skinner 2012) | Case control design, 1995–2008, Canada | Cases were children (ages 36–143 months) injured in private homes (n = 6549), and controls were children (ages 36–143 months) injured in a public space including schools parks and other institutions (n = 33,181). Cases and controls reported to a Canadian ER (participating in CHIRPP§) from a playground-related injury, 53% were male | Playground type (home vs. public) | Severe injury (defined by level of treatment received), defined as any injury requiring admission, remaining in the emergency department | A logistic regression was used to determine the odds of sustaining a severe injury or fracture, at home compared to a public setting | Children falling from playgrounds at home had greater odds of severe injuries (OR = 1.30; 95% CI 1.23–1.37) and fractures (OR = 1.47; 95% CI 1.39–1.55) than those from public playgrounds. Children ages 3–5 years falling off slides at home, compared to slides in public settings, had the greatest odds of severe injuries (OR = 1.72; 95% CI 1.41–2.09) and fractures (OR = 2.17; 95% CI 1.79–2.64) | 16 |
| Severity of fall injuries on sand or grass in playgrounds (Laforest et al. 2000) | Retrospective cohort design, May–September, 1991 and 1995, Canada | Children (n = 930) presenting to two Montreal Children’s Hospitals, ages 1–14 years (mean = 5.8 years) involved in a playground fall injury, 54% male | Playground surfacing (sand, grass, other), playground type (municipal, school, daycare, residential), equipment type, age, supervision, location, sex, playground use | Nature of injury was the reported outcome (in two categories): fracture and head injury (includes concussions, skull fractures, and contusions) and fractures and dislocations). Other injury was defined as all other injuries | Multivariable logistic regression was used to estimate the association with surface, nature, and severity of injury, adjusting for age, location of the accident, month of the injury, number of medical visits for an injury per year, playground use | Falls onto grass surfaces had a 1.74 increased odds of having a fracture or head injury (95% CI 1.21–2.0) over sand. When controlling for surface type, residential playgrounds increased the odds of injury over municipal playgrounds, although this was not statistically significant (OR = 1.29, 95% CI 0.91–1.84) | 12 |
| Surface characteristics, equipment height, and the occurrence and severity of playground injuries (Laforest et al. 2001) | Nested case control design, May–September 1991 and 1995, Canada | Cases (n = 111) were children who suffered a severe injury and controls (n = 74) suffering a minor injury from a playground fall, both presenting to two Montreal Children’s Hospitals, ages 1–14 years (mean = 5.8 years) | Playground surfacing (g-max, recommended/not recommended), sex, age, supervision, playground use, equipment type, equipment height | Injury occurrence (yes/no) and injury severity (cases-controls) | Logistic regression was used to determine predictors of injury severity and occurrence of injury | Injury occurrence (adjusted|| results): Injuries were 2.56 times more likely (95% CI 1.07–6.14) to occur on equipment higher than 2 m compared with equipment lower than 1.5 m. Falls occurring on a surface exceeding 200 g had a risk of injury 3.03 (95% CI 1.45–6.35) times greater than for < 150. For surfaces between 150 and 200 g, injuries were 1.8 times higher (95% CI 0.91–3.57) | 14 |
| Risk factors for severe injuries associated with falls from playground equipment (Macarthur et al. 2000) | Case control design, May–October, 1995–1996, Canada | Cases (n = 67) were children with at least one severe injury from playground fall. Controls (n = 59) were children with a minor injury. Cases and controls were matched on age (> 18 months–14 years), sex (males, 51%; females, 49%), day of presentation | Playground equipment height, condition (wet vs. dry), use, prior to playground injury, supervision, surfacing (impact vs. non-impact) | Severe injury was defined as an injury with an AIS score ≥ 2, minor injury was defined as an injury with an AIS score < 2 | Multivariable logistic regression | The majority of cases (73%) fell from heights greater than 150 cm, compared to 54% of controls. The odds of severe injury was 2.30 (95% CI 1.09–4.84) times higher for falls over 150 cm, compared to falls lower than 150 cm | 18 |
| Safety standards and socioeconomic disparities in school playground injuries: a retrospective cohort study (Macpherson et al. 2010) | Retrospective cohort design, 1998–1999 (pre) and 2004–2007 (post), Canada | Study involved 374 elementary schools (population ≈ 145,000). Students attending the schools ranged from 62 to 1600 students (mean = 388 students) | Socio-economic status (SES) (via Learning Opportunity Index—LOI) | Injuries reported to the Ontario School Board Insurance Exchange. Injuries categorized as outdoor, outdoor non-equipment, and on play equipment | Poisson regression analysis was used to determine the relationship between the rate of playground injuries and LOI scores for both time periods | Prior to equipment upgrade, there was a significant relationship between SES and equipment-related injury, with children at poorer schools being at increased risk (RR = 1.52, 95% CI 1.24–1.86). After unsafe equipment was replaced, no relationship existed (RR = 1.13, 95% CI 0.95–1.32) | 11 |
| Measuring parent attributes and supervision behaviours relevant to child injury risk: examining the usefulness of questionnaire measures (Morrongiello and House 2004) | Cross-sectional design with observational measures, Canada | Participants were 48 parent-child dyads (46% female, 54% male children, 71% mothers, and 29% fathers) randomly selected from 7 local parks (to include variability in parental education and SES) | Parental supervision | Injury risk taking behaviour (total number of injury risk activities) | Correlation between observed behaviours and responses to behaviour checklist | Greater physical proximity to the child was associated with less risk taking (− 0.31) | 11 |
| Patterns of injuries to children on public playgrounds (Mott et al. 1994) | Retrospective cohort design, two, 6-month periods—1992–1993, Wales | Children presenting to the emergency room from playground injury (n = 162), ages 0–14 years | Playground surfacing (bark vs. concrete) | Injury was defined as one suffered on a public playground that required emergency room attention | Comparison of injury rates to expected injury rates (playground exposure) | There was a significantly different pattern of injury on bark and concrete/tarmac surfaces (p < 0.005); however, no statistically significant difference was found between surfacings | 12 |
| Safety of surfaces and equipment for children in playgrounds (Mott et al. 1997) | Retrospective cohort design, 1992–1994, Wales | Children presenting to the emergency room from playground injury 1992–1993 (summer) and all of 1994 (n = 330), ages 0–14 years | Playground surfacing (rubber, bark/rubber, bark, bark/tarmac, concrete/tarmac), equipment type, and height | Injury was defined as any injury on a public playground that required emergency room attention | A Poisson regression analysis was used to estimated the risk of injury by play surface, equipment type, and height, controlling for playground use | The risk of injury by surface type (RR, 95% CI): bark/rubber = 1.81, 0.66–4.98, bark = 1.98, 0.87–4.52, bark/tarmac = 4.63, 1.49–14.4, and concrete/tarmac = 5.11, 2.09–12.5 compared to rubber surfacing. RR of injury on equipment: swing = 1.27, 0.64–2.51; climbing frame = 3.47, 1.97–6.11; monkey bars = 7.49, 4.14–13.5. There was a linear relationship with height of fall from equipment and the number of fractures (p < 0.005) | 13 |
| A case-control study of risk factors for playground injuries among children in Kingston and area (Mowat et al. 1998) | Case control design, 1995, Canada | Cases (n = 45) were children (ages 1–16 years) presenting to one of two emergency rooms. Controls (two groups, n = 45, n = 45) were defined as (1) children injured not from playgrounds and (2) children non-injured needing emergency care. Children were age (mean = 8.3 years, SD = 2.2), sex (n = 26 male, n = 19 female), and day presented matched | Playground hazards (surfacing, handrails, use, number of hazards, sharp edges, protrusion, non-encroachment zones, entrapment, pinch points) playground type (municipal vs. school) and SES (family income) | Injury was defined as any injury on a playground that required attendance to of two hospitals emergency room | A multivariable conditional logistic regression was used to estimate the association with playground hazards and injury risk | Inadequate handrails and guardrails were associated with an increased odds of sustaining an injury requiring emergency room attendance (OR = 6.7, 95% CI 2.6–7.5). Surfacing material not recommended by the Canadian Standards Association and the US Consumer Product Safety Commission (CPSC) had an increased odds of injury (OR = 21.0, 95% CI 3.4–128.1). Surface depth not meeting CPSC has an increased odds of injury (OR = 18.2, 95% CI 3.3–99.9) | 17 |
| Composite playground safety measure to correlate the rate of supracondylar humerus fractures with safety: an ecologic study (Park et al. 2010) | Retrospective cohort design, 1998–2006, USA | Children (n = 48) 3–16 years (mean = 5.9 years, SD = 3.3) who sustained supracondylar humerus fractures (ICD-9), males 52% | Compliance to safety codes using a composite safety measure (including surfacing, supervision, age-appropriate design) | Supracondylar humerus fractures | The difference in fracture rate by ZIP code was calculated using Yates Chi-squared and OR. Linear regression was used to calculate playground safety with injury rates | Children in the least safe zone had a 4.77 increased odds of injury (95% CI 2.36–9.63) compared to the most safe zone. Linear regression analysis showed thataverage playground safety value (safety index) was inversely related to injury rates (r = 0.98, p = 0.04) | 13 |
| Injuries in public and private playgrounds: the relative contribution of structural, equipment, and human factors (Petridou et al. 2002) | Nested case control design, September – December 1999, Greece | Cases (n = 777) and controls (n = 294) were children, 0–14 years presenting to the emergency room. Cases were children with a playground injury, controls were non-injured children | Nationality (Greek vs. non-nationals), playground type (public vs. private), age and sex | All injuries occurring in a public or private playground | A multivariable logistic regression analysis was used to estimate the odds of injury, controlling for age, sex, nationality | There was a 2.22 times higher risk of injury in public playgrounds over private (OR = 2.22, 95% CI 1.61–3.07). Age greater than 10 years was protective of injury (OR = 0.50, 95% CI 0.34–0.73) compared to 5–9 year olds, and those defined as non-nationals were at an increased odds of injury (OR = 2.00, 95% CI 1.13–3.54) | 16 |
| Playground hazards in Atlanta child care centers (Sacks et al. 1990) | Prospective cohort design, June 1987–October 1988, USA | Randomly selected licensed childcare centres with playgrounds (n = 66) | Playground hazards (rock, concrete, root, or stump in fall zone; inadequate clearance; sharp protrusion; contaminants in under surface, missing parts, blunt protrusions, trip-and-fall, tip-over, entrapment hazards, broken or loose parts, open S-hooks, hard swing seats), equipment height | Medically treated injury (injury treated within 48 h) | Results were presented as the number of hazards and the number of reported injuries | Of 21 centres with ≤ 5 hazards, 43% reported a playground-related injury in the previous year; of 25 centres with 6–11 hazards, 52% reported a playground-related injury; and of 20 centres with ≥ 12 hazards, 60% reported a playground-related injury. Reported that climbing equipment ≥6 ft had inadequate impact-absorbing under-surfacing and over twice the rate of fall injuries as climbing equipment ≤6 ft | 12 |
| Out on a limb: risk factors for arm fracture in playground equipment falls (Sherker et al. 2005) | Case control, design (unmatched), 2000–2002, Australia | Cases and controls were children (< 13 years) who fell from school playground equipment and landed on their arm. Cases (n = 402) sustained an upper limb fracture and controls (n = 283) had minor to no injury | Equipment height, surfacing type, surface depth, substrate material, arm load | Upper arm fractures | A multivariable logistic regression was used to estimate the association between fractures and exposures, controlling for covariates | Falls from equipment heights greater than 1.5 m had a 2-fold increased odds (OR = 2.39, 95% CI 1.49–3.84 of sustaining an arm fracture compared to 1.5 m or less. Fall heights >1 m were 2.96 times more likely to sustain an arm fracture over those who fell less than 1.0 m (95% CI 1.71–5.15 | 16 |
*Quality appraisal
†Adjusted for place of occurrence, age, gender, child weight, child height, type of equipment, and height of fall (broad categories)
‡Adjusted for age, gender, place of occurrence, child weight, child height, and type of equipment
§Canadian Hospitals Injury Reporting Prevention Program
||Adjusted for height, surface, g-max, equipment type, and density of children in the playground area
Table 3.
Data extraction for included intervention studies aimed to reduce playground injury or injury-related outcomes
| Title (author, year of publication) | Study (design, duration, year(s), country) | Participants (age, sample size, sex) (SD = standard deviation) | Intervention | Outcome | Data analysis | Effect estimates [presented relative risk (RR) with 95% confidence intervals (CI)] | QA* score |
|---|---|---|---|---|---|---|---|
|
The Stamp-in-Safety Program, an intervention to promote better supervision of children on childcare centre playgrounds: an evaluation in an urban setting (Chelvakumar et al. 2010) |
Quasi-experimental design (same centre had both control and intervention groups; 3-week intervention period, 2-week pre and 2-week post) unknown years of data collection, USA | Children ranged from 3 to 5 years (mean age = 4.16 years, SD ± 0.72) and 58% were girls (n = 44). Teachers (n = 12) mean age of 38 years (SD = 14, range = 20–58 years) and were primarily white (83% Caucasian, 8% African American, and 8% Asian American) | Stamp-in-Safety Program (supervision including proximity to the child, continuity of supervision, and attention to children’s activities) | Quality of teacher supervision on playgrounds | Paired t tests, with teacher as the unit of analysis and ANOVA for child risk-taking behaviours, with session as the unit of analysis | Total behaviour change (teacher verbalization + child risk-taking behaviour) in the intervention group, pre-intervention 4.8, post-intervention 3.8 (p = 0.168), control group, pre-intervention 5.1, intervention 4.9 (p = 0.918) | 12 |
| Assessment of a pilot child playground injury prevention program in New York state (Fisher et al. 1980) | Quasi-experimental design, pre- and post-workshops, 1977 and 1978, USA | Recreational leaders, day care centres, and elementary school teachers, nurses, and parent/teacher reps (n = 100 knowledge scores, n = 78 surveys, number of children reporting injuries, not reported) | A 40-min workshop for voluntary correction of playground hazards | Change in knowledge (supervisors, change in number of hazards, change in the number of injury reports | Change in knowledge scores, number of hazards, and number of injuries (difference in mean percent before and after) | There was a 17% increase in scores post-workshops, 42% reduction in number of hazards, and 22% reduction in the number of injuries compared to base year | 9 |
| Decreasing children’s risk taking on the playground (Heck et al. 2001) | Quasi-experimental design, 5-day intervention, unknown years of data collection, USA | First graders (n = 117), second graders (n = 122), and kindergarten/third-grade students (n = 140) from a suburban school (kindergarten through grade 5) | Program to model safe and unsafe behaviour with children receiving rewards for changing risky behaviour | Risky behaviour observed on slides in playgrounds | Change in number of risky behaviours observed in the playground | There was a decrease in slide misuse for second and third graders (results reported in a figure, not in the results section) | 3 |
| The effect of safer play equipment on playground injury rates among school children (Howard et al. 2005) | Non-randomized quasi-experimental design, 10-months before and after the change in playgrounds in the Toronto District School Board (TDSB), 1998–2002, Canada | Intervention schools (n = 34,557 attending students), control schools (n = 88,417 attending students), grades 1–6. Injuries were collected in children ages 4–11 years that occurred in the school playground | Change in the playground structures to Canadian Standards Association (CSA) standard equipment (reduction of height to less than 1.5 m and change in surfacing from concrete/non-impact absorbing to wood chips) | Injuries reported to the Ontario School Board Insurance Exchange that occurred in TDSB playgrounds | The RR (with 95% CI) was estimated using random-effect meta-analytic methods | The injury rate in the intervention schools decreased from 2.61 (95% CI 1.93–3.29) injuries per 1000 students per month before equipment removal to 1.68 (95% CI 1.31–2.05) per 1000 per month after the equipment was replaced (RR = 0.70, 95% CI 0.62–0.78) | 20 |
| School playground surfacing and arm fractures inchildren: a cluster randomized trial comparing sand towood chip surfaces (Howard et al. 2009) | Cluster, randomized controlled trial design, January 2005–June 2007, Canada | Children from 37 elementary schools (n = 15,074) | Change in playground surfacing comparing Fibar—wood fibre and granite sand surfacing | Arm fracture rates per 100,000 student-months from falls onto playground surfaces | A Poisson regression analysis, controlling for cluster was used to estimate the difference in arm fracture rate (per 100,000 student-months) comparing sand to Fibar surfaces | Among all schools, the arm fracture rate was 4.5 (95% C: 0.26–15.9) per 100,000 student-months for falls into sand compared with 12.9 (95% CI 5.1–30.1) for falls onto Fibar surfaces | 29 |
|
An evaluation of the Cool 2 Be Safe Program: an evidence-based community-disseminated program to positively impact children’s beliefs about injury risk on playgrounds (Morrongiello and Kane 2015) |
Randomized controlled trial design, 4-week intervention, Canada | Four Boys and Girls Clubs were recruited for participation. Intervention group (n = 80 children), 43 male, 36 female, and 1 unidentified), 7 to 12 years old (mean age = 9.67 years, SD = 1.65). Control group (n = 24 children (5 male and 19 female), 7 to 12 years old (mean age = 9.62 years, SD = 1.56) | Cool to be Safe Program: Four lessons (hazard recognition, hypocrisy thinking, acting safe, injury vulnerability, and peer pressure, each designed to address key injury beliefs and/or peer group pressure) | Injury risk beliefs on playgrounds | An ANOVA on each subscale (fate, vulnerability, severity), with condition (intervention, control) as a between-participant factor and time (pre, post) as a within-participant factor. Follow-up t tests applied a Bonferroni correction | The intervention group (children’s attributions for injuries to bad luck) decreased significantly following the intervention, t(74) = − 3.74, p < 0.001. In the control group, this belief increased significantly pre to post test, t(23) = 2.97, p < 0.01. Children’s injury vulnerability recognition significantly increased post-intervention, t(74) = 4.93, p < 0.001 with no change in the control group (p > 0.05) | 17 |
|
“Practice what you preach”: induced hypocrisy as an intervention strategy to reduce children’s intentions to risk take on playgrounds (Morrongiello and Mark 2008) |
Randomized controlled trial design, Canada | Children from 5 schools, ages 7–13 years, grades 2–6, n = 238 (intervention = 138, control = 101, 98% Caucasian) | An induced-hypocrisy intervention comprised three parts: generation of a list of previous risky behaviours, signing of the Safe Play on Playgrounds poster, and making a radio commercial advocating for safe play | Perception of injury risk-taking behaviours on playgrounds | Pre to post intervention indices to risk take (before and after Likert scale) | 87% of the children in the intervention group moved at least one targeted risk pictures to the “would not do” side, compared to 20% of control children (x2 (1) = 100.37, p < 0.001). In addition, the reduction in the intention to risk take for children in the intervention group was greater compared to the control group [F(1, 207) = 10.08, p < 0.01, d = 0.38, 95% CI 1.10–2.32], [mean = 3.22 (intervention), SD = 5.20, mean = 0.59, SD = 3.30 (control) | 16 |
| Addressing the issue of falls off playground equipment: an empirically-based intervention to reduce fall-risk behaviours on playgrounds (Morrongiello and Matheis 2007) | Randomized controlled trial design, Canada | Intervention group (n = 191 children), 91 males and 100 females) and the control group (n = 67 children), 32 males, 35 females, ages 6–11 years | Poster and video program for risky playground behaviour. Post-intervention poster of if/how children would change their poster after seeing the video images | Perception of injury risk-taking behaviours on playgrounds | Pre to post intervention indices to risk take (before and after Likert scale) | Significant decreases in intentions to risk take were obtained in the intervention, but not the control group. Intervention effectiveness did not vary with children’s age or sex, but was greater for those scoring high in sensation seeking | 15 |
| Strategies for playground injury prevention: an overview of a playground project (Olsen et al. 2010) | Quasi experimental design, pre- to post-program data collection, over 3 years (2002–2005), USA | 8 schools were randomly assigned to the intervention. There is no information on the children that data were collected on | Change in playground surfacing to rubberized surfaces, with playground injury prevention education | Playground injuries reported to schools | No analyses reported, just a change in injuries pre- to post-program implementation | There was a 70, 71, 27, and 30% reduction in injuries (medical, sent home, returned to class, total) compared to the previous year (before the surfacing) | 10 |
|
An intervention to reduce playground equipment hazards (Roseveare et al. 1999) |
Randomized controlled trial design, pre- to post-program data collection over 2 time periods (19 and 27 months), New Zealand | Children in the intervention schools (12 schools, n = 3013) and control schools (12 schools, n = 2694), total hazards at baseline, intervention schools = 155, control schools = 136 | Recommended action report (hazards and improvements to make), Public Health Service project worker worked with the intervention schools for 9 months to help enact the report | Playground hazards | Mann-Whitney test to examine the change in the number of hazards | After 19 months, there was a significantly greater reduction in hazards in the intervention schools, compared to controls (z = 2.21, p = 0.027). There was a 52% reduction in height/surface hazards in the intervention schools, compared to 8% in controls | 16 |
| The Stamp-in-Safety Program: a behavioural intervention to reduce behaviours that can lead to unintentional playground injury in a preschool setting (Schwebel et al. 2006) | Quasi-experimental, time series design, data collection pre-, during and post-program, as well as 6-month post-intervention, USA | Teachers (n = 12) from 1 childcare centre, supervising 3–4-year olds, women, mean age = 38 years (SD = 14, range = 20–58), childcare professionals for a mean of 5 years (SD = 6, range = 0–19), mostly white (83% Caucasian, 8% African American, and 8% Asian American). Children: more girls (64%) than boys, from middle- to upper-middle-class backgrounds (52% reported income over $80,000; 71% Caucasian) | Stamp-in-Safety Program (supervision including proximity to the child, continuity of supervision, and attention to children’s activities) | Behaviour change program to reduce children’s risk of unintentional playground injury | Univariate ANOVA (phase as a random within-subject factor) was used to examine the change in mean number of observations made (warnings, explanations, redirections) | Teachers offered more warnings to children on dangerous behaviours before the intervention, decreasing in frequency during and after the intervention (pre-intervention mean = 1.89, intervention mean = 0.56, post-intervention mean = 0.13, follow-up mean = 0.33, F(3, 32) = 3.40, p < 0.05. The explanation rate decreased pre-intervention to post-intervention (not statistically significant) and returned to pre-intervention levels at 6-month follow-up. Redirections showed a pattern similar to explanations | 13 |
| Evaluation of an intervention to reduce playground hazards in Atlanta child-care centers (Sacks et al. 1992) | Randomized controlled trial design, baseline data collected between July and October, 1988, post-data collection occurred between May and June 1990, USA | Intervention childcare centres (n = 58), with 123 playgrounds with 1171 pieces of equipment, and 71 control childcare centres with 135 playgrounds and 1021 pieces of equipment | Program showing the director of playgrounds, the hazards, and distributing safety information | Playground hazards | Mean change in number of hazards before and after the intervention | Playground hazards increased from 85 to 104 for 53 intervention centres and decreased from 81 to 73 for 57 control centres. Intervention centres had 9.4 hazards per playground and control centres had 8.0 | 11 |
*Quality Index
Data synthesis
Due to the heterogeneity of the study designs and interventions, a meta-analysis could not be completed. A qualitative analysis was performed, and the results were summarized in evidence tables.
Results
Study characteristics
Risk and/or protective factors
A total of 311,639 studies were retrieved from the literature search, with 183,776 excluded at the title stage, 571 at the abstract review stage, and a further 19 at the eligibility stage. This included one article identified from a reference review (Macarthur et al. 2000). A final total of 15 articles were included (Fig. 1). There were 8 case-control studies (Macarthur et al. 2000; Chalmers et al. 1996; Fiissel et al. 2005; Keays and Skinner 2012; Laforest et al. 2001; Mowat et al. 1998; Petridou et al. 2002; Sherker et al. 2005), 5 retrospective cohorts (Laforest et al. 2000; Macpherson et al. 2010; Mott et al. 1994, 1997; Park et al. 2010), 1 prospective cohort (Sacks et al. 1990), and 1 cross-sectional study (Morrongiello and House 2004) included in the final analyses.
Fig. 1.
Adapted PRISMA flow diagram for risk and/or protective study inclusion
Twelve studies examined associations with potential risk factors among children (Macarthur et al. 2000; Chalmers et al. 1996; Fiissel et al. 2005; Keays and Skinner 2012 Laforest et al. 2000, 2001; Mowat et al. 1998; Petridou et al. 2002; Sherker et al. 2005; Mott et al. 1994, 1997; Park et al. 2010), two studies provided descriptions of the centres where the playgrounds were located (childcare centres, elementary schools) (Macpherson et al. 2010; Sacks et al. 1990), and one study examined parent-child dyads (Morrongiello and House 2004). Children in the included studies ranged in age from 0 to 16 years, with most children being less than 14 years of age (Macarthur et al. 2000; Chalmers et al. 1996; Laforest et al. 2000, 2001; Petridou et al. 2002; Mott et al. 1994, 1997). Sample sizes ranged from 48 to 39,730 children with a median of 330 children (IQR 135, 1070). The median quality index score for these studies was 14 (IQR 12, 16) of a possible 22.
Interventions
A total of 311,337 studies were retrieved from the literature search, with 243,440 excluded at the title stage and 242 at the abstract review stage, leaving a total of 12 articles included in this study (Fig. 2). There were 6 randomized controlled trials (Morrongiello and Kane 2015; Howard et al. 2009; Morrongiello and Mark 2008; Morrongiello and Matheis 2007; Roseveare et al. 1999; Sacks et al. 1992) and 6 quasi-experimental designs (Chelvakumar et al. 2010; Fisher et al. 1980; Heck et al. 2001; Howard et al. 2005; Olsen et al. 2010; Schwebel et al. 2006). Of the studies that provided information on the children included, ages ranged from 3 to 13 years, with sample sizes ranging from 44 to 122,974, with a median of 238 children (IQR 104, 5707). The median critical appraisal score for these studies was 14 (IQR 11, 16) of a possible 32.
Fig. 2.
Adapted PRISMA flow diagram for intervention study inclusion
Risk and/or protective factors for injury in playgrounds
All studies included in this review examined the association with a risk factor and injury or an injury-related outcome in equipment- and structure-based playgrounds. Eight studies examined the association between surface type and injury (Macarthur et al. 2000; Chalmers et al. 1996; Laforest et al. 2000, 2001; Mowat et al. 1998; Sherker et al. 2005; Mott et al. 1994, 1997). Seven studies examined associations between injury outcomes and equipment height (Macarthur et al. 2000; Chalmers et al. 1996; Fiissel et al. 2005; Laforest et al. 2001; Sherker et al. 2005; Mott et al. 1997; Sacks et al. 1990), and four studied playground type (e.g. residential vs. municipal playgrounds, municipal vs. school) (Keays and Skinner 2012; Mowat et al. 1998; Petridou et al. 2002; Laforest et al. 2000) and supervision (Macarthur et al. 2000; Laforest et al. 2000, 2001; Morrongiello and House 2004). Three studies examined the association between equipment type (Laforest et al. 2000, 2001; Mott et al. 1997), sex (Laforest et al. 2000, 2001; Petridou et al. 2002), age (Laforest et al. 2000, 2001; Petridou et al. 2002), and playground use (Macarthur et al. 2000; Laforest et al. 2000, 2001). Two studies examined playground hazards (Mowat et al. 1998; Sacks et al. 1990) and socio-economic status (SES) (Mowat et al. 1998; Macpherson et al. 2010) with injury or injury-related outcomes. One study measured compliance with safety codes using a composite safety measure (including surfacing, supervision, age-appropriate design) (Park et al. 2010), one study examined associations between playground injury and playground conditions (wet vs. dry) and prior playground injury (Macarthur et al. 2000), and another study examined the injury location (in the city compared to out of the city) (Laforest et al. 2000) with another comparing injury in those identified as residents of the country, compared to non-residents (Petridou et al. 2002).
The majority of studies examined the association between exposure and playground injury, with injury as the outcome. Injury definitions varied across risk factor studies, including ten studies reporting severe injury (injury definitions included emergency department-attended injuries, fractures, medically attended injuries, and severe injuries from falls) (Macarthur et al. 2000; Fiissel et al. 2005; Laforest et al. 2000, 2001; Sherker et al. 2005; Macpherson et al. 2010; Mott et al. 1994, 1997; Park et al. 2010; Sacks et al. 1990). Six studies reported outcomes by playground injury (including injury definitions of all playground-related injuries and only fall-related injury) (Chalmers et al. 1996; Keays and Skinner 2012; Laforest et al. 2000, 2001; Mowat et al. 1998; Petridou et al. 2002). One study reported outcomes specific to injury risk-taking behaviour (Morrongiello and House 2004).
Surfacing
All but one study that examined playground surfacing demonstrated an increased risk of injury. In Laforest et al. (2001), the risk of injury was 3.03 times higher (95% CI 1.45–6.35) for surface absorption level deceleration exceeding 200 g-max compared to less than 150 g-max, and the risk of injury was lower below 100 g-max (OR = 0.67, 95% CI 0.45–0.99), as demonstrated by Sherker et al. (2005). On surfaces having absorption levels between 150 and 200 g-max, injuries were 1.8 times more likely (95% CI 0.91–3.57) (Laforest et al. 2001). When comparing surface types, the risk of injury on bark/rubber, bark, bark/tarmac, and concrete/tarmac was higher when compared to that on rubber surfacing [OR = 1.81 (95% CI 0.66–4.98), OR = 1.98 (95% CI 0.87–4.52), OR = 4.63 (95% CI 1.49–14.4), and OR = 5.11 (95% CI 2.09–12.5)] (Mott et al. 1997). Chalmers et al. (1996) demonstrated that non-impact-absorbing surfaces, including concrete, asphalt, paving, earth, and grass, had an odds ratio of injury 2.28 times higher (95% CI 1.04–4.96) when compared to playgrounds with impact-absorbing fill (loose fill or synthetic). When examining this relationship further, the authors found an increased odds ratio of injury requiring medical attention with non-impact-absorbing surfaces compared with loose fill surfaces only (adjusted OR = 2.27, 95% CI 1.04–4.97). Laforest et al. (2000) and Sherker et al. (2005) demonstrated similar results when comparing sand to grass surfaces, with an increased odds ratio of severe injury on playgrounds with grass surfacing (OR = 1.74, 95% CI 1.21–2.52; OR = 0.29, 95% CI 0.16–0.53—protective for sand vs. soil). Mowat et al. (1998) demonstrated an increased risk of injury in playgrounds with inappropriate surface materials under and around equipment (OR = 21.0, 95% CI 3.4–128.1).
Mott et al. (1994) examined injuries on playgrounds, comparing injury types across bark and concrete surfaces and noted that fractures and sprains were more common on bark surfaces; 60% of all injuries on bark surfaces were fractures compared to 17% on concrete. Although this study demonstrated that there were fewer injuries than expected on bark surface playgrounds, these differences were not significant (Mott et al. 1994).
Equipment height
Several studies point to the height of playground equipment as a contributing factor to injuries resulting from falls. Two studies demonstrated over twice the risk of injury from equipment that was 2 and 1.8 m high, compared to less than 1.5 m (Laforest et al. 2001; Sacks et al. 1990); however, other studies have demonstrated a critical fall height of 1.5 m. Mott et al. (1997) demonstrated a significant linear increase in the number of fractures from playground equipment with heights greater than 1.5 m. Other studies support this finding including Macarthur et al. (2000) and Sherker et al. (2005), where there were significant increases in severe injuries from falls greater than 1.5 m (OR = 2.30, 95% CI 1.09–4.84, and OR = 2.39, 95% CI 1.49–3.84, respectively), even at playgrounds with impact-absorbing surfaces (OR = 3.80, 95% CI 2.01–7.17) (Chalmers et al. 1996). When further examining the severity of injuries from falls at playgrounds, fractures from playground height falls were found to be 3.9 times (95% CI 2.76–5.54) more likely to require surgical reduction compared to those from standing-height falls (Fiissel et al. 2005).
Residential playgrounds
There were four studies that support the protective effect of public playgrounds over playgrounds in residential areas (Keays and Skinner 2012; Mowat et al. 1998; Petridou et al. 2002; Laforest et al. 2000). In Laforest et al. (2000), there was a 70% increase in the risk of severe injury at residential playgrounds compared to municipal playgrounds (OR = 1.69, 95% CI 1.15–2.47) with another study demonstrating an increased risk of severe injury (OR = 1.30, 95% CI 1.23–1.37) and fracture (OR = 1.47, 95% CI 1.39–1.55) at residential playgrounds compared to parks, schools, or daycare centres (Keays and Skinner 2012).
Other risk factors
There is less evidence of other risk factors as independent predictors of injury including age, supervision, playground hazards, sex, playground use, SES, prior playground injury, playground condition (wet vs. dry), location, and resident status. Three studies examined multiple risk factors, such as age, supervision, sex, playground use, equipment type, as well as playground equipment height and surfacing (Macarthur et al. 2000; Laforest et al. 2000, 2001). The only statistically significant effect estimates (other than surfacing and equipment height) were found by age, location, and resident status. Children aged 5–9 years old had twice the risk of suffering a severe injury compared to 1–4 year olds (OR = 1.99, 95% CI 1.47–2.74), and 10–14 year olds were at a 60% increased risk of severe injury (OR = 1.59, 95% CI 1.01–2.39) (Laforest et al. 2000). This is supported by Petridou et al. (2002) who demonstrated a protective effect from playground injury in children greater than 10 years of age, compared to children ages 5–9 years (OR = 0.50, 95% CI 0.34–0.73). Two studies demonstrated that children reported to be non-residents, or those children playing at playgrounds outside of the city, were at increased risk for playground injury (Petridou et al. 2002; Laforest et al. 2000). Two studies demonstrated an increased risk of injury with increased playground hazards, including playgrounds without adequate handrails and guardrails (Mowat et al. 1998; Sacks et al. 1990). One study examined risky playground behaviour with adult supervision and showed that greater physical proximity to children in playgrounds was associated with less risk-taking behaviour (Morrongiello and House 2004); other studies that examined parental supervision with injury outcomes demonstrated no association (Macarthur et al. 2000; Laforest et al. 2000, 2001). Four studies reported no significant risk of injury with increased playground use (Macarthur et al. 2000; Laforest et al. 2000, 2001; Mowat et al. 1998). There were two studies that examined proxy measures for SES with increased risk of injury (Mowat et al. 1998; Macpherson et al. 2010). When controlling for compliance with playground standards, schools with lower SES measures did not demonstrate an increased risk for injury (Macpherson et al. 2010). Mowat et al. (1998) also demonstrated no significant increase in risk of injury with increasing household income.
Interventions
All intervention studies included in this review examined the effectiveness of the intervention in equipment- and structure-based playgrounds. Outcomes for intervention studies included injury risk beliefs (Morrongiello and Kane 2015) or behaviour (Morrongiello and Mark 2008; Morrongiello and Matheis 2007; Chelvakumar et al. 2010; Heck et al. 2001; Schwebel et al. 2006), injuries reported to schools (Howard et al. 2005, 2009; Olsen et al. 2010), national injury surveillance systems (Fisher et al. 1980), or the change in the number of playground hazards (Roseveare et al. 1999; Sacks et al. 1992; Fisher et al. 1980).
Interventions to decrease risky playground behaviour
Four studies examined the effectiveness of interventions designed to decrease risky playground behaviour. One study modelled safe playground behaviour and provided rewards for reductions in risky behaviour (Heck et al. 2001), and three studies examined hazard-recognition interventions (Morrongiello and Kane 2015; Morrongiello and Mark 2008; Morrongiello and Matheis 2007). These studies demonstrated varying levels of effectiveness, with two studies showing null effects (Morrongiello and Kane 2015; Heck et al. 2001) and two studies demonstrating positive changes in perceptions of risk-taking behaviours when comparing pre- to post-intervention estimates (Morrongiello and Mark 2008; Morrongiello and Matheis 2007).
Interventions to increase playground supervision
Two studies included in this review examined the effectiveness of interventions to decrease risky child behaviour by increasing adult supervision at playgrounds (Chelvakumar et al. 2010; Schwebel et al. 2006). Neither study demonstrated statistical significance in reducing injury or risky playground behaviour.
Interventions to change playground surfacing, height, and hazards
Three studies examined the effectiveness of changes to playground surfacing (Howard et al. 2009; Olsen et al. 2010), or surfacing and height (Howard et al. 2005). Howard et al. (2005) used a quasi-experimental design to examine differences in injury rates, pre- to post-change in surfacing (from concrete, non-impact-absorbing surfaces to wood chips) and height (reduced to 1.5 m) at playgrounds in Toronto, Canada. The injury rate at the intervention schools decreased 30% post equipment replacement (RR = 0.70, 95% CI 0.62–0.78) (Howard et al. 2005). In 2009, Howard et al. further examined playground surfacing, and demonstrated a reduction in arm fracture rate for sand surfacing compared to engineered wood chip surfacing (IR = 4.5, 95% CI 0.26–15.9 per 100,000 student-months vs. 12.9, 95% CI 5.1–30.1). Olsen et al. (2010) demonstrated a 30% reduction in total injuries (including medically attended, child being sent home, and child returning to class injuries) post-implementation of rubberized surfacing.
Interventions to reduce playground hazards
Two studies included in this review examined the effectiveness of programs to bring attention to and decrease playground hazards (Roseveare et al. 1999; Sacks et al. 1992). A study by Roseveare et al. (1999) demonstrated effectiveness in reducing the number of playground hazards, post-implementation of a health promotion program consisting of information about the hazards, an engineer’s report, regular contact, and encouragement to act on the report, and assistance in obtaining funding to support the program. Post-program, there was a significant reduction in the number of hazards in the intervention schools compared with the control schools (p = 0.027) (Roseveare et al. 1999); however, in Sacks et al. (1992), the intervention designed to reduce playground hazards, by revealing the hazards to the playground directors and providing safety information, was ineffective.
Study quality assessment
Consensus was achieved between the two independent reviewers for the critical appraisal of all articles included in this review. Overall, studies included in this review were of moderate quality. The median quality index score for risk factor and intervention studies was 14, with similar interquartile ranges (IQR 12, 16, and 11, 16, respectively). The majority of studies that examined risk factors did not clearly describe the distributions of principal confounders across subject groups (Keays and Skinner 2012; Laforest et al. 2000, 2001; Macpherson et al. 2010; Mott et al. 1994, 1997; Sacks et al. 1990; Morrongiello and House 2004), did not describe the children that were lost to follow-up (Chalmers et al. 1996; Fiissel et al. 2005; Laforest et al. 2000, 2001; Petridou et al. 2002; Sherker et al. 2005; Macpherson et al. 2010; Mott et al. 1994, 1997; Park et al. 2010; Morrongiello and House 2004), and lacked adjustment for confounding variables in the analyses (Keays and Skinner 2012; Macpherson et al. 2010; Mott et al. 1994, 1997; Sacks et al. 1990; Morrongiello and House 2004). One study reported adjusting for confounders without reporting what these variables were (Sherker et al. 2005). In addition, many studies did not report taking into account potential losses to follow-up in their analyses (Laforest et al. 2000, 2001; Macpherson et al. 2010; Mott et al. 1997; Park et al. 2010; Sacks et al. 1990; Morrongiello and House 2004) or calculating risk based on exposure, or the number of children actually playing in playgrounds. For example, only one study calculated the risk of injury using exposure data (Mott et al. 1997), and this was estimated based on the mean number of children playing in playgrounds from 1-year observations. The study by Laforest et al. (2000) controlled for playground use in their final model, using a dichotomous variable describing rare and frequent use, calculated from the mean number of weekly playground visits. Another study, by this same author, considered exposure of play using a proxy measure that included the number of visits to the playground in the summer, their favourite equipment, and measures related to the height and surfacing of the equipment used (Laforest et al. 2001). This measure was not used in the final analyses.
Intervention studies suffered similar reporting issues, including: not describing distributions of principal confounders (Morrongiello and Kane 2015; Morrongiello and Mark 2008; Morrongiello and Matheis 2007; Roseveare et al. 1999; Sacks et al. 1992; Chelvakumar et al. 2010; Fisher et al. 1980; Heck et al. 2001; Howard et al. 2005; Olsen et al. 2010; Schwebel et al. 2006), the characteristics of patients lost to follow-up (Morrongiello and Kane 2015; Morrongiello and Mark 2008; Morrongiello and Matheis 2007; Roseveare et al. 1999; Sacks et al. 1992; Chelvakumar et al. 2010; Fisher et al. 1980; Heck et al. 2001; Howard et al. 2005; Olsen et al. 2010), any sample bias (Morrongiello and Kane 2015; Morrongiello and Matheis 2007; Roseveare et al. 1999; Chelvakumar et al. 2010; Fisher et al. 1980; Heck et al. 2001; Howard et al. 2005; Olsen et al. 2010), adjustment for confounding or losses to follow-up in the analysis, and whether there were any power calculations (Morrongiello and Kane 2015; Morrongiello and Mark 2008; Morrongiello and Matheis 2007; Roseveare et al. 1999; Sacks et al. 1992; Chelvakumar et al. 2010; Fisher et al. 1980; Heck et al. 2001; Howard et al. 2005; Olsen et al. 2010; Schwebel et al. 2006). Other reporting deficiencies included: not reporting adverse events that may be a consequence of the intervention (Morrongiello and Mark 2008; Morrongiello and Matheis 2007; Roseveare et al. 1999; Sacks et al. 1992; Chelvakumar et al. 2010; Fisher et al. 1980; Heck et al. 2001; Olsen et al. 2010), reporting of actual probability values (Morrongiello and Mark 2008; Morrongiello and Matheis 2007; Sacks et al. 1992; Fisher et al. 1980; Heck et al. 2001; Olsen et al. 2010; Schwebel et al. 2006), attempts made to blind study subjects to the intervention they received as well as those measuring the study outcomes (Morrongiello and Kane 2015; Morrongiello and Mark 2008; Roseveare et al. 1999; Sacks et al. 1992; Chelvakumar et al. 2010; Fisher et al. 1980; Heck et al. 2001; Olsen et al. 2010; Schwebel et al. 2006), and assuring the concealment of the randomized assignment of groups (Morrongiello and Kane 2015; Morrongiello and Mark 2008; Roseveare et al. 1999; Sacks et al. 1992; Chelvakumar et al. 2010; Fisher et al. 1980; Heck et al. 2001; Olsen et al. 2010; Schwebel et al. 2006).
Discussion
As injury prevention researchers, we aim to provide evidence around the risk and protective factors for injury, given an exposure, in addition to evidence of intervention effectiveness used to mitigate these risks. This review included equipment- or structure-based playground studies of moderate quality, revealing a number of risk factors for injury. These risk factors included the lack of handrails and guardrails on playground equipment, non-impact-absorbing surfacing, and critical fall heights. Other risk factors included the lack of adult supervision, playground hazards, privately owned and operated playgrounds, age, location, and resident status. Effective interventions included modifying playground surfacing and reducing equipment height to less than 1.5 m.
Understanding the risk factors and effective prevention strategies for playgrounds plays an important role in the prevention of injury. This information must be used in equipment-based playground design and installation, as well as toward guidelines for maintenance. There are organizations that provide voluntary guidelines for playground safety, including the Canadian Standards Association (CSA) and the US Consumer Product Safety Commission (CPSC). The CSA (CSA Z614-14) (Canadian Standards Association 2015) and the CPSC (U.S. Consumer Product Safety Commission 2010) outline many recommendations to prevent playground injuries, and compliance to such standards is recommended in many of the studies included in this review (Chalmers et al. 1996; Mowat et al. 1998; Macpherson et al. 2010; Howard et al. 2005) and is supported by those that examined one or more of the recommendations as a risk factor for injury (Macarthur et al. 2000; Chalmers et al. 1996; Fiissel et al. 2005; Laforest et al. 2000, 2001; Mowat et al. 1998; Sherker et al. 2005; Mott et al. 1994, 1997; Park et al. 2010; Sacks et al. 1990; Morrongiello and House 2004). The majority of studies reviewed examined two specific recommendations: height and surfacing type; however, there are eight other key recommendations in the CSA-Z614-14 (for example, user age groups and inspection and maintenance) that lack review of effectiveness.
Based on the results of this review, playground designs that include play equipment and structures must comply with playground standards given the evidence to support the reduction of serious injury; however, there may be a case to rethink playground design to support a healthy and safe environment for play. In completing this systematic review, several studies were found that examined and reported on local and national compliance to playground standards. All of the studies found reported a significant lack of compliance (CDC, Centers for Disease Control and Prevention 1999; Acik et al. 2004; Allen et al. 2013; Chalmers et al. 2001; Cradock et al. 2010; Hudson et al. 2008; Kotch et al. 2003; Martin and Cooper 2005; Pickett et al. 1996; Sherker and Ozanne-Smith 2004; Sherker et al. 2009; Uskun et al. 2008). For example, in the study by Pickett et al. (1996), playgrounds in the Kingston, Ontario, area were investigated for adherence to CSA standards using a standardized checklist. Few playgrounds in the survey met the accepted safety standards, for both the equipment and ground surface material below and around the equipment.
The lack of concordance between supporting CSA playground equipment standards and meeting their recommendations may be due to several factors. One may include the absence of enforcement mechanisms. Currently, adherence to CSA standards for playgrounds in Canada is voluntary (Canadian Standards Association 2015); however, there are some exceptions, including provincially licensed daycare centres. Most of the compliance studies reviewed cite the lack of a coordinated approach, time, funding, and personnel to support meeting and maintaining safety standards. Thus, the burden that the recommendations place on playground providers may also be a reason for non-compliance. Moving toward nature-based play spaces, spaces that allow children to play in an environment without play equipment and structures, can provide the necessary benefits associated with outdoor play, as well as reduce barriers to compliance for playground safety. This is supported by Tremblay et al. (2015), as well as Brussoni et al. (2015), who report the significant health benefits from nature-based play spaces, including more self-directed play (Brussoni et al. 2015; Tremblay et al. 2015). In addition, these reviews report that the majority of injuries that occur in outdoor play spaces are minor injuries, requiring minimal or no medical treatment (Brussoni et al. 2015).
Strengths and limitations
There are several strengths of this review. This is a rigorous systematic review of the risk and protective factors and interventions to reduce injury in playgrounds. Second, two reviewers critically appraised the included studies for internal and external validity using a previously validated and widely used quality appraisal tool. In addition, all applicable articles, regardless of significance and publication status, were considered for review. Finally, articles examining compliance with playground standards were also found while completing the systematic searches. These articles, although outside of the scope of this review, gave context to the discussion on compliance with recommended standards. There are, however, some limitations to this review. First, only published articles were included, thus increasing the potential for publication bias. We included search of non-published studies in the grey literature; however, we were unable to find unpublished studies or data (including published abstracts without full-text publications). In addition, only articles in English were considered for inclusion, thus potentially decreasing the breadth of studies included. Of the articles systematically searched and included, however, reviewers also scanned reference lists as well as a number of grey literature sources, including playground-specific resource websites. Other limitations of this review lie within the primary studies themselves. Of most significance was the lack of the calculation of risk based on exposure data and the lack of analytical control for confounding. Exposure-based risk calculations provide a more accurate representation of the true risk of injury, per child playing in a playground or per hour of play. Given exposure may be an independent predictor of playground injury, lack of these data may have led to erroneous associations. The two studies that considered exposure in their modelling were either unable to use the measures in the final model (Laforest et al. 2001) or used a binary variable not able to represent risk of injury per hour of play, or the number of children playing (Laforest et al. 2000). The lack of control for confounding in the analysis presents the inability to make conclusions about the independence of the risk factor under study. These studies lack the strength to report associations confidently. For intervention studies, a significant limitation was the inability to conduct true experimental studies. It is impossible to randomize study groups to an intervention given the known risk factors for injury in playgrounds (particularly with playground equipment/structure height and playground surfacing). The ethical considerations for this type of intervention would exceed the impact of the results from a true experimental design. Research groups interested in conducting effectiveness studies in playground design should create collaborations with local and municipal playground practitioners and implementers in order to provide evaluation opportunities as they arise. An example includes the work undertaken by Howard et al. (2005), where a quasi-experimental design was used to evaluate the impact of changes to surfacing and height of playground equipment in school playgrounds that had previously decided on playground restructuring.
Conclusion
The results of this study support non-impact-absorbing surfacing, fall height, and the lack of handrails and guardrails as critical risk factors for serious injury in equipment-based playgrounds. Interventions that support changes to playground equipment to mitigate these risks were also supported by the existing literature. Less support was gathered for adult supervision, sex, and SES as risk factors for playground injuries. In addition, there was less evidence for adult supervision as an effective intervention for reducing risky behaviour or playground-related injury. Those responsible for providing playgrounds for children have an important role to create both engaging and safe play spaces. Future work should include investigation of nature-based over equipment-based playgrounds and its relationship to injury and health outcomes in children and youth.
Compliance with ethical standards
Conflict of interest
The authors declare that they have no competing interests.
References
- American Academy of Pediatrics, American Public Health Association, National Resource Center for Health and Safety in Child Care and Early Education. (2011). Caring for our children: national health and safety performance standards; guidelines for early care and education programs, 3rd edn. Elk Grove Village, IL: American Academy of Pediatrics; Washington, DC: American Public Health Association.
- (CDC), Centers for Disease Control and Prevention Playground safety—United States, 1998-1999. MMWR Recomm Rep. 1999;48(16):329–332. [PubMed] [Google Scholar]
- Acik Y, Gulbayrak C, Turaci CG. Investigation of the level of safety and appropriateness of playgrounds in Elazig city in Turkey. Int J Environ Health Res. 2004;14(1):75–82. doi: 10.1080/09063120310001633868. [DOI] [PubMed] [Google Scholar]
- Allen EM, Hill AL, Tranter E, Sheehan KM. Playground safety and quality in Chicago. Pediatrics. 2013;131(2):233–241. doi: 10.1542/peds.2012-0643. [DOI] [PubMed] [Google Scholar]
- Brussoni M, Gibbons R, Gray C, et al. What is the relationship between risky outdoor play and health in children? A systematic review. Int J Environ Res Public Health. 2015;12(6):6423–6454. doi: 10.3390/ijerph120606423. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Canadian Standards Association (2015). CAN/CSA-Z614-14. Children’s playspaces and equipment.
- Chalmers DJ, Marshall SW, Langley JD, et al. Height and surfacing as risk factors for injury in falls from playground equipment: a case-control study. Inj Prev. 1996;2(2):98–104. doi: 10.1136/ip.2.2.98. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chalmers DJ, Parry ML, Crawford AI, Wright CS. Compliance of Dunedin school playground equipment with the New Zealand playground standard. Aust N Z J Public Health. 2001;25(3):253–255. doi: 10.1111/j.1467-842X.2001.tb00572.x. [DOI] [PubMed] [Google Scholar]
- Chelvakumar G, Sheehan K, Hill AL, Lowe D, Mandich N, Schwebel DC. The stamp-in-safety programme, an intervention to promote better supervision of children on childcare centre playgrounds: an evaluation in an urban setting. Inj Prev. 2010;16(5):352–354. doi: 10.1136/ip.2009.025056. [DOI] [PubMed] [Google Scholar]
- Cradock A, O'Donnell E, Benjamin SE, Walker E, Slining M. A review of state regulations to promote physical activity and safety on playgrounds in child care centers and family child care homes. J Phys Act Health. 2010;7(Suppl 1):S108–S119. doi: 10.1123/jpah.7.s1.s108. [DOI] [PubMed] [Google Scholar]
- Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J Epidemiol Community Health. 1998;52(6):377–384. doi: 10.1136/jech.52.6.377. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fiissel D, Pattison G, Howard A. Severity of playground fractures: play equipment versus standing height falls. Inj Prev. 2005;11(6):337–339. doi: 10.1136/ip.2005.009167. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fisher L, Harris VG, VanBuren J, Quinn J, DeMaio A. Assessment of a pilot child playground injury prevention project in New York State. Am J Public Health. 1980;70(9):1000–1002. doi: 10.2105/AJPH.70.9.1000. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fuselli P, Yanchar NL. Preventing playground injuries. Paediatr Child Health. 2012;17(6):328. doi: 10.1093/pch/17.6.328. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Heck A, Collins J, Peterson L. Decreasing children’s risk taking on the playground. J Appl Behav Anal. 2001;34(3):349–352. doi: 10.1901/jaba.2001.34-349. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Herrington S, Nicholls J. Outdoor play spaces in Canada: the safety dance of standards as policy. Critical Social Policy. 2007;27(1):128–138. doi: 10.1177/0261018307072210. [DOI] [Google Scholar]
- Howard AW, MacArthur C, Willan A, Rothman L, Moses-McKeag A, MacPherson AK. The effect of safer play equipment on playground injury rates among school children. CMAJ. 2005;172(11):1443–1446. doi: 10.1503/cmaj.1041096. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Howard AW, Macarthur C, Rothman L, Willan A, Macpherson AK. School playground surfacing and arm fractures in children: a cluster randomized trial comparing sand to wood chip surfaces. PLoS Med. 2009;6(12):e1000195–e1000195. doi: 10.1371/journal.pmed.1000195. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hudson SD, Olsen HM, Thompson D. An investigation of school playground safety practices as reported by school nurses. J Sch Nurs. 2008;24(3):138–144. doi: 10.1177/1059840511223344. [DOI] [PubMed] [Google Scholar]
- Keays G, Skinner R. Playground equipment injuries at home versus those in public settings: differences in severity. Inj Prev. 2012;18(2):138–141. doi: 10.1136/injuryprev-2011-040240. [DOI] [PubMed] [Google Scholar]
- Kotch JB, Hussey JM, Carter A. Evaluation of North Carolina child care safety regulations. Inj Prev. 2003;9(3):220–225. doi: 10.1136/ip.9.3.220. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Laforest S, Robitaille Y, Dorval D, Lesage D, Pless B. Severity of fall injuries on sand or grass in playgrounds. J Epidemiol Community Health. 2000;54(6):475–477. doi: 10.1136/jech.54.6.475. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Laforest S, Robitaille Y, Lesage D, Dorval D. Surface characteristics, equipment height, and the occurrence and severity of playground injuries. Inj Prev. 2001;7(1):35–40. doi: 10.1136/ip.7.1.35. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Macarthur C, Hu X, Wesson DE, Parkin PC. Risk factors for severe injuries associated with falls from playground equipment. Accid Anal Prev. 2000;32(3):377–382. doi: 10.1016/S0001-4575(99)00079-2. [DOI] [PubMed] [Google Scholar]
- MacKay M. Playground injuries. Inj Prev. 2003;9:194. doi: 10.1136/ip.9.3.194. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Macpherson AK, Jones J, Rothman L, Macarthur C, Howard AW. Safety standards and socioeconomic disparities in school playground injuries: a retrospective cohort study. BMC Public Health. 2010;10:542–542. doi: 10.1186/1471-2458-10-542. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Martin J, Cooper CD. Playground safety in south western Sydney. J Paediatr Child Health. 2005;41(11):587–591. doi: 10.1111/j.1440-1754.2005.00727.x. [DOI] [PubMed] [Google Scholar]
- Morrongiello BA, House K. Measuring parent attributes and supervision behaviors relevant to child injury risk: examining the usefulness of questionnaire measures. Inj Prev. 2004;10(2):114–118. doi: 10.1136/ip.2003.003459. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Morrongiello BA, Kane A. An evaluation of the Cool 2 Be Safe program: an evidence-based community-disseminated program to positively impact children's beliefs about injury risk on playgrounds. Prev Sci. 2015;16(1):61–69. doi: 10.1007/s11121-014-0484-0. [DOI] [PubMed] [Google Scholar]
- Morrongiello BA, Mark L. “Practice what you preach”: induced hypocrisy as an intervention strategy to reduce children’s intentions to risk take on playgrounds. J Pediatr Psychol. 2008;33(10):1117–1128. doi: 10.1093/jpepsy/jsn011. [DOI] [PubMed] [Google Scholar]
- Morrongiello BA, Matheis S. Addressing the issue of falls off playground equipment: an empirically-based intervention to reduce fall-risk behaviors on playgrounds. J Pediatr Psychol. 2007;32(7):819–830. doi: 10.1093/jpepsy/jsm029. [DOI] [PubMed] [Google Scholar]
- Mott A, Evans R, Rolfe K, Potter D, Kemp KW, Sibert JR. Patterns of injuries to children on public playgrounds. Arch Dis Child. 1994;71(4):328–330. doi: 10.1136/adc.71.4.328. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mott A, Rolfe K, James R, et al. Safety of surfaces and equipment for children in playgrounds. Lancet (London, England) 1997;349(9069):1874–1876. doi: 10.1016/S0140-6736(96)10343-3. [DOI] [PubMed] [Google Scholar]
- Mowat DL, Wang F, Pickett W, Brison RJ. A case-control study of risk factors for playground injuries among children in Kingston and area. Inj Prev. 1998;4(1):39–43. doi: 10.1136/ip.4.1.39. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Norton C, Nixon J, Sibert J. Playground injuries to children. Arch Dis Child. 2004;89:103–108. doi: 10.1136/adc.2002.013045. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Olsen H, Hudson S, Thompson D. Strategies for playground injury prevention: an overview of a playground project. Am J Health Educ. 2010;41:187–192. doi: 10.1080/19325037.2010.10598861. [DOI] [Google Scholar]
- Park MJ, Baldwin K, Weiss-Laxer N, et al. Composite playground safety measure to correlate the rate of supracondylar humerus fractures with safety: an ecologic study. J Pediatr Orthop. 2010;30(2):101–105. doi: 10.1097/BPO.0b013e3181d07689. [DOI] [PubMed] [Google Scholar]
- Petridou E, Sibert J, Dedoukou X, Skalkidis I, Trichopoulos D. Injuries in public and private playgrounds: the relative contribution of structural, equipment and human factors. Acta Paediatr. 2002;91(6):691–697. doi: 10.1111/j.1651-2227.2002.tb03304.x. [DOI] [PubMed] [Google Scholar]
- Pickett W, Carr PA, Mowat DL, Chui A. Playground equipment hazards and associated injuries in Kingston and area. Can J Public Health. 1996;87(4):237–239. [PubMed] [Google Scholar]
- Roseveare CA, Brown JM, Barclay McIntosh J, Chalmers DJ. An intervention to reduce playground equipment hazards. Injury prevention : journal of the International Society for Child and Adolescent Injury Prevention. 1999;5(2):124–128. doi: 10.1136/ip.5.2.124. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sacks JJ, Holt KW, Holmgreen P, Colwell LS, Jr, Brown JM., Jr Playground hazards in Atlanta child care centers. Am J Public Health. 1990;80(8):986–988. doi: 10.2105/AJPH.80.8.986. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sacks JJ, Brantley MD, Holmgreen P, Rochat RW. Evaluation of an intervention to reduce playground hazards in Atlanta child-care centers. Am J Public Health. 1992;82(3):429–431. doi: 10.2105/AJPH.82.3.429. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Schwebel DC, Summerlin AL, Bounds ML, Morrongiello BA. The stamp-in-safety program: a behavioral intervention to reduce behaviors that can lead to unintentional playground injury in a preschool setting. J Pediatr Psychol. 2006;31(2):152–162. doi: 10.1093/jpepsy/jsj001. [DOI] [PubMed] [Google Scholar]
- Sherker S, Ozanne-Smith J. Are current playground safety standards adequate for preventing arm fractures? Med J Aust. 2004;180(11):562–565. doi: 10.5694/j.1326-5377.2004.tb06092.x. [DOI] [PubMed] [Google Scholar]
- Sherker S, Ozanne-Smith J, Rechnitzer G, Grzebieta R. Out on a limb: risk factors for arm fracture in playground equipment falls. Inj Prev. 2005;11(2):120–124. doi: 10.1136/ip.2004.007310. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sherker S, Ritchie J, Eager D, Dennis R. Soft landings: encouraging compliance with safety standards in local government authority playgrounds. Health Promot J Austr. 2009;20(1):31–36. doi: 10.1071/HE09031. [DOI] [PubMed] [Google Scholar]
- Tremblay MS, Gray C, Babcock S, et al. Position statement on active outdoor play. Int J Environ Res Public Health. 2015;12(6):6475–6505. doi: 10.3390/ijerph120606475. [DOI] [PMC free article] [PubMed] [Google Scholar]
- U.S. Consumer Product Safety Commission. (2010). Public playground safety handbook.
- Uskun E, Kisioglu AN, Altay T, Cikinlar R, Kocakaya A. Assessment of the current status of playground safety in the Midwestern region of Turkey: an effort to provide a safe environment for children. Turk J Pediatr. 2008;50(6):559–565. [PubMed] [Google Scholar]
- Vollman D, Witsaman R, Comstock RD, Smith GA. Epidemiology of playground equipment-related injuries to children in the United States, 1996-2005. Clin Pediatr (Phila) 2009;48(1):66–71. doi: 10.1177/0009922808321898. [DOI] [PubMed] [Google Scholar]


