Abstract
BACKGROUND
Childhood exposure to adverse experiences has been associated with adult asthma, smoking, sexually transmitted disease, obesity, substance use, depression, and sleep disturbances. Conceptualizing bullying as an adverse childhood experience, 2011 Youth Risk Behavior Survey (YRBS) data were used to examine the relationship between in-person and electronic bullying victimization among US high school students and health risk behaviors and conditions related to violence, substance use, sexual risk, overweight and physical activity, sleep, and asthma.
METHODS
Data were from the 2011 national YRBS among students who answered questions about in-person and electronic bullying (N=13,846). The YRBS is a biennial, nationally representative survey of students in grades 9–12 (overall response rate=71%). Logistic regression analyses, stratified by sex and controlling for race/ethnicity and grade, examined the association between bullying victimization and health risk behaviors or conditions.
RESULTS
Rates of victimization varied; 9.4% of students reported being bullied in-person and electronically, 10.8% only bullied in-person, 6.8% only electronically bullied, and 73.0% uninvolved. Bullying was associated with nearly all health risk behaviors and conditions studied.
CONCLUSION
Assessing the broad functioning and behaviors of victims of bullying could enable educators and health practitioners to intervene early and promote the long-term health of youth.
Keywords: adolescents, bullying, electronic bullying, adverse childhood experiences, health risk behaviors
Survey data, media reports, and legislation passed in the United States all suggest that a significant percentage of young people in the United States experience bullying victimization. The 2011 national Youth Risk Behavior Survey (YRBS) found that 20.1% of youth in grades 9–12 reported being bullied on school property and 16.2% reported being bullied electronically.1 Reports of bullying in the popular media and the number of laws passed related to bullying have increased national attention to this health issue. From 1999 to 2010 state legislatures in the United States enacted more than 120 bills introducing new or amending existing bullying statutes.2 These steps reflect awareness that bullying victimization can have a significant impact on young people’s health and well-being.
Prior research has examined the relationship between in-person bullying victimization and a limited range of physical and mental health risk behaviors. Being a victim of bullying is associated with depression3,4 and psychological distress,5 particularly when the bullying occurs in multiple arenas5 or when the perpetrator is the same in multiple contexts.6 Bullying victimization is also associated with poor social problem-solving skills;7 use of alcohol,8 cigarettes,8,9 and illicit drugs;9 and use of over-the-counter medication for psychosomatic symptoms.10,11 Young people who are the victims of more than one type of bullying (physical, electronic, or verbal) are more likely than those experiencing fewer types to be depressed, have medically attended injuries, and to use medicine for sleeping problems or nervousness.10
The purpose of this study was to examine the relationship between in-person and electronic bullying victimization among US high school students and health risk behaviors and conditions related to violence; cigarette, alcohol, and illicit drug use; sexual activity; overweight and physical activity; sleep; and asthma using data from the 2011 YRBS. A wide-range of health behaviors was included in this analysis because we hypothesized that for some youth, bullying victimization, a repeated exposure to violence, is an adverse and traumatic childhood experience that may mirror the effects of other prolonged or repeated violent childhood experiences. Although there is no direct evidence to support this hypothesis, there is evidence that childhood bullying victimization is associated with negative adult mental health outcomes including depressive disorders, anxiety disorders, generalized anxiety, panic disorder, and agoraphobia.12 In addition, adverse childhood experiences such as being the victim of physical, verbal, or sexual abuse; being exposed to substance abuse; witnessing domestic violence; and witnessing criminal behavior in the home are associated with adult asthma, smoking, sexually transmitted disease and early pregnancy, obesity, substance use, depression, sleep disturbances, and premature mortality.13,14 Research examining associations between trauma and health risk behaviors during childhood is less plentiful, but suggests a similar relationship as found with adults. For example, a study of children in Head Start programs found that children exposed to violence with high levels of traumatic stress had a 4-fold increased risk for asthma and gastrointestinal problems compared with other peers within the Head Start program.15 If bullying is a traumatic experience for young people, then it is reasonable to anticipate similar associations between being a victim of bullying and a variety of health risk behaviors.
We hypothesize based upon the prior evidence that any bullying victimization will be significantly associated with the health risk behaviors examined. The analysis of sex differences is exploratory. Prior work identified differences in the types of bullying victimization males versus females experience, with males more likely to experience physical bullying and girls more like to experience electronic bullying.10 However, because the bullying question in this study did not separate physical bullying from other types of in-person bullying such as verbal or relational bullying, we were unsure whether there would be differences by sex between in-person bullying and electronic bullying.
This article fills 2 gaps in the literature. First, there is little understanding of the independent associations between one type of bullying victimization and health risk behaviors and multiple types of bullying victimization, such as in-person and electronic, and health risk behaviors. Some have theorized that exposure to electronic victimization has more adverse effects than in-person bullying because it can happen at any time;16 others have said that electronic victimization differs little from other types of victimization, but is just a new mechanism for delivery.17 Still others have suggested that what is important is the number of ways one is victimized, with those experiencing multiple types of victimization having a greater likelihood of negative health behaviors.5,10 Second, this article examines the associations between these types of bullying victimization and a significantly wider range of health risk behaviors than has previously been examined.
METHODS
The national YRBS, developed by the Centers for Disease Control and Prevention (CDC), monitors 6 categories of priority health risk behaviors—unintentional injury and violence; tobacco use; alcohol and other drug use; sexual behaviors that contribute to unintended pregnancy and sexually transmitted diseases, including human immunodeficiency virus (HIV) infection; unhealthy dietary behaviors; and physical inactivity. In addition, the YRBS monitors the prevalence of overweight, obesity, and asthma. The national YRBS sampling strategies and the psychometric properties of the questionnaire have been described in detail elsewhere.18–20
Using independent, 3-stage cluster samples, the YRBS is administered biennially to a nationally representative sample of private and public school students in grades 9–12. Participation in the survey is anonymous and voluntary and local parental permission procedures are used. Students record their responses directly on a self-administered computer-scannable questionnaire or answer sheet. In 2011, the sample size was 15,425, the school response rate was 81%, the student response rate was 87%, and the overall response rate was 71%. Missing data were not statistically imputed.
The YRBS asked high school students whether they had been a victim of bullying on school property and whether they had been a victim of electronic bullying. Bullying was defined on the questionnaire as “when 1 or more students tease, threaten, spread rumors about, hit, shove, or hurt another student over and over again. It is not bullying when 2 students of about the same strength or power argue or fight or tease each other in a friendly way.” Students were asked, “During the past 12 months, have you ever been bullied on school property?” Response options were yes or no. Hereafter, this will be referred to as bullied in-person or in-person bullying. Students were also asked, “During the past 12months, have you ever been electronically bullied? (Include being bullied through e-mail, chat rooms, instant messaging, Web sites, or texting).” Response options were yes or no. Hereafter, this will be referred to as electronic bullying.
Data Analysis
This analysis was conducted among cases in which the student record included an answer for both the in-person bullying and electronic bullying questions (N=13,846). Responses were classified into 1 of 4 mutually exclusive bullying categories: had been both bullied in-person and electronically bullied (referred to hereafter as both kinds of bullying) (N=1122), had been bullied in-person but not electronically bullied (N=1372), had been electronically bullied but not bullied in-person (N=935), and had been neither bullied in-person nor electronically bullied (referred to hereafter as uninvolved) (N=10,417). To examine differences in the prevalence of the 4 bullying categories by sex, race/ethnicity, and grade, t-tests were used for pairwise comparisons. Because prior work identified gender differences in the types of bullying victimization males and females experience,10 this study presents data stratified by sex. Logistic regression analyses that controlled for race/ethnicity and grade were used to examine the relationship between the 4 bullying categories and a variety of health risk behaviors and conditions among female and male students.
A weight based on students’ sex, race/ethnicity, and grade was applied to each record to adjust for school and student nonresponse and oversampling of Black and Hispanic students, making weighted estimates nationally representative. SUDAAN version 9.0.1 (Research Triangle Institute, Research Triangle Park, North Carolina), which accounts for weighted data and the complex multistage cluster sample design of the survey, was used for all data analysis. Alpha was set at p < .05.
RESULTS
The prevalence of having been a victim of bullying in-person (20.1%) and electronic bullying (16.2%) has been previously reported.18 This analysis found that 9.4% of high school students had been victims of both kinds of bullying, 10.8% had been only bullied in-person, 6.8% had been only electronically bullied, and 73.0% were uninvolved (Table 1). Students’ experience with being bullied in-person or electronically bullied varied by demographic characteristics. For example, the prevalence of being a victim of both kinds of bullying and having been only electronically bullied were higher among females than males, but the prevalence of having been only bullied in-person was higher among males than females. The prevalence of each type of bullying was higher among White and Hispanic students than Black students and, in addition, the prevalence of being a victim of both kinds of bullying was higher among White students than Hispanic students. Differences by grade were also identified. While in-person bullying only was most prevalent at the lowest grade levels, there were no differences by grade for electronic bullying only.
Table 1.
Prevalence of Having Been Bullied In-Person* and Electronically Bullied† Among US High School Students, by Sex, Race/Ethnicity, and Grade—Youth Risk Behavior Survey, 2011
| Both Kinds of Bullying*,† (N = 1122§) % (95% CI‖) |
In-Person Bullied Only (N = 1372) % (95% CI) |
Electronically Bullied Only (N = 935) % (95% CI) |
Uninvolved With Bullying‡ (N = 10,417) % (95% CI) |
|
|---|---|---|---|---|
| Total | 9.4 (8.6–10.3) | 10.8 (9.8–11.8) | 6.8 (6.3–7.5) | 73.0 (71.6–74.3) |
| Sex | ||||
| Female | 13.0 (11.9–14.2‖ | 9.2 (8.2–10.2)‖ | 9.1 (8.3–10.0)‖ | 68.7 (67.1–70.3)‖ |
| Male | 6.0 (4.9–7.4) | 12.2 (10.9–13.6) | 4.7 (4.0–5.5) | 77.0 (75.3–78.6) |
| Race/ethnicity | ||||
| White | 11.5 (10.3–12.9)¶,# | 11.5 (10.3–12.9)¶ | 7.1 (6.2–8.0)¶ | 69.9 (68.2–71.5)¶,# |
| Black | 3.9 (3.0–5.2)** | 7.5 (5.9–9.5)** | 5.0 (4.0–6.1)** | 83.5 (81.1–85.7)** |
| Hispanic | 7.1 (6.0–8.4) | 10.4 (8.8–12.2) | 6.4 (5.5–7.5) | 76.1 (73.4–78.7) |
| Grade | ||||
| 9th | 9.3 (8.0–10.7) | 15.0 (13.3–16.9)††,# | 6.2 (5.2–7.4) | 69.5 (67.3–71.7)‡‡,§§ |
| 10th | 11.6 (9.6–13.9)††, ¶¶, ‖‖ | 11.1 (9.6–12.7)‖‖,¶¶ | 6.6 (5.4–7.9) | 70.8 (68.5–73.0)‖‖,¶¶ |
| 11th | 8.7 (7.0–10.6) | 8.6 (7.3–10.0) | 7.3 (6.2–8.6) | 75.5 (72.3–78.4) |
| 12th | 7.8 (6.6–9.2) | 7.5 (6.2–9.0) | 7.2 (6.2–8.3) | 77.6 (75.5–79.5) |
CI, confidence interval.
During the 12 months before the survey, were bullied on school property.
During the 12 months before the survey, were electronically bullied (including through e-mail, chat rooms, instant messaging, Web sites, or texting).
Neither bullied in-person nor electronically bullied.
Ns are unweighted.
Significant difference between male and female students (p <.05).
Significant difference between White and Black students (p < .05).
Significant difference between White and Hispanic students (p < .05).
Significant difference between Black and Hispanic students (p < .05).
Significant difference between 9th- and 10th-grade students (p < .05).
Significant difference between 9th- and 11th-grade students (p < .05).
Significant difference between 9th- and 12th-grade students (p < .05).
Significant difference between 10th- and 11th-grade students (p < .05).
Significant difference between 10th- and 12th-grade students (p < .05).
Violence-Related Factors
Among both female and male students, being a victim of bullying was positively associated with all of the other violence-related variables examined in this study, with some notable differences for different kinds of bullying victimization (Table 2). Having been only bullied in-person was associated with being injured in a physical fight and with dating violence victimization among male but not female students. Having been only bullied in-person was associated with carrying a weapon on school property and with being the victim of forced sexual intercourse among female but not male students. Having been only electronically bullied was associated with having carried a weapon on school property among male but not female students.
Table 2.
Prevalence and Adjusted Odds Ratios (AOR) for Violence-Related Risks Among US High School Students, by Bullied In-Person and Electronically Bullied Status—Youth Risk Behavior Survey, 2011
| Female | Male | |||
|---|---|---|---|---|
| % (95% CI) | AOR† (95% CI) | % (95% CI) | AOR† (95% CI) | |
| In a physical fight‡ | ||||
| Both kinds of bullying‡, ‖ | 38.1 (33.8–42.6) | 2.8 (2.2–3.4)** | 55.7 (46.3–64.6) | 2.1 (1.4–3.2)** |
| In-person bullied only | 29.6 (25.5–34.0) | 1.7 (1.3–2.1)** | 45.7 (40.4–51.2) | 1.3 (1.0–1.7)* |
| Electronically bullied only | 32.7 (27.2–38.8) | 2.1 (1.6–2.9)** | 56.5 (50.5–62.4) | 2.1 (1.6–2.7)** |
| Uninvolvedwith bullying¶ | 20.0 (18.1–21.9) | 1.0 | 37.8 (36.2–39.5) | 1.0 |
| Injured in a physical fight# | ||||
| Both kinds of bullying | 6.4 (4.9–8.5) | 4.8 (3.2–7.2)** | 14.0 (9.5–20.2) | 5.0 (2.9–8.5)** |
| In-person bullied only | 2.4 (1.3–4.4) | 1.6 (0.8–3.2) | 6.9 (5.4–8.9) | 2.1 (1.5–2.9)** |
| Electronically bullied only | 4.8 (2.7–8.3) | 3.5 (1.8–6.8)** | 10.9 (7.6–15.5) | 2.8 (1.8–4.3)** |
| Uninvolvedwith bullying | 1.6 (1.2–2.0) | 1.0 | 3.6 (3.0–4.4) | 1.0 |
| In a physical fight on school property‡ | ||||
| Both kinds of bullying | 15.4 (12.6–18.7) | 3.5 (2.5–4.9)** | 32.8 (26.0–40.5) | 3.5 (2.4–4.9)** |
| In-person bullied only | 9.6 (7.2–12.7) | 1.8 (1.3–2.6)** | 21.2 (17.8–25.0) | 1.6 (1.3–2.1)** |
| Electronically bullied only | 11.8 (8.4–16.3) | 2.5 (1.7–3.7)** | 25.2 (20.2–31.0) | 2.3 (1.7–3.1)** |
| Uninvolved with bullying | 5.6 (4.8–6.6) | 1.0 | 13.4 (12.3–14.6) | 1.0 |
| Carried a weapon on school property†† | ||||
| Both kinds of bullying | 5.9 (4.5–7.7) | 4.1 (2.5–6.4)** | 19.3 (14.2–25.5) | 3.2 (2.2–4.6)** |
| In-person bullied only | 2.7 (1.5–4.6) | 1.9 (1.0–3.5)* | 10.0 (7.2–13.7) | 1.4 (0.9–2.2) |
| Electronically bullied only | 1.9 (0.8–4.3) | 1.2 (0.5–3.1) | 12.5 (8.6–17.9) | 1.9 (1.1–3.1)* |
| Uninvolved with bullying | 1.6 (1.2–2.0) | 1.0 | 6.9 (5.7–8.3) | 1.0 |
| Dating violence‡‡ | ||||
| Both kinds of bullying | 17.4 (14.5–20.8) | 3.5 (2.7–4.5)** | 24.4 (17.9–32.4) | 4.8 (3.5–6.7)** |
| In-person bullied only | 7.8 (5.5–11.0) | 1.3 (0.9–2.0) | 11.0 (8.0–14.9) | 1.9 (1.3–2.7)** |
| Electronically bullied only | 18.4 (14.5–22.9) | 3.4 (2.5–4.5)** | 19.8 (14.9–25.9) | 3.1 (2.1–4.5)** |
| Uninvolved with bullying | 6.8 (5.6–8.2) | 1.0 | 6.9 (6.1–7.9) | 1.0 |
| Forced to have sexual intercourse when they did not want to | ||||
| Both kinds of bullying | 23.7 (19.6–28.2) | 3.8 (2.7–5.3)** | 13.8 (8.9–20.7) | 6.0 (3.9–9.3)** |
| In-person bullied only | 14.9 (11.3–19.4) | 2.3 (1.7–3.1)** | 4.1 (2.5–6.5) | 1.5 (0.9–2.3) |
| Electronically bullied only | 19.1 (16.3–22.3) | 2.8 (2.0–3.7)** | 13.9 (10.0–18.9) | 5.0 (3.3–7.7)** |
| Uninvolved with bullying | 8.1 (6.7–9.7) | 1.0 | 2.9 (2.3–3.5) | 1.0 |
| Attempted suicide‡ | ||||
| Both kinds of bullying | 22.6 (19.4–26.0) | 4.9 (3.8–6.4)** | 17.9 (14.0–22.8) | 6.8 (4.8–9.7)** |
| In-person bullied only | 12.1 (9.3–15.6) | 2.1 (1.5–2.7)** | 7.7 (5.5–10.6) | 2.4 (1.6–3.6)** |
| Electronically bullied only | 14.8 (11.3–19.1) | 2.9 (2.0–4.2)** | 14.8 (9.9–21.5) | 4.7 (3.0–7.4)** |
| Uninvolved with bullying | 6.1 (5.3–7.0) | 1.0 | 3.4 (2.8–4.1) | 1.0 |
| Suicide attempt treated by a doctor or nurse§§ | ||||
| Both kinds of bullying | 6.8 (4.8–9.5) | 4.6 (2.9–7.3)** | 3.9 (2.0–7.4) | 3.2 (1.5–6.4)** |
| In-person bullied only | 2.8 (1.2–6.4) | 1.7 (0.7–4.2) | 1.8 (1.0–3.4) | 1.5 (0.8–2.8) |
| Electronically bullied only | 4.8 (2.8–8.0) | 3.2 (1.6–6.4)** | 6.7 (4.1–10.9) | 4.6 (2.5–8.7)** |
| Uninvolved with bullying | 1.6 (1.3–2.1) | 1.0 | 1.3 (0.9–1.7) | 1.0 |
p<.05;
p<.01.
CI, confidence interval.
Odds ratio adjusted for race/ethnicity and grade.
One or more times during the 12 months before the survey.
During the 12 months before the survey, were bullied on school property.
During the 12 months before the survey, were electronically bullied (including through e-mail, chat rooms, instant messaging, Web sites, or texting).
Neither bullied in-person nor electronically bullied.
One or more times during the 12 months before the survey, had injuries that had to be treated by a doctor or nurse.
Such as a gun, knife, or club on at least 1 day during the 30 days before the survey.
Hit, slapped, or physically hurt on purpose by their boyfriend or girlfriend during the 12 months before the survey.
During the 12 months before the survey.
Weight-Related and Physical Activity Factors
Being a victim of bullying was not associated with being overweight as measured by self-reported height and weight (or with being obese, data not shown); however, perceived overweight was associated with having been a victim of both kinds of bullying among female students and associated with having been only bullied in-person among both female and male students (Table 3). Unhealthy weight control practices were associated with all categories of bullying victimization for both female and male students.
Table 3.
Prevalence and Adjusted Odds Ratios (AOR) for Weight-Related and Physical Activity Factors Among US High School Students, by Bullied In-Person and Electronically Bullied Status—Youth Risk Behavior Survey, 2011
| Female | Male | |||
|---|---|---|---|---|
| % (95% CI) | AOR†(95% CI) | % (95% CI) | AOR†(95% CI) | |
| Overweight‡ | ||||
| Both kinds of bullying§, ‖ | 16.4 (13.8–19.3) | 1.3 (1.0–1.6) | 13.2 (9.9–17.3) | 0.8 (0.6–1.1) |
| In-person bullied only | 16.8 (13.7–20.5) | 1.2 (0.9–1.7) | 14.8 (12.3–17.7) | 0.9 (0.8–1.2) |
| Electronically bullied only | 15.7 (12.1–20.2) | 1.2 (0.8–1.6) | 15.1 (10.7–21.0) | 1.0 (0.7–1.4) |
| Uninvolved with bullying¶ | 14.6 (13.1–16.4) | 1.0 | 15.3 (14.3–16.3) | 1.0 |
| Describe themselves as overweight | ||||
| Both kinds of bullying | 38.9 (34.5–43.4) | 1.3 (1.0–1.7)* | 26.6 (23.0–30.6) | 1.2 (1.0–1.5) |
| In-person bullied only | 39.8 (35.3–44.5) | 1.4 (1.1–1.7)** | 29.8 (25.5–34.4) | 1.5 (1.2–1.8)** |
| Electronically bullied only | 36.0 (30.1–42.3) | 1.1 (0.9–1.5) | 24.3 (18.8–30.7) | 1.1 (0.8–1.6) |
| Uninvolved with bullying | 33.4 (31.2–35.8) | 1.0 | 22.8 (21.4–24.2) | 1.0 |
| Unhealthy weight control practices# | ||||
| Both kinds of bullying | 40.4 (36.3–44.6) | 3.5 (2.9–4.3)** | 23.0 (16.9–30.4) | 3.5 (2.4–5.0)** |
| In-person bullied only | 28.9 (25.2–32.8) | 2.1 (1.6–2.6)** | 14.4 (11.6–17.8) | 2.0 (1.5–2.7)** |
| Electronically bullied only | 33.1 (28.2–38.2) | 2.5 (2.0–3.1)** | 23.0 (17.5–29.7) | 3.2 (2.3–4.4)** |
| Uninvolved with bullying | 16.9 (15.5–18.3) | 1.0 | 8.2 (7.3–9.2) | 1.0 |
| Watched television ≥3 hours/day†† | ||||
| Both kinds of bullying | 29.3 (25.2–33.7) | 1.1 (0.8–1.4) | 38.7 (33.5–44.1) | 1.5 (1.1–1.9)** |
| In-person bullied only | 31.5 (26.6–36.7) | 1.0 (0.8–1.4) | 35.8 (31.5–40.3) | 1.3 (1.1–1.6)* |
| Electronically bullied only | 30.4 (25.5–35.8) | 1.1 (0.8–1.4) | 29.5 (22.7–37.4) | 0.9 (0.6–1.4) |
| Uninvolved with bullying | 31.8 (29.2–34.5) | 1.0 | 32.4 (30.1–34.7) | 1.0 |
| Used computers ≥3 hours/day‡‡ | ||||
| Both kinds of bullying | 33.1 (28.7–37.8) | 1.6 (1.2–2.1)** | 47.5 (40.8–54.3) | 1.8 (1.4–2.3)** |
| In-person bullied only | 29.5 (23.2–36.8) | 1.3 (0.9–1.8) | 36.8 (32.1–41.8) | 1.2 (0.9–1.4) |
| Electronically bullied only | 27.0 (23.2–31.2) | 1.2 (0.9–1.4) | 47.4 (39.2–55.8) | 1.9 (1.3–2.7)** |
| Uninvolved with bullying | 25.3 (22.9–27.8) | 1.0 | 33.8 (31.7–35.9) | 1.0 |
| Physically active for at least 60 minutes/day on 0 of the past 7 days§§ | ||||
| Both kinds of bullying | 16.6 (14.0–19.6) | 1.1 (0.8–1.4) | 13.9 (10.0–19.1) | 1.7 (1.2–2.6)** |
| In-person bullied only | 18.2 (14.6–22.4) | 1.2 (0.9–1.6) | 9.6 (7.1–12.8) | 1.1 (0.8–1.5) |
| Electronically bullied only | 18.5 (14.2–23.8) | 1.1 (0.8–1.6) | 10.8 (7.7–15.1) | 1.2 (0.8–1.8) |
| Uninvolved with bullying | 17.7 (16.1–19.5) | 1.0 | 9.2 (8.1–10.4) | 1.0 |
p < .05;
p < .01.
CI, confidence interval.
Odds ratio adjusted for race/ethnicity and grade.
Students who were ≥85th percentile but <95th percentile for body mass index, by age and sex, based on reference data.
During the 12 months before the survey, were bullied on school property.
During the 12 months before the survey, were electronically bullied (including through e-mail, chat rooms, instant messaging, Web sites, or texting).
Neither bullied in-person nor electronically bullied.
Took diet pills, powders, or liquids; vomited or took laxatives; or did not eat for 24 or more hours to lose weight or to keep from gaining weight, during the 30 days before the survey.
On an average school day.
Played video or computer games or used a computer for something that was not school work.
Were physically active doing any kind of physical activity that increased their heart rate and made them breathe hard some of the time for at least 60 minutes on 0 of the 7 days before the survey.
Among male but not female students, having been a victim of both kinds of bullying, as well as having been only in-person bullied, was associated with watching television 3 or more hours per day. While having been a victim of both kinds of bullying was positively associated with using computers 3 or more hours per day among female and male students, having been electronically bullied only also was associated with computer use among male students. Being physically active for at least 60minutes/day on 0 of the past 7 days was associated with having been a victim of both kinds of bullying among male students but not female students.
Sexual Risk Factors
Among female students, having had sexual intercourse with 4 or more people during their lifetime was associated with being a victim of both kinds of bullying as well as having been only electronically bullied (Table 4). This relationship also was true among male students; however, among male students, being in-person bullied only was negatively associated with having had sexual intercourse with 4 or more persons during their life. Having been a victim of both kinds of bullying was associated with failing to use a condom at last sexual intercourse among both female and male students.
Table 4.
Prevalence and Adjusted Odds Ratios (AOR) for Sexual Risk Factors, Substance Use, Sleep, and Current Asthma Among US High School Students, by Bullied In-Person and Electronically Bullied Status—Youth Risk Behavior Survey, 2011
| Female | Male | |||
|---|---|---|---|---|
| % (95 % CI) | AOR† (95% CI) | % (95% CI) | AOR† (95% CI) | |
| Had sexual intercourse with 4 or more persons during their life | ||||
| Both kinds of bullying‡, § | 18.3 (15.2–21.8) | 2.2 (1.7–2.8)** | 23.2 (16.4–31.7) | 1.7 (1.1–2.4)* |
| In-person bullied only | 8.7 (6.0–12.6) | 1.0 (0.6–1.6) | 9.7 (7.7–12.0) | 0.6 (0.5–0.8)** |
| Electronically bullied only | 20.5 (15.5–26.8) | 2.2 (1.5–3.4)** | 22.8 (17.1–29.7) | 1.5 (1.0–2.3)* |
| Uninvolved with bullying‖ | 10.9 (9.3–12.6) | 1.0 | 18.2 (16.5–20.0) | 1.0 |
| Currently sexually active¶ | ||||
| Both kinds of bullying | 45.1 (39.9–50.4) | 2.2 (1.7–2.9)** | 38.2 (29.8–47.3) | 1.3 (0.9–1.8) |
| In-person bullied only | 26.8 (22.8–31.3) | 1.0 (0.8–1.3) | 23.3 (20.2–26.7) | 0.7 (0.5–0.8)** |
| Electronically bullied only | 47.8 (41.7–53.9) | 2.2 (1.7–2.9)** | 37.2 (29.7–45.3) | 1.2 (0.9–1.8) |
| Uninvolved with bullying | 30.9 (28.4–33.6) | 1.0 | 34.4 (31.9–37.1) | 1.0 |
| Did not use a condom, among currently sexually active¶ | ||||
| Both kinds of bullying | 52.0 (45.3–58.7) | 1.4 (1.0–1.9)* | 47.2 (37.3–57.4) | 1.9 (1.3–2.9)** |
| In-person bullied only | 43.2 (33.9–53.1) | 1.0 (0.7–1.5) | 36.3 (28.2–45.1) | 1.3 (0.8–2.0) |
| Electronically bullied only | 48.6 (40.2–57.1) | 1.2 (0.8–1.7) | 40.7 (28.6–54.1) | 1.6 (0.9–2.5) |
| Uninvolved with bullying | 44.8 (41.7–47.9) | 1.0 | 30.3 (26.8–34.1) | 1.0 |
| Current cigarette use# | ||||
| Both kinds of bullying | 27.3 (23.1–32.0) | 2.3 (1.8–3.0)** | 33.7 (26.8–41.3) | 2.2 (1.6–3.0)** |
| In-person bullied only | 10.8 (8.1–14.2) | 0.8 (0.6–1.1) | 18.7 (15.8–22.0) | 1.0 (0.8–1.3) |
| Electronically bullied only | 27.0 (22.4–32.2) | 2.3 (1.7–3.2)** | 30.1 (24.6–36.3) | 1.7 (1.3–2.3)** |
| Uninvolved with bullying | 13.5 (11.7–15.6) | 1.0 | 19.3 (17.3–21.3) | 1.0 |
| Current alcohol use†† | ||||
| Both kinds of bullying | 48.3 (43.9–52.8) | 1.9 (1.5–2.3)** | 48.0 (38.5–57.6) | 1.5 (1.0–2.3) |
| In-person bullied only | 32.9 (28.7–37.3) | 1.0 (0.8–1.2) | 37.3 (32.5–42.3) | 1.0 (0.8–1.3) |
| Electronically bullied only | 54.9 (48.6–61.1) | 2.4 (1.8–3.1)** | 54.4 (48.1–60.5) | 1.9 (1.4–2.7)** |
| Uninvolved with bullying | 34.4 (32.1–36.9) | 1.0 | 38.5 (36.0–41.1) | 1.0 |
| Ever used illicit drugs‡‡ | ||||
| Both kinds of bullying | 44.9 (40.0–50.0) | 2.5 (1.9–3.2)** | 49.9 (41.6–58.1) | 2.5 (1.8–3.4)** |
| In-person bullied only | 29.4 (24.8–34.6) | 1.3 (1.1–1.6)* | 31.8 (28.7–35.1) | 1.2 (1.0–1.4) |
| Electronically bullied only | 42.4 (36.6–48.5) | 2.2 (1.7–2.8)** | 43.9 (38.5–49.5) | 1.9 (1.5–2.5)** |
| Uninvolved with bullying | 25.5 (22.7–28.6) | 1.0 | 29.6 (27.3–31.9) | 1.0 |
| Get fewer than 8hours of sleep on an average school night | ||||
| Both kinds of bullying | 76.3 (72.3–79.8) | 1.5 (1.2–1.9)** | 73.9 (67.7–79.3) | 1.6 (1.2–2.1)** |
| In-person bullied only | 67.3 (61.3–72.7) | 1.0 (0.8–1.4) | 65.8 (61.2–70.2) | 1.2 (0.9–1.5) |
| Electronically bullied only | 76.5 (72.1–80.5) | 1.4 (1.2–1.8)** | 73.0 (62.4–81.5) | 1.5 (0.9–2.5) |
| Uninvolved with bullying | 69.5 (67.4–71.6) | 1.0 | 65.4 (63.3–67.4) | 1.0 |
| Current asthma§§ | ||||
| Both kinds of bullying | 18.7 (15.6–22.2) | 1.6 (1.3–2.0)** | 12.8 (9.9–16.5) | 1.4 (1.1–1.9)* |
| In-person bullied only | 13.8 (10.0–18.6) | 1.2 (0.8–1.6) | 14.4 (11.9–17.4) | 1.7 (1.2–2.3)** |
| Electronically bullied only | 15.8 (11.8–21.0) | 1.3 (0.9–1.9) | 15.2 (10.3–21.9) | 1.9 (1.2–3.0)* |
| Uninvolved with bullying | 12.4 (10.9–14.1) | 1.0 | 9.2 (8.0–10.4) | 1.0 |
p < .05;
p < .01.
CI, confidence interval.
Odds ratio adjusted for race/ethnicity and grade.
During the 12 months before the survey, were bullied on school property.
During the 12 months before the survey, were electronically bullied (including through e-mail, chat rooms, instant messaging, Web sites, or texting).
Neither bullied in-person nor electronically bullied.
Had sexual intercourse with at least 1 person during the 3 months before the survey.
Smoked cigarettes on at least 1 day during the 30 days before the survey.
Had at least 1 drink of alcohol on at least 1 day during the 30 days before the survey.
Ever used cocaine, inhalants, heroin, methamphetamines, ecstasy, hallucinogenic drugs, or a prescription drug without a doctor’s prescription.
Ever told by a doctor or nurse that they had asthma and still have asthma.
Substance Use and Other Risk Factors
Having been a victim of bullying was associated with current cigarette use, current alcohol use, and having ever used illicit drugs (Table 4). The patterns were similar among female and male students except that among female, but not male students, having been a victim of both kinds of bullying was associated with current alcohol use, and having been bullied in-person only was associated with ever having used illicit drugs.
Having been a victim of both kinds of bullying was associated with getting fewer than 8 hours of sleep on an average school night among female and male students but among female students, having been only electronically bullied also was associated with getting fewer than 8 hours of sleep on an average school night. Having been a victim of both kinds of bullying was associated with current asthma for male and female students, but among male students, having been only bullied in-person or having been only electronically bullied also were associated with current asthma.
DISCUSSION
There has been an increasing recognition by the public that bullying is not a rite of passage but is a damaging and unacceptable behavior. However, prior work has focused primarily on the links between in-person bullying victimization and other violence, mental health, and substance use related behaviors. Relatively little is known about similar links with electronic bullying. This study replicates and significantly extends previous findings demonstrating that bullying does not occur in isolation, but is part of a constellation of risk behaviors or conditions, such as asthma, inadequate sleep, substance use, sexual risk behaviors, unhealthy weight control practices, and physical inactivity and related behaviors, as well as dating violence, suicide, and sexual violence. Further, the findings of this study and others13–15 also suggest support for our hypothesis that bullying victimization is an adverse childhood experience and a public health problem.
With few exceptions, being a victim of both in-person bullying and electronic bullying was associated with the risk behaviors examined in this study, even when being a victim of only one type of bullying was not. This is consistent with the existing literature suggesting that those who are victims of bullying both in school and electronically are at highest risk.5,10 However, there were some unique findings specific to the type of victimization. This study is the first to find that although those who are victims of bullying in any way are more likely to attempt suicide than those who are uninvolved in bullying, both males and females who were only bullied in-person were not any more likely than uninvolved youth to make a suicide attempt that required attention by a doctor or a nurse. Both males and females who were victims of electronic bullying only and who were victims of both kinds of bullying were more likely than uninvolved youth to make a suicide attempt that required attention by a doctor or a nurse. More research is needed to determine if this finding is consistent over time and with different populations and, if so, to understand why this may be the case. One possible explanation is the inescapability of electronic bullying, potentially prolonging the experience of victimization and publicizing it.
Ybarra et al. reported that young people who were bullied online and in-person by the same person were more likely to report distress (46%) compared with those who did not know the identity of the online perpetrator (18%), those who were bullied by different people online and in-person (15%), and those who were only bullied online (20%).6 In fact, this study found that among both female and male students, for some behaviors, electronic bullying only seemed more detrimental than in-person bullying only. Among female students, being a victim of electronic bullying only was associated with increased odds of being injured in a physical fight, dating violence, making a suicide attempt needing treatment by a doctor or nurse, having 4 or more sexual partners, being currently sexually active, current cigarette use, current alcohol use, and getting fewer than 8 hours of sleep each night. However, these findings were not significant for females who were victims of in-person bullying only. Males who were bullied electronically were more likely to report carrying a weapon on school property, being forced to have sexual intercourse, making a suicide attempt needing treatment by a doctor or a nurse, using a computer for 3 or more hours per day, currently using cigarettes or alcohol, or ever using illicit drugs; this was not true for males who were only victims of in-person bullying. One study found that nearly 48% of those who were bullied online did not know the identity of their perpetrator,20 so perhaps uncertainty about with whom or where the threat lies may influence the weapon-carrying behaviors found among male students who were victims of electronic bullying only. It is possible that the males who are bullied electronically spend more time online than their nonelectronically bullied counterparts. If so, this increased time may be one explanation for the increased likelihood of sexual victimization. Other studies have identified associations between time spent online, self-reported importance of these activities, and sexual solicitations (requests from someone online to engage in sexual activities or sexual talk or to give personal sexual information that were unwanted or, whether wanted or not, were made by an adult).21 These findings need further research to test these relationships.
Given the prevalence of bullying and the growing evidence that bullying is associated with other negative health outcomes, it is clear that bullying, including electronic bullying, is a public health problem. Preventing all forms of bullying requires the availability of effective prevention strategies. Few programs have been rigorously tested in the United States and those that have been evaluated in the United States have shown limited to no benefits.22–26 A recent meta-analysis concluded, “To be sure, the evidence is sufficiently strong to indicate that bullying interventions can be effective. At the same time, many programs appear to be ineffective.”25(p536) Meta-analyses of bullying prevention programs suggest promising approaches include whole-school interventions and approaches such as parent training and information, playground supervision, school conferences, work with peers, and classroom rules and management.24,25 To date, the effects of school-based bullying prevention programs on electronic bullying is unknown.
Limitations
The findings of this analysis should be considered in the context of several limitations. First, students were asked only about whether they had been a victim of in-person bullying on school property or electronically (location not specified), but not if they had been a perpetrator of bullying nor the frequency or severity of the bullying. The relationship between risk behaviors and being a perpetrator and the extent to which a dose-dependent relationship was present were not possible to analyze. Second, the YRBS examines behaviors among high school students; thus, results cannot be generalized to other age groups. In-person bullying is most prevalent among middle school age youth,20 so the relationship between bullying victimization and other risk behaviors in a younger population warrants exploration. Third, these data apply only to youth attending school and, therefore, are not representative of all persons in this age group. Nationwide, in 2009, of persons aged 16–17 years, approximately 4% were not enrolled in a high school program and had not completed high school.27 Fourth, the extent of underreporting or overreporting of behaviors cannot be determined, although YRBS questions generally demonstrate good test-retest reliability.18 Finally, the YRBS is a cross-sectional study and unable to determine temporal relationships between bullying victimization and risk behaviors.
Conclusion
The findings of this study have significant implications for practice. Mental health and health professionals in schools and other settings may consider assessing students who are victims of bullying for involvement in other types of violence, suicidality, and other health risk behaviors and conditions. In addition, given limited resources, in lieu of implementing both bullying prevention programs with mixed evidence of effectiveness and general violence prevention programs with significant evidence of effectiveness,28 schools and researchers could consider implementing and evaluating general violence prevention programs for impact on bullying behaviors and comprehensive programs that may have an impact on multiple types of health risk behaviors.
IMPLICATIONS FOR SCHOOL HEALTH
Because of the literature’s mixed results on the effectiveness of bullying prevention programs in the United States, program developers and implementers in school and other settings could consider whether it is more appropriate to implement a strategy focused solely on bullying prevention or to implement a broader, general youth violence prevention strategy, perhaps one that is focused on improving the school climate. Prior work suggests that the relationship between bullying and violent and nonviolent health risk behaviors is modified by protective factors such as connectedness to school, parental support and monitoring, and by depression.29,30 There is emerging evidence that some general youth violence prevention programs show promise in preventing bullying as well as other forms of violence.31–33 Alternatively, prevention and intervention efforts could address common risk and protective factors, such as substance use or parental monitoring, behaviors correlated with violence and sexual risk behaviors,29,34 Regardless of a school’s choice of intervention, school personnel as well as health professionals in community settings should be alert to the signs and symptoms of bullying and follow-up with counseling and referrals as appropriate.
Footnotes
Indicates that continuing education hours are available. Visit www.ashaweb.org and click on Continuing Education for more information.
Human Subjects Approval Statement
CDC’s Institutional Review Board approved the protocol for the national YRBS.
Disclaimer
The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.
Contributor Information
Marci Feldman Hertz, Email: MHertz@cdc.gov.
Sherry Everett Jones, Email: sce2@cdc.gov.
Lisa Barrios, Email: LBarrios@cdc.gov.
Corinne David-Ferdon, Email: CFerdon@cdc.gov.
Melissa Holt, Email: holtm@bu.edu.
REFERENCES
- 1.Centers for Disease Control and Prevention. Youth risk behavior surveillance, United States, 2011. MMWR Morb Mortal Wkly Rep. 2012;61(SS-#4):1–162. [PubMed] [Google Scholar]
- 2.U.S. Department of Education, Office of Planning, Evaluation and Policy Development, Policy and Program Studies Service. Washington DC.: 2011. [Accessed April 4, 2014]. Analysis of State Bullying Laws and Policies. Available at http://www2.ed.gov/rschstat/eval/bullying/state-bullying-laws/state-bullying-laws.pdf. [Google Scholar]
- 3.Kaltiala-Heino R, Rimpelä M, Marttunen M, Rimpelä A, Rantanen P. Bullying, depression, and suicidal ideation in Finnish adolescents: school survey. BMJ. 1999;319:348–351. doi: 10.1136/bmj.319.7206.348. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Wang J, Nansel TR, Iannotti RJ. Cyber and traditional bullying: differential association with depression. J Adolesc Health. 2011;48(4):415–417. doi: 10.1016/j.jadohealth.2010.07.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Schneider SK, O’Donnell L, Stueve A, Coulter R. Cyberbullying, school bullying, and psychological distress: a regional census of high school students. Am J Public Health. 2012;102(1):171–177. doi: 10.2105/AJPH.2011.300308. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Ybarra ML, Diener-West M, Markow D, Leaf PJ. Examining the overlap in internet harassment and school bullying: Implications for school intervention. J Adolesc Health. 2007;41:S42–S50. doi: 10.1016/j.jadohealth.2007.09.004. [DOI] [PubMed] [Google Scholar]
- 7.Haynie DL, Nansel T, Eitel P, et al. Bullies, victims, and bully/victims: distinct groups of at- risk youth. J Early Adolesc. 2001;21(1):29–49. [Google Scholar]
- 8.Nansel TR, Overpeck MD, Pilla RS, Ruan WJ, Simmons-Morton B, Scheidt D. Bullying behaviors among US youth: prevalence and association with psychosocial adjustment. JAMA. 2001;285(16):2094–2100. doi: 10.1001/jama.285.16.2094. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Niemelä S, Brunstein-Klomek A, Sillanmäki L, et al. Childhood bullying behaviors at age eight and substance use at age 18 among males. A nationwide prospective study. Addict Behav. 2011;36(3):256–260. doi: 10.1016/j.addbeh.2010.10.012. [DOI] [PubMed] [Google Scholar]
- 10.Wang J, Iannotti R, Luk JW, Nansel TR. Co-occurrence of victimization from five subtypes of bullying: physical, verbal, social exclusion, spreading rumors, and cyber. J Pediatr Psychol. 2010;35(10):1103–1112. doi: 10.1093/jpepsy/jsq048. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Due P, Hansen EH, Merlo J, Andersen A, Holstein BE. Is victimization from bullying associated with medicine use among adolescents? A nationally representative cross-sectional survey in Denmark. Pediatrics. 2007;120(1):110–117. doi: 10.1542/peds.2006-1481. [DOI] [PubMed] [Google Scholar]
- 12.Copeland WE, Wolke D, Angold A, Costello EJ. Adult psychiatric outcomes of bullying and being bullied by peers in childhood and adolescence. JAMA Psychiatry. 2013;70(4):419–426. doi: 10.1001/jamapsychiatry.2013.504. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Anda RF, Felitti VJ, Bremner JD, et al. The enduring affects of abuse and related adverse childhood experiences in adulthood. Eur Arch Psychiatry Clin Neurosci. 2006;256(3):174–186. doi: 10.1007/s00406-005-0624-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Brown DW, Anda RF, Tiemeier H, et al. Adverse childhood experiences and the risk of premature mortality. Am J Prev Med. 2009;37(5):389–396. doi: 10.1016/j.amepre.2009.06.021. [DOI] [PubMed] [Google Scholar]
- 15.Graham-Bermann SA, Seng J. Violence exposure and traumatic stress symptoms as additional predictors of health problems in high-risk children. J Pediatr. 2005;146(3):349–354. doi: 10.1016/j.jpeds.2004.10.065. [DOI] [PubMed] [Google Scholar]
- 16.Raskauskas J, Stoltz AD. Involvement in traditional and electronic bullying among adolescents. Dev Psychol. 2007;43(3):564–575. doi: 10.1037/0012-1649.43.3.564. [DOI] [PubMed] [Google Scholar]
- 17.Li Q. New bottle but old wine: a research of cyberbullying in schools. Comput Human Behav. 2007;23(4):1777–1791. [Google Scholar]
- 18.Brener ND, Kann L, McManus T, Kinchen SA, Sundberg EC, Ross JG. Reliability of the 1999 Youth Risk Behavior Survey questionnaire. J Adolesc Health. 2002;31(4):336–342. doi: 10.1016/s1054-139x(02)00339-7. [DOI] [PubMed] [Google Scholar]
- 19.Eaton DK, Kann L, Kinchen S, et al. Youth risk behavior surveillance - United States, 2011. MMWR Surveill Summ. 2012;61(4):1–162. [PubMed] [Google Scholar]
- 20.Kowalski RM, Limber SP. Electronic bullying among middle school students. J Adolesc Health. 2007;41(suppl 1):S22–S30. doi: 10.1016/j.jadohealth.2007.08.017. [DOI] [PubMed] [Google Scholar]
- 21.Mitchell KJ, Finkelhor D, Wolak J. Risk factors for and impact of online sexual solicitation of youth. JAMA. 2001;285(23):3011–3014. doi: 10.1001/jama.285.23.3011. [DOI] [PubMed] [Google Scholar]
- 22.Merrell KW, Gueldner BA, Ross SW, Isava DM. How effective are school bullying intervention programs? A meta-analysis of intervention research. Sch Psychol Q. 2008;23(1):26–42. [Google Scholar]
- 23.Ferguson CJ, San Miguel C, Kilburn JC, Sanchez P. The effectiveness of school-based anti-bullying programs: a meta-analytic review. Crim Justice Rev. 2007;32(4):401–414. [Google Scholar]
- 24.Vreeman RC, Carroll AE. A systematic review of school-based interventions to prevent bullying. Arch Pediatr Adolesc Med. 2007;161(1):78–88. doi: 10.1001/archpedi.161.1.78. [DOI] [PubMed] [Google Scholar]
- 25.Ttofi MM, Farrington DP. What works in preventing bullying: effective elements of anti-bullying programs. J Aggress Confl Peace Res. 2009;1(1):13–25. [Google Scholar]
- 26.Evans CBR, Fraser MW, Cotter KL. The effectiveness of school-based bullying prevention programs: a systematic review. Aggress Violent Behav. 2014;19(5):532–544. [Google Scholar]
- 27.Chapman C, Laird J, Ifill N, Kewal-Ramani A. Washington, DC: National Center for Education Statistics, US Department of Education; [Accessed April 5, 2012]. Trends in high school dropout and completion rates in the United States: 1972–2009 (NCES 2012-006) Available at http://nces.ed.gov/pubs2012/2012006.pdf. [Google Scholar]
- 28.Hahn R, Fuqua-Whitley D, Wethington H, et al. The effectiveness of universal school-based programs for the prevention of violent and aggressive behavior: a report on recommendations of the task force on community preventive services. MMWR Morb Mortal Wkly Rep. 2007;56(RR07):1–12. [PubMed] [Google Scholar]
- 29.Resnick MD, Ireland J, Borowsky I. Youth violence perpetration: What protects? What predicts? Findings from the National Longitudinal Study of Adolescent Health. J Adolesc Health. 2004;35(5):424.e1–424.e10. doi: 10.1016/j.jadohealth.2004.01.011. [DOI] [PubMed] [Google Scholar]
- 30.Duong J, Bradshaw C. Associations between bullying and engaging in aggressive and suicidal behaviors among sexual minority youth: the moderating role of connectedness. J Sch Health. 2014;84(10):636–645. doi: 10.1111/josh.12196. [DOI] [PubMed] [Google Scholar]
- 31.Waasdorp TE, Bradshaw CP, Leaf PJ. The impact of schoolwide positive behavioral interventions and supports on bullying and peer rejection: a randomized controlled effectiveness trial. Arch Pediatr Adolesc Med. 2012;166(2):149–156. doi: 10.1001/archpediatrics.2011.755. [DOI] [PubMed] [Google Scholar]
- 32.Hawkins JD, Kosterman R, Catalano RF, Hill KG, Abbott RD. Promoting positive adult functioning through social development intervention in childhood long-term effects from the Seattle social development project. Arch Pediatr Adolesc Med. 2005;159(5):25–31. doi: 10.1001/archpedi.159.1.25. [DOI] [PubMed] [Google Scholar]
- 33.Catalano RF, Berglund ML, Ryan JAM, Lonczak HS, Hawkins JD. Positive youth development in the United States: research findings on evaluations of positive youth development programs. [Accessed April 4, 2014];Prevention & Treatment. 2002 5(1):591–598. Available at http://psycnet.apa.org/journals/pre/5/1/15a.pdf. [Google Scholar]
- 34.Burrus B, Leeks KD, Sipe TA, et al. Person-to-person interventions targeted to parents and other caregivers to improve adolescent health: a community guide systematic review. Am J Prev Med. 2012;42(3):316–326. doi: 10.1016/j.amepre.2011.12.001. [DOI] [PubMed] [Google Scholar]