Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Jun 1.
Published in final edited form as: Health Educ Behav. 2020 Apr 12;47(3):402–411. doi: 10.1177/1090198120914250

School Climate as a Universal Intervention to Prevent Substance Use Initiation in Early Adolescence: A Longitudinal Study

Shay M Daily 1, Michael J Mann 2, Christa L Lilly 1, Thomas K Bias 1, Megan L Smith 2, Alfgeir L Kristjansson 1
PMCID: PMC7427833  NIHMSID: NIHMS1615331  PMID: 32281413

Abstract

Initiation of substance use often starts during adolescence, with tobacco and alcohol use frequently preceding the use of marijuana and other illicit drugs. Studies suggest that a positive school climate may prevent substance use while promoting healthy student behaviors. The purpose of this study was to determine the longitudinal associations between school climate and substance use initiation in a group of middle school students. Parallel latent growth curve modeling was used to examine changes among study variables longitudinally using a sample of 2,097 sixth-, seventh-, and eighth-grade students across 16 regional schools located in three counties in West Virginia. Results suggest that a positive school climate may prevent substance use initiation (β = −0.07 to −0.25, p < .01). However, perceptions of school climate decreased on their own over time (β = −0.28 to −0.66, p < .01). Furthermore, substance use initiation also increased as students grew older (β = 0.96 to 0.99, p < .01) and reduced the effects of school climate longitudinally (β = −0.07 to −0.24, p < .01). Early substance use initiation may be a warning sign of other underlying student issues and requires additional school support to foster student success. Findings suggest that a positive school climate may delay substance use initiation and promote school success. School climate may, therefore, be useful as an intervention to support school-based health promotion.

Keywords: middle school, high school, parallel latent growth model, substance use prevention

Initiation of substance use, defined as “the first use of a particular substance” (Substance Abuse and Mental Health Services Administration, 2018, p. 21), often starts during adolescence, with tobacco and alcohol use frequently preceding the use of marijuana and other illicit drugs (Miller & Hurd, 2017). Nationally, alcohol continues to be the most commonly initiated substance by adolescents (62%; Johnston et al., 2018). While combustible cigarette use is at an all-time low (9% among eighth graders and 27% among 12th graders), electronic cigarette (vaping) use appears to be increasing, with 13.5% of early adolescents and 37.7% of young adults reporting initiation of substances.

Preventing adolescent substance use is essential to reduce the likelihood of drug-related disorders and health problems in adulthood (Caetano et al., 2014). Research suggests that schools may play an important role in substance use prevention (Sigfusdottir et al., 2011). Studies also suggest that a safe and supportive school climate may ameliorate the likelihood of substance use initiation (Cornell & Huang, 2016). Furthermore, schools that foster a positive school climate have shown to encourage healthy student behaviors (Durlak et al., 2011). In a recent review, Michael et al. (2015) not only advocated the importance of school climate as a protective factor to prevent risky student behavior like substance use but also highlighted the shortage of comprehensive evidence of the importance of school climate, especially over time.

A school’s climate provides an important example of how a social-emotional environment can shape behavior (Allensworth et al., 2011). Although descriptions vary, our definition aligns with Cohen et al. (2009) who state, “School climate refers to the quality and character of school life. School climate is based on patterns of people’s experiences of school life and reflects norms, goals, values, interpersonal relationships, teaching and learning practices, and organizational structures” (p. 182). Furthermore, the tenets of school climate research emphasize concepts like teacher relationships, such as bonding with teachers and peers; order and safety, such as respect for school rules; academic engagement, such as a sense of accomplishment and emotional investment with school; and school connectedness, such as attachment and building social bonds to school (Wang & Degol, 2016).

A positive school climate is important because formal education often occurs during critical stages of growth and maturity when negative experiences during these stages may develop into cycles of deviant behavior that may affect not only school performance but also transition into adulthood (Mann et al., 2016). Studies have found that school climate may reduce substance use, while promoting healthy psychosocial development and well-being (Ruus et al., 2007). However, empirical attempts to explain the relationship between school settings and substance use have been drawn primarily from cross-sectional studies (Wang & Dishion, 2012). Currently, the literature remains unclear regarding school climate’s longitudinal association and potential impact on substance use initiation (Wang & Degol, 2016).

School-based substance use prevention strategies have demonstrated success at reducing initiation within a multitiered system of support (MTSS; OSEP Technical Assistance Center on Positive Behavioral Interventions and Supports, 2017). MTSS may be defined as “an evidence-based model of education that employs data-based problem-solving techniques to integrate academic and behavioral instruction and intervention” (Eagle et al., 2015, p. 161; Gamm et al., 2012, p. 4). Delivery of MTSS in schools often follows recommendations outlined by the Institute of Medicine’s prevention and treatment of mental disorders guidelines (National Research Council & Institute of Medicine, 2009). Tiered models, by design, use progressive levels of treatment to improve the likelihood of positive outcomes (Durlak et al., 2011). In addition, tiers are integrated to supplement and support one another as a collective whole (Macklem, 2011).

Universal (Tier 1) strategies in tiered models are designed so every child receives the benefit of an intervention simply by being in school (Wilson & Lipsey, 2007). Universal strategies are often implemented in school settings and focus on students’ social-emotional learning to prevent problems like substance use and other deviant behaviors (Stanis & Andersen, 2014). For students who are deemed as “higher risk,” selective (Tier 2) provides targeted group-based strategies such as supplemental instruction, while indicated (Tier 3) uses intensive individualized approaches such as one-on-one behavioral counseling (Domitrovich et al., 2010). Although, this study does not measure or address MTSS directly, the framework should be acknowledged as many school districts across the United States are implementing MTSS in various ways (March et al., 2016).

Universal models that integrate health and education are gaining ground among school districts (Lewallen et al., 2015). A shared goal of school climate research has been to empirically highlight the complementary impact of nonacademic factors to support desired student outcomes (Piscatelli & Lee, 2011). Although components of school-based frameworks are likely to improve student outcomes, integrating measures of school climate to stimulate actionable steps for school personnel presents some challenges (Mann et al., 2018). Models like MTSS often require substantial preparation to establish a coordinated system of leadership and monitoring of framework activities (Bradshaw et al., 2009). Furthermore, perceptions of school climate have been empirically shown to decrease longitudinally starting in early adolescence (Holfeld & Leadbeater, 2017). A study by Niehaus et al. (2012) demonstrated declines in school connectedness among Midwestern sixth-graders, noting males to be at higher risk than females. Improving aspects of school climate, therefore, seems practical as part of universal (Tier 1) strategies because of its inherent potential to improve positive characteristics of the learning environment that affects all students without much alteration to system-level change. Nevertheless, aside from a few notable exceptions (see Sugai & Horner, 2006), interventions meant to promote and sensibly explain school climate as a potential Tier 1 intervention are relatively nonexistent.

The purpose of this study was to determine the longitudinal associations between school climate and initiation of licit substances and marijuana across three waves in a sample of middle school students. We hypothesized that (1) perceptions of school climate (i.e., positive teacher relationships) would decrease over time, (2) self-reported substance use initiation would increase over time, and (3) positive perceptions of school climate would significantly change students’ self-reported substance use initiation over time.

Method

Participants, Procedures, and Handling of Missing Data

Annual data collections occurred from 2015 to 2017 using clustered sampling techniques from evenly distributed groups of students in sixth (37.8%), seventh (32.5%), and eighth (29.7%) grades across 16 regional schools from three counties in West Virginia. The three West Virginia counties represented a triangulation of diverse population characteristics ranging from families living in severe isolation/poverty to those living in modest privilege/affluence (Appalachian Regional Commission, 2011–2015). All aspects of each annual collection were approved by the host university’s institutional review board (Protocols 2015 #1406345394, 2016 #1406345394R002, and 2017 #1406345394R004). In line with best practices recommended by Chartier et al. (2008), a letter was sent to parents to provide an opportunity to exclude their children (parental opt out rate <1%). Surveys were administered by classroom teachers with oversight from a coordinator at each school to ensure delivery protocols. Participation was voluntary and made available to all students. Students were free to answer all or part of the survey and opt out at any time. For expanded details on data collection procedures, see Kristjansson et al. (2013).

In 2015 (T1), students at baseline provided 6,364 eligible observations (response rate = 82.6%). In 2016 (T2), students provided 6,336 observations (response rate = 82.0%). In 2017 (T3), students provided 6,278 observations (response rate = 81.3%). Data management and cleaning removed 507 observations due to a temporal violation of substance initiation from T1 to T3. Student data were then matched over time using a unique self-reported state administered identification number yielding a final sample of 2,097 retained cases (three observations per participant). With participant dropout rates being common in longitudinal studies, this level of attrition was anticipated (Gustavson et al., 2012). To ensure data quality and accuracy, omnibus tests under the unrestricted latent class indicator models (Little, 1988; Little & Rubin, 2002) for data missing completely at random were tested and shown to be nonsignificant (all ps > .05). Preliminary tests additionally demonstrated less than 5% missing on dependent variables using pairwise techniques on retained cases, which has been shown to produce stable estimates (Marshall et al., 2010). However, because of the inclusion of covariates, we chose a more conservative route and assumed missing to be a function of missing at random (Little & Rubin, 2002). Missing data were then handled using full information maximum likelihood estimation (Mazza et al., 2015), which yields unbiased estimates under missing at random hypotheses and is often equivalent to imputation techniques (Cham et al., 2017).

Measures

Substance Use Initiation.

Substance use initiation items asked participants: Have you ever “tried cigarette smoking, even just one or two puffs?,” “tried electronic cigarettes (e-cigarettes or vapors), even just one or two puffs?,” “had a drink of alcohol, other than a few sips?,” and “tried marijuana (also called weed or pot) or hashish (also called hash or hash oil)?” Response options for all substance use initiation variables were binary with “no” (coded 0) and “yes” (coded 1).

School Climate.

Students’ perceptions of school norms, practices, and relationships were measured using three subscales (positive student–teacher relationships, order and safety, and opportunities for student engagement, 20 items) from the School Climate Measure (SCM). The three subscales were selected based on previous studies that demonstrated robust psychometric support for the SCM (see Zullig et al., 2010; Zullig et al., 2014; Zullig et al., 2015) and measurement invariance among early adolescents (Daily et al., 2018). SCM items use a 5-point Likert-type scale with response options “strongly disagree” (coded 1) to “strongly agree” (coded 5). Higher scores indicate a positive perception of school climate.

Covariates

Study covariates were chosen based on a review of school climate literature and supported with citations.

Biological Sex.

Biological sex was assessed by asking respondents “Are you a boy or girl?” Male (coded 0) and female (coded 1) were represented as a dichotomous time-invariant covariate (see Marcenaro-Gutierrez et al., 2017).

Family Structure.

Respondents were asked to indicate their family structure using a multiresponse question, “Which of the following persons live in your home?” For analysis, student responses were dichotomized into “lives with both biological parents” (coded 1) and “other arrangements” (coded 0) and represented as a dichotomous time-invariant covariate (see O’Malley et al., 2015).

Maternal Education.

Maternal education was captured by asking students to select one of nine response options from a singular question, “What is the highest level of schooling your mother has completed?” Responses were pooled into categories to simplify analyses, “college graduate” (coded 3), “high school graduate” (coded 2), “less than high school” (coded 1), and “I don’t know” (coded 0), and represented as a nominal time-invariant covariate (see Holmes et al., 2018).

Data Analysis

Descriptive frequencies, central tendency measures, scale internal consistency, and confirmatory factor analysis estimates were analyzed using SAS 9.4 (SAS Institute, 2013). Parallel latent growth curve modeling (PLGM) was selected to examine changes in school climate subscales and substance use initiation while controlling for sex, family structure, and maternal education over time (Cheong et al., 2003). All PLGM analyses were performed in Mplus 8.0 (Muthén & Muthén, 2017) using a two-tailed distribution (p ≤ 05). Model specifications used the complex option to accommodate nonnormality and nonindependence of participants clustered in schools with robust standard errors and probit transformation (Masyn et al., 2014). Numerical integration was also used to account for latent binary and continuous dependent variable interactions between initial start points (intercepts) and growth trajectory (slopes). Latent substance use initiation intercept and slope residuals were constrained to 1 and 0 to support model convergence (Muthén & Muthén, 2017). Intercept and slope means represent between-person initial starting points and growth. Standardized latent regression path estimates (β) and standard errors (SE) signify predictive relationships and practical importance between school climate and substance use initiation over time. Model fits were determined using the deviance statistic (−2LL, free parameters), Akaike information criteria, and Bayesian information criteria. Last, we ran sensitivity analyses for group comparisons on a sample of middle school students who did not move into high school (n = 818) and a sample of high school students (n = 471) in ninth and 10th grade.

Results

School climate subscale mean scores ranged from 3.3 (SD = 0.9) to 3.7 (SD = 0.8) from T1 to T3, with all scales reporting acceptable internal consistency (α = .85–.94). Additionally, confirmatory factor analyses across T1 to T3 demonstrated excellent measurement fits and ranged as follows: χ2 = 929.17 to 950.61 (all df = 157, p = < .01), comparitive fit index (CFI) = 0.97 to 0.98, Tucker–Lewis index (TLI) = 0.96 to 0.97, standardized root mean residual (SRMR) = 0.02 to 0.03, root mean square error of approximation (RMSEA) = all 0.044 [± 0.041, 0.047]. Alcohol was reported as the most initiated substance ranging from 7.6% at T1 to 27.8% at T3. Electronic cigarettes (4.5% at T1 to 20% at T3), combustible cigarettes (4.1 % at T1 to 14.6% at T3), and marijuana (3.2% at T1 to 13.7% at T3) followed with similar trends. Additional descriptive statistics are reported in Table 1.

Table 1.

Sample Frequencies, Scale Means, Standard Deviation, and Reliability Coefficients, n = 2097

2015
 2016
 2017
Variable n % n % n %
Biological sex
  Female/male 1138/950 54.5/45.5
Race
  White/all other races 1817/280 86.7/13.3
Maternal education
  College graduation/high school graduation 696/615 34.2/30.2
  Less than high school/not sure 146/581 7.2/28.4
Family structure (lives with)
  Biological parents/other arrangement 1195/902 57.0/43.0
Ever tried e-cigarettes
  No/yes 1784/67 96.38/3.6 1626/145 91.8/8.2 1530/313 83.0/17.0
Ever tried cigarettes
  No/yes 1796/56 96.9/3.1 1670/107 93.9/6.1 1619/228 87.6/12.4
Ever drank alcohol
  No/yes 1740/194 88.8/11.2 1397/433 76.3/23.7 1397/433 76.3/23.7
Ever tried marijuana
  No/yes 1789/31 98.3/1.7 1668/67 96.1/3.9 1625/200 89.0/11.0
Scale variable M (SD) α M (SD) α M (SD) α
Student-teacher relationships 3.6 (0.8) .91 3.5 (0.9) .93 3.3 (0.9) .94
Order, safety, and discipline 3.8 (0.8) .85 3.6 (0.9) .88 3.5 (0.9) .90
Student engagement 3.8 (0.8) .86 3.7 (0.9) .88 3.6 (0.9) .89
Open in a new tab

Note. HS = high school. Missingness of observations due to pairwise techniques not reported. (Cronbach & Meehl, 1955).

All results for PLGMs are described in Tables 2 through 4, with a conceptual model depicted in Figure 1. For reporting parsimony and clarity, please refer to Tables 2 through 4 for between-person intercept and slope means. Table 2 reports associations between student-teacher relationships and substance use initiation. Student-teacher relationship growth estimates between intercepts and slopes across all substance use initiation models ranged β = −0.26 to −0.66 (SE = 0.09–0.10, all ps < .01). Substance initiation βs for students’ growth trajectories ranged from 0.92 to 0.99 (all SEs = 0.02, p < .01). Cross-lagged βs between student-teacher relationships on substance models ranged from −0.07 to −0.25 (SE = 0.01–0.04, all ps < .01). Cross-lagged βs for substance models on teacher relationships ranged from −0.16 to −0.20 (SE = 0.01–006, p < .01). Standardized covariance between intercepts ranged from −0.20 to −0.32 (SE = 0.05– 0.08, all ps < .01).

Table 2.

Parallel Latent Growth Model Intercepts, Slopes, and Unstandardized/Standardized Regression Estimates for Teacher Relationships, n = 2,022.

E-cigarettes
 SC1
 Cigarettes
 SC1
 Alcohol
 SC1
 Marijuana
 SC1
Parameter B (SE) B (SE) B (SE) B (SE) B (SE) B (SE) B (SE) B (SE)
Intercept −2.45 (.23)** 3.51 (.06)** −2.51 (.18)** 3.51 (.06)** −2.50 (.18)** 3.51 (.05)** −3.08 (.17)** 3.50 (.06)**
Slope 3.80 (.42)** 0.12 (.10) 3.68 (.33)** 0.11 (.11) 3.56 (.30)** 0.10 (.14) 3.83 (.23)** 0.58 (.09)**
Int.→ slope 1.42 (.20)** −0.12 (.04)** 1.55 (.06)** −0.11 (.04)** 1.13 (.08)** −0.11 (.04)* 0.96 (.16)** −0.29 (.02)**
SC1 × drug −0.05 (.01)** −0.22 (.04)** −0.05 (.02)** −0.19 (.03)** −0.04 (.02)* −0.18 (.03)** −0.09 (.02)** −0.28 (.06)**
Int. ↔ int. −0.15 (.03)** −0.17 (.04)** −0.19 (.03)** −0.20 (.06)**
Std. est. β (SE) β (SE) β (SE) β (SE) β (SE) β (SE) β (SE) β (SE)
Int.→ slope 0.99 (.02)** −0.28 (.09)** 0.99 (.02)** −0.29 (.09)** 0.96 (.02)** −0.26 (.10)** 0.93 (.02)** −0.66 (.04)**
SC1 × drug −0.18 (.04)** −0.10 (.02)** −0.20 (.07)** −0.07 (.01)** −0.16 (.06)* −0.10 (.01)** −0.20 (.04)** −0.25 (.05)**
Int. ↔ int. −0.25 (.05)** −0.28 (0.7)** −0.31 (.06)** −0.20 (.07)**
−2LL(FP) 15904.81 (26) 15399.22 (26) 16461.02 (26) 15388.62 (25)
AIC/BIC 15956.81/16102.72 15451.22/15597.13 16513.02/16658.93 15438.62/15438.92
Open in a new tab

Note. −2LL = deviance; FP = free parameters; Std. est. = standardized estimate; SC1 = teacher relationships; Int. = intercept; AIC = Akaike information criterion; BIC = Bayesian information criterion. Estimates include covariate adjustment. School clusters = 16.

*

p < .05.

**

p < .01.

Figure 1.

Figure 1.

Open in a new tab

Parallel growth model for school climate and substance initiation.

Table 3 describes the standardized association between order and safety and substance use initiation models. Growth estimates for order and safety across all substance use initiation models ranged from β = −0.29 to −0.32 (SE = 0.10–0.11, all ps < .01). Substance initiation βs reported relatively consistent growth estimates and ranged from 0.95 to 0.99 (SE = 0.02–0.03, all ps < .01). Cross-lagged βs for order and safety on all substance models ranged from −0.07 to −0.17 (SE = 0.01–0.06, p < .01) and βs for substances on order and safety ranged from −0.09 to −0.20 (SE = 0.03– 0.08, p < .01). Alcohol was the only exception (β = −0.09, SE = 0.05). The correlation between intercepts ranged from −0.29 to −0.37 (SE = 0.04–0.06, all ps < .01).

Table 3.

Parallel Latent Growth Model Intercepts, Slopes, and Unstandardized/Standardized Regression Estimates for Order and Safety, n = 2,023.

E-cigarettes
 SC2
 Cigarettes
 SC2
 Alcohol
 SC2
 Marijuana
 SC2
Parameter B (SE) B (SE) B (SE) B (SE) B (SE) B (SE) B (SE) B (SE)
Intercept −2.27 (.26)** 3.72 (.07)** −2.48 (.18)** 3.72 (.07)** −2.49 (.19)** 3.72 (.06)** −3.08 (.18)** 3.72 (.07)**
Slope 3.62 (.38)** 0.28 (.18) 3.80 (.35)** 0.28 (.18) 3.38 (.29)** 0.23 (.18) 3.86 (.24)** 0.38 (.17)**
Int.→ slope 1.38 (.09)** −0.15 (.05)** 1.56 (.07)** −0.15 (.05)** 0.97 (.08)** −0.13 (.06)** 1.06 (.06)** −0.15 (.05)**
SC2 × drug −0.05 (.02)** −0.22 (.04)** −0.05 (.02)** −0.22 (.06)** − 0.02 (.02) −0.21 (.04)** −0.04 (.02)* −0.28 (.06)**
Int. ↔ int. −0.16 (.04)** − 0.17 (.04)** −0.20 (0.3)** −0.18 (.04)**
Std. est. β (SE) β (SE) β (SE) β (SE) β (SE) β (SE) β (SE) β (SE)
Int.→ slope 0.99 (.02)** −0.32 (.10)** 0.99 (.02)** −0.32 (.10)** 0.98 (.11)** −0.29 (.09)** 0.95 (.03)** −0.32 (.10)**
SC2 × drug −0.17 (.06)** −0.09 (.02)** −0.20 (.08)** −0.08 (.02)** −0.09 (.05) −0.07 (.01)** −0.15 (.07)* −0.15 (.03)**
Int. ↔ int. −0.28 (.07)** −0.28 (.07)** −0.37 (.04)** −0.30 (.06)**
−2LL(FP) 16069.51 (26) 15573.36 (26) 16632.91 (26) 15286.61 (26)
AIC/BIC 16121.51/16267.43 15625.36/15771.28 16684.91/16830.83 15338.61/15484.53
Open in a new tab

Note. −2LL = deviance; FP = free parameters; Std. est. = standardized estimate; SC2 = order and safety; Int. = intercept; AIC = Akaike information criterion; BIC = Bayesian information criterion. Estimates include covariate adjustment. School clusters = 16.

*

p < .05.

**

p < .01.

Table 4 summarizes βs between student engagement and substance use initiation. Student engagement βs between intercepts and slopes were significant and ranged from −0.09 to −0.37 (SE = 0.02–0.06, p < .01) Substance initiation growth models ranged from 0.96 to 0.99 (all SEs = 0.2, p < .01). All cross-lagged βs were significant for student engagement on substance use initiation and ranged from −0.07 to −0.15 (SE = 0.01–0.03, p < .01). Cross-lagged βs for substances on student engagement were all significant and ranged from −0.07 to −0.24 (SE = 0.01–0.03). The correlations between student engagement intercepts were all significant and ranged from −0.20 to −0.30 (SE = 0.04– 0.07, p < .01).

Table 4.

Parallel Latent Growth Model Intercepts, Slopes, and Unstandardized/Standardized Regression Estimates for Student Engagement, n = 2,023.

E-cigarettes
 SC3
 Cigarettes
 SC3
 Alcohol
 SC3
 Marijuana
 SC3
Parameter B (SE) B (SE) B (SE) B (SE) B (SE) B (SE) B (SE) B (SE)
Intercept −2.45 (.23)** 3.79 (.05)** −2.49 (.17)** 3.79 (.06)** −2.48 (.16)** 3.79 (.05)** −3.12 (.18)** 3.79 (.05)**
Slope 3.91 (.41)** 0.34 (.12)** 3.82 (.25)** 0.33 (.11)* 3.36 (.23)** 0.37 (.12)** 3.85 (.29)** 0.33 (.13)**
Int. → slope 1.40 (.07)** −0.18 (.03)** 1.56 (.07)** −0.18 (.03)** 1.05 (.07)** −0.17 (.03)** 1.29 (.09)** −0.16 (.03)**
SC3 × drug −0.06 (.01)** −0.23 (.02)** −0.07 (.02)** −0.19 (.03)** −0.04 (.02)** −0.17 (.03)** −0.03 (.02)** −0.18 (.03)**
Int. ↔ int. −0.13 (.03)** −0.13 (.04)** −0.17 (.02)** −0.19 (.03)**
Std. est. β (SE) β (SE) β (SE) β (SE) β (SE) β (SE) β (SE) β (SE)
Int.→ slope 0.99 (.02)** −0.36 (.06)** 0.99 (.02)** −0.37 (.05)** 0.96 (.02)** −0.35 (.06)** 0.97 (.02)** −0.36 (.06)**
SC3 × drug −0.20 (.03)** −0.10 (.01)** −0.24 (.05)** −0.07 (.01)** −0.12 (.05)** −0.10 (.01)** −0.17 (.06)** −0.15 (.03)**
Int. ↔ int. −0.20 (.05)** −0.21 (.07)** −0.30 (.04)** −0.21 (.06)**
−2LL (FP) 16023.09 (26) 15519.03 (26) 16588.87 (26) 15243.63 (26)
AIC/BIC 16075.09/16221.02 15571.03/15716.95 16640.87/16786.79 15358.63/15358.95
Open in a new tab

Note. −2LL = deviance; FP = free parameters; Std. est. = standardized estimate, SC3 = student engagement; Int. = intercept; AIC = Akaike information criterion; BIC = Bayesian information criterion. Estimates include covariate adjustment. School clusters = 16.

*

p < .05.

**

p < .01.

Discussion

This study sought to determine the associated growth trajectories between school climate and adolescent substance use initiation. Our results suggest a possible causal direction that supports the importance of a positive school climate to prevent substance use initiation. Analyses produced three main findings: (1) school climate may delay substance use initiation over time, (2) perceptions of school climate decreased over time, and (3) substance use initiation increased over time.

Regarding school climate specifically, positive student–teacher relationships presented slightly stronger effects than the other subscales across all models. Cross-sectional studies have consistently suggested that nurturing relationships between students and teachers is a salient factor that contributes to student outcomes (Zullig et al., 2015). A systematic review by Fletcher et al. (2008) outlined the importance of a school’s culture and how it may affect drug use. Their review generally concluded that when students mutually respect and feel connected to school, they are less likely to use drugs. Our findings support their conclusion and suggest that a positive school climate may be important for the duration of schooling. Additionally, by reviewing the correlations between intercepts, we are better able understand how school climate affects substance use initiation between years. Our findings suggest that students who maintained higher perceptions of school climate at each time point were less likely to have initiated substance use between years.

Consistent with other studies (see Heilbrun et al., 2018; Niehaus et al., 2012), school climate subscales also decreased over time. This finding is important because if a positive school climate has the ability to delay substance use initiation, preserving or improving a positive school climate may be a goal on its own. This study’s findings may also suggest that a “transition effect” may be present and may interrupt students’ connection with school as they acclimate to a new environment (Madjar & Cohen-Malayev, 2016). Hence, perceptions of school climate may take on a new meaning as students get older. For middle school students, strengthening a positive school climate may entail promoting activities that improve bonding relationships and connectedness in school (Goodenow, 1993). For high school students, supporting academic focus and subject specialization as they prepare for adulthood may be more beneficial (Scott et al., 2014). This study’s findings suggest a potential transitionary effect, but more studies are needed to better understand how school climate changes between middle and high school and how this may affect systemic outcomes.

As students grew older, the possibility of initiating any substance increased. This finding was anticipated given what is known about developmental impacts among at-risk youth (Mason et al., 2017). In addition, students who initiated any substance use demonstrated reduced perceptions of a positive school climate. As mentioned previously, student perceptions of a positive school climate may delay substance use initiation but decline over time as students grow older. Thus, substance use initiation may be an important indicator for students who require additional support (Briesch et al., 2017).

Lastly, group comparisons from sensitivity analysis revealed a few notable differences between a middle school only and a high school only sample. In the middle school only group, students who initiated either electronic or combustible cigarettes use at an early age presented stronger diminishing effects of perceptions of school climate, but school climate’s impact on substance use initiation was also more substantial. However, without any influence from initiation of substances, high school students’ perceptions of school climate demonstrated steeper declines over time. These findings support the position that there may be a difference in school climate’s importance between middle and high school students. From a primary prevention point of view, our findings suggest that middle school may the best last time to intervene (Mann et al., 2016). This may hold especially true when considering school climate as a universal strategy in a MTSS framework. More research in this area is needed to better understand the relationships between school climate, substance use initiation, and potential group differences.

Limitations

First, our sample was obtained from a large group of mostly White middle school students from 16 schools in West Virginia. Findings may be limited in their representativeness to other regions of the United States. Nevertheless, the three counties sampled contain a spectrum of students who live in chronic rural poverty and report high rates of adolescent substance use that exceed national estimates (U.S. Department of Health & Human Services & Office of Adolescent Health, 2017). Second, the potential issue of nonmatched students may exacerbate the limited representativeness. However, the mechanism that may cause nonmatching may be difficult to distinguish. Fortunately, even when participant attrition is high, parameter estimates are still likely to be accurate (Gustavson et al., 2012). Third, student self-reported information is subject to acquiescence and recall bias. However, testing of similar risk factors such as substance use in cross-sectional and longitudinal studies have demonstrated strong accuracy with reliability coefficients ranging from .61 to .82 (Brener et al., 2002; Zullig et al., 2006). Fourth, substance use initiation measures were relatively unbalanced and may diminish precision. Modeling actual number or order categories of substances used may provide additional and/or alternative information for prevention research (Buu et al., 2012). Fifth, although a binary response option to indicate students’ biological sex is common practice in school health research, including a measure that assesses multiple levels of gender may provide additional and useful information. Last, our study only modeled three aspects of school climate. The school climate literature is diverse and considers many aspects of the socio-organizational structures of schools. However, the three subscales chosen for this study are consistently used and found in other school climate studies.

Implications for Practice

Primary Finding 1.

A positive school climate may delay substance use initiation.

Recommendation 1.

School-based prevention frameworks like MTSS require continued research to better understand how nonacademic factors like school climate may influence desired student outcomes (Rooney et al., 2015). Preventing substance initiation at early ages will continue to be shaped by forces beyond the classroom, but as our findings suggest, what happens in school may affect what happens outside school. To better understand the complexity within school systems, future research should compare perceptions of school climate among substance initiators and noninitiators by using comprehensive student profiles.

Primary Finding 2.

Positive perceptions of school climate appeared to decrease over time.

Recommendation 2.

Based on what we have learned, bonding teacher relationships, respect for school rules, and positive motivation toward academics all seem to be important to students throughout school (Mann et al., 2018). Investing in school climate as a universal intervention may bolster new opportunities for schools to prevent substance use initiation (Resnick et al., 2012). In middle schools, it may be practical to select strategies specific to improving healthy student-teacher relationships that enhance classroom connectedness and reduce alienation (Blum, 2005). In high school, classroom instruction or programmatic policy that encourages preparation for adulthood may be more useful (Larusso et al., 2008). Although more research and practice-tested guidelines are necessary, how school climate information is used may support school-based policy and prevent less-than-ideal student outcomes.

Acknowledgments

The authors would like to extend their gratitude to the funder for their generous support. The authors would like to recognize Dr. Angela M. Dyer and Dr. Danielle M. Davidov for their thoughtful review and commentary of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the West Virginia Department of Education (WVDOE) in Charleston, West Virginia, and the Substance Abuse Mental Health Service Administration (SAMHSA) in Washington, D.C.; Project AWARE 2015–2020 (Prime Award #: GRTAWD04021500003266). The content is solely the responsibility of the authors and does not necessarily represent the official views of the WVDOE or SAMHSA.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

References

  1. Allensworth D, Lewallen TC, Stevenson B, & Katz S. (2011). Addressing the needs of the whole child: What public health can do to answer the education sector’s call for a stronger partnership. Preventing Chronic Disease, 8(2), Article A44 http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3073437/ [PMC free article] [PubMed] [Google Scholar]
  2. Appalachian Regional Commission. (2011–2015). Persons below poverty level, 2011–2015. https://www.arc.gov/reports/custom_report.asp?REPORT_ID=70 [Google Scholar]
  3. Blum RW (2005). A case for school connectedness. Educational Leadership, 62(7), 16–19. [Google Scholar]
  4. Bradshaw CP, Koth CW, Thornton LA, & Leaf PJ (2009). Altering school climate through school-wide positive behavioral interventions and supports: Findings from a grouprandomized effectiveness trial. Prevention Science, 10(2), 100–115. 10.1007/s11121-008-0114-9 [DOI] [PubMed] [Google Scholar]
  5. Brener ND, Kann L, McManus T, Kinchen SA, Sundberg EC, & Ross JG (2002). Reliability of the 1999 youth risk behavior survey questionnaire. Journal of Adolescent Health, 31(4), 336–342. 10.1016/S1054-139X(02)00339-7 [DOI] [PubMed] [Google Scholar]
  6. Briesch AM, Chafouleas SM, & Chaffee RK (2017). Analysis of state-level guidance regarding school-based, universal screening for social, emotional, and behavioral risk. School Mental Health, 10(2), 147–162. 10.1007/s12310-017-9232-5 [DOI] [Google Scholar]
  7. Buu A, Li R, Tan X, & Zucker RA (2012). Statistical models for longitudinal zero-inflated count data with applications to the substance abuse field. Statistics in Medicine, 31(29), 4074–4086. 10.1002/sim.5510 [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Caetano R, Mills BA, Vaeth PAC, & Reingle J. (2014). Age at first drink, drinking, binge drinking, and DSM-5 alcohol use disorder among Hispanic national groups in the United States. Alcoholism: Clinical Experimental Research, 38(5), 1381–1389. 10.1111/acer.12354 [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Cham H, Reshetnyak E, Rosenfeld B, & Breitbart W. (2017). Full information maximum likelihood estimation for latent variable interactions with incomplete indicators. Multivariate Behavioral Research, 52(1), 12–30. 10.1080/00273171.2016.1245600 [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Chartier M, Stoep AV, McCauley E, Herting JR, Tracy M, & Lymp J. (2008). Passive versus active parental permission: Implications for the ability of school-based depression screening to reach youth at risk. Journal of School Health, 78(3), 157–164. 10.1111/j.1746-1561.2007.00278.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Cheong J, MacKinnon D, & Khoo ST (2003). Investigation of mediational processes using parallel process latent growth curve modeling. Structural Equation Modeling, 10(2), 238–262. 10.1207/S15328007SEM1002_5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Cohen J, McCabe L, Michelli NM, & Pickeral T. (2009). School climate: Research, policy, practice, and teacher education. Teachers College Review, 111(1), 180–213. [Google Scholar]
  13. Cornell D, & Huang F. (2016). Authoritative school climate and high school student risk behavior: A cross-sectional multi-level analysis of student self-reports. Journal of Youth and Adolescence, 45(11), 2246–2259. 10.1007/s10964-016-0424-3 [DOI] [PubMed] [Google Scholar]
  14. Cronbach LJ, & Meehl PE (1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 281–302. 10.1037/h0040957 [DOI] [PubMed] [Google Scholar]
  15. Daily SM, Zullig KJ, Myers EM, Smith ML, Kristjansson AL, & Mann MJ (2018). Preliminary validation of the SCM in a sample of early adolescent public school children. Assessment for Effective Intervention. Advance online publication. 10.1177/1534508418815751 [DOI] [Google Scholar]
  16. Domitrovich CE, Bradshaw CP, Greenberg MT, Embry D, Poduska JM, & Ialongo NS (2010). Integrated models of school-based prevention: Logic and theory. Psychology in Schools, 47(1), 71–88. 10.1002/pits.20452 [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Durlak JA, Weissberg RP, Dymnicki AB, Taylor RD, & Schellinger KB (2011). The impact of enhancing students’ social and emotional learning: A meta-analysis of school-based universal interventions. Child Development, 82(1), 405–432. 10.1111/j.1467-8624.2010.01564.x [DOI] [PubMed] [Google Scholar]
  18. Eagle JW, Dowd-Eagle SE, Snyder A, & Holtzman EG (2015). Implementing a multi-tiered system of support (MTSS): Collaboration between school psychologists and administrators to promote systems-level change. Journal of Educational and Psychological Consultation, 25(2–3), 160–177. 10.1080/10474412.2014.929960 [DOI] [Google Scholar]
  19. Fletcher AMS, Bonell CPD, & Hargreaves JPD (2008). School effects on young people’s drug use: A systematic review of intervention and observational studies. Journal of Adolescent Health, 42(3), 209–220. 10.1016/j.jadohealth.2007.09.020 [DOI] [PubMed] [Google Scholar]
  20. Gamm S, Elliott J, Halbert JW, Price-Baugh R, Hall R, Walston D, Gabriela U, & Casserly M. (2012). Common core state standards and diverse urban students: Using multitiered systems of support (ED537476). ERIC. https://eric.ed.gov/?id=ED537476 [Google Scholar]
  21. Goodenow C. (1993). Classroom belonging among early adolescent students: Relationships to motivation and achievement. Journal of Early Adolescence, 13(1), 21–43. 10.1177/0272431693013001002 [DOI] [Google Scholar]
  22. Gustavson K, von Soest T, Karevold E, & Roysamb E. (2012). Attrition and generalizability in longitudinal studies: Findings from a 15-year population-based study and a Monte Carlo simulation study. BMC Public Health, 12, Article 918 10.1186/1471-2458-12-918 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Heilbrun A, Cornell D, & Konold T. (2018). Authoritative school climate and suspension rates in middle schools: Implications for reducing the racial disparity in school discipline. Journal of School Violence, 17(3), 324–338. 10.1080/15388220.2017.1368395 [DOI] [Google Scholar]
  24. Holfeld B, & Leadbeater BJ (2017). Concurrent and longitudinal associations between early adolescents’ experiences of school climate and cyber victimization. Computers in Human Behavior, 76, 321–328. 10.1016/j.chb.2017.07.037 [DOI] [Google Scholar]
  25. Holmes EK, Holladay HM, Hill EJ, & Yorgason JB (2018). Are mothers’ work-to-family conflict, school involvement, and work status related to academic achievement? Journal of Child and Family Studies, 27(6), 1881–1898. 10.1007/s10826-018-1021-8 [DOI] [Google Scholar]
  26. Johnston LD, Miech RA, O’Malley PM, Bachman JG, Schulenberg JE, & Patrick ME (2018). Monitoring the future: National survey results on drug use, 1975–2017: Overview: Key findings on adolescent drug use. http://www.monitoringthefuture.org//pubs/monographs/mtf-overview2017.pdf [Google Scholar]
  27. Kristjansson AL, Sigfusson J, Sigfusdottir ID, & Allegrante JP (2013). Data collection procedures for school-based surveys among adolescents: The Youth in Europe Study. Journal of School Health, 83(9), 662–667. 10.1111/josh.12079 [DOI] [PubMed] [Google Scholar]
  28. Larusso MD, Romer D, & Selman RL (2008). Teachers as builders of respectful school climates: Implications for adolescent drug use norms and depressive symptoms in high school. Journal of Youth and Adolescence, 37(4), 386–398. 10.1007/s10964-007-9212-4 [DOI] [Google Scholar]
  29. Lewallen TC, Hunt H, Potts-Datema W, Zaza S, & Giles W. (2015). The whole school, whole community, whole child model: A new approach for improving educational attainment and healthy development for students. Journal of School Health, 85(11), 729–739. 10.1111/josh.12310 [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Little RJA (1988). A test of missing completely at random for multivariate data with missing values. Journal of the American Statistical Association, 83(404), 1198–1202. 10.1080/01621459.1988.10478722 [DOI] [Google Scholar]
  31. Little RJA, & Rubin DB (2002). Statistical analysis with missing data (2nd ed.). Wiley; 10.1002/9781119013563 [DOI] [Google Scholar]
  32. Macklem GL (2011). Evidence-based Tier 1, Tier 2, and Tier 3 mental health interventions in schools In Macklem GL (Ed.), Evidence-based school mental health services (pp. 19–37).Springer; 10.1007/978-1-4419-7907-0_2 [DOI] [Google Scholar]
  33. Madjar N, & Cohen-Malayev M. (2016). Perceived school climate across the transition from elementary to middle school. School Psychology Quarterly, 31(2), 270–288. 10.1037/spq0000129 [DOI] [PubMed] [Google Scholar]
  34. Mann MJ, Kristjansson AL, Smith ML, Daily SM, Thomas S, & Murray S. (2018). From tactics to strategy: Creating and sustaining social conditions that demand and deliver effective school health programs. Journal of School Health, 88(5), 333–336. 10.1111/josh.12614 [DOI] [PubMed] [Google Scholar]
  35. Mann MJ, Kristjansson AL, Smith ML, & Sigfusdottir ID (2016). The Influence of negative life events and problem behavior on grades in early adolescence: Pathways to academic risk in the middle grades. RMLE Online, 39(10), 1–13. 10.1080/19404476.2016.124440229098057 [DOI] [Google Scholar]
  36. Marcenaro-Gutierrez O, Lopez–Agudo LA, & Ropero-García MA (2017). Gender differences in adolescents’ academic achievement. Young, 26(3), 250–270. 10.1177/1103308817715163 [DOI] [Google Scholar]
  37. March AL, Castillo JM, Batsche GM, & Kincaid D. (2016). Relationship between systems coaching and problem-solving implementation fidelity in a response-to-intervention model. Journal of Applied School Psychology, 32(2), 147–177. 10.1080/15377903.2016.1165326 [DOI] [Google Scholar]
  38. Marshall A, Altman DG, & Holder RL (2010). Comparison of imputation methods for handling missing covariate data when fitting a Cox proportional hazards model: A resampling study. BMC Medical Research Methodology, 10, Article 112 10.1186/1471-2288-10-112 [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Mason WA, Patwardhan I, Smith GL, Chmelka MB, Savolainen J, January S-AA, Miettunen J, & Järvelin M-R (2017). Cumulative contextual risk at birth and adolescent substance initiation: Peer mediation tests. Drug and Alcohol Dependendence, 177(August), 291–298. 10.1016/j.drugalcdep.2017.03.045 [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Masyn KE, Petras H, & Liu W. (2014). Growth curve models with categorical outcomes In Bruinsma G. & Weisburd D. (Eds.), Encyclopedia of criminology and criminal justice (pp. 2013–2025). Springer; 10.1007/978-1-4614-5690-2_404 [DOI] [Google Scholar]
  41. Mazza GL, Enders CK, & Ruehlman LS (2015). Addressing item-level missing data: A comparison of proration and full information maximum likelihood estimation. Multivariate Behavioral Research, 50(5), 504–519. 10.1080/00273171.2015.1068157 [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Michael SL, Merlo CL, Basch CE, Wentzel KR, & Wechsler H. (2015). Critical connections: Health and academics. Journal of School Health, 85(11), 740–758. 10.1111/josh.12309 [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Miller ML, & Hurd YL (2017). Testing the gateway hypothesis. Neuropsychopharmacology, 42(5), 985–986. 10.1038/npp.2016.279 [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Muthén LK, & Muthén BO (2017). Mplus user’s guide (8th ed.). Muthén & Muthén. [Google Scholar]
  45. National Research Council & Institute of Medicine. (2009). Preventing mental, emotional, and behavioral disorders among young people: Progress and possibilities Committee on the Prevention of Mental Disorders and Substance Abuse Among Children, Youth, and Young Adults: Research Advances and Promising Interventions. In O’Connell ME, Boat T, & Warner KE (Eds.). National Academies Press; https://www.ncbi.nlm.nih.gov/books/NBK32775/pdf/Bookshelf_NBK32775.pdf [PubMed] [Google Scholar]
  46. Niehaus K, Rudasill KM, & Rakes CR (2012). A longitudinal study of school connectedness and academic outcomes across sixth grade. Journal of School Psychology, 50(4), 443–460. 10.1016/j.jsp.2012.03.002 [DOI] [PubMed] [Google Scholar]
  47. O’Malley M, Voight A, Renshaw TL, & Eklund K. (2015). School climate, family structure, and academic achievement: A study of moderation effects. School Psychology Quarterly, 30(1), 142–157. 10.1037/spq0000076 [DOI] [PubMed] [Google Scholar]
  48. OSEP Technical Assistance Center on Positive Behavioral Interventions and Supports. (2017). Positive behavioral interventions & supports: Multi-tiered system of support. https://www.pbis.org/pbis/tiered-framework [Google Scholar]
  49. Piscatelli J, & Lee C. (2011, July). State policies on school climate and bully prevention efforts: Challenges and opportunities for deepening state policy support for safe and civil schools (ED566375). ERIC. https://eric.ed.gov/?id=ED566375 [Google Scholar]
  50. Resnick MD, Catalano RF, Sawyer SM, Viner R, & Patton GC (2012). Seizing the opportunities of adolescent health. Lancet, 379(9826), 1564–1567. 10.1016/S0140-6736(12)60472-3 [DOI] [PubMed] [Google Scholar]
  51. Rooney LE, Videto DM, & Birch DA (2015). Using the whole school, shole community, whole child model: Implications for practice. Journal of School Health, 85(11), 817–823. 10.1111/josh.12304 [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Ruus V-R, Veisson M, Leino M, Ots L, Pallas L, Sarv E-S, & Veisson A. (2007). Students’ well-being, coping, academic success, and school climate. Applied Research in Quality of Life, 35(7), 919–936. 10.2224/sbp.2007.35.7.919 [DOI] [Google Scholar]
  53. SAS Institute. (2013). Base SAS 9.4 procedures guide: Statistical procedures (2nd ed.). [Google Scholar]
  54. Scott TM, Hirn RG, & Alter PJ (2014). Teacher instruction as a predictor for student engagement and disruptive behaviors. Preventing School Failure, 58(4), 193–200. 10.1080/1045988X.2013.787588 [DOI] [Google Scholar]
  55. Sigfusdottir ID, Kristjansson AL, Gudmundsdottir ML, & Allegrante JP (2011). Substance use prevention through school and community-based health promotion: A transdisciplinary approach from Iceland. Global Health Promotion, 18(3), 23–26. 10.1177/1757975911412403 [DOI] [PubMed] [Google Scholar]
  56. Stanis JJ, & Andersen SL (2014). Reducing substance use during adolescence: A translational framework for prevention. Psychopharmacology, 231(8), 1437–1453. 10.1007/s00213-013-3393-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  57. Substance Abuse and Mental Health Services Administration. (2018). Key substance use and mental health indicators in the United States: Results from the 2017 National Survey on Drug Use and Health. https://www.samhsa.gov/data/sites/default/files/cbhsq-reports/NSDUHFFR2017/NSDUHFFR2017.pdf [Google Scholar]
  58. Sugai G, & Horner RR (2006). A promising approach for expanding and sustaining school-wide positive behavior support. School Psychology Review, 35(2), 245–259. [Google Scholar]
  59. U.S. Department of Health & Human Services & Office of Adolescent Health. (2017, January 27). West Virginia adolescent substance use facts. https://www.hhs.gov/ash/oah/facts-and-stats/national-and-state-data-sheets/adolescents-and-substance-abuse/west-virginia/index.html [Google Scholar]
  60. Wang MT, & Degol JL (2016). School climate: A review of the construct, measurement, and impact on student outcomes. Educational Psychology Review, 28(2), 315–352. 10.1007/s10648-015-9319-1 [DOI] [Google Scholar]
  61. Wang MT, & Dishion TJ (2012). The trajectories of adolescents’ perceptions of school climate, deviant peer affiliation, and behavioral problems during the middle school years. Journal of Research on Adolescence, 22(1), 40–53. 10.1111/j.1532-7795.2011.00763.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  62. Wilson SJ, & Lipsey MW (2007). School-based interventions for aggressive and disruptive behavior: Update of a meta-analysis. American Journal of Preventative Medicine, 33(2 Suppl.), S130–S143. 10.1016/j.amepre.2007.04.011 [DOI] [PMC free article] [PubMed] [Google Scholar]
  63. Zullig KJ, Collins R, Ghani N, Hunter AA, Patton JM, Huebner ES, & Zhang J. (2015). Preliminary development of a revised version of the school climate measure. Psychological Assessment, 27(3), 1072–1081. 10.1037/pas0000070 [DOI] [PubMed] [Google Scholar]
  64. Zullig KJ, Collins R, Ghani N, Patton JM, Huebner ES, & Ajamie J. (2014). Psychometric support of the school climate measure in a large, diverse sample of adolescents: A replication and extension. Journal of School Health, 84(2), 82–90. 10.1111/josh.12124 [DOI] [PubMed] [Google Scholar]
  65. Zullig KJ, Koopman TM, Patton JM, & Ubbes VA (2010). School climate: Historical review, instrument development, and school assessment. Journal of Psychoeducational Assessment, 28(2), 139–152. 10.1177/0734282909344205 [DOI] [Google Scholar]
  66. Zullig KJ, Pun S, Patton JM, & Ubbes VA (2006). Reliability of the 2005 middle school youth risk behavior survey. Journal of Adolescent Health, 39(6), 856–860. 10.1016/j.jadohealth.2006.07.008 [DOI] [PubMed] [Google Scholar]

RESOURCES