Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Jan 1.
Published in final edited form as: Prev Sci. 2015 Jan;16(1):80–89. doi: 10.1007/s11121-013-0457-8

The Process of Adoption of Evidence-based Tobacco Use Prevention Programs in California Schools

Melissa A Little 1,*, Pallav Pokhrel 1, Steve Sussman 2, Louise Ann Rohrbach 2
PMCID: PMC4090291  NIHMSID: NIHMS554354  PMID: 24398826

Abstract

Although there are a number of research-validated substance use prevention programs available for wide-scale dissemination, very little is known about the factors that influence adoption of evidence-based prevention programs in schools. We tested a model of the mechanisms of program adoption in schools that was guided by diffusion of innovations and social ecological theories. Cross-sectional data were collected from a sample of school district and county office of education tobacco use prevention education coordinators throughout California. Structural equation modeling was used to test the effects of community- and organizational variables on the adoption of prevention programs via school administrators’ beliefs and the organization’s receipt of funding for the program. Results supported the hypothesis that the process of adoption begins with forming beliefs about the program, leading to adoption through the receipt of funding. In addition, we found direct effects of various community- and organizational-level factors on beliefs, receipt of funding, and adoption. These results are likely to inform policies that affect school districts’ use of evidence-based substance use prevention programming, which should ultimately lead to reductions in negative health outcomes among adolescents. Specifically, this study identifies various factors that could be targeted for improvement to enhance evidence-based program adoption. To our knowledge, this is the first study to empirically elucidate the process of adoption of evidence-based tobacco prevention programs in schools.

Keywords: adoption, school-based, tobacco prevention, substance use prevention, translation

INTRODUCTION

There is now substantial empirical evidence demonstrating long term positive effects of school-based prevention programs in reducing adolescent tobacco use (Flay, 2009; Skara & Sussman, 2003), and a number of effective prevention programs have been made available for wide-scale dissemination. However, despite the empirical evidence demonstrating the effectiveness of these programs, the literature on factors influencing adoption of evidence-based prevention programs in schools remains sparse (Rohrbach, Grana, Sussman, & Valente, 2006). Of particular interest are the community-, organizational-, and individual-level factors that are responsible for promoting program adoption.

Given the obligatory nature of school attendance, schools have been the primary context for delivering health interventions targeting adolescents (Ellickson, 1995; Gregory, Henry, & Schoeny, 2007). As a result, numerous effective prevention programs have been made available for purchase and use in schools; however, many school districts fail to adopt these programs. In a recent national study of school district administrators, Ringwalt et al. (2008) found that only 10.3% of school districts reported using one of six evidence-based substance use prevention curricula listed as effective by national registries in their high schools, and only 5.7% reported using one of these curricula the most (Ringwalt et al., 2008). This is despite years of federal and state policies (e.g., Safe and Drug Free Schools Act of 1999 and the No Child Left Behind Act of 2001) mandating the use of evidence-based programs (Hallfors & Godette, 2002) and publication of several lists of ‘best practice,’ ‘research-validated,’ and ‘evidence-based’ programs that have proven effectiveness in reducing youth risk behaviors (Fagan, Hanson, Hawkins, & Arthur, 2008; National Cancer Institute & SAMHSA, 2010; United States Department of Health and Human Services & SAMHSA, 2010). Although the criteria for identifying evidence-based programs are derived from agreed upon scientific standards, such as the rigor of evidence (e.g., appropriateness of methodology), the methods used to collect and analyze the data, the magnitude and consistency of the effects, and the generalizability of the findings (e.g., Center for Substance Abuse Prevention, 2009; Flay et al., 2005), the application of these standards varies across registries and lists. Generally, programs are given a score from weak to strong, which practitioners must interpret when they decide whether or not to adopt. As a result, some programs with weaker evidence might be adopted and implemented in the community. To further complicate the problem, resources for prevention program implementation have been reduced over the past decade, and in 2010 the main source of funding for school-based substance use prevention programming, the Safe and Drug Free Schools and Communities (SDFSC) grants program, was eliminated from the federal budget. In the state of California, recent changes to the Health and Safety Code removed the annual entitlements for tobacco use education in schools and replaced them with a competitive grants program, thereby significantly limiting the resources available for implementation of evidence-based prevention programs.

The Tobacco Use Prevention Education (TUPE) program began in California in 1989, after the passage of a statewide referendum (Proposition 99) that increased the tax on tobacco products and established the California Tobacco Control Program (Rohrbach et al., 2002). The TUPE funds, administered by the California Department of Education (CDE), are designated to support school-based tobacco-specific research-validated student instruction, reinforcement activities, special events, and intervention and cessation programs. Schools that are awarded competitive TUPE funds are expected to select and implement research-validated programs as well as research-based activities listed in the CDE’s publication Getting Results (California Department of Education 2001). Given these policy stipulations, it is essential for researchers to identify factors that lead school districts to adopt evidence-based prevention programs so that strategies may be developed to improve such factors.

Diffusion of Innovations theory posits that the adoption of innovations, like evidence-based public health programs, is preceded by a process of gaining information and forming an opinion about the innovation (Rogers, 1983). Studies have shown that school district administrators pay greater attention to evidence that is in line with their preexisting beliefs, which in turn affects their decision-making about new programs and curricula (Coburn, Toure, & Yamashita, 2009). In one national study assessing decisions about adoption of substance use prevention programs, mid-level district administrators were the most important decision-makers (Rohrbach, Ringwalt, Ennett, & Vincus, 2005). These administrators employed a variety of sources to gather information to guide their decisions, and decision-making was also shaped by organizational priorities and available resources. Other studies have suggested that due to the nature of school settings, with multiple levels of decision-making dispersed among a central administration and multiple schools, decisions about implementation of evidence-based programs are rather complex (Shinn, 2003; Spillane, 1998).

To date, most research examining the use of evidence-based programs is narrowly focused on the individual adopters of programs, such as school district administrators, rather than the organizational- and community-level contexts in which decisions about program implementation are made. These relatively molar factors need consideration because they serve as structural barriers or supports for any changes entertained within the system (Estabrooks & Glasgow, 2006; Green & Glasgow, 2006; Sallis, Owen, & Fisher, 2008). If they are unyielding barriers, administrative decisions may tend to be adopted weakly; if they are supportive of innovation, administrative decisions will tend to be adopted quickly and more forcefully. However, little research has been conducted on the service delivery systems in which decisions about prevention programming are made (Rohrbach et al., 2006; Spoth et al., 2013). Thus, as a field we still do not fully understand how to translate health interventions from research to practice.

To address these limitations, researchers have begun to look at the translation process from a social ecological or multilevel systems perspective (Estabrooks & Glasgow, 2006). This type of approach considers how components from multiple levels within a system interact to produce changes over time (Ginexi et al., 2010). In school-based research, for example, one might examine how factors from multiple systems (e.g., community-, organizational-, and individual-level factors) affect decision-making about prevention programming.

In previous studies, several characteristics of the community and organizational context have been associated with the use of evidence-based programs in schools (Durlak & DuPre, 2008). Available funding is often cited as a key factor influencing school districts’ adoption of evidence-based prevention programs, given the high costs associated with purchasing training and program materials (Cho, Hallfors, Iritani, & Hartman, 2009; Hallfors & Godette, 2002; Rohrbach et al., 2006). In the past, because the distribution of public funds for tobacco education has been based on enrollment, larger districts have been better able than smaller districts to offset the costs involved in adopting and implementing evidence-based tobacco prevention programs (Cho et al., 2009; Ennett et al., 2003; Ringwalt et al., 2002; Rohrbach et al., 2005). Use has also been greater among schools with a positive external environment (e.g., greater stability outside of school, less opposition to prevention, more mandates and policies supporting prevention programs and less bureaucracy) (Gingiss, Roberts-Gray, & Boerm, 2006; Roberts-Gray, Gingiss, & Boerm, 2007). The presence of a program champion has also been found to influence the use of evidence-based prevention programs in schools (Fagan et al., 2008; Fagan & Mihalic, 2003; Gingiss et al., 2006; Mihalic, Fagan, & Argamaso, 2008; Roberts-Gray et al., 2007; Rohrbach et al., 2006; Rohrbach et al., 2005).

Based on the diffusion of innovations model (Rogers, 1983), the process of adoption of evidence-based prevention programs in schools begins with individual decision-makers establishing beliefs about the program, which influences their intentions to adopt the program and subsequently leads to adoption. Integrating a social ecological or multilevel systems perspective (Estabrooks & Glasgow, 2006), community and organizational factors would be influential in shaping the beliefs that administrators form about the effectiveness of evidence-based tobacco prevention programs. Furthermore, these contextual factors would influence the overall process of adoption through their influence on beliefs.

Despite the growing body of research exploring the diffusion of evidence-based programs in schools, there are a number of areas where the research is lacking. Very few studies have examined the process of adoption from an ecological approach, and those that have accounted for the influence of contextual factors have done so without accounting for the varying levels of influence of these factors (Blake et al., 2005; Cho et al., 2009; Fagan et al., 2008; Hallfors & Godette, 2002; Rohrbach, Gunning, Grana, Gunning, & Sussman, 2010; Rohrbach et al., 2005). Thus, the influence of contextual factors on the use of evidence-based programs in schools may have been masked. Furthermore, previous research has failed to explore the co-varying relationships among community-, organizational-, and individual-level factors, and how these relationships influence adoption decisions.

The Present Study

The present study builds upon previous research by considering the simultaneous effects of community- and organizational-level factors (organizational size, organizational mandate, presence of a program champion, organizational support, community priority, and funding) on the process of adoption of evidence-based tobacco prevention programs in schools. We hypothesized that:

  1. Adoption of evidence-based tobacco use prevention programs occurs through the following process: Administrators establish beliefs about the effectiveness of evidence-based tobacco prevention interventions, which if they receive funding will lead them to adopt.

  2. Beliefs about the effectiveness of evidence-based prevention interventions mediate the effects of five contextual variables (community priority for tobacco and substance use prevention education, larger organizational size, having an organizational mandate to use a specific prevention curriculum, organizational support for tobacco use prevention, and the presence of a program champion) on receiving funding to adopt prevention programs.

DESIGN AND METHODS

Target Sample

Data for this study come from a cross-sectional survey of administrators and Tobacco Use Prevention Education (TUPE) coordinators in school districts and county offices of education across the state of California. In addition to 956 school districts, there are 58 county offices of education in the state that play a vital role in providing technical assistance, staff development, curriculum and instructional support and oversight to the school districts located within their counties. Because county offices of education also are responsible for providing tobacco use prevention education to youth in a number of smaller school districts, we included them in the current sample (California Department of Education, 2011a). School districts, direct-funded charter schools, and county offices of education that serve students in grades six through twelve and have met the tobacco-free school district requirements are eligible to apply for competitive TUPE funds.

In order to reduce bias from potential confounders (e.g., population density, organizational size, and student ethnicity), the sample was drawn from two matched groups of school districts and county offices of education: (1) those that submitted an application to the California Department of Education (CDE) in 2009 for competitive TUPE funds (N=134) and (2) a matched group that did not apply (n=134). The two groups were matched on three key demographic variables obtained from the Quality Education Direct California District 2008 file (Quality Education Direct Inc., 2009), including population density, organizational size, and student ethnicity. The final target sample was 268 (n=240 school districts and n=28 country offices of education).

Data Collection

Data collection occurred between January and October 2011. The target respondent in each county office of education was the TUPE coordinator, and in each school district was the substance use prevention or TUPE coordinator. Previous research suggests that this administrator is the key decision-maker in regards to tobacco use prevention education within the organization (Rohrbach et al., 2005). However, because of the recent elimination of Safe and Drug Free Schools (SDFS) funds, many school districts no longer have a substance abuse prevention program coordinator. In those cases, the respondent was the individual within the district that knew the most about the tobacco prevention programs currently being offered to students. To establish the appropriate respondent for our survey, organizations were contacted by telephone in advance of data collection. Next, this individual was invited by phone and/or email to participate in the study and complete a 25-minute online survey containing 89 self-report items. Respondents were provided a $20 gift card as compensation for their time spent participating in the study. We made a maximum of five attempts to reach the target respondent and obtain his/her informed consent to participate in the survey. Respondents (n=7) who did not complete the online survey after repeated contacts were interviewed via telephone using the same standardized questionnaire used in the online survey.

Final Study Sample

Of the target school district administrators and county office of education TUPE coordinators that we reached (n=247), a total of 229 (92.7%) agreed to participate in the study. Of those agreeing to participate, we surveyed 205 administrators who represented 183 school districts and 22 county offices of education. The majority of participants (87.8%) represented regular school districts. One half (49.8%) of the organizations had 7 or more schools, with 35.6% having more than 18 schools. Over one third (38.6%) of students were White and 36.5% were of Hispanic ethnicity. Nearly one half of the organizations were located in suburban areas (46.3%). To examine differences between the final samples of county offices of education and school districts, we utilized chi-square or t-tests to compare the two groups on key demographic variables (p value at the .05 level, two-tailed). Relative to the sample of county offices, the school districts were comprised of a larger number of schools and had a smaller proportion of White students. There were no significant differences on any of the other key demographics. Therefore, we chose to combine the two samples for analytic purposes.

Measures

Dependent Variables

To assess adoption of an evidence-based prevention program, we first asked respondents to list which prevention program they were using the most with their students in middle schools. Districts that serve grades 9–12 only were asked to list which prevention program they were using the most with their students in high schools. These items were then coded as evidence-based (1) or not (0). Programs were determined to be evidence-based if they were identified in one or more of the following state or federal registries as a school-based program that is effective in preventing tobacco use. The registries included: California Healthy Kids Resource Center (California HealthyKids Resource Center, 2010), National Registry of Evidence-based Programs and Practices (NREPP) (Substance Abuse and Mental Health Services Administration & United States Department of Health and Human Services, 2010), Exemplary and Promising: Safe, Disciplined, and Drug-Free Schools Programs (Office of Safe and Drug Free Schools & United States Department of Education, 2010), Research-Tested Intervention Programs (National Cancer Institute & SAMHSA, 2010), and Preventing Drug Use Among Children and Adolescents: A Research-Based Guide (National Institute on Drug Abuse, National Institute of Health, & U.S. Department of Health and Human Services, 2010).

Received funding was based on whether districts were awarded a competitive TUPE grant between 2008 and 2011 (0=not funded, 1=funded). The names of school districts and county offices of education that had received a competitive TUPE grant were provided by the CDE (California Department of Education, 2011b). Two items comprised the beliefs index assessing the perceived effectiveness of evidence-based interventions (EBIs) (e.g., classroom curricula, substance use policies) (4-point scales; 1=not effective at all to 4=very effective). Items asked, “In your opinion, how effective are (e.g., classroom curricula) in reducing tobacco, alcohol, and other drug use among (middle/high) school students in your district?”

Independent Variables

Variables measuring community characteristics, including population density (urban, suburban, or rural), organization size (number of schools in the district or districts, 1=1 school, 2=2–3 schools, 3=4–6 schools, 4=7–11 schools, 5 = 11–18 schools, 6 = >18 schools), and student ethnicity were obtained from Quality Education Direct’s California District 2008 file (Quality Education Direct Inc., 2009). Community priority was assessed by asking participants the degree to which tobacco and substance use prevention was a priority in the community (1=not a priority at all to 4=high priority).

Organizational factors were assessed through three items including: the presence of a program champion (0=no to 1=yes); an organizational mandate to use a specific prevention curriculum (0=no to 1=yes); and organizational support for tobacco use prevention, which was assessed through four items such as, “Principals and teachers support our districts’ tobacco/substance use prevention and cessation efforts” (1=strongly disagree to 5=strongly agree; α =.79).

DATA ANALYSIS

Descriptive statistics were analyzed with the SAS (v.9.1.3) statistical package (SAS Institute Inc. SAS/C Online Doc TM, 2000) and hypotheses were tested in Mplus (6.11) (Muthén & Muthén, 1998–2012) using structural equation modeling (SEM). Missing data were estimated using MPLUS (6.11) (Muthén & Muthén, 1998–2012). MPLUS computes missing data using maximum likelihood estimation. The SEM analysis was conducted in the following steps. First, the model was specified with organizational mandate, organizational support, community priority, organizational size, and presence of a program champion as exogenous variables. Two variables were specified as mediators: perceived effectiveness of EBIs and received funding. Actual adoption of a program was specified as the criterion variable. Second, paths were specified from all exogenous variables to perceived effectiveness of EBIs, from perceived effectiveness of EBIs to funding, and from funding to adoption. In addition, paths were specified from each exogenous variable to funding and adoption variables in order to test for any direct effect that the organizational and community variables may have on funding and adoption. Next, we ran the resulting model with covariance specified among all exogenous variables. The goodness of fit of the model to the data was determined based on χ2 statistic, comparative fit index (CFI), root mean square error of approximation (RMSEA), and weighted root mean square residual (WRMR) (Bentler, 2007; Yu & Muthén, 2002). The following cut-offs were employed for establishing adequate fit: CFI ≥.95 (Hu & Bentler, 1999); RMSEA<.05 (Ho, 2006); WRMR<1.0 (Yu & Muthén, 2002). Path effects were determined based on standardized path coefficients. The final model was estimated after removing the statistically non-significant paths from original model. Among hypothesized pathways, a statistically significant mediation was determined in terms of a statistically significant indirect effect (MacKinnon, Fairchild, & Fritz, 2007). Indirect effects and corresponding standard errors were computed in Mplus using the delta method (MacKinnon et al., 2007; Muthén & Muthén, 1998–2012).

RESULTS

Figure 1 shows the TUPE funding status and evidence-based program use among the organizations in the study. Out of 205 participating organizations, nearly two thirds (61.5%) applied for competitive TUPE funds and of those, two-thirds received funding (65.9%). Among these 83 organizations that were awarded TUPE grants, over three-fourths (79.5%) used evidence-based tobacco use prevention programs. Among the 43 organizations that applied but did not receive competitive TUPE funding, roughly one half (51.2%) used evidence-based tobacco use prevention programs. Among the 79 organizations that did not apply for competitive TUPE funding, slightly more than one third (35.4%) used evidence-based tobacco use prevention programs. Among all organizations (both TUPE-funded and non-TUPE-funded) that reported using a tobacco prevention program, the most prevalent programs were Project Alert (42.0%), Project Towards No Drug Use (26.3%), Too Good for Drugs (22.4%), Life Skills Training (13.7%) and Project Towards No Tobacco Use (8.8%). In addition, the majority of these organizations reported holding Red Ribbon Week and distributing pamphlets, posters or other tobacco use prevention materials (68.5% and 64.7%, respectively).

Figure 1.

Figure 1

Open in a new tab

TUPE funding status and evidence-based program use among organizations

Table 1 presents a correlation matrix between all variables. Organizational mandate was relatively highly correlated with the presence of a program champion and program adoption, and as expected, funding was strongly correlated with whether the organization adopted an evidence-based program. Figure 2 presents the final path model. The final model fit indices were χ2df10=16.94, RMSEA=.06, CFI=.98, WRMR=.51.

Table 1.

Summary descriptive statistics and correlations (N=205)

Variable M SD 1 2 3 4 5 6 7 8
1. Community priority 2.60 0.84 −0.01 0.09 0.06 0.18 0.23 −0.08 0.15
2. Organizational size 4.40 1.47 0.25 0.23 0.09 0.17 0.11 0.36
3. Organizational mandate 0.50 0.50 0.44 0.14 0.23 0.25 0.58
4. Program champion 0.40 0.49 0.15 0.27 0.30 0.47
5. Organizational support 3.73 0.71 0.31 0.26 0.18
6. Perceived effectiveness of EBIs 3.00 0.50 0.27 0.35
7. Received funding 0.23 0.42 0.59
8. Adoption 0.57 0.50
Open in a new tab

Notes. Correlations are standardized; M = mean; SD = standard deviation; Abbreviations: EBIs = Evidence-based interventions

Figure 2.

Figure 2

Open in a new tab

Path model. Only significant paths are shown. Covariance was specified between all exogenous variables, however it was not shown for graphical simplicity. Abbreviations: EBIs = Evidence-based interventions. Significance: *p< .05, **p< .0001. Model fit indices: χ2 = 16.94, d.f. = 10, p=0.08, RMSEA = 0.06, 90% CI = 0.01, 0.11, CFI = 0.98, WRMR = 0.51.

The present data supported our hypothesis that beliefs about the effectiveness of evidence-based tobacco prevention programs are significantly related to whether organizations received funding (β=.24; p=.007), which in turn, is related to program adoption (β=.52; p<.0001). Receiving funding fully mediated the effect of perceived effectiveness of EBIs on adoption (β=.13, SD=.05, p=.01) (see Table 2). Identifying tobacco and substance use as a community priority (β=.16; p=.024), having organizational support for tobacco use prevention (β=.27; p<.0001), and presence of a program champion (β=.26; p<.0001) were associated with perceived effectiveness of EBIs. The second hypothesis regarding perceived effectiveness of EBIs mediating the relationships between community- and organizational-level factors and receiving funding to adopt prevention programs was partially supported. Perceived effectiveness of EBIs mediated the effects of community priority (β=.04, SD=.02; p=.07), having a program champion (β=.06, SD=.03; p=.02), and organizational support (β=.07, SD=.03; p=.02) on receiving funding. However, organizational mandate and size were not significantly related to perceived effectiveness of EBIs, although organizational mandate (β=.47; p<.0001) and organizational size (β=.21; p=.005) were both directly related to adoption. Having a program champion was also directly related to receiving funding (β=.28; p=.001).

Table 2.

Summary of indirect effects

Effects B (SE) p-value
Community priority Inline graphic Perceived effectiveness of EBIs Inline graphic Received funding 0.04 (0.02) 0.07
Program Champion Inline graphic Perceived effectiveness of EBIs Inline graphic Received funding 0.06 (0.03) 0.02
Organizational Support Inline graphic Perceived effectiveness of EBIs Inline graphic Received funding 0.07 (0.03) 0.02
Perceived effectiveness of EBIs Inline graphic Received funding Inline graphic Adoption 0.13 (0.05) 0.01
Open in a new tab

Abbreviations: EBIs = Evidence-based interventions

DISCUSSION

School-based tobacco education is an integral part of tobacco control among youth. However, due to the complex nature of decision-making in schools, decisions to adopt tobacco education programs are influenced by a variety of factors at the community, organization, and individual decision-maker levels. By understanding the factors that lead schools to adopt tobacco prevention programs, researchers and policymakers can work together to shape policies that will increase the use of evidence-based programs, which will ultimately lead to reductions in tobacco use among adolescents.

The main objective of this study was to explore the simultaneous influence of community- and organizational-level factors on the process of adoption of evidence-based tobacco prevention programs in schools. The structural equation model tested in this study showed that the process of adoption begins with forming beliefs about the program, which leads to adoption by way of receiving funding; while community- and organizational-level factors influence beliefs, receiving funding, and adoption.

Administrators’ beliefs were found to influence the process of adoption of evidence-based tobacco prevention programs. The diffusion of innovations model posits that an individual first forms an opinion about an innovation, which subsequently influences their decision to use the innovation (Rogers, 1983). As hypothesized, in the current study community and organizational factors were associated with forming beliefs about the effectiveness of evidence-based tobacco prevention programs and beliefs mediated the effect of these factors on receiving funding to adopt prevention programs. These findings suggest that administrators responsible for selecting prevention programs are concerned with ensuring that the program they select is not only responsive to community priorities and has organizational support, but also there is someone within the school system that is willing to champion the program. These factors should be targeted in interventions aimed at increasing the adoption of evidence-based prevention programs in schools.

Interestingly, having an organizational mandate to use a specific prevention curriculum and size of the organization were not related to beliefs, but instead were directly related to adoption. These findings are not surprising. Previous research has found a direct association between size of the organization and adoption (Cho et al., 2009; Rohrbach et al., 2005), suggesting that larger organizations have greater infrastructure to support the implementation of prevention education. With regard to organizational mandates to use a specific prevention curriculum, if an organization has such a mandate it does not matter whether the administrator responsible for selecting the program holds favorable or unfavorable beliefs about the program; teachers are still required to use the mandated program. Therefore, the direct path from having an organizational mandate to adoption is a logical finding.

Funding has long been cited as an important indicator of whether schools are able to provide evidence-based prevention education. In the current study, receiving funding mediated the effect of beliefs about the effectiveness of evidence-based tobacco prevention programs on adoption, suggesting that the availability of dedicated funds is critical to the decision to adopt programs with proven effectiveness in schools. These findings highlight the importance of external funding to support school-based tobacco use prevention education. In order to increase the use of evidence-based prevention programs in schools, additional public funds need to be made available for programming and organizations need to be encouraged to apply for them.

The findings have several implications for the adoption of evidence-based tobacco use prevention programs in schools. First, they suggest that organizational factors are important in promoting adoption of evidence-based programs, possibly independently of funding. Our data indicated that although funding greatly increases the adoption of evidence-based tobacco prevention programs in schools, organizational factors show direct influence on adoption as well. Given the limited funding available to implement evidence-based prevention programs, specific efforts should be made to strengthen the existing infrastructure in schools and school districts to support prevention education. Additionally, continued efforts to disseminate information to district, county, and state office of education administrators about the benefits of using evidence-based programs will be integral to increasing the adoption of evidence-based tobacco prevention programs in schools.

One way to make prevention programs more appealing to school administrators is to make them more compatible with schools’ values and needs. For example, there is now a growing body of research demonstrating that many negative outcomes among youth (e.g. psychopathology, substance abuse, delinquency and school failure) have related risk factors and co-morbidity (Greenberg, 2010). In addition, there is evidence that prevention programs can improve academic outcomes as well as reduce health risk behaviors (Snyder et al. 2010). Promoting the benefits of substance abuse prevention programming to prevent a broad range of social and emotional problems and potentially improve academic achievement could influence key decision-makers in school districts to adopt evidence-based programs, as well as strengthen organizational support for prevention programming (National Research Council and Institute of Medicine, 2009). If administrators are aware of the benefits of prevention programs in promoting improved behavior and well-being among students (Greenberg, 2010), they may be more likely to put in the effort to adopt these programs. Similarly, given that administrators’ beliefs about the program were related to organizations’ adoption of evidence-based prevention programs, developers should take prospective adopters’ preferences, such as teachers’ instructional styles, into account from the earliest phases of program development (Kreuter and Bernhardt, 2009).

Taking into account these findings, several limitations should be considered. First, the data are cross-sectional. Therefore, while the relationships of structural variables to dependent variables are implied as directional effects, the relationships are still associational. However, the results of the current study using cross-sectional data help inform directional tests for future studies that should incorporate longitudinal data. A second limitation is that there were only two items comprising the beliefs index assessing the perceived effectiveness of EBIs. Because this construct was an integral part of the process of adoption in schools, it should be assessed in greater detail in future studies. It should also be noted that the funding variable used in the current analyses may not be generalizable to all funding mechanisms, since organizations had to have a high level of intentions to adopt a tobacco prevention program in order to compete for TUPE funding. Another limitation pertains to the individual respondent within each organization. We intended to survey the SDFS or TUPE coordinator in each district because previous research suggests that this administrator is the key decision-maker with regard to substance use prevention education (Rohrbach et al., 2005). However, due to cuts in funding many districts no longer have a SDFS or TUPE coordinator, which made it difficult to identify the most appropriate person to complete the survey. Although we attempted to find the administrator that was the most knowledgeable about substance use prevention education, it is possible that some respondents were less familiar with these programs overall than were designated prevention coordinators.

In the current study, it would have been interesting to see how the amount of TUPE funds districts received directly influenced the process of adoption. TUPE funds are distributed based on size of the organization; therefore, it is likely that organizational size and TUPE funds would have been statistically redundant in the current analyses. However, future studies should explore how the amount of funding directly affects the adoption of evidence-based programs in schools.

Lastly, it should be noted that in the current study we operationalized adoption as the use of an evidence-based prevention program (yes/no) in schools as reported by an administrator at the district or county office of education. Due to limited resources, we were unable to collect data on program implementation at the school and teacher level. Thus, we do not know the extent to which students in the study districts were exposed to the evidence-based programs and whether the programs were implemented with fidelity. In order to advance the field of prevention science, it is important that future studies include comprehensive assessments of both adoption and implementation of evidence-based programs.

In the current study, we sought to fill a gap in the school-based tobacco prevention literature by investigating the impact of community, organizational and individual-level factors as well as funding on district decisions to adopt tobacco prevention programs. While previous studies have used structural equation modeling to explore the process of implementation in schools (Beets et al., 2008; Payne, Gottfredson, & Gottfredson, 2006), this is the first study we are aware of to use an advanced statistical analysis technique to explore the process of adoption. We found that adoption of evidence-based tobacco prevention programs in schools occurs through a process by which administrators first establish beliefs about the effectiveness of evidence-based tobacco prevention programs, which if they receive funding will lead to them to adopt. This entire process is influenced by community- and organizational-level factors, which supports a multilevel ecological approach. These results should be used to inform policies that affect the use of evidence-based tobacco prevention programs in schools. Ultimately, the increased use of evidence-based prevention programs should lead to reductions in negative health outcomes among adolescents.

Acknowledgments

This research was supported by a dissertation grant from the Tobacco-Related Disease Research Program (grant #19DT-0002) to Melissa Little (P.I.). During development of the project, Dr. Little was supported as a predoctoral fellow on a training grant from the National Cancer Institute (grant #T32-CA 009492; Pentz, P.I.). While writing the manuscript, Dr. Little was supported by a postdoctoral fellowship from the National Cancer Institute (grant #R25 CA90956).

References

  1. Beets M, Flay B, Vuchinich S, Acock A, Li KK, Allred C. School climate and teachers’ beliefs and attitudes associated with implementation of the Positive Action program: A Diffusion of Innovations model. Prevention Science. 2008;9:264–275. doi: 10.1007/s11121-008-0100-2. [DOI] [PubMed] [Google Scholar]
  2. Bentler PM. On tests and indices for evaluating structural models. Personality and Individual Differences. 2007;42:825–829. [Google Scholar]
  3. Blake SM, Ledsky RA, Sawyer RJ, Goodenow C, Banspach S, Lohrmann DK, Hack T. Local school district adoption of state-recommended policies on HIV prevention education. Preventive Medicine: An International Journal Devoted to Practice and Theory. 2005;40:239–248. doi: 10.1016/j.ypmed.2004.05.028. [DOI] [PubMed] [Google Scholar]
  4. California Department of Education. Getting results: Developing safe and healthy kids update 2. Sacramento, CA: Healthy Kids Program Office, California Department of Education; 2001. [Google Scholar]
  5. California Department of Education. County offices of education. 2011a Retrieved from http://www.cde.ca.gov/re/sd/co/coes.asp.
  6. California Department of Education. Tobacco Use Prevention Education (TUPE) competitive grant applicants. Sacramento, CA: 2011b. Prepared by the Safe and Healthy Kids Program Office, California Department of Education. [Google Scholar]
  7. California HealthyKids Resource Center. Research validated titles. 2010 Retrieved from http://www.californiahealthykids.org/c/@MxTzHUbt3Nbj2/Pages/searchres.html.
  8. Center for Substance Abuse Prevention. HHS Pub. No. (SMA)09-4205. Rockville, MD: Center for Substance Abuse Prevention, Substance Abuse and Mental Health Services Administration; 2009. Identifying and selecting evidence-based interventions revised guidance document for the strategic prevention framework state incentive grant program. [Google Scholar]
  9. Cho H, Hallfors DD, Iritani BJ, Hartman S. The influence of “No Child Left Behind” legislation on drug prevention in U.S. schools. Evaluation Review. 2009;33:446–463. doi: 10.1177/0193841X09335050. [DOI] [PubMed] [Google Scholar]
  10. Coburn C, Toure J, Yamashita M. Evidence, interpretation, and persuasion: Instructional decision making at the district central office. Teachers College Record. 2009;111:1115–1161. [Google Scholar]
  11. Durlak J, DuPre E. Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology. 2008;41:327–350. doi: 10.1007/s10464-008-9165-0. [DOI] [PubMed] [Google Scholar]
  12. Ellickson PL. Schools. In: Coombs RH, Ziedonis D, editors. Handbook on drug abuse prevention: A comprehensive strategy to prevent the abuse of alcohol and other drugs. Needham Heights, MA: Ally & Bacon; 1995. pp. 93–120. [Google Scholar]
  13. Ennett ST, Ringwalt CL, Thorne J, Rohrbach LA, Vincus A, Simons-Rudolph A, Jones S. A comparison of current practice in school-based substance use prevention programs with meta-analysis findings. Prevention Science. 2003;4:1–14. doi: 10.1023/a:1021777109369. [DOI] [PubMed] [Google Scholar]
  14. Estabrooks PA, Glasgow RE. Translating effective clinic-based physical activity interventions into practice. American Journal of Preventive Medicine. 2006;31:45–56. doi: 10.1016/j.amepre.2006.06.019. [DOI] [PubMed] [Google Scholar]
  15. Fagan AA, Hanson K, Hawkins JD, Arthur MW. Bridging science to practice: Achieving prevention program implementation fidelity in the community youth development study. American Journal of Community Psychology. 2008;41:235–249. doi: 10.1007/s10464-008-9176-x. [DOI] [PubMed] [Google Scholar]
  16. Fagan AA, Mihalic S. Strategies for enhancing the adoption of school-based prevention programs: Lessons learned from the blueprints for violence prevention replications of the Life Skills Training program. Journal of Community Psychology. 2003;31:235–253. [Google Scholar]
  17. Flay BR, Biglan A, Boruch RF, Castro FG, Gottfredson D, Kellam S, Ji P. Standards of evidence: Criteria for efficacy, effectiveness and dissemination. Prevention Science. 2005;6:151–175. doi: 10.1007/s11121-005-5553-y. [DOI] [PubMed] [Google Scholar]
  18. Flay B. School-based smoking prevention programs with the promise of long-term effects. Tobacco Induced Diseases. 2009;5:6. doi: 10.1186/1617-9625-5-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Ginexi E, Mabry P, Collins L, Rivera D, Ridenour T, Valente T, Gest S, Biglan A. Systems science methodology for prevention science. Paper presented at the Society for Prevention Research 18th Annual Meeting; Denver, Colorado. 2010. [Google Scholar]
  20. Gingiss PM, Roberts-Gray C, Boerm M. Bridge-It: A system for predicting implementation fidelity for school-based tobacco prevention programs. Prevention Science. 2006;7:197–207. doi: 10.1007/s11121-006-0038-1. [DOI] [PubMed] [Google Scholar]
  21. Green LW, Glasgow RE. Evaluating the relevance, generalization, and applicability of research: Issues in external validation and translation methodology. Evaluation and the Health Professions. 2006;29:126–153. doi: 10.1177/0163278705284445. [DOI] [PubMed] [Google Scholar]
  22. Greenberg MT. School-based prevention: Current status and future challenges. Effective Education. 2010;2:27–52. [Google Scholar]
  23. Gregory A, Henry DB, Schoeny ME. School climate and implementation of a preventive intervention. American Journal of Community Psychology. 2007;40:250–260. doi: 10.1007/s10464-007-9142-z. [DOI] [PubMed] [Google Scholar]
  24. Hallfors D, Godette D. Will the ‘Principles of Effectiveness’ improve prevention practice? Early findings from a diffusion study. Health Education Research. 2002;17:461–470. doi: 10.1093/her/17.4.461. [DOI] [PubMed] [Google Scholar]
  25. Ho R. Handbook of univariate and multivariate data analysis and interpretation with SPSS. New York, NY: Chapman and Hall/VRV, Taylor & Francis Group; 2006. [Google Scholar]
  26. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling. 1999;6:1–55. [Google Scholar]
  27. Kreuter MW, Bernhardt JM. Reframing the dissemination challenge: A marketing and distribution perspective. American Journal of Public Health. 2009;99:2123–2127. doi: 10.2105/AJPH.2008.155218. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. MacKinnon DP, Fairchild AJ, Fritz MS. Mediation analysis. Annual Review of Psychology. 2007;58:593. doi: 10.1146/annurev.psych.58.110405.085542. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Mihalic S, Fagan A, Argamaso S. Implementing the Life Skills Training drug prevention program: Factors related to implementation fidelity. Implementation Science. 2008;3:5. doi: 10.1186/1748-5908-3-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Muthén LK, Muthén BO. Mplus: User’s guide. 7. Los Angeles, CA: Muthén & Muthén; 1998–2012. [Google Scholar]
  31. National Cancer Institute, & Substance Abuse and Mental Health Services Administration. Research-tested intervention programs. 2010 Retrieved from http://rtips.cancer.gov/rtips/index.do.
  32. National Institute on Drug Abuse, National Institute of Health, & United States Department of Health and Human Services. Preventing drug use among children and adolescents: A research-based guide. 2010 Retrieved from http://www.nida.nih.gov/Prevention/Prevopen.html.
  33. National Research Council and Institute of Medicine. Preventing mental, emotional, and behavioral disorders among young people: Progress and possibilities. In: O’Connell ME, Boat T, Warner KE, editors. Committee on the Prevention of Mental Disorders and Substance Abuse among Children, Youth, and Young Adults: Research Advances and Promising Interventions. Washington DC: The National Academies Press; 2009. [PubMed] [Google Scholar]
  34. Office of Safe and Drug Free Schools, & United States Department of Education. Exemplary and promising: Safe, disciplined, and drug-free schools programs. 2010 Retrieved from http://www2.ed.gov/admins/lead/safety/exemplary01/report_pg9.html.
  35. Payne AA, Gottfredson DC, Gottfredson GD. School predictors of the intensity of implementation of school-based prevention programs: Results from a national study. Prevention Science. 2006;7:225–237. doi: 10.1007/s11121-006-0029-2. [DOI] [PubMed] [Google Scholar]
  36. Quality Education Direct Inc. QED California district 2008 database: Data users guide. Denver, CO: Quality Education Direct Inc; 2009. [Google Scholar]
  37. Ringwalt C, Hanley S, Vincus A, Ennett S, Rohrbach L, Bowling J. The prevalence of effective substance use prevention curricula in the nation’s high schools. Journal of Primary Prevention. 2008;29:479–488. doi: 10.1007/s10935-008-0158-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Ringwalt CL, Ennett S, Vincus A, Thorne J, Rohrbach LA, Simons-Rudolph A. The prevalence of effective substance use prevention curricula in U.S. middle schools. Prevention Science. 2002;3:257–265. doi: 10.1023/a:1020872424136. [DOI] [PubMed] [Google Scholar]
  39. Roberts-Gray C, Gingiss PM, Boerm M. Evaluating school capacity to implement new programs. Evaluation and Program Planning. 2007;30:247–257. doi: 10.1016/j.evalprogplan.2007.04.002. [DOI] [PubMed] [Google Scholar]
  40. Rogers EM. Diffusion of innovations. 2. New York, NY: The Free Press; 1983. [Google Scholar]
  41. Rohrbach LA, Howard-Pitney B, Unger JB, Dent CW, Howard KA, Cruz TB, Johnson CA. Independent evaluation of the California tobacco control program: Relationships between program exposure and outcomes, 1996–1998. American Journal of Public Health. 2002;92:975–984. doi: 10.2105/ajph.92.6.975. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Rohrbach LA, Grana R, Sussman S, Valente TW. Type II translation: Transporting prevention interventions from research to real-world settings. Evaluation and the Health Professions. 2006;29:302–333. doi: 10.1177/0163278706290408. [DOI] [PubMed] [Google Scholar]
  43. Rohrbach LA, Gunning M, Grana R, Gunning G, Sussman S. Dissemination of Project Towards No Drug Abuse (TND): Findings from a survey of program adopters. Substance Use & Misuse. 2010;45:2551–2566. doi: 10.3109/10826081003725278. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Rohrbach LA, Ringwalt CL, Ennett ST, Vincus AA. Factors associated with adoption of evidence-based substance use prevention curricula in U.S. school districts. Health Education Research. 2005;20:514–526. doi: 10.1093/her/cyh008. [DOI] [PubMed] [Google Scholar]
  45. Sallis JF, Owen N, Fisher EB. Ecological models of health behavior. In: Glanz K, Rimer BK, Viswanath K, editors. Health behavior and health education: Theory, research, and practice. 4. San Francisco, CA: Jossey-Bass; 2008. pp. 465–486. [Google Scholar]
  46. SAS Institute Inc. SAS/C Online Doc TM. Release 9.0. Cary, NC: SAS Institute; 2000. [Google Scholar]
  47. Shinn M. Understanding implementation of programs in multilevel systems. Prevention & Treatment. 2003:6. [Google Scholar]
  48. Skara S, Sussman S. A review of 25 long-term adolescent tobacco and other drug use prevention program evaluations. Preventive Medicine. 2003;37:451–474. doi: 10.1016/s0091-7435(03)00166-x. [DOI] [PubMed] [Google Scholar]
  49. Snyder F, Vuchinich S, Acock A, Washburn I, Beets M, Li KK. Impact of the Positive Action program on school-level indicators of academic achievement, absenteeism, and disciplinary outcomes: A matched-pair, cluster randomized, controlled trial. Journal of Research on Educational Effectiveness. 2010;3:26–55. doi: 10.1080/19345740903353436. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Spillane JP. State policy and the non-monolithic nature of the local school district: Organizational and professional considerations. American Educational Research Journal. 1998;35:33–63. [Google Scholar]
  51. Spoth R, Rohrbach L, Greenberg M, Leaf P, Brown CH, Fagan A, Hawkins JD. Addressing core challenges for the next generation of type 2 translation research and systems: The Translation Science to Population Impact (TSci Impact) framework. Prevention Science. 2013;14:319–351. doi: 10.1007/s11121-012-0362-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Substance Abuse and Mental Health Services Administration, & United States Department of Health and Human Services. SAMHSA model programs. 2010 Retrieved from http://nrepp.samhsa.gov/
  53. United States Department of Health and Human Services, & Substance Abuse and Mental Health Services Administration. National registry of evidence-based programs and practices (NREPP) 2010 Retrieved from http://nrepp.samhsa.gov/index.asp.
  54. Yu C-Y, Muthén B. Evaluation of model fit indices for latent variable models with categorical and continuous outcomes. Paper presented at the annual meeting of the American Educational Research Association; New Orleans, LA. 2002. [Google Scholar]

RESOURCES