State-level education standards for substance use prevention programs in schools: a systematic content analysis

Abstract

Purpose

Three-fourths of public schools in the U.S. maintain instructional programs to discourage alcohol, tobacco, and other drug (ATOD) use. State-sanctioned instructional standards attempt to direct this ATOD preventive education. No existing research, however, systematically codes these standards across all grades and states. We performed such an analysis.

Methods

We retrieved ATOD standards information from all 50 states and the District of Columbia from multiple sources, including the National Association of State Boards of Education's State School Health Policy website. Three independent researchers classified and cross-validated ATOD standards (inter-rater agreement = 98%) based on recommended content domains and pedagogic delivery methods.

Results

We find substantial grade-level variation in standards. Elementary schools emphasize generic social skills and affective skills, whereas middle and high school standards focus on knowledge about biological and behavioral consequences of ATOD use. States also vary widely in their content and coverage of standards. Two-thirds of states do not include standards in all content areas considered “evidence-based.”

Conclusions

The ATOD curricular agenda for the majority of states falls well below recommended content and delivery benchmarks. We intend for our harmonized dataset—the first of its kind—to promote research that examines the relation among state ATOD standards, actual classroom instruction, and adolescent ATOD use.

Introduction

Despite a general secular decline since the 1990s, adolescent illicit drug use remains more prevalent in the United States than in other high-income countries [1]. The most recent national survey of high school youth (2011), moreover, reports that by grade twelve 19% of students currently smoke, 39% currently drink alcohol, and 25% currently use illicit drugs [2]. A substantial portion of these youth also experiment with alcohol, tobacco, and other drugs (ATOD) by 8^th grade. Adolescent ATOD use, moreover, varies substantially by state of residence. Prevalence estimates of “ever smoked” by high school, for example, range from 23.1% in Utah to 59.5% in Louisiana [2]. The relatively high prevalence of adolescent ATOD use in the U.S., in conjunction with extensive research that finds increased risk of lifetime addiction and other attendant adverse physical and mental health outcomes, underscores the importance of implementing evidence-based efforts to delay, prevent, or reduce adolescent ATOD use [3].

Some researchers contend that school-based ATOD prevention efforts may effectively complement broad social, economic and policy initiatives [4]. Federal and state governmental agencies, moreover, routinely direct funds to schools to augment ATOD prevention efforts. For instance, the federal government recently allocated an estimated $600 million per year towards the Safe and Drug Free Schools and Communities Act, the largest source of school-based ATOD education funds [3]. This funding, however, remains controversial in that it often results in the enactment of programs deemed by education and public health scholars as ineffective and/or lacking a research base [5,6]. A recent U.S. Department of Education analysis further reports that less than 10% of youth substance use programs in middle and high schools show evidence of research-proven effectiveness [7]. Systematic reviews, moreover, suggest that the lack of a long-term benefit of most school-based ATOD prevention programs may warrant their termination [8,9]. These reviews and other analyses call into question the cost-effectiveness of current ATOD prevention efforts in schools [10].

Whereas extensive literature in health policy finds that specific state legislation (e.g., cigarette taxes) precedes a reduction in adolescent ATOD use [11], we know of no work that examines whether state school-based health education requirements influence adolescent ATOD use. A key first step to analyzing this relation involves a comprehensive assessment of whether, and to what extent, state ATOD instructional standards reflect current evidence regarding ATOD prevention. This paper thus builds on previous work [12,13] to systematically evaluate the prevalence of content germane to ATOD prevention in state health education standards—the detailed documents that states produce and distribute to guide health instruction in public schools. The objective of this paper is to determine whether ATOD-related instructional standards vary by state and grade level.

Instructional standards attempt to establish a set of shared expectations in the highly decentralized context of American public education. In core academic areas such as mathematics and English, state and federal educational accountability policies enforce instructional standards by testing student mastery of standards and providing sanctions and rewards to schools based on student performance [14]. By contrast, instructional standards in health education are largely informational. Nevertheless, the standards carry the force of law in most states and may shape ATOD instruction by influencing the health curricula that schools adopt and by providing guidelines for instructors on appropriate topics for each grade level. States have increasingly adopted ATOD educational standards over time (see Figure 1). Currently, 44 states outline some form of school-based ATOD education standard, as compared with only six states in 1970.

Figure 1.

Cumulative frequency plot over time of the number of states that implemented school-based ATOD educational standards, 1970-2010.

A recent U.S. Department of Education report on youth substance use endorses 22 school programs as evidence-based [7]. We used characteristics of these programs, as well as results from a meta-analysis of over 200 programs by Tobler and colleagues [15], to code state health education standards documents. Based on literature in child psychology highlighting the importance of age-specific and life-course patterns in cognitive development, decision-making, and the social environment, our systematic analysis takes a developmental perspective [16,17]. We analyze each state ATOD education standard by grade level (i.e., a proxy for developmental stage).

Our systematic approach to categorizing ATOD educational standards may uncover substantial variation across states in agenda setting for school-based prevention efforts. We, moreover, make our dataset publicly available (http://inid.gse.uci.edu/public-use-data/) to encourage further analyses.

Methods

Variables and Data

We used the National Association of State Boards of Education's State School Health Policy website (http://www.nasbe.org/healthy_schools/hs/), as well as the websites of state boards of education, and direct communication with state educational and health and human services departments, to collect state standards related to health and behavioral education from all 50 states and the District of Columbia. We focused on only the current (i.e., 2010) standards from each of the states with the assumption that these standards influenced instruction in the fall of 2012. We did not analyze any standards issued before 2010. We did not study human subjects; therefore no human subjects approval by the Institutional Review Board was required.

In most cases, state departments of education distribute the standards documents to administrators and health instructors to influence the design and content of health instruction in K-12 schools. These standards carry the force of law in most states. Thirty-eight states legally mandate ATOD instruction in public schools, and legislation in 20 states explicitly requires schools to enact the instruction described in ATOD educational standards. Few states, however, have fiscal or other mechanisms in place to enforce these standards. Approximately half of the local school districts in the U.S. may circumvent state standards and design their own ATOD instruction. Nonetheless, research about the implementation of instructional standards in academic subjects such as mathematics or English Language arts indicates that even weakly-enforced standards exert a modest influence on instruction [13]. Therefore, we suspect that ATOD educational standards shape health instructor training, inform the adoption of health education curricular materials and prevention programs at schools, and guide teachers as they plan their day-to-day instruction.

The literature does not converge on specific educational components that prevent or reduce adolescent ATOD use. Whereas several scholars assert benefits of various school-based programs [18,19], evaluations of school-based prevention programs typically include small samples and utilize non-experimental designs. Accordingly, critics contend that these evaluations provide limited evidence to establish best practices in ATOD prevention programming [20,21,8-10]. We, therefore, relied on Tobler and colleagues' systematic review of over 200 school-based ATOD programs rather than any particular program touted as exemplary, to derive curricular themes of ATOD instruction [15]. Tobler and colleagues classify programs based on four content areas (Knowledge, Affective skills, Refusal skills, and Generic social skills) and two modes of instructional delivery (Non-interactive or Interactive). Table 1 describes these content areas and modes of instructional delivery and provides examples of representative questions within ATOD standards across grades and states.

Table 1.

Classification of school-based ATOD instructional standards by content area and delivery method (adapted from Table 4 of Tobler et al., 2000).

Knowledge (Biology and Behavior): Short and long-term physiological and behavioral effects of ATOD
Kentucky, Grade 4:  Students will distinguish between the use and misuse of drugs, alcohol, and tobacco, and identify the effects each use might have on the body.	Massachusetts, Grade 8: List the potential outcomes of prevalent early and late adolescent risk behaviors related to tobacco, alcohol, and other drugs, including the general pattern and continuum of risk behaviors involving substances that young people might follow.	New York, Grades 9-12: Identify the consequences associated with engaging in high-risk behaviors which compromise health, such as smoking, violent behavior, or driving under the influence of alcohol/drugs.
Knowledge (Context): Empirical prevalence and causes of ATOD use
Michigan, Grade 4:  Analyze data that supports that most young people in middle school do not use tobacco, alcohol, or other drugs.	Georgia, Grade 7: Examine how information from the media influences health behaviors (Ex: Examine a variety of alcohol/tobacco advertisements and the messages being sent to adolescents).	New Hampshire, Grades 9-12: Present/examine a variety of externalelements [influencing ATOD use], such as media, parents, ethnic, legal, peers, geographic, societal.
Affective: Self-esteem, personal self-awareness, attitudes, beliefs and values
Washington, Grade 3:  Understands emotions and how they affect self and others. Discusses strategies to deal with different emotions.	Indiana, Grade 8: Explain how personal values and beliefs can encourage abstinence from risk behaviors (e.g., bullying, sexual activity, using tobacco, alcohol, or other drugs, etc.).	Colorado, Grades 9-12: Develop self-management skills to improving health by staying tobacco, alcohol, and drug-free.
Refusal: Information-gathering, saying no, healthy alternatives, public commitment
Indiana, Grade 4:  Illustrate how to assist others to make positive health choices. Example: Design a bumper sticker to encourage others to abstain from tobacco use (e.g., personal use of smokeless tobacco, cigarettes, or cigars; secondhand smoke; etc.)	Florida, Grade 7-8: Identify health-related situations that require the application of a thoughtful decision-making process such as prescription drug use and abuse.	Oklahoma, Grades 9-12: Demonstrate refusal and resistance skills for unhealthy risk behaviors (e.g., alcohol, tobacco, drugs).
Generic social skills: Communication skills, assertiveness, decision-making, coping skills, social/dating skills, goal-setting
California, Grade 2:  Demonstrate communication skills to alert an adult about unsafe situations involving drugs or medicines.	New Jersey, Grade 8: Predict social situations that may require the use of decision-making skills. Justify when individual or collaborative decision-making is appropriate.	Kansas, Grades 9-12: The student will demonstrate the ability to utilize various strategies when making decisions related to health needs and risks of young adults.
Interactive delivery method: role-playing, discussions, problem-solving.
Tennessee, Pre-K – Grade 2:  The teacher may… provide plastic drink cups or distortion glasses for students to look through to simulate the affects of alcohol on vision and have the students describe how their vision is affected.	Maryland, Grade 8: Compose a personal health goal and measure progress towards its achievement. Devise a plan that addresses personal strengths, needs, and health risks.	Colorado, Grades 9-12: “Inquiry question: What would I say if my best friend wanted to drive home after drinking alcohol at a party?”

For our classification of state-level ATOD educational standards, we built on Tobler and colleagues' framework but make two key refinements. First, we create two subgroups of the Knowledge content category: “Biology and Behavior” and “Context.” Knowledge-Biology and Behavior captures the mechanisms by which ATOD causes short- and long-term effects on the body and on behavior and mood. By contrast, Knowledge-Context describes the prevalence of ATOD among adolescents and adults, as well as the importance of parents, peers, society and the media in influencing ATOD use. While Knowledge-Biology and Behavior standards aim to discourage drug and alcohol abuse by familiarizing students with the attendant risks, Knowledge-Context standards attempt to help students understand the role peers and social influences play in ATOD use decisions.

Our second refinement to Tobler and colleagues' framework involved classification of ATOD standards by grade level of intended instruction. This refinement coheres with a developmental perspective which builds on earlier work on ATOD standards [13] to emphasize prevention efforts tailored to age-related patterns of competence and ATOD use [17]. This perspective, as applied to prevention, intends to match specific skills to environmental and social contexts that vary according to childrens' stage of cognitive and social development. Each state separates their educational standards by grade level, which allows for clear coding of ATOD prevention content by grade.

Approach

We generated a list of topics and approaches for ATOD education based upon several reviews of in-school ATOD education [15,16,22-24]. Many of these refer explicitly to ATOD, such as instructional standards about the long-term health risks associated with drug use. Others do not explicitly refer to ATOD but remain central to ATOD prevention approaches, such as standards related to students' self-esteem and self-management. In addition, we conducted a preliminary review of the standards to add ATOD topics explicitly covered in three or more states.

After reaching agreement on a comprehensive set of topics, three investigators independently reviewed each state document and coded for the presence or absence of each of the topics in each grade's standard (i.e., Kindergarten through 12^th grade). If we found no standards for a given grade, we coded all topics as not included. We copied codes across grades when standards repeated for multiple grades. In the rare cases in which investigators returned inconsistent codes, we resolved disagreements via iterative discussion and by adding codes. Inter-rater agreement for coding decisions was 98%.

We then summed content codes across the following five general areas adapted from Tobler and colleagues [15]: Knowledge-Biology and Behavior, Knowledge-Context, Affective skills, Refusal skills, and Generic social skills. These content codes thus gauge the intensity, by grade level, of content coverage in each of these five areas by summing all relevant concepts included in the standards. If a standard covered several content areas, we allowed for the single standard to count across multiple areas.

Content codes in the Knowledge-Biology and Behavior area include general instruction about the risks associated with ATOD use, as well as specific instruction about short-term health consequences, long-term health consequences, psychological risk, economic risks, risks for friends and family, legal risks, potential costs for youth's life plans, and consequences of ATOD use for decision-making. The Knowledge-Context area includes instruction about the causes and social influences that lead to ATOD use as well as information about the empirical prevalence of ATOD use. The Affective Skills area includes instruction about self-awareness and self-management skills. The Refusal Skills area includes instruction about decision-making that refers specifically to ATOD, as well as instruction in strategies for saying no, gathering information about ATOD, alternatives to ATOD, and public commitment and other approaches that youth can take to discourage peer ATOD use. The General Social Skills codes cover similar areas more generally, without making specific reference to ATOD, such as general instruction about social awareness, relationships, decision-making, goal-setting, and other social skills. Finally, Interactive delivery serves as an indicator variable for the presence of explicit wording that encourages interactive methods in ATOD instruction. For each state/grade combination, this code asks whether the state standards mention “role-playing,” “practicing skills,” “discussing,” or similar practices in the context of ATOD education. In the descriptive analyses, we tabulated and plotted the resulting data to assess state and grade-level variation in ATOD instructional standards.

Results

Table 1 provides examples of standards in each of our content and delivery themes by state and grade level. A qualitative reading of the standards indicates distinct language across the elementary, middle, and high school grades. Within the Generic social skills area, elementary school standards tend to emphasize communication skills to parents and teachers. By contrast, middle and high school standards emphasize communication with friends and peers. This shift of communication from authority figures to peer groups with advancement in grade level conforms with literature in child development. The literature finds that in pre-adolescence, authority figures influence behaviors more than do peer groups. The relative influence of authority figures on ATOD behaviors, however, reportedly diminishes by mid-adolescence (i.e., middle school) [25].

We find substantial variation across states in content and level of prescriptive detail of instructional standards. The volume of ATOD-related mentions in the standards, when aggregated within state but across grade levels, ranged from 20 (Iowa) to 298 (New Hampshire). Of the 44 states with standards, we find no discernable regional clustering in the volume of ATOD standards.

Next, we examined the intensity of ATOD language from a developmental perspective by plotting mentions of various content areas by grade level (Figure 2). The volume of ATOD standards is lowest in kindergarten and rises through 9^th grade. The volume of standards by content area differs by grade level. In kindergarten and first grade, standards emphasize Generic social skills and, to a lesser extent, Affective skills. Elementary school standards also mention Generic social skills more frequently than middle and high school standards. However, Knowledge-Behavior and Biology standards are most prevalent in middle and high school. The prevalence of Refusal skills gradually rises from 3^rd grade until 9^th grade, although Refusal skills standards are rarer than Knowledge-Biology and Behavior standards.

Figure 2.

Volume of ATOD prevention standards, by content area and grade level, for the 44 states with instructional standards.

Tobler and colleagues' meta-analysis of over 200 school-based ATOD education programs indicates that interactive instructional methods confer greater program effectiveness than do non-interactive methods [15]. Only 16 state standards documents refer to interactive instructional methods at every grade level. By contrast, 17 states with an ATOD standard have no language—at any grade level—regarding interactive instructional delivery.

Figure 3 provides a sense of the state-level variation of the presence of ATOD standards on our derived “best practice” score. We used the coding results of the ATOD questions to derive a summary score which, consistent with previous reviews, may promote effective ATOD instruction [22, 23]. Each state may score up to six points on the composite index. States could gain a possible five points for coverage in most grades (i.e., seven or more grades from K through 12), one point for each of the five content areas listed in Table 1 (e.g., Knowledge-Biology and Behavior, Knowledge-Context, Affective skills, Refusal skills, and Generic social skills). States could receive an additional point if they promoted interactive instructional delivery at all 13 grades. We refer the reader to our website (see Introduction) for the six detailed state-by-grade level tables, and attendant data documentation, that we used to derive the “best practice” score.

Figure 3.

Summary “best practice” score of ATOD instructional standards, by state, based on coverage of five content areas and interactive delivery method. The “best practice” score ranges from zero to six.

The map indicates substantial variation in our “best practice” score (median = 4; standard deviation = 1.8). Twenty-one states scored a five or higher. California, Michigan, New Hampshire, Rhode Island, South Carolina, Tennessee, Utah and Wyoming scored the highest possible value (i.e., six). These states represent a diverse cross-section of the U.S.; that is, they include ethnically diverse populations in rural and urban, as well as coastal and interior, settings. We observe a geographic pattern in that Western and Mountain regions, as well as New England, contain a disproportionately large number of states that score in the top tier of the “best practice” score (Figure 3). Color maps of ATOD content areas by school level (elementary, middle, and high school) also reveal substantial variation across states (available upon request).

Discussion

State instructional standards regarding ATOD prevention may influence health curricula that schools adopt by providing guidelines for instructors regarding topics and delivery methods in health education classes. Federal and state governments spend $1 billion annually on universal, school-based ATOD prevention programs in the U.S. [3], and youth spend approximately 14% of their waking hours in school [26]. However, we know of no work that evaluates whether state instructional standards influence health education class content and, in turn, student ATOD use [3]. We aim to conduct such analyses as part of a multi-year research project. As a necessary first step, we systematically coded, and now make publicly available, the ATOD curricular agenda of each state and grade level. We find that two-thirds (i.e., 34 of 51) of states do not include standards in all content areas identified as evidence-based [15,22,23]. In addition, only 16 states discuss the “best practice” of interactive content delivery across all grades. In sum, we identify many states in which the curricular agenda regarding ATOD prevention in schools falls well below the content and delivery expectations from the literature.

We also find substantial variation across grade level in the content and volume of ATOD standards. Standards at the kindergarten and first grade levels emphasize general social skills such as communication and assertiveness, whereas by middle and high school, the standards focus on instruction relating to knowledge about the biological and behavioral consequences of drugs. The volume of ATOD standards also rises in high school, when experimentation of ATOD during adolescence also peaks [27]. The grade level differences in content and focus area conform with a developmental perspective of ATOD prevention that emphasizes general social and individual skill development at early ages. The high volume of the Knowledge–Biology and Behavior content in middle and high school, however, appears at odds with the findings of Tobler and colleagues' meta-analysis [15] and others in that instruction in this area does not vary positively with desirable youth ATOD outcomes [22,23].

Our content analysis complements recent work by Seitz and colleagues [13] in two important ways. First, we classify standards before high school and by grade level. This classification conforms with a developmental perspective [16]. ATOD use often begins in middle school, and problematic substance use in high school correlates positively with several social and emotional factors as early as the 6^th grade. The developmental perspective asserts that gains in social and emotional learning in elementary and middle school may reduce the risk of subsequent ATOD use. Second, we address an additional dimension of school policy: interactive vs. didactic content delivery. This facet of instructional standards may exert a stronger influence on effective instruction than does any content domain [10,23].

Our classification of standards by specific grade level permits examination of more refined hypotheses regarding the age-appropriateness of universal curricula depending on the child's stage of cognitive development. For instance, if our grade-level data (shown in Figure 2) correspond with actual ATOD instruction, this circumstance raises the intriguing question of whether middle and high school ATOD education classes should emphasize refusal skills more than knowledge of biological and behavioral effects. Another strength includes the public dissemination of our data which we hope will lead to greater state accountability between curricular standards and actual “in the classroom” instruction.

Our content analysis of state standards points to potentially important variation in the intended curricula for ATOD prevention education across states and grades. However, we do not know whether enacted curricula covaries with the state standards. Previous research indicates that instructors often do not closely adhere to ATOD prevention program curricula [3,28]. In addition, given the varying degree to which school districts adhere to state ATOD standards, limitations include lack of information on these standards across school districts. We know of no dataset that would allow such a district-level comparison. States also may differ in terminology used to describe ATOD-related instruction which may lead to difficulty in comparing state standards. In practice, however, our high inter-rater agreement in coding content areas (98%) supports the notion of comparable language constructs across all states.

Future research should consider the ways that state historical and legislative environments shape their ATOD educational agendas. Illinois, for instance, outlines detailed standards for the development of socio-emotional skills in pre-adolescence, but does not mention ATOD prevention in their standards (http://www.isbe.state.il.us/ils/social_emotional/standards.htm). We expect that a closer examination of legislation over time—of the historical dataset shown in Figure 1 which differs from the 2010 standards data we analyze in this manuscript— may identify a broader policy context under which ATOD policies in the U.S. evolved since the 1970s.

More broadly, we hope that our database stimulates a new line of research regarding the relative effectiveness of various approaches to adolescent ATOD prevention. If research on state educational standards also finds a positive influence on instructional practices, exploring the relation between these standards and student behaviors will provide a new, population-based mechanism for evaluating various ATOD prevention strategies. In particular, given the variation that we observe in state ATOD standards across grade levels, we believe that these standards may allow important insights into the optimal timing of various forms of ATOD education.

We note, however, that systematic reviews call into question school-based ATOD efforts. In a review of 56 alcohol intervention studies, Foxcroft and colleagues did not find any robustly designed, school-based study that showed a long-term reduction in alcohol misuse [8]. Thomas and colleagues' review of tobacco interventions, moreover, reports that while social competence interventions reduce smoking initiation in the long term, many trials yielded no demonstrable benefit [9]. We expect that next generation, innovative school-based ATOD programs may learn from these past studies and hone their approaches consistent with a developmental perspective [10].

State instructional standards represent a central component to the policy response of adolescent ATOD use in the U.S. Thirty-eight states mandate ATOD instruction in public schools and 44 states have explicit instructional standards related to ATOD instruction. Approximately three-fourths of U.S. public schools have at least one ATOD prevention program in place [29]. Yet, to date, researchers have not rigorously evaluated the standards that states created to shape ATOD prevention education. By compiling and describing these standards, we aim to stimulate research in this neglected arena.