Assessing campus alcohol policies: Measuring accessibility, clarity, and effectiveness

Abstract

Background:

Excessive alcohol consumption poses significant hazards to health and safety on college campuses. While substantial research exists regarding effective policies for preventing alcohol-related problems in the communities surrounding campuses, on-campus alcohol policies have received far less attention.

Methods:

Official campus alcohol policies (CAPs) were retrieved from the websites of the 15 member schools of the Maryland Collaborative to Reduce College Drinking and Related Problems, a voluntary statewide collaborative. CAPs were assessed for accessibility, clarity, and effectiveness. In addition to assessing whether campuses were in compliance with federal regulations for comprehensiveness of policies, a measure of likely policy effectiveness was developed through the use of two Delphi panels drawing on alcohol policy researchers and on campus and community practitioners, respectively. The panels rated 35 potential policies and 13 possible sanctions; lists of policies and sanctions were compiled primarily from what was already in existence at one or more member schools.

Results:

For most campuses, the CAPs could be located within 30 seconds, but tended to be spread across multiple web pages. Language used to communicate the policies tended to be complex and above the reading level of someone with a high school education. At least half of the schools had less than half of the possible policies rated most or somewhat effective by the Delphi panels. Schools were more likely to employ the most effective sanctions, but somewhat and ineffective sanctions were also not uncommon.

Conclusions:

CAPs are an important element in reducing negative consequences of alcohol consumption on college campuses. A higher level of research scrutiny is warranted to understand the extent to which CAPs are associated with excessive drinking, but this research describes an evidence- and expert-informed assessment approach that colleges can use to regularly analyze and update their CAPS.

INTRODUCTION

According to the 2017 National Survey on Drug Use and Health (NSDUH), 53.6% of college students between the ages of 18 and 22 drank alcohol during the past month, compared with 48.2% of their non-college attending peers (Substance Abuse and Mental Health Services Administration, 2018). Binge drinking, defined as consuming five or more drinks on the same occasion, was reported by 35% of college students (Substance Abuse and Mental Health Services Administration, 2018).

College students who drink heavily are at increased risk for causing significant harms to themselves and others. Nationwide in 2005 (the latest year for which data are available) an estimated 1,825 students ages 18 to 24 attending 2- and 4-year colleges died from alcohol-related injuries, while 599,000 were injured, 696,000 were hit or assaulted, and 97,000 were victims of sexual assault or date rape by a student who had been drinking (Hingson et al., 2009). The more students drink, the more likely they are to experience acute negative consequences, such as blackouts, getting physically sick, or having had sex they regretted (Barnett et al., 2014); and to perform less well academically (Singleton, 2007). Excessive alcohol consumption can also result in long-term harms including vulnerability to addiction, impaired neurocognitive function (e.g., learning, memory, and information processing deficits), cancer, and liver disease (Bagnardi et al., 2015; Hingson et al., 2009; Yoon et al., 2014; Zeigler et al., 2005). Beyond the individual student, it is worth noting that excessive drinking can result in substantial damages and costs for colleges and their surrounding communities, such as property damage, crime, and noise disturbances (Wechsler et al., 2002a; Wechsler et al., 1995).

Previous research indicates that state- and community-level policies aimed at altering the drinking environment are associated with reduced drinking and drinking-related problems within the college population. College students in states with more restrictive alcohol policies exhibit significantly lower binge drinking and drinking participation (Chaloupka and Wechsler, 1996; Nelson et al., 2005a). Environmental strategies, which use policies and enforcement to influence drinking patterns in communities around the campus, can also be effective in reducing college drinking (Nelson et al., 2005b; Toomey et al., 2007). On campus, schools employ a wide range of policies to try to reduce and prevent excessive alcohol consumption (Lenk et al., 2012). However, on-campus policies have been subjected to far less rigorous examination of effectiveness than policies affecting the surrounding communities.

In 2015 the National Institute on Alcohol Abuse and Alcoholism released the College Alcohol Intervention Matrix (CollegeAIM), outlining a set of individual- and environmental-level approaches that college administrators can use as a guide to create alcohol intervention strategies unique to their campuses. According to CollegeAIM, the most effective environmental-level policies include increasing alcohol excise taxes and community-wide restrictions on alcohol price and physical availability (National Institute on Alcohol Abuse and Alcoholism, 2015).

In addition to state- and community-wide policy measures, colleges and universities are federally required to have their own set of campus alcohol policies (CAPs). To receive federal funds, an institution of higher education must, according to Part 86 of the Education Department General Administrative Regulations (EDGAR), at minimum provide students with a written policy that: 1) bans unlawful possession and use of alcohol; 2) states applicable local, state, and federal laws; 3) describes the health risks associated with alcohol consumption; 4) cites any available alcohol counseling/treatment programs; and 5) clearly enumerates the sanctions to be imposed in instances of policy violation (U.S. Government Printing Office, 2018).

Judicial decisions have further indicated that colleges and universities have the same responsibilities as other property owners of ensuring activities and programs meet minimum standards to safeguard students by taking protective measures that are reasonable, comprehensive, and enforceable (Bickel and Lake, 1999). Colleges and universities might fulfill this responsibility in part through enacting and enforcing CAPs beyond those required by EDGAR. Certain CAPs that reduce the environmental availability of alcohol have shown promise for reducing student binge drinking and alcohol-related harms. Offering substance-free housing represents one example of a CAP that is associated with lower levels of binge drinking and both direct and secondhand consequences. Students living in substance-free housing have been found to be less likely to binge drink, less likely to be heavy episodic drinkers, and experience less alcohol-related problems compared with students in substance-unrestricted housing (Wechsler et al., 2001b). Banning alcohol on campus is another effective policy: students attending colleges that ban alcohol on campus have been found to be 30% less likely to binge drink and more likely to abstain from alcohol than students attending colleges that allow alcohol consumption (Wechsler et al., 2001a).

However, despite the various legal requirements, little research exists on the effectiveness of the majority of CAPs. It can be difficult to extricate the individual effects of one policy from a set of CAPs and/or from the state and federal policy context in which a college or university exists. For example, the national “A Matter of Degree” program funded by the Robert Wood Johnson Foundation encouraged implementation of a range of on- and off-campus policy interventions including keg registration, responsible beverage service training, campus-community police collaboration, and substance-free residence halls at 10 campuses (Weitzman et al., 2004). While the campuses that implemented the most policies influencing alcohol environments were found to have reduced binge drinking and alcohol-related injuries among students relative to students attending 32 comparison universities, it is unclear which interventions—or whether the general introduction of a package of environmental-level interventions accompanied by campus-community collaboration, as opposed to any single policy measure—were effective. Similarly, the Study to Prevent Alcohol Related Consequences (SPARC) led to 11 on-campus policy changes across the experimental campuses, and the campuses with the highest “dose” of the intervention saw reductions in interpersonal consequences due to others’ drinking and drinkers causing alcohol-related injuries to others. However, again it was not clear which specific policies were associated with which outcomes (Wolfson et al., 2012). Consequently, most CAPs are implemented with little evidence regarding their effectiveness or overall impact.

Further, student awareness (or lack thereof) of CAPs might attenuate their effectiveness. Several studies have surveyed students and school administrators about their institution’s CAPs. Wechsler et al. (2002b) surveyed students at 119 schools, asking if they received specific information from their college regarding state and local laws and policies concerning alcohol sale, use, and consumption. The authors found that students at schools with more binge drinking had greater indirect exposure to these policy components—such as through mailings, handouts, posters, and signs—than students attending schools with less binge drinking. Direct exposure to the policy components—such as through lectures or workshops—did not differ among schools with high- and low-binge drinking. These cross-sectional findings might imply that schools with a higher prevalence of problem drinking are more proactive at informing students of their CAPs. Rhodes et al. (2005) surveyed students at five historically black colleges and universities about the existence and specific components of their CAPs. Among male students in this population, lack of awareness of the school’s CAPs or of the health risks of alcohol was associated with a significant increase in risk for binge drinking.

To our knowledge there are three published instruments available to assess various aspects of CAPs. The first, devised by Mitchell et al. (2005), assessed the comprehensiveness and completeness of online alcohol policies at all 4-year colleges in Minnesota and Wisconsin by surveying high-level college administrators and collecting from each school’s website written alcohol policies having to do with five specific measures: 1) total ban of campus alcohol consumption; 2) ban of alcohol consumption at campus sport events; 3) ban on keg delivery to residence halls; 4) restrictions on alcohol sponsorship; and 5) alcohol-free student housing options. The authors compared information provided in the written policies with that acquired through administrator interviews. This comparison revealed poor agreement between the two sources, as many colleges did not provide their CAPs in detail on their websites.

The second instrument was created and used by Faden and Baskin (2002) in evaluating the online accessibility of CAPs at the 52 “top” universities listed in the 2002 U.S. News and World Report ranking. The instrument encompassed 35 policy items grouped in four categories: policy information, restrictions/requirements, possible sanctions to individual students, and possible sanctions to student groups. The authors found that alcohol policies were difficult to find online at the majority of the 52 schools. Policy components were often dispersed among multiple pages on the schools’ websites, and the information provided was often incomplete according to interviews with college administrators at the institutions. Faden et al. (2009) repeated this assessment five years later and found the same 52 schools’ CAPs were more accessible and contained more complete content.

The third and final instrument, developed by Hirschfeld et al. (2005) incorporated components of the instruments devised by Mitchell et al. (2005) and by Faden and Baskin (Faden and Baskin, 2002; Faden et al., 2009). It employed an evaluation scheme using two raters to compare CAPs across four dimensions: 1) accessibility, 2) comprehensiveness, 3) enforcement procedures, and 4) clarity. The Hirschfeld et al. (2005) instrument rated accessibility on a six-point scale based on how “difficult” raters perceived finding a school’s policy to be through a web-based search. Comprehensiveness was rated on a six-point scale according to how many of 21 pre-defined policy criteria each school’s campus policy contained, such as permitting only certain types of alcohol on campus or establishing substance-free dormitories. Enforcement procedures were rated on a six-point scale based on the clarity and certainty with which disciplinary actions and sanctions were tied to policy violations in the campus policy document. Clarity was rated on a six-point scale and defined as the perceived “clearness” of a policy to the reader. Hirschfeld et al. (2005) used these four dimensions to evaluate substance abuse policies available online at 24 colleges and universities. The authors found that the schools’ CAPs were generally accessible, and—as a proof of concept—that their instrument could be used to review them.

However, major components of these three instruments rely on subjective criteria, which lessen these tools’ utility for evaluating CAPs. The instruments designed by Faden et al. (Faden and Baskin, 2002; Faden et al., 2009) and Mitchell et al. (2005) do not provide systematic criteria for assessing the online accessibility of a school’s CAP. This hinders meaningful measurement of online policy accessibility both within and among different institutions, as neither of these measures of accessibility provide any methodological guard against variations in policy reviewer capability or subjectivity. While Hirschfeld et al. (2005) do provide systematic criteria for assessing online policy accessibility, these criteria hinge on the policy reviewer’s subjective perception of the relative difficulty of finding a school’s CAP online, with the rating system based on how “obscure” or “confusing” finding a school’s CAP is to a particular reviewer. Hirschfeld et al. (2005) measure of clarity is similarly subjective, relying on the interpretability of policy language to a reviewer whose own capacity for understanding the text might be influenced by numerous personal demographic characteristics and life experiences. Furthermore, none of the existing instruments provides any measure of the effectiveness of the policies assessed. While there is merit in measuring the accessibility, clarity, comprehensiveness and sanctions of a policy, it is perhaps equally—if not more important—to understand first whether the policy itself is likely to be effective. Together, these limitations weaken the capacity of these instruments to generate informative metrics regarding a school’s CAPs.

To address this gap, the authors collected CAPs from 15 post-secondary member schools from the Maryland Collaborative to Reduce College Drinking and Related Problems [described in greater detail elsewhere (Arria and Jernigan, 2018)], and evaluated them for their accessibility and clarity. The authors then used a Delphi panel process to evaluate the policies and sanctions for likely effectiveness. We undertook this assessment of CAPs to address the dearth of literature on effective CAPs, and to develop a standardized, objective instrument that could be used by colleges to make their CAPs clearer, accessible, and effective. By assessing the CAPs of 2- and 4-year universities in Maryland in the context of comprehensive evidence-based recommendations known to reduce excessive alcohol consumption, we sought to offer meaningful feedback that colleges could use to reduce excessive drinking and provide a campus environment that promotes the health, safety, and success of their students.

Our specific objectives were to: 1) identify commonly used CAPs among member schools in the Maryland Collaborative; 2) rate these CAPs’ accessibility and clarity; 3) rate the likely effectiveness of each policy measure and associated sanctions through a Delphi panel review; and 4) develop an evaluative tool that incorporates these variables and can be used by schools to assess their policies.

MATERIALS AND METHODS

Data Collection

Online CAPs from the 15 member schools of the Maryland Collaborative were gathered from school websites. School names were de-identified in analyses to preserve anonymity. Student enrollment varied from 900 to 19,049 full-time undergraduate students. The schools varied with respect to gender distribution, from 23% to 95% female; racial composition, from 15% to 88% minority; and community context, from rural to urban. A total of approximately 73,000 full-time, undergraduate students attended the 15 member schools that comprised the Maryland Collaborative at the time this research was conducted.

Each school’s CAPs were collected in December 2016 using an approach a late adolescent prospective or current college student (or their parent) might use: a Google search including the school name and the term “alcohol policy.” All data were current as of December 15, 2016. If that initial search did not generate appropriate results, it was followed by a second Google search using the school name and the term “student handbook.” If these two initial searches did not produce the most recent or comprehensive policies, the search terms “alcohol policy,” “alcohol,” and “student handbook” were employed using the school website’s search feature until the most recent and complete policy documents were retrieved. Policy documents were saved on a shared drive as pdf files in folders by school. All scores were maintained in a common Excel spreadsheet.

Each school’s policies were then assessed based on four distinct factors: 1) accessibility, 2) clarity, 3) effectiveness, and 4) sanctions for policy violations.

Accessibility

The accessibility measure, adapted from Hirschfeld et al. (2005) addressed both the extent to which policy information was dispersed across several documents or websites and how much time it took to locate the policy online. Accessibility score metrics are described in Table 1. While students might have alternate access to a school’s CAP, such as by receiving a physical copy in the mail or through a student-only online portal, general online availability was used to gauge accessibility for the broader public (including prospective students and parents). CAP accessibility was assessed by four independent reviewers, whose scores were averaged to produce a single score for each school.

Table 1.

Accessibility scoring metric for campus alcohol policies

Score	Meaning
1	Policy was spread across multiple locations and took more than 30 seconds to find
2	Policy was spread across multiple locations and took less than 30 seconds to find
3	Policy existed in one location and took less than 30 seconds to find

Clarity

Clarity was measured using the Flesch readability score from Microsoft Word, which assesses sentence length and number of syllables per word (Flesch, 1948). Flesch scores range from 0 to 100, with higher scores indicating greater text clarity: for example, a score of 60 to 70 designates “standard” or “plain” English; a score of 30 to 50 is considered “difficult,” confusing, and best understood by those with some college education; and a score below 30 indicates the text is “very difficult” and best understood by college graduates.

Comprehensiveness/Effectiveness

Each school’s CAPs were assessed based on: 1) the presence of 40 specific policy elements, and 2) the likely relative effectiveness of 35 of those elements at preventing excessive alcohol consumption as determined by two Delphi panels, described below. The 35 policy elements evaluated by the Delphi panels were compiled from the schools’ existing CAPs, previous literature and existing instruments (Faden and Baskin, 2002; Hirschfeld et al., 2005; Mitchell et al., 2005; Wechsler et al., 2001a), promising emerging measures (Bormann and Stone, 2001; Demers et al., 2013), and other commonly used policies (Marchell et al., 2013; Sanburn, 2015).

Expert and practitioner evaluation of likely effectiveness.

Delphi panel methodology is used to provide expert guidance in research areas where data are incomplete, imprecise, or controversial (Dalkey and Helmer, 1963; de Meyrick, 2003). We employed a Delphi panel of five alcohol policy experts who evaluated the 35 specific policy elements for their likely effectiveness at preventing excessive alcohol consumption among the general college student population. Of note, in this study policy elements were analyzed according to their anticipated effectiveness, not their efficacy. The question posed to Delphi panelists was, “Were this policy fully implemented and fully enforced, what would its maximum level of effectiveness be?” Recognizing that varying levels of implementation might accompany even ideal policy language, panelists were asked to disregard the impact that potential variation in policy implementation might have on policy effectiveness in the interest of conducting the panel expediently.

The Delphi panelists in this study first rated each policy element on a five-point Likert scale ranging from 0 (least effective) to 5 (most effective). Then, using the average rating of each policy element as a reference, the panelists participated in a round-table conference call in which they re-categorized each policy element according to group consensus into one of three categories: most effective, somewhat effective, or ineffective. Certain policy elements were placed in an additional “not scored” category if they were determined to be important for reasons other than to modify college student drinking behavior.

We supplemented the alcohol policy expert panel with a practitioner Delphi panel, comprised of seven members of the Maryland Collaborative’s Advisory Board. This voluntary group included three vice-presidents of student affairs, a campus Alcohol and Other Drug Center Director, a campus Alcohol and Drug Education Officer, a member of a local town-gown coalition, and a local prevention coordinator. These practitioners offered a unique perspective as they play key roles in implementing the policy measures on campus and have had a first-hand look at what has been successful. There was a high level of agreement between the two Delphi panels, with the exception of seven policies that were termed somewhat effective by the expert panel and most effective by the practitioners; these were ultimately categorized as most effective and the disagreement between the two panels was footnoted in the final report to the schools.

Policy elements classified as most effective were those determined by panelists as likely to comprehensively affect the physical and/or normative drinking environment on campus. Banning alcohol consumption in public places, for example, was classified as most effective both because it restricts campus alcohol consumption and is likely to influence social norms around drinking. Likewise, banning alcohol at student organization recruitment events both sets the normative tone for the school year (as recruitment often happens soon after students arrive on campus) and actively restricts student alcohol consumption.

Policy elements classified as somewhat effective generally reflect little or mixed research regarding their effects on student drinking behavior and were perceived by panelists as likely having lesser reach than most effective elements. These include measures such as bans on hard alcohol or kegs, which, while they would affect the whole student body, have been shown in the literature to result in beverage switching by students [i.e., shifting from hard liquor to beer or beer to hard liquor, accordingly (Kilmer et al., 1999)]. Policy measures with lesser reach include mandated registration of campus events with alcohol and hosting alcohol-free events, which would only affect select segments of the student body.

Ineffective policy measures or elements were those that were considered unlikely to significantly influence drinking behavior across the general student body based on expert panelist consensus and prior research. For instance, keg registration was classified this way, given recent literature demonstrating that keg registration policies, if not accompanied by other efforts to reduce the environmental availability of alcoholic beverages, can inadvertently contribute to high-risk drinking (Fell et al., 2015a; Fell et al., 2015b).

Policy measures categorized as not scored included bans on drinking paraphernalia, which were considered important symbolically but not practically, and existence of recovery houses, which were agreed to be an important policy measure for addiction treatment and recovery but unlikely to have a large impact on the student body as a whole.

Evaluating Sanctions for Policy Violations

Both sets of Delphi panelists additionally evaluated the likely effectiveness of sanctions for violating CAPs, using the same rating methodology. These sanctions, as they are often termed in the schools’ alcohol policies, were gathered from each school’s CAPs. Delphi panelists classified sanctions as most effective, somewhat effective, ineffective, or not scored through the same process as described above. A total of 13 sanctions were evaluated, including parental notification, individual and organizational probation, suspension, expulsion, and alcohol education.

Sanctions classified as most effective were those that panelists agreed would have a strong, population-wide deterrent effect as standalone measures. These included student organization probation and loss of student organization status, which panelists agreed were potent deterrents due to their permanent, structural effects on the physical and normative drinking environments on campus. Sanctions categorized as somewhat effective included those that would have some effect as a stand-alone sanction, but whose effectiveness would be heightened as part of a “package” of graduated sanctions or stepped-care procedures working in tandem to deter policy violations, and appropriately intervene with individuals found in violation, respectively. These included individual suspension and probation, which, because of their severity and the extended deliberative process often required to enforce them, become less swift and certain, and consequently less effective, at the population level as stand-alone measures. Additionally, panelists determined that alcohol evaluation/screening would be most effective as a part of a stepped-care model with a clear referral in the policy language to an evidence-based program for the students in need. While this consequence does not necessarily have a population-level impact, it could prove crucial for students benefitting from brief interventions and referral to treatment.

Ineffective sanctions were those that were considered by panelists to be ineffective on their own, such as providing students solely with warnings in the event of CAP violations. However, the panelists did identify that ineffective sanctions might have some effectiveness if folded into a structured or graduated array of sanctions. In this case, panelists expressed a preference for graduated sanctions beginning, for example, with probation with the threat of ultimate suspension, combined with other sanctions such as community service, and culminating in expulsion.

The expert panel classified alcohol treatment as not scored, viewing it as impractical on a large scale as a standalone measure but important in unique cases; the practitioner panel scored this as most effective for students who need it.

RESULTS

Accessibility

The mean average accessibility score rating across the 15 member schools of the Maryland Collaborative was 2.14, meaning that the CAPs within the collaborative generally were spread across multiple locations and took less than 30 seconds to find.

Clarity

The median Flesch-readability score across the 15 schools was 29. This score suggests that the text in the CAPs among the 15 Maryland Collaborative schools is considered very difficult and best understood by a college graduate. The maximum Flesch-readability score was 38.1, indicating that even the most “clear” CAP would be considered difficult, confusing, and best understood by someone with at least some college education. The lowest Flesch-readability score was 20.8, implying that the text was very difficult to understand and best understood by a college graduate (see Figure 1).

Figure 1.

Clarity (Flesch-readability Scores) of Campus Alcohol Policies, Maryland Collaborative Member Schools, 2016

Policy Measure Effectiveness

Of the 35 individual policy measures evaluated by the Delphi Panel, 17 were categorized as most effective, 13 as somewhat effective, 3 as ineffective, and 2 as not scored (see Table 2). The median number of most effective policy measures among Maryland Collaborative member schools was 6, of somewhat effective measures was 5, of ineffective measures was 1, and of not scored measures was 0.

Table 2.

Campus alcohol policies by effectiveness level

Most Effective	Somewhat Effective	Ineffective	Not Scored
1. Prohibition of alcohol in public places on campus 2. Prohibition of alcohol consumption in the college’s stadium/arena 3. Prohibition of tailgating on campus 4. Prohibition of alcohol at student organization member recruitment events 5. Prohibition of drinking games (including activities/objects that promote them) 6. Prohibition of alcohol delivery to campus 7. Mandated ID check at campus events with alcohol 8. Mandated limit on total alcohol provided at campus events with alcohol 9. Explicit mention of campus police patrolling off-campus neighborhoods regularly 10. Explicit mention that there will be campus consequences for off-campus violations 11. Prohibition of sponsorship by alcohol manufacturers or alcohol outlets* 12. Prohibition of kegs on campus* 13. Prohibition of campus advertising for alcohol/alcohol outlets* 14. Prohibition of picturing/mentioning alcohol in flyers for events on campus* 15. Mandated registration of campus events with alcohol* 16. Mandated server training at campus events with alcohol* 17. Explicit mention that college receives names of students cited or arrested off campus from local police*	1. Prohibition of hard alcohol on campus 2. Prohibition of alcohol consumption in private dorm rooms 3. Prohibition of alcohol consumption in common rooms in residence halls 4. Mandated food at campus events with alcohol 5. Mandated non-alcoholic beverages at campus events with alcohol 6. Mandated host training (in safe alcohol service/alcohol abuse) at campus events with alcohol 7. Mandated security at campus events with alcohol 8. Alcohol-free events 9. Required Friday classes (or shifting more required classes to Fridays) 10. Optional substance-free residence halls/floors 11. Explicit mention of campus security’s ability to patrol off-campus neighborhoods 12. Restriction on use of student funds for purchase of alcohol 13. Explicit mention that some or all campus alcohol restrictions extend off campus	1. If kegs are allowed, required registration of kegs on campus 2. No mention of relationship with local police force 3. Use of student funds to purchase alcohol over the phone	1. Prohibition of alcohol paraphernalia 2. Recovery houses on campus

^*Scored by the panel of practitioners as most effective and by the panel of alcohol policy experts as somewhat effective.

Sanctions Effectiveness

Of the 13 individual sanctions evaluated by the Delphi Panel, 5 were categorized as most effective, 6 as somewhat effective, 2 as ineffective, and 1 as not scored (see Table 3). Across all 15 Maryland Collaborative schools, the median number of most effective sanctions for policy violations was 4, of somewhat effective sanctions was 5, and of ineffective sanctions was 2.

Table 3.

Campus alcohol policy sanctions by effectiveness level

Most Effective	Somewhat Effective	Ineffective
1. Parental notification	1. Fine	1. Warning
2. Dismissal from housing	2. Community service	2. Alcohol education
3. Student organization probation	3. Alcohol evaluation/screening
4. Loss of student organization status	4. Individual probation
5. Alcohol treatment*	5. Individual suspension
	6. Expulsion

^*Scored by the panel of practitioners as most effective and not scored by the panel of alcohol policy experts.

DISCUSSION

We successfully collected and scored the CAPs for a diverse group of 15 institutions of higher education. CAPs were generally spread across multiple pages on school websites, but could usually be located within 30 seconds. However, the language of the policies tended to be legalistic and difficult for a reader who had not already completed a college education to understand.

Schools had a mix of policies considered most and somewhat effective, but the fact that the median number of policies in each category was less than half of the total showed that many schools were missing policies. Overall the effectiveness ratings demonstrated that, based on what is in place in peer institutions, CAPs could be substantially revised to be more effective. In contrast, schools were much more likely to include somewhere in their written policies the sanctions termed effective by the expert and practitioner panels; however, they were also likely to include the sanctions termed ineffective.

Furthermore, to be most effective, sanctions should be clearly tied to specific policy violations. Deterrence is critical to effective enforcement, and rests on the perception that violations will incur swift, certain, and sufficiently severe sanctions (Decker and Kohfield, 1990; Grogger, 1991; Ross, 1984). Certainty has been shown to be the most relevant and effective factor in deterring behaviors that warrant sanctions, especially among college students (Nagin and Pogarsky, 2001; Tittle, 1969). Assessment of certainty was deemed beyond the scope of this research; however, recommendations about increasing certainty, often in the form of tying specific sanctions to violation of specific policies, were included in the final recommendations to the schools.

The goal of this effort was to provide institutions of higher education with clear and actionable recommendations for improving their CAPs. To this end, individual reports, including the results of the assessment as well as specific recommendations for improvement in each school’s written policies, were developed for every school. Individual school results were compared with the median scores for the 15 schools as a group, to provide a basis for comparison with peer institutions. Delivery of the individual school reports was followed by meetings with relevant officials at each school. These meetings provided school officials the opportunity to correct findings from the report if policies had changed since the research was done, in recognition of the fact that CAPs change over time and require vetting from multiple stakeholders.

The meetings also offered a forum for discussing specific recommendations tailored to each campus, based both on the specific environment of the campus and the results of the assessment in comparison with other schools. These recommendations typically provided feedback to the schools regarding each area of the assessment. Thus, for accessibility, a common recommendation was for the schools to create a single on-line document containing all college alcohol policies and associated consequences, along with information about the health risks of consuming alcohol and available counseling resources. For clarity, to improve Flesch-readability scores, specific text was provided to the schools showing how readable policy content would look. For effectiveness, each school received a list of the policies scored most effective that were missing in their written policies, and asked to consider adopting those policies if appropriate to their campus.

The data collection exercise will be repeated in the 2018–2019 school year to evaluate whether the assessment influenced the school’s written CAPs.

This study has several limitations. Accessibility was assessed by multiple reviewers and scores were averaged; however, it is still not possible to remove an element of subjectivity from this process. Moreover, schools might use methods other than their websites to distribute and make students aware of their CAPs; this assessment did not attempt to capture these methods. Additionally, although the policy measures and sanctions included in this assessment were representative of what we found on the participating campuses, and while we supplemented them with policies that research has found to be effective, the lists of policies and consequences were not exhaustive. Further, these as well as other policy measures not currently included in this assessment might be contextually more or less appropriate depending on the campus. While the level of enforcement of existing CAPs is critical to their effectiveness, assessing that was deemed beyond the scope of this study, which examined what was available to students and others in written form on college web sites. Assessment of the level of enforcement of the existing policies on each campus was not attempted. Finally, the Delphi panelists made their determinations about the likely effectiveness of policy measures/sanctions—not about the real-world efficacy of those measures or sanctions. Efficacy relies on the extent to which policy measures are implemented and enforced on a campus, which are variables not captured in this analysis and which warrant further research.

In response to the dearth of research on effective CAPs, we successfully collected, rated and provided to a diverse group of post-secondary educational institutions peer-oriented feedback on their CAPs. The goal of providing campus-specific feedback is to revise CAPs to be more accessible, clear, and effective. Given that excessive alcohol consumption can impede the health, safety, and academic success of students, this kind of assessment should be conducted more widely and more frequently. CAPs can and should be the first line of defense against alcohol problems on campus. Further research is needed into the effectiveness of specific policies and combinations of policies. Colleges should also regularly enforce their CAPs, and assess the perception of certainty among their students of enforcement and consequences resulting from violations of CAPs. This analysis provides a model for how such an assessment can be done, providing practical and campus-specific feedback that colleges and universities can use to improve their CAPs and further reduce excessive drinking in their campus communities.