The Sacramento Neighborhood Alcohol Prevention Project: Outcomes From a Community Prevention Trial

Abstract

Objective

This article reports the results of the Sacramento Neighborhood Alcohol Prevention Project (SNAPP). SNAPP set as its goal the reduction of alcohol access, drinking, and related problems in two low-income, predominantly ethnic minority neighborhoods, focusing on individuals between the ages 15 and 29, an age group identified with high rates of alcohol-involved problems.

Method

Two neighborhoods in Sacramento were selected to be the intervention sites because they were economically and ethnically diverse and had high rates of crime and other drinking-related problems. The quasi-experimental design of the study took a “phased” approach to program implementation and statistical examination of outcome data. Outcome-related data were collected in the intervention sites as well as in the Sacramento community at large. Five project interventions included a mobilization component to support the overall project, a community awareness component, a responsible beverage-service component, an underage-access law enforcement component, and an intoxicated-patron law enforcement component. Archival data were collected to measure and evaluate study outcomes and to provide background and demographic information for the study.

Results

Overall, we found significant (p < .05) reductions in assaults as reported by police, aggregate emergency medical services (EMS) outcomes, EMS assaults, and EMS motor vehicle accidents.

Conclusions

Results from the Sacramento Neighborhood Alcohol Prevention Project demonstrate the effectiveness of neighborhood-based interventions in the reduction of alcohol-related problems such as assaults, motor vehicle crashes, and sale of alcohol to minors.

Much enthusiasm has been generated by recent efforts to reduce alcohol-related problems with the use of environmentally based strategies. Included among these strategies are efforts to alter serving policies at on-premise alcohol outlets; increased enforcement of laws regulating drinking and driving; enforcement of alcohol-service and sales-to-minors laws; regulation of the proliferation of alcohol outlets; and, increasingly, efforts to reduce social access of alcohol to underage persons (Casswell and Gilmore, 1989; Hauritz et al., 1998; Hingson et al., 1996; Holder et al., 1997; Holmila, 1995, 1997; Homel et al., 1997;Midford et al., 1999; Putnam et al., 1993; Stewart and Casswell, 1993; Treno et al., 2005; Wagenaar et al., 2000).

The enthusiasm is merited, as these strategies have been generally demonstrated across a series of studies to be effective in the reduction of youth access to alcohol, subsequent drinking by young people, problematic consumption patterns among drinkers, rates of drinking and driving, and increases in community mobilization and media advocacy surrounding alcohol use and drinking-related problems.

However, such programs have generally been implemented across entire communities, leaving the question of whether they may be effectively applied at the neighborhood level. Although one study did demonstrate the effectiveness of environmental strategies implemented at the neighborhood level in bringing about changes in youth access to alcohol (Treno et al., 2005), the potential impact of such programs on outcomes, specifically drinking patterns and related problems, is not known.

The efficacy of implementing environmental strategies at the neighborhood level in terms of problem outcomes, although to date untested, shows considerable promise for a number of reasons. First, neighborhoods often possess considerable resources in the form of local community-based organizations (CBOs), which have extensive ties both within the neighborhood and throughout the broader community. From this standpoint they possess considerable social capital which may be expended in the community mobilization. Second, because the needs and interests may differ between neighborhoods within the same community, programs may be more narrowly tailored and targeted in terms of problems addressed, general strategies, and particular tactics. Third, it may be argued that certain programs are most effectively and appropriately addressed at specific levels. For example, the problem of drinking and driving may be seen largely as a community-level problem, because the problem occurs as drinkers traverse the community-wide roadway system (Gruenewald et al., 1996), whereas illegal drinking in public or assaultive violence may be seen as more of a neighborhood problem (Alaniz et al., 1998).

Fourth, whereas more formal responses to alcohol-related problems (e.g., enforcement of underage sales laws) might require the sort of political clout typically found at the city administrative level, less formal responses (e.g., training of parents on risks associated with drinking at underage parties) might be best implemented in contexts closer to those in which people live their daily lives—for example, trainings delivered through local CBOs such as parent-teacher organizations, church groups, or social-service programs. Finally, neighborhood-based interventions have shown promising results in other areas of health behaviors (O'Loughlin et al., 1999; Fisher et al., 1998).

Sacramento Neighborhood Alcohol Prevention Project (SNAPP)

This article reports the results of a project designed to test the efficacy of an environmentally based program to reduce alcohol access to young people and thereby reduce traumatic injuries associated with alcohol use. The primary goal of SNAPP was to implement and evaluate neighborhood-level interventions intended to reduce youth and young-adult access to alcohol, risky drinking, and associated problems, particularly in low-income ethnically diverse neighborhoods.

This prevention trial used an environmental approach that was developed in an earlier Community Trials Project (Holder et al., 2000). SNAPP represented an extension of Community Trials in that it posed three basic questions. First, could an environmental approach be tailored to the unique needs of economically and ethnically diverse populations? Second, could environmental strategies address the problem of intentional injuries (i.e., assaultive violence) in the context of more economically and ethnically diverse settings? Finally, could these more specifically tailored interventions be implemented at the neighborhood level?

To address these questions, SNAPP set as its goal the reduction of alcohol access, drinking, and related problems in two low-income, predominantly ethnic minority neighborhoods, focusing on individuals between ages 15 and 29. (This age group was identified with high rates of alcohol-involved problems that greatly concerned local community stakeholders.)

Method

Two neighborhoods in Sacramento, CA, identified here as the South and North sites, were selected to be the intervention sites because they were economically and ethnically diverse and had high rates of crime and other drinking-related problems. Of importance, these neighborhoods also had local CBOs sympathetic to—but inexperienced in—environmental prevention. These CBOs served as liaisons to other community agencies and organizations to facilitate community mobilization efforts around project goals while providing the local knowledge and energy for program implementation. As the key problem areas identified by neighborhood stakeholders were youth access to alcohol and alcohol-related violence, the interventions designed to address these issues centered around neighborhood alcohol outlets.

One contribution of SNAPP to the evaluation of community-based interventions was the opportunity it provided to apply geostatistical techniques to efficiently evaluate neighborhood-level outcomes. Rather than viewing communities as a whole, this approach views neighborhoods, and ultimately individuals and individual events within neighborhoods, as the units of analysis, each having specific geographic (i.e., spatial) relationships to one another. From this perspective, one could think of the geostatistical portion of the study as addressing the question of whether data from South and North census block groups (n = 37) change with respect to those from all other block groups in Sacramento (n = 243).

In selecting specific census block groups, a number of factors were taken into consideration. First, they were to be within the general geographic area served by the community partner in the project. Second, to the extent possible, they were to be roughly comparable in terms of size and demographic composition. Third, base rates for outcome measures needed to be sufficiently powered.

Table 1 details the differences in assault rates, sociodemographics, and alcohol outlets by study area. Both the North and South study sites had higher rates of assaults (per 1,000 population) as recorded by both police and emergency medical service (EMS) incidents than the at-large area. Additionally, 2000 census data showed that 39.3% and 31.0% of the population in the South and North study sites, respectively, was Hispanic compared with only about 20.8% in the at-large area. Similarly, about one third of residents in both study sites lived in poverty (35.6% for South, 33.0% for North) compared with 18.3% of the population of the at-large area. Data from the California Department of Alcoholic Beverage Control indicated that the two study areas had higher concentrations of both bars and off-premise alcohol outlets per roadway mile. Thus, although not comparable to the city of Sacramento at large, the North and South sites were roughly comparable to each other.

Table 1. Descriptive information for dependent and independent measures, by study area.

Variable	At large	South	North
Assaults, per 1,000 population
Crime incidents	8.46	9.77	12.58
Emergency medical service events	3.28	5.10	6.47
Sociodemographics, %
Black	17.2	17.4	19.5
Hispanic	20.8	39.3	31.0
Persons living in poverty	18.3	35.6	33.0
Alcohol outlets, per roadway mile
Bars	0.029	0.057	0.052
Restaurants	0.231	0.216	0.187
Off premise	0.133	0.362	0.245

Project logic model

Figure 1 presents a simplified logic model indicating both the relationship between alcohol outlets and alcohol problems and the manner in which the SNAPP components intervene. As indicated by the solid lines, alcohol outlets lead directly to formal access (i.e., purchases made directly from outlets). Formal access may lead to alcohol-related problems directly, as purchasers consume what they purchase, or indirectly, by increasing informal access (e.g., alcohol availability in the social environment). This distinction is important to emphasize, particularly when attempting to prevent youth access to alcohol. Environmental prevention efforts may fail by focusing exclusively on underage sales and ignoring social or informal access to alcohol, which have been identified as critical sources for underage youth (Harrison et al., 2000).

Figure 1. Logic Model.

The model also notes that formal access may directly contribute to alcohol-related problems. For example, alcohol outlets might serve as “magnets” that attract large numbers of individuals predisposed to engage in problem behaviors. In support of this idea is the finding that neighborhoods with high levels of formal access (i.e., high densities of alcohol outlets) are often characterized by alcohol-related problems such as violence (Gorman et al., 2001; Lipton and Gruenewald, 2002) and driving after drinking (Gruenewald et al., 1996, 2002).

Although most alcohol-prevention projects focus solely on changing individual drinking behaviors to reduce alcohol-related problems, SNAPP aimed to decrease alcohol-related problems at the community level by using environmental interventions to decrease both formal and social or informal access to alcohol. As indicated by the broken lines in Figure 1, the project interventions (indicated in circles) played three different and crucial roles in the reduction of overall alcohol access. The combined enforcement and merchant training activities targeted the reduction of alcohol sales to minors and beverage service to intoxicated patrons, thus limiting formal access. In contrast, the community awareness activities targeted the reduction of social or informal access. The community mobilization activities operated in support of the other project interventions by supporting the governmental sectors of the community (enforcement and licensing) with the nongovernmental sectors (CBOs and voluntary resident groups).

Following the logic of the project conceptual model, it was anticipated that program interventions would reduce both formal and social or informal access to alcohol that was likely to result in drinking and related problem outcomes. As reported elsewhere (Treno et al., 2005), the project seemed generally effective in terms of reducing youth access to alcohol. However, the question remained whether this intermediary effect translated into reductions of problem outcomes such as assaults, automobile crashes, or other forms of injuries.

Study design

The quasi-experimental design of the study took a “phased” approach to program implementation and statistical examination of outcome data. In both North and South sites as well as the Sacramento community at large, there was a baseline data collection phase. In the first year (between July 2000 and July 2001), while interventions were applied in the South neighborhood, the North neighborhood served as a no-treatment comparison site. Subsequently, application of the same interventions in the North neighborhood—between January 2002 and January 2003— provided a replication test of intervention effectiveness, and data from the South neighborhood allowed for testing of long-term impacts of the interventions. Data collected from Sacramento at large served to control for historical conditions affecting outcomes at both sites. (See Table 2 for project design.)

Table 2. Intervention and data collection timeline.

Study area	Preprogram	First wave	Second wave	Third wave
South	baseline	intervention	short-term posttest	long-term posttest
North	baseline	baseline	intervention	short-term posttest
At large	baseline	baseline	baseline	baseline

Project interventions

There were a total of five project interventions, including a mobilization component, a community awareness component, a responsible beverage service component, an underage-access law enforcement component, and an intoxicated-patron law enforcement component.

Community mobilization effort

To mobilize the neighborhoods in support of the overall project goals and interventions, project lead agencies worked with collaborative advisory committees, composed of members drawn from each of the two geographical areas and that worked to ensure intervention implementation and fidelity to project design, as well as subadvisory groups addressing specific project interventions. Research staff at the Prevention Research Center coordinated with these committees, which met on a monthly basis at each site, to implement project interventions and to collect local data for the evaluation. The advisory committees were multisectoral, with memberships including local law enforcement officers, medical service providers, and alcohol-beverage control agents as well as community activists, youth, parents, and CBO staff. Committee members were identified through consultations between the research team, who drew on prior experiences in community mobilization to identify critical community sectors and agents, and the lead agencies in each community, who drew on their knowledge of key stakeholders and local gatekeepers.

Although the overall study design was developed by the research team and lead agencies, the committees provided valuable guidance into appropriate means to conduct intervention and data collection activities, which shaped the actual implementation. Thus, for example, specific strategies for the scheduling of police stings were worked out based on both the needs of the project's evaluation and available police resources. The committees served in an outreach capacity, disseminating information about the intervention goals and activities to their constituent groups and linking program staff to appropriate venues for community awareness activities. The committee meetings, moreover, provided an effective way for community members to leverage increased commitment to addressing alcohol-related issues by law enforcement and other local authorities. In addition, committee members were often able to provide researchers with access to key data sources used for baseline and outcome measures, as described below. The community mobilization activities were implemented by a local coordinator and outreach staff based in the lead agencies and working closely with the scientific coordinator from the Prevention Research Center.

Community awareness

This component intended to increase awareness of the problems associated with youth and young-adult drinking to catalyze support for community mobilization efforts. The specific objectives of the component were as follows: (1) to increase parent and related adults' knowledge of underage and young-adult drinking and related problems, along with risks associated with providing alcohol to minors through social sources (e.g., at parties in one's home), (2) to motivate concerned adults to participate in activities to reduce underage drinking and problematic drinking among young adults, and (3) to alter youth perceptions of community norms about drinking in general and drinking in high-risk situations in particular.

Activities included neighborhood presentations of research findings and local statistics related to underage and problematic drinking to parents and other community groups, distribution of informational flyers and brochures, and youth participation as volunteers in data collection activities related to neighborhood alcohol availability. The presentations and brochures presented findings on drinking patterns and problems from previous studies on ethnic minority drinking at the state and national level as well as local data on alcohol availability derived from the project's data collection activities.

These project activities were adapted to the various ethnic groups inhabiting the intervention sites; for example, because many residents were not native speakers of English, the presentations and brochures were translated into the most common non-English languages (e.g., Spanish and Hmong). The staff for these project activities included the local program coordinator together with four part-time outreach workers who were residents of the study sites. Research scientists at the Prevention Research Center, in conjunction with the local coordinator, developed the outreach materials and provided training and oversight to the outreach staff to ensure the scientific validity of the presentations.

The local coordinator and outreach staff identified and contacted local community groups, ensured that the materials were presented in formats that were accessible as well as culturally appropriate, and conducted the distribution and presentation of material. For example, presentations were formatted as slide shows and included images of ethnic marketing of alcohol and depictions of underage youth drinking at community events as well as bullet-point presentations of statistical data, followed by open discussions of the issues and community responses.

Project staff and neighborhood volunteers distributed more than 3,000 brochures at community meetings and door-to-door throughout the two neighborhoods. Staff conducted 44 presentations at gatherings attended by 727 residents and 23 in-home presentations attended by 144 residents. The gatherings included parent-teacher organizations and parent-teacher associations from elementary, middle, and high schools attended by youth in the study areas; church groups and other voluntary associations; and staff and board meetings of neighborhood CBOs. We estimate that brochures and flyers reached one in five households, and face-to-face presentations reached one in 20 residents of the targeted neighborhoods.

Responsible beverage service (RBS)

The RBS program was designed to help retailers develop policies and train staff to reduce alcohol sales to minors and intoxicated persons. The RBS component was designed to increase compliance with existing alcohol policies through a combination of measures: obtaining sponsorship and support from local and state hospitality organizations, providing manager and server training for all on- and off-premise licensed alcohol outlets in the selected neighborhoods using a standard curriculum, developing a process to increase enforcement of existing laws regarding service to intoxicated customers, and obtaining endorsement of RBS from neighborhood bodies and organizations. The central feature of the component was the RBS training, which focused on (1) raising awareness of the problems of youthful drinking and the sale of alcohol to minors and intoxicated persons; (2) raising awareness of community norms against the sale of alcohol to minors and intoxicated persons; (3) training in methods of age identification, estimating intoxication levels, and techniques for interacting with customers; (4) providing information about laws and associated penalties for sales to minors and intoxicated persons; and (5) reviewing with owners and managers existing policies and providing new policy options to reduce sales to minors and intoxicated persons.

This RBS component was implemented in a systematic fashion. Specifically, lists of neighborhood outlets were obtained from the California Department of Alcoholic Beverage Control. Owners and managers were invited by outreach staff to attend RBS workshops conducted by a subcontracted professional RBS trainer who was well experienced in alcohol service as well as in policy research and training. The trainings were held in half-day sessions and included skills building as well as lecture components that focused on specific details of the alcohol service industry. For example, servers working in on-premise locations were instructed in (1) monitoring customer consumption of alcohol beverages and pacing heavier drinkers to prevent intoxication, (2) preventing intoxicated patrons from driving or engaging in other risky behaviors, (3) serving drinks in standard serving sizes, (4) promoting food and nonalcoholic beverages, (5) avoiding price promotions (e.g., “happy hours”), and (6) checking age identification for all customers appearing to be below 30 years of age.

Owners and managers of establishments who did not attend RBS trainings were strongly encouraged to attend by repeated visits from outreach staff accompanied by law enforcement officers. In the South site, staff members from 40% of the on-premise outlets (n = 52) and 69% of the off-premise outlets (n = 54) attended RBS trainings, whereas in the North site staff from 56% of the on-premise outlets (n = 45) and 70% of the off-premise outlets (n = 37) participated in trainings.

Underage-access law enforcement

This component focused on increasing actual and perceived enforcement of laws prohibiting alcohol sales to minors and was accomplished by working with neighborhood police to increase the number of off-premise sting operations. The operations occurred on weekend evenings (Friday and Saturday). A specially recruited and trained minor worked with the police and attempted to purchase a six-pack of beer. If asked for identification, the youth would claim not to have any but would attempt to complete the purchase. A police officer would be present outside the outlet, but hidden from view, during the purchase attempt. All outlets that sold to the youth were immediately cited.

Letters announcing increased enforcement were sent to all outlets in the study sites and surrounding areas to inform owners and managers of increased police activity. The letters stated that increased enforcement would be continued into the future. Follow-up letters were sent to the outlets after they were visited in the sting operation. This letter informed the owner or manager that their outlet was, in fact, visited as part of the intervention effort. Owners and managers of outlets that did not sell to the underage minor were congratulated and informed that the sting operations would continue on a random basis. During the course of the project, 37 establishments in the South and 25 establishments in the North were visited, with sales occurring in 20.5% of attempts in the South and 30.8% in the North. There were also sting operations in the at-large sites, where sales occurred 37.5% of the time.

Intoxicated-patron law enforcement

Enforcement efforts targeting on-premise establishments were designed to parallel those of the off-premise intervention. Letters from the local police were sent to all premises in and around the study sites informing them that stepped-up enforcement of laws regarding sales to intoxicated patrons and minors would become a regular police activity, that undercover police would visit all establishments at least once during the following year, and that minors may be used to determine if sales were made to underage persons.

At least six such operations were performed each month during the intervention period. Undercover police officers and minors employed by the police were deployed to on-premise establishments to assess whether intoxicated or underage patrons were being served. The police officers documented the results of these operations in written reports made available to the research staff. A total of 33 establishments were visited by the Sacramento Police Department, primarily in the North site during the second intervention phase.

Establishments found in compliance received a letter congratulating them on being in compliance with the law but would also be informed that regular visits to bars and restaurants would continue into the future. Premises found in violation of the law were warned they could be cited. Implementation of this component was hindered by reluctance on the part of local police, in part because of departmental understaffing resulting from a combination of unforeseen factors. Thus, it could be argued that the on-premise intervention targeting intoxicated-patron services was not strictly comparable to the more aggressive off-premise intervention targeting youth access.

Process evaluation

The general notion underlying the project logic model was that these interventions would alter aspects of the alcohol environment associated with acute alcohol-related problems. Mobilization, community awareness, and law enforcement activities were tracked on a monthly basis using a management information system developed for this purpose (Treno et al., 2005). In addition, changes in access to alcohol were monitored through the use of Apparent Minor and Pseudo Intoxicated Patron Surveys.

The Apparent Minor Survey documented the frequency of sales of alcohol to apparent minors in the experimental and comparison neighborhoods. The three waves of the survey were conducted during Years 1, 3, and 5, respectively. The survey involved a census of all off-premise outlets, including liquor, convenience, and grocery stores, within the study neighborhoods as well as within a 0.5-mile buffer zone surrounding them and of a random sample of off-premise outlets from the remainder of Sacramento. The samples were obtained from readily available Department of Alcoholic Beverage Control data containing information about all off-premise licenses in the selected regions.

The Apparent Minor Surveys were conducted using protocols similar to those used in previous research (Forster et al., 1994, 1995; Grube, 1997). The buyers were young women who were at least age 21 but who were judged by an independent panel as appearing younger. Women were used rather than men for this survey because prior research has indicated that female apparent minors are more likely to be able to purchase from off-premise establishments than male apparent minors (Forster et al., 1995). Panel members were selected based on consideration of their experience in judging young people's ages (e.g., youth workers, high school teachers, police).

The survey consisted of two stages. First, an off-premise establishment scouting survey was conducted to provide important information to facilitate the conduct of the Apparent Minor Survey (e.g., establishment hours of operation, relative safety of area) and also to provide information on the establishment and its immediate environment (e.g., presence of other alcohol establishments in the area, general maintenance of the area). Next, a buyer and driver team was sent to each sampled outlet on two occasions, once on a Friday night and once on a Saturday night. A different buyer was sent on each occasion.

On arriving at the outlet, the driver parked the car and the buyer entered the outlet and attempted to purchase a six-pack of beer. In those cases where beer was not available, the buyer attempted to purchase an inexpensive bottle of wine. After leaving the outlet, the buyer recorded whether the purchase attempt was successful, the gender, ethnicity, and approximate age of the clerk, and whether identification had been requested. The buyer also recorded additional descriptive information about the interior of the outlet (e.g., number of cash registers, presence of signs stating age identification policies, etc.).

The Pseudo Intoxicated Patron Survey documented the frequency of service of alcohol to “pseudo-intoxicated” patrons in the experimental and comparison neighborhoods. Pseudo-intoxicated patrons were actors who had been trained to follow a protocol involving attempts to purchase alcohol while appearing intoxicated. Using a sampling procedure similar to that used in the Apparent Minor Survey, the Pseudo Intoxicated Patron Survey involved a census of all on-premise alcohol retail outlets (i.e., bars and restaurants licensed to sell alcohol) within the two study neighborhoods as well as within a 0.5-mile buffer zone surrounding them and of a random sample of on-premise outlets from the remainder of Sacramento City. Alcohol-license data were obtained from Department of Alcoholic Beverage Control and contained information about all on-sale alcohol licenses in the target areas. Similar to the Apparent Minor Survey, an on-premise establishment survey was conducted before the Pseudo Intoxicated Patron Survey to determine hours of operation, the type of patrons using the establishment, and the general safety of the area.

The Pseudo Intoxicated Patron Survey involved the use of pairs of trained project staff persons—one an observer and another who acted as if he were intoxicated. For this survey, all pseudo-intoxicated patrons were played by male staff members. The use of female staff members was considered but rejected for reasons of safety. Care was taken to match ethnicity of the actor with the predominant ethnicity of the patrons of the establishment. The specific protocol involved the pseudo-intoxicated patron entering the bar and interacting with the bartender, including ordering a drink, while displaying a loss of gross motor coordination. This included swaying on his stool and exhibiting slurred speech, speech changes, apparent difficulty understanding and remembering, and emotional changes. The actor would refuse the drink if served or thank the employee if not served, reassuring the server that he was not going to drive. The pseudo-intoxicated patron would then exit as soon as possible.

The observer entered the establishment separately from the actor and left after observing the interaction with the bartender. Both the actor and the observer recorded information about the encounter and the establishment, including the details and results of the purchase attempt.

Outcome evaluation

An evaluation of projects such as SNAPP requires consideration of effects not only on intermediary measures (e.g., youth access to alcohol, intoxicated-patron service) but also on targeted outcomes, in this case alcohol-related injuries and police incidents. These outcomes were indexed by police calls for assaults and public drunkenness and EMS reports of assaults, motor vehicle accidents, and calls relative to alcohol and other drugs. For purposes of analysis, these data were aggregated on a monthly basis. This aggregation allowed for examination of changes in the neighborhoods over time.

Sacramento City Police crime data were obtained for the period from January 1996 through December 2003; for the first 2 years of the project, police data were provided by the Sacramento City Police Department as electronic (D:Base) files (Sacramento Police Department, 1991-2003). Thereafter, these data were downloaded from the Internet at www.sacpd.org/databases.asp. Similar data were provided by the Sacramento Sheriff's Department. The 1999 data were available only as paper printouts and were scanned into an electronic database. Later, the Sheriff's Department converted from a proprietary mainframe database to the Records Access Database (RAD) and data were downloaded directly (Sacramento Sheriff Department, 1996-2003).

Data were also provided by Sacramento Fire Department EMS for the period from January 1995 through December 2003. These data consisted of EMS responses to injuries in which incident reports were generated identifying the source of injury as an assault, motor vehicle accident, alcohol or other drug problem, or suicide. These data were stored in an Access database (Sacramento Fire Incident Data, 1995-2003). We were able to obtain downloads onto 4-mm tape cartridges directly from their archival database.

One issue that emerged concerned whether to limit our study to outcomes experienced by our target groups (i.e., 15-29 year olds) or to consider effects relative to the broader public. The decision was made to consider outcomes irrespective of the age of parties involved, for theoretical and practical reasons. First, although a specific population was targeted by program activities, it was expected that effects would be experienced neighborhoodwide. Second, the de-identified police data did not have age as a data element, and, in the EMS data, age was generally missing for injury victims. Third, analyses of smaller subpopulations would have introduced problems related to reduced statistical power.

The data from the Sacramento City Police Department, Sheriff's Department, and EMS all contained the address or nearest intersection to which the police or EMS providers responded. The police and EMS records were geocodable to 99%. However, the sheriff's data varied in quality from a worst year of 90% to a best of >99% (average over 8 years = 94.6%). All five outcome measures were aggregated within the South, North, and at-large areas of Sacramento and denominated by 1,000 population age 15 years and older. A logarithmic transformation of each of these measures was sufficient to provide conditional normality for these rates over time.

The statistical evaluation of the impacts of SNAPP compared the effects of two phased interventions in the South and North sites relative to the same outcomes measured in Sacramento at large. The two phased interventions represented the peak of mobilization and awareness activities in each area (July 2001 for the South; October 2002 for the North) and of responsible beverage service and enforcement activities in each area (February 2002 for the South; February 2003 for the North; Treno et al., 2005).

Using a quasi-experimental design, the question was asked whether the time series of outcomes diverged significantly between South and at-large and North and at-large areas at the time of either or both of each phased intervention. Statistical analyses of data from this design relied on seemingly unrelated regression equation (SURE) models that treat each time series separately; compare intervention effects in terms of the relative impacts of the program between experimental and control series (i.e., South vs at-large and North vs at-large); allow each series to have separate time-trend (linear), seasonal (monthly), and temporal autoregressive components; and account for correlated measurement error between equations (Greene, 1993). The last control is particularly important, as the outcome data were from relatively small areas nested within common community, county, and state political and regulatory systems and, for this reason, are likely to be correlated over time. Although not reported in detail here, temporal autocorrelated error was significant in most of the individual time series and ranged from 0.0334 to 0.2871. Bartlett tests of residual correlations between errors for every series were significant, indicating a substantial degree of spatial autocorrelation between areas.

Results

We found a one-third reduction in sales to apparent minors in the South site relative to baseline (from 49% to 32%) compared with increases of 205% in the North (from 20% to 61%) and 23.7% (from 38% to 47%) in the at-large area. We found increases in service to pseudo-intoxicated patrons in the South, North, and at-large areas of 26.5%, 8.8%, and 46.0%, respectively, between Waves 1 and 2. Between Waves 2 and 3 we found decreases of 28.1%, 59.0%, and 76.6%, respectively for sales to apparent minors and of 3.5%, 5.7% and 17.4% for Pseudo Intoxicated Patron Surveys. Declines in sales to apparent minors appeared in the final project year across all conditions. (See Table 3 for complete results across all waves and both instruments.)

Table 3. Comparison of pre- and postintervention premise survey results for both off-premise (apparent minor) and on-premise (pseudo-intoxicated patron) surveys.

Survey	Pre	Post 1	Post 2	Post 2-pre
	Wave1	Wave2	Wave3
Apparent minor survey
South	0.49	0.32	0.23	-0.26
North	0.20	0.61	0.25	0.05
At large	0.38	0.47	0.11	-0.27
Pseudo-intoxicated
patron survey
South	0.68	0.86	0.83	0.15
North	0.80	0.87	0.82	0.02
At large	0.63	0.92	0.76	0.13

These findings suggest that project activities were successful in reducing sales to minors in both the South and North neighborhoods, along with some contamination in the control area. The Sacramento Police Department, having determined the effectiveness of the project, implemented underage sting operations citywide, ironically with funds obtained by the assistance of project staff.

Unfortunately, in the absence of an on-premise enforcement intervention targeting service to intoxicated patrons parallel to the off-premise underage stings, program efforts to reduce intoxicated-patron service appear to have been less successful. Rates of such service remained unaltered and high (above 68%) in all three conditions and across time.

The results of the SURE analyses are summarized in Table 4. Wald tests were used to assess the impacts, first, of all interventions across both South and North sites to at-large areas of Sacramento (one 4-df test) and, second, within each of the two areas separately (two 2-df tests). The summary of intervention effects assessed for each model and entered into the table represents comparisons of effects between two or more different time series regression equations. The Wald statistic tests the constraints that no significant differences related to the intervention effects appeared between corresponding time series. Table 4 reports the chi-square statistic from each Wald test, its significance level, and an estimate of the effect size related to intervention impacts. Effect sizes were calculated as the ratio of the differences between change scores in the South, North, and at large divided by residualized standard deviations of the measured outcomes over time.

Table 4. Overall statistical assessment of impacts of south and north interventions corrected for seasonal effects and temporal autocorrelated error (seemingly unrelated regression equations).

	Overall effects (4 df)			South site (2 df)			North site (2 df)
			Effect			Effect			Effect
Outcome	π2	p	sizea	π2	p	sizea	π2	p	Sizea
Police incident reports
Assault	27.09	<.001	-.475	10.10	.011	-.412	18.05	.002	-.837
Public drunkenness	7.25	ns	–	4.07	ns	–	4.43	ns	–
Emergency medical services
Aggregate outcomes	14.72	.005	-.695	10.79	.005	-.761	4.35	ns	–
Assault	11.82	.019	-.571	3.15	ns	–	8.98	.011	-.833
Motor vehicle accidents	10.92	.028	-.548	10.11	.006	-.740	1.48	ns	–
Alcohol and other drugs	9.16	ns	–	6.77	.034	-.466	4.84	ns	–
Suicide	4.81	ns	–	1.23	ns	–	4.11	ns	–

^aEffect sizes calculated as the ratio of relative change scores between South and North and at-large sites to residualized standard deviations of relative outcomes over time. Overall effect sizes refer to both South and North time series.

In essence, each series was pre-whitened by controlling for effects related to time trends, seasonality, and autoregression; intervention impacts were assessed by comparing effects estimates between series, and these effects estimates referred to the standard deviation of residuals of the series to obtain effect sizes. Overall effect sizes refer to both South and North time series. Overall, we found significant (p < .05) reductions in assaults as reported by police, aggregate EMS outcomes, EMS assaults, and EMS motor vehicle accidents.

Table 5 represents the savings in injuries as measured by police and EMS calls. Specifically, we found an estimated reduction of 3.9% in police calls involving assaults and reductions of 33.4% in EMS calls involving motor vehicle accidents in the South. Similarly, we found an estimated reduction of 36.5% in police calls involving assaults and an estimated reduction of 37.4% in EMS calls involving assaults in the North. Although it is difficult to estimate the exact cost saving associated with these reductions in medical treatment, lost wages, pain and suffering, and other costs, the minimal program costs associated with program implementation, essentially the salary of one full-time project director assisted by several community outreach workers, along with costs associated with RBS trainings and community presentations, appear justified.

Table 5. Relative differences in rates of problem outcomes by South and North sites (significant effects only).

		South site		North site
	Annual	Percent		Percent
Outcome	ratea	change	Savingsb	change	Savingsb
Police incident reports
Assault	8.796	-3.9%c	0.34	-36.5%	3.21
Emergency medical services
Aggregate outcomes	12.864	-33.7%	4.34	–	–
Assault	4.428	–	–	-37.4%	1.66
Motor vehicle accidents	6.864	-33.4%	2.29	–	–
Alcohol and other drugs	1.056	-47.7%	.50	–	–

^aBase rates established pre-intervention per 1,000 population per year

^bsavings calculated per 1,000 population per year

^canomalous finding for South site; significant chi-square test due to positive enforcement pulse at beginning of South intervention.

Discussion

Although prior research had indicated that broad-based community-level interventions can reduce problems such as youth access to alcohol, underage drinking, heavy drinking among adults, and drinking while driving, these interventions had not been applied at the smaller scale of the neighborhood. On the other hand, the neighborhood-level interventions have shown promising results in other areas of health behavior. The results from SNAPP reported here provide support for the efficacy of neighborhood-based interventions in the reduction of alcohol-related problems.

From a combination of interventions focusing on community mobilization, community awareness, and responsible beverage services, involving multiple sectors of the study neighborhoods, we were able to show reductions in problem outcomes such as assaults and motor vehicle crashes.

These results are encouraging, given the lead agencies' relative lack of experience in implementing community-based environmental interventions. What is particularly impressive is the relatively large magnitude of effect sizes, particularly given that statistical estimates of effects from the SURE equations are relatively conservative as they may overcontrol for a variety of potential confounders in these analysis models (time trend effects and seasonality).

Several areas of investigation remain. One is the question of dosage. The evaluation design did not allow for an assessment of separate effects. Were some program components more effective than others? For example, is community mobilization necessary to implement the other program interventions? For the larger-scale Community Trials Project on which SNAPP was modeled, community mobilization was necessary to support other interventions. It is not clear that this step is necessary at the smaller scale of the neighborhood, where activities between community sectors such as police and local business are less difficult to coordinate. In fact, the rather large effect sizes found may be due to the fact that the mobilization efforts were more geographically focused than is possible in larger community-wide efforts such as the Community Trials Project. Here the selection of the appropriate geographic scale for implementation of environmental strategies emerges as an important, if under-researched, issue.

Another area to be further explored is the issue of effects with different subgroups. The SNAPP project has shown the efficacy of neighborhood-level intervention in low-income neighborhoods because the intervention frame was the neighborhood. But the efficacy of the intervention at the subgroup level—for example, among the various ethnic minority groups within the neighborhood—is not known. Controlling for subgroup differences was not possible given the scale of this project.

Finally, the SNAPP project represents an efficacy trial, in that, like its predecessor Community Trials and other community-based alcohol interventions, it was implemented under ideal rather than generalized circumstances. The question of whether the interventions are effective in general cannot be addressed. Similarly, the cost benefit of these interventions is yet undefined. Although the savings in terms of injuries and health consequences to residents are clear, the savings in community resources devoted to enforcement, emergency facilities, and hospital and justice system costs relative to the project expenses have yet to be determined.

A number of serious limitations to the study must be acknowledged. The first involves differences in type and impact of the interventions. Of note, although underage stings appeared to be effective when combined with trainings to reduce underage access at off-premise establishments as measured by our Apparent Minor Survey data collection efforts, no comparable intervention could be instituted targeting service to intoxicated patrons in on-premise establishments. Although business compliance checks were done, primarily in the North, it may be argued that these combined with warnings about service to intoxicated patrons are not strictly comparable to the aggressive enforcement possible with the more clearly defined legal violation of selling to a minor.

Second, although it might be argued that the contamination of the at-large area in the area of enforcement of underage-access laws, while suggesting the power of interventions to spread, seriously compromised interpretation of our findings as pertains to intermediary effects on youth access. Third, although efforts were made to provide comparability of the South and North sites, their general lack of comparability with the at-large area remains a serious concern. A fourth limitation concerns the absence of consideration of the project's intermediary effects as they were hypothesized to impact the 15 to 29-year-old target group specifically. Thus, although it is true that it was expected that project outcomes would be expressed in terms of the broader community (e.g., in reductions of assaults of older persons perpetrated by younger persons), the absence of an effort to index target group-specific effects remains a serious issue.

One alternative would have been the inclusion of analyses of additional data. For example, data indexing driving while intoxicated, gang violence, or domestic violence would have provided a stronger case. Unfortunately, this was not feasible. To be specific, available data indicating driving under the influence of alcohol, domestic violence, and gang activity provide low base rates and are subject to problems of reliability. In addition, data on domestic violence provide unique problems related to victim protection. And the lack of uniformity across effects by community also remains a concern. Why some effects were observed in one site and not the other unfortunately cannot be determined given the methodology implemented. It should be noted, however, that the reductions in motor vehicle accidents in the South site was an unintended, although welcome, outcome of the project, suggesting that the results produced by community efforts such as SNAPP may extend well beyond their original intents.