Multisite Cost Analysis of a School-Based Voluntary Alcohol and Drug Prevention Program

Abstract

Objective:

This article estimates the societal costs of Project CHOICE, a voluntary after-school alcohol and other drug prevention program for adolescents. To our knowledge, this is the first cost analysis of an after-school program specifically focused on reducing alcohol and other drug use.

Method:

The article uses microcosting methods based on the societal perspective and includes a number of sensitivity analyses to assess how the results change with alternative assumptions. Cost data were obtained from surveys of participants, facilitators, and school administrators; insights from program staff members; program expenditures; school budgets; the Bureau of Labor Statistics; and the National Center for Education Statistics.

Results:

From the societal perspective, the cost of implementing Project CHOICE in eight California schools ranged from $121 to $305 per participant (Mdn = $238). The major cost drivers included labor costs associated with facilitating Project CHOICE, opportunity costs of displaced class time (because of in-class promotions for Project CHOICE and consent obtainment), and other efforts to increase participation. Substituting nationally representative cost information for wages and space reduced the range to $100–$206 (Mdn = $182), which is lower than the Substance Abuse and Mental Health Services Administration's estimate of $262 per pupil for the "average effective school-based program in 2002." Denominating national Project CHOICE costs by enrolled students instead of participants generates a median per-pupil cost of $21 (range: $14—$28).

Conclusions:

Estimating the societal costs of school-based prevention programs is crucial for efficiently allocating resources to reduce alcohol and other drug use. The large variation in Project CHOICE costs across schools highlights the importance of collecting program cost information from multiple sites.

School-based prevention programs are a popular strategy intended to delay initiation and reduce alcohol and other drug (AOD) use. The Substance Abuse and Mental Health Services Administration (SAMHSA) reports that 80% of American youth participated in a school-based prevention program in 2005 (SAMHSA, 2009). Although typical prevention programs are inexpensive when compared with other efforts intended to reduce substance use (e.g., treatment and law enforcement; Caulkins et al., 1999), they require expenditures by school districts who have many AOD use prevention programming options. Those making these programming decisions need information about program costs and well as program outcomes.

Most of the early studies of school-based prevention efforts did not include information about program costs (Tobler, 1997). Fortunately, this situation has changed. Aos et al.'s (2004) cost-benefit analysis of prevention and early intervention programs for youth summarized the costs for a number of school-based AOD programs. This study made an important contribution by collecting and reporting cost information for several school-based programs; however, for many of these programs, costs associated with displacing class time to teach AOD prevention instead of regular subjects (e.g., mathematics) were not considered. Although this omission is not a problem if the analysis is conducted from the perspective of the school district, it is an important omission for those who prefer to consider the societal perspective (Gold et al., 1996). The societal perspective would place a value on the lost classroom time and consider it a cost of the AOD program.

Reviews also highlight that the school-based cost literature is focused exclusively on programs that occur during school. We are not aware of any cost studies that examine after-school programs that specifically target AOD use. Thus, this article presents the first cost analysis of a voluntary after-school program focused on reducing adolescent AOD use: Project CHOICE.

Project CHOICE is a promising brief intervention that has been implemented in nine middle schools. A pilot study of Project CHOICE found that it reduced alcohol and marijuana use not only among voluntary participants but also throughout the school where the program was implemented (D'Amico and Edelen, 2007). Preliminary results from an experimental evaluation also suggest that students in schools that adopt Project CHOICE are less likely to report alcohol use (D'Amico et al., 2011).

The current study uses microcosting methods based on the societal perspective and includes a number of sensitivity analyses to assess how the results change with alternative assumptions. Additionally, this study makes three other important contributions. First, by calculating the costs of adopting the program in eight schools, it highlights the variation in costs associated with implementing the same program across different settings. Second, it addresses the opportunity costs associated with students who volunteer for an after-school program as well as costs imposed on parents and other adults who may have to adjust their schedules to pick up the student from the school. Third, the study considers the often-neglected costs associated with displacing class time. Although the program occurred after school, some class time was displaced because of in-class advertising and ensuring that consent forms were distributed and collected.

Method

This section describes Project CHOICE and the methods used for calculating the cost of implementing the program in eight schools. Our baseline estimates considered the societal perspective (Gold et al., 1996) and followed methods recommended in peer-reviewed cost studies of interventions intended to reduce AOD use (e.g., Zarkin et al., 2003, 2005). We generated cost data from a number of sources, including surveys of students (both Project CHOICE participants and nonparticipants), Project CHOICE facilitators and supervisors, Project CHOICE program expenditures, school budgets, and data from the Bureau of Labor Statistics (2009) and the National Center for Education Statistics (Snyder and Dillow, 2010). After reporting costs for the baseline scenario, we then examined how results would change under several alternative assumptions (e.g., how the per-participant costs would differ if national wage and space costs were used instead of costs specific to southern California). "Research costs" related to data collection and study support were excluded from this cost analysis, as were the fixed costs of training facilitators.

Description of Project CHOICE: Theory and implementation

Project CHOICE is a voluntary after-school program that targets middle school youth and focuses on reducing AOD use. It provides youth with normative information to help them get a better understanding of their peers' AOD use, helps youth examine the pros and cons of AOD use, and provides skills training (D'Amico and Edelen, 2007).

Each "cycle" of Project CHOICE involved five half-hour sessions, once per week, repeated throughout the year until approximately six cycles (30 sessions) were completed. Two facilitators, trained in motivational interviewing by a member of the Motivational Interviewing Network of Trainers, presided over each session. Free pizza and juice as well as Project CHOICE promotional materials (e.g., pens) were made available at every session. Thirty-five percent of the participants attended only one session, 27% attended two to four sessions, and 38% attended all five sessions. If a student attended all five sessions during the course of the year, he or she "graduated" and received a $5 gift card. At the end of each session, participants completed a brief feedback survey regarding satisfaction and protocol adherence. To ensure fidelity to Project CHOICE's protocols, two staff members also were trained on the protocol to act as fidelity monitors, and they observed a random selection of sessions across all facilitators.

This article estimates the cost of implementing Project CHOICE in eight middle schools in three school districts in southern California. Table 1 displays demographic information for students who responded to a school-wide survey conducted before Project CHOICE was implemented. Each school included Grades 6 through 8, and in every school more than 90% of the students were between 11 and 13 years old. With respect to ethnic and racial composition, Hispanic students were the largest group at six of the eight schools (range: 19.3%–86.4%). Asians (range: 3.2%–40.7%) and non-Hispanic Whites (range: 1.6%–36.9%) were the next largest groups, with few students reporting being non-Hispanic Black (range: 1.3%–7.5%).

Table 1.

Demographic characteristics of eight California middle schools that offered Project CHOICE

Variable	District #1			District #2				District #3
	School 1	School 2	School 3	School 4	School 5	School 6	School 7	School 8
Survey sample size	736	561	667	311	525	431	473	539
Age, in years
≤10	3.67%	2.50%	5.10%	2.91%	5.10%	4.18%	3.39%	2.23%
11	26.90%	31.25%	32.83%	32.04%	31.68%	30.86%	35.17%	30.67%
12	33.42%	31.43%	35.23%	35.92%	32.63%	32.25%	31.14%	32.53%
13	31.39%	29.11%	23.54%	26.54%	31.49%	29.00%	28.18%	31.23%
14	4.48%	5.71%	3.30%	2.59%	2.10%	3.71%	1.91%	3.35%
15	0.14%	0.00%	0.00%	0.00%	0.00%	0.00%	0.21%	0.00%
Sex
Male	45.38%	54.82%	47.08%	47.25%	47.05%	49.42%	48.41%	53.16%
Female	54.62%	45.18%	52.92%	52.75%	52.95%	50.58%	51.59%	46.84%
Grade
6	28.53%	37.08%	38.08%	31.83%	32.19%	34.11%	35.52%	35.44%
7	34.65%	30.12%	34.03%	40.19%	33.19%	32.28%	32.28%	32.28%
8	36.82%	32.80%	27.89%	27.97%	31.29%	31.79%	31.29%	32.28%
Race/ethnicity
Hispanic or Latino	84.22%	86.41%	53.16%	19.28%	24.05%	31.85%	24.57%	57.46%
NH American Indian or Alaska Native	0.27%	0.36%	0.45%	0.98%	0.57%	1.17%	1.07%	0.37%
NH Native Hawaiian or Pacific Islander	0.14%	0.55%	0.60%	0.98%	0.57%	3.26%	1.07%	0.19%
NH Asian or Asian American	3.15%	5.46%	8.41%	40.66%	33.59%	28.21%	24.46%	3.93%
NH Black or African American	4.24%	2.55%	7.51%	3.61%	1.34%	3.96%	1.50%	5.79%
NH White	3.42%	1.64%	20.72%	25.90%	29.96%	20.98%	36.91%	24.30%
NH other	4.38%	3.10%	9.01%	8.20%	9.73%	10.02%	9.44%	7.85%

Note:NH=non-Hispanic

For each school, sessions began in October 2008 and were delivered weekly until June 2009. Two facilitators were assigned to each school: the primary facilitator was a salaried employee at the research institution where Project CHOICE was developed, and the secondary facilitator was an hourly contractor hired by the research institution. Facilitators advertised the program to recruit students and led the sessions. Advertising entailed making brief announcements in homeroom classes, physical education classes, and health classes, over school public-address systems, and at a designated table in the cafeteria during lunchtime. During implementation, Project CHOICE staff members stayed in contact with school employees (such as principals and administrative assistants) to coordinate logistical details (such as finding locations for sessions or sending announcements to be read over the public-address system). School employees also helped Project CHOICE staff members by unlocking classrooms for facilitators, making sure that classrooms were clean, helping facilitators set up promotional tables in lunch areas, and keeping facilitators informed about events at the school.

Procedures for microcosting

Labor costs for supervisors and facilitators.

Throughout the implementation period, the program supervisor met regularly with pairs of facilitators and as a group for ongoing training and supervision during implementation. Hours spent were multiplied by the supervisor's total compensation figure (inclusive of benefits) and were divided equally among the eight Project CHOICE schools. Facilitator time fell into three categories: facilitating, training, and "all else." Total hours per facilitator were attributed to each facilitator's two assigned schools in proportion to the school's share of the facilitator's total number of sessions. This figure was multiplied by the facilitator's total hourly compensation.

Transportation costs for facilitators.

Project CHOICE facilitators were not school employees and therefore made special trips to schools for each session. We administered two surveys to facilitators covering two separate periods, which asked questions about transportation choices and time spent in transit. Per-mile travel costs were obtained from the U.S. General Services Administration's (2010) privately owned vehicle mileage reimbursement rate. For facilitators who reported carpooling “all of the time,” we cut per-mile costs in half; for those who reported carpooling “some of the time,” we adjusted per-mile costs downward by 33%.

Fidelity monitoring.

We divided fidelity monitoring costs evenly among the schools, in light of the fact that the benefit of fidelity monitoring would apply to all schools served by a monitored facilitator.

Consent forms.

The cost of copying the 7,708 consent forms was based on what it would cost to make the copies at FedEx Office.

Advertising and incentives.

Advertising costs included the time and resources devoted to creating and posting flyers, developing informational videos (digital media production), and student incentives (e.g., gift cards, pens). In our cost model, this amount was distributed proportionally to each school's share of the total enrollment across the eight schools.

General supplies.

The costs of general supplies—such as rolling crates, flip charts, office supplies, and paper stock—also were taken from program expenditures. Again, instead of dividing the total cost by eight to construct per school costs, these expenditures were distributed proportionally to each school's share of the total enrollment figure in September.

School labor costs.

We fi elded a small survey to all eight school principals asking them about Project CHOICE– related time costs for themselves and for staff members other than teachers and administrative assistants. We also obtained information from Project CHOICE staff members about school-specific time cost estimates for physical education teachers and for administrative staff. We combined these time cost estimates with average salary figures for physical education teachers, principals, and administrative assistants, using publicly available salary information for each district, and we added average figures for fringe benefits for public employees from the Bureau of Labor Statistics (2009; for “other” employees, we assumed compensation equal to that received by physical education teachers). To calculate hourly compensation figures, we divided annual salaries by 1,760 in the case of principals and superintendents, and 1,440 in the case of teachers and administrative assistants. Because District 2 was larger, we had a consultant who assisted with program implementation by acting as a liaison between Project CHOICE staff members and school staff members.

Opportunity costs for students participating in Project CHOICE.

To calculate opportunity costs for adults, an hourly wage figure (adjusted for age, gender, and other characteristics) is typically used and multiplied by the time consumed by a given activity. This calculation is more difficult for adolescents, especially those who are below the legal working age (Luce et al., 1996). Yet, even if one could easily place a dollar value on adolescents' time, it is not entirely clear whether this dollar value would be appropriate for a voluntary program such as Project CHOICE. In a cost analysis of Big Brothers Big Sisters, Belfield (2003) argues that if the little brothers and sisters enjoy participating, "then the opportunity costs are trivial" (p. 9). However, the argument that enjoyment nullifies costs is not a universally accepted notion (Foster et al., 2007).

To gain a better understanding of the opportunity cost of students' after-school time, we fielded a survey to middle school students 1 year after Project CHOICE ended. One of the questions asked was: "During the 30 minutes after school yesterday, did you do anything that earned you money?" Only 10.2% reported that they did something that earned money. Based on this low percentage, we followed the published literature and assumed that the opportunity cost for the adolescents is $0 in the baseline estimate. This assumption is revisited in the sensitivity analyses.

Transportation costs for students participating in Project CHOICE.

Transportation issues feature more prominently in after-school programs than they do in interventions that take place during school hours. For example, participants who normally rely on the school bus to take them to and from school may have to arrange alternate transportation to participate in an after-school program. If parents (or other drivers) now have to pick up these students because of their participation in Project CHOICE, then we should include the opportunity cost of time for adults and children, as well as actual travel expenditures (such as the cost of gasoline and vehicle depreciation).

To learn about this, we asked students how they got home the previous day and how long it took. We then asked how they thought they would have gotten home had they decided to participate in a new club or activity for 30 minutes after school the previous day. For both questions, we calculated the percentage of students who had (or would have had) a parent, older sibling, or "other" person drive them home in a private automobile (hereinafter referred to as "adults"). We generated these figures for each school (looking only at those students who also participated in Project CHOICE). In six of the eight Project CHOICE schools, the percentage of Project CHOICE participants who would have an adult drive them home was larger for the hypothetical "participating in a new activity" compared with how they got home the previous day. We interpreted this discrepancy as suggesting that participating in a program such as Project CHOICE would impose additional societal costs for these six schools (the negative values for the other two schools are interpreted as reductions in the societal cost). However, in our baseline analysis, we considered cost differences only in schools where the percentage of burdened adults was significantly different from the number of adults that would have been burdened (two-sided t test, p ≤ .05). Only two schools saw statistically significant changes in the adult travel burden; one had an increased burden and the other a decreased burden.

For the drivers, we assumed a wage equal to the per capita income of their children's schools' respective ZIP codes (based on the 2000 census, brought forward to 2009 using the Consumer Price Index inflation calculator; Bureau of Labor Statistics, 2010), divided by 2,000 hours, and multiplied that figure by the time cost reported by the student. To compute distance costs, we estimated mileage by multiplying the student's reported travel time in fractions of an hour by 20 miles per hour (the average speed used by the Environmental Protection Agency [2010] to test fuel economy in city driving) and multiplied the resulting figure by the General Services Administration's (2010) per-mile reimbursement rate.

Value of displaced class time.

During the program, a certain amount of class time was displaced by three Project CHOICE-related activities. First, consent forms were collected from students by physical education teachers. The time displaced by consent form administration was retroactively estimated by a member of the Project CHOICE staff, who worked in close contact with staff members at the eight schools and had specific information about the burden it placed on those staff members. Her estimates were based on the product of the following school-specific figures: the number of physical education teachers at each school; the number of physical education classes in each week; the number of weeks that particular school spent dealing with consent forms; the number of minutes spent per class dealing with consent forms (if consent forms were dealt with during that class); and the proportion of this total burden that fell during class time versus outside of class time. In our baseline analysis, the in-class portion was counted as displaced class time (whereas the out-of-class portion was counted as a staff labor cost).

Second, some class time was used to make in-person promotional announcements. These time costs were estimated based on parameters provided to us by a primary facilitator, who informed us that at each school, every 4–6 weeks during the program, facilitators presented for 5 minutes at five to six health and physical education class sessions, for a total of 1 hour at each school for the year.

Third, time was displaced by announcements made over the public-address system during the homeroom period. This time was estimated based on approximate announcement length (15 seconds), number of announcements (two per program week), and classrooms per school (enrollment divided by class size). In all three cases, we inferred the implicit value of the lost classroom time by dividing each school district's annual budget by the total number of class hours (in all of that district's schools and classrooms) over the course of a year.

Space and utility costs.

From the societal perspective, school space represents a real resource with value. Although the classroom space used for Project CHOICE was "donated" by the school, the value of this space was included in the baseline analysis. Many researchers recommend pricing school space by identifying its replacement price as determined by what one would have to pay in the local rental market (Chatterji et al., 2001, 2004). As we did for the costs of training space, we followed the existing literature and used hourly commercial rental rates for one square foot of space (including maintenance, depreciation, and utilities).

Results

Costs were calculated for each school and are listed in Table 2. Table 2 also includes input values for key categories so that readers can understand which factors are driving the variation across schools. Additional details about how these costs were generated are in a Technical Appendix that is available on request (kilmer@rand.org). Throughout this article, 2009 U.S. dollars are used, and the term "median" refers to the median value for a given item or category across the eight middle schools.

Table 2.

Baseline societal costs of using Project CHOICE in eight California middle schools

Variable	District #1			District #2				District #3	Median
	School 1	School 2	School 3	School 4	School 5	School 6	School 7	School 8
Supervisor	$155.15	$155.15	$155.15	$155.15	$155.15	$155.15	$155.15	$155.15	$155.15
Total hours: Facilitator #1	155.81	82.77	85.43	151.06	155.43	155.94	150.57	150.79	150.93
Hourly compensation: Facilitator #1	$64.75	$49.93	$49.93	$64.75	$43.14	$64.75	$43.14	$64.75	57.34
Facilitator #1	$10,089.29	$4,132.38	$4,265.68	$9,781.74	$6,705.81	$10,097.28	$6,496.25	$9,763.83	$8,234.82
Total hours: Facilitator #2	106.99	100.40	103.63	80.45	54.40	83.05	52.70	103.54	91.72
Hourly compensation: Facilitator #2	$18.00	$18.00	$18.00	$18.00	$18.00	$18.00	$18.00	$18.00	$18.00
Facilitator #2	$1,925.89	$1,807.12	$1,865.42	$1,448.14	$979.20	$1,494.86	$948.60	$1,863.77	$1,650.99
Facilitators – travel costs	$1,593.29	$735.58	$759.31	$1,285.47	$939.98	$1,275.70	$910.60	$1,592.47	$1,107.84
Fidelity monitoring	$97.13	$97.13	$97.13	$97.13	$97.13	$97.13	$97.13	$97.13	$97.13
Fidelity monitors – travel costs	$288.16	$288.16	$288.16	$288.16	$288.16	$288.16	$288.16	$288.16	$288.16
Consent forms (paper)	$87.85	$102.72	$71.25	$53.20	$54.10	$51.46	$50.00	$64.73	$59.41
Advertising and incentives	$3,685.47	$4,308.94	$2,989.17	$2,231.68	$2,269.55	$2,158.84	$2,097.66	$2,715.31	$2,492.43
General supplies	$39.13	$45.75	$31.74	$23.69	$24.10	$22.92	$22.27	$28.83	$26.46
Total hours: PE teacher	15.75	19.50	7.50	5.77	2.50	2.00	3.50	7.75	6.63
Hourly compensation: PE teacher	$67.04	$67.04	$67.04	$66.51	$66.51	$66.51	$66.51	$62.32	$66.51
Subtotal: PE teacher	$1,055.81	$1,307.19	$502.76	$383.57	$166.29	$133.03	$232.80	$483.01	$433.29
Total hours: Principal	0.00	2.33	4.83	15.00	0.00	5.00	2.50	2.33	2.42
Hourly compensation: Principal	$101.52	$101.52	$101.52	$85.69	$85.69	$85.69	$85.69	$95.67	$90.68
Subtotal: Principal	$0.00	$236.88	$490.67	$1,285.28	$0.00	$428.43	$214.21	$223.22	$230.05
Total hours: Consultant	–	–	–	7.20	7.20	7.20	7.20	–	3.60
Hourly compensation: Consultant	$155.14	$155.14	$155.14	$155.14	$155.14	$155.14	$155.14	$155.14	$155.14
Subtotal: Consultant	$0.00	$0.00	$0.00	$1,117.01	$1,117.01	$1,117.01	$1,117.01	$0.00	$558.51
Total hours: Admin. assts.	0.00	4.67	0.00	10.00	0.00	0.00	0.00	2.33	0.00
Hourly compensation: Admin. assts.	$30.17	$30.17	$30.17	$59.46	$59.46	$59.46	$59.46	$39.49	$49.48
Subtotal: Admin. assts.	$0.00	$140.77	$0.00	$594.64	$0.00	$0.00	$0.00	$92.15	$0.00
Total hours: Other staff	0.00	4.67	0.00	0.00	0.00	0.00	2.50	2.33	0.00
Hourly compensation: Other staff	$67.04	$67.04	$67.04	$66.51	$66.51	$66.51	$66.51	$62.32	$66.51
Subtotal: Other staff	$0.00	$312.83	$0.00	$0.00	$0.00	$0.00	$166.29	$145.42	$0.00
School staff – labor	$1,055.81	$1,997.67	$993.43	$3,380.50	$1,283.30	$1,678.47	$1,730.31	$943.80	$1,480.88
Student afterschool time (op. cost)	$0.00	$0.00	$0.00	$0.00	$0.00	$0.00	$0.00	$0.00	$0.00
Student travel	$0.00	$0.00	−$570.20	$0.00	$0.00	$656.61	$0.00	$0.00	$0.00
Total hours: In-class form collection	18.00	22.50	7.20	6.40	6.00	4.50	6.00	12.00	6.80
Cost per hour of classroom time	$370.40	$370.40	$370.40	$288.04	$288.04	$288.04	$288.04	$363.47	$325.76
Subtotal: Consent form collection	$6,667.25	$8,334.07	$2,666.90	$1,843.45	$1,728.24	$1,296.18	$1,728.24	$4,361.68	$2,255.18
Interrupted classes per year	33.00	30.80	31.90	33.00	34.10	34.10	33.00	30.80	33.00
Interruption length in hours	0.083	0.083	0.083	0.083	0.083	0.083	0.083	0.083	0.083
Subtotal: Interruptions for announcements	$1,018.61	$950.70	$984.65	$792.11	$818.51	$818.51	$792.11	$932.92	$875.71
Public address (PA) announcements per year	60	56	58	60	62	62	60	56	60
PA announcement length, hours	0.0042	0.0042	0.0042	0.0042	0.0042	0.0042	0.0042	0.0042
No. classrooms, enrollment / class size	42.17	49.3	34.2	21.89	22.26	21.17	20.57	31.07
Subtotal: PA announcements	$65.08	$76.09	$52.78	$26.27	$26.71	$25.41	$24.69	$47.05
Displaced class time	$7,750.94	$9,360.85	$3,704.34	$2,661.83	$2,573.46	$2,140.10	$2,545.04	$5,341.65	$3,183.08
School space	$45.14	$14.06	$43.14	$76.71	$46.12	$46.12	$44.29	$42.96	$44.71
TOTAL	$26,813.25	$23,045.52	$14,693.73	$21,483.40	$15,416.06	$20,162.80	$15,385.47	$22,897.78	$20,823.10
Total no. sessions	30	28	29	30	31	31	30	28	30
Per session	$893.78	$823.05	$506.68	$716.11	$497.29	$650.41	$512.85	$817.78	$683.26
Total no. unique attendees	88	92	121	76	72	100	68	86	87
Per unique attendee	$304.70	$250.49	$121.44	$282.68	$214.11	$201.63	$226.26	$266.25	$238.38
Total no. attendees	284	362	313	206	186	298	202	239	261.5
Per nonunique attendee	$94.41	$63.66	$46.94	$104.29	$82.88	$67.66	$76.17	$95.81	$79.52
Total no. enrolled at school	1,265	1,479	1,026	766	779	741	720	932	855.5
Per student enrolled in the school	$21.20	$15.58	$14.32	$28.05	$19.79	$27.21	$21.37	$24.57	$21.28

Notes: Because of space constraints, detailed cost information is presented only for select categories. The technical appendix with additional information can be obtained by contacting the fi rst author at kilmer@rand.org. PE = physical education.

Microcosting results

Labor costs for supervisors and facilitators.

The median costs for the supervisor, the primary facilitator who was employed by the study institution, and the secondary facilitator who contracted with the study institution were $155.15, $8,234.82 (range: $4,132.38–$10,097.28), and $1,650.99 (range: $948.60–$1,925.89), respectively, per school. The variation in primary facilitator costs was largely driven by the fact that the primary facilitator who served Schools 2 and 3 reported working about half as many hours as the primary facilitators in the other six schools. The difference in hours is largely seen in the "other" category that included work outside of training and facilitation (e.g., preparing for promotional efforts).

Transportation costs for facilitators.

Median transportation costs for the two facilitators totaled $1,107.84 per school (range: $735.58–$1,593.29).

Fidelity monitoring.

With total compensation assumed to be the same as it was for the primary facilitator ($64.75/hour = $43.90/hour plus benefits equal to $20.85/hour), the median cost for fidelity monitoring was estimated to be $97.13 per school. Travel costs for the fidelity monitors were an additional $288.16 per school. Costs for fidelity monitoring and travel were divided equally among the eight schools under the assumption that the benefits from ensuring program adherence would accrue to all eight schools.

Consent forms.

Median costs for consent forms were $59.41 per school (range: $50.00–$102.72). Variation in this figure is based solely on enrollment size.

Advertising and incentives.

Total advertising and incentive costs of $22,456.62 were taken directly from program expenditures, which included copy costs, food and drinks at each session, gift cards, and other incentives such as water bottles and pens. After distributing these costs across each school according to its share of the total enrollment across the eight schools, median costs added up to $2,492.43 per school (range: $2,097.66–$4,308.94). All variation in this figure comes from differences in enrollment figures.

General supplies.

The total cost for supplies was $238.43. This cost also was distributed proportionally to each school's share of total enrollment, yielding a median cost of $26.46 per school (range: $22.27–$45.75). All variation in this figure comes from differences in enrollment figures.

School labor costs.

We estimated median school-based labor costs (principals, physical education teachers, administrative assistants, and other staff members) to be $1,480.88 per school (range: $943.80–$3,380.50). Table 1 highlights how the hours and wages vary for the different positions. School 4 represents the upper bound and this value is largely attributable to the fact that the principal reported dedicating 15 hours to helping with Project CHOICE implementation (the median value across all eight schools was 2.42 hours).

Opportunity costs for students participating in Project CHOICE.

These costs were assumed to be $0 in the baseline analysis.

Transportation costs for students participating in Project CHOICE.

We found that Project CHOICE could have increased transportation costs at one of eight schools while decreasing costs at another (with students at this school saying that they actually would be less likely to need an adult's help to get home in the event of a new club or activity). Thus, median student transportation costs were $0.00 per school (range: -$570.20–$656.61).

Value of displaced class time.

Our estimated values of class time equaled $370.40 per hour in School District 1, $288.04 per hour in School District 2, and $363.47 per hour in School District 3. These values resulted in a median cost of $3,183.08 for displaced class time per school (range: $2,140.10–$9,360.85). Variation in the value of the opportunity cost of displaced class time is largely driven by the amount of class time spent dealing with consent forms. We consider a scenario that excludes the costs associated with collecting consent forms in the sensitivity analysis.

Space and utility costs.

Median space and utility costs came to $44.71 per school (range: $14.06–$76.71). Variation in this figure is driven solely by classroom size.

Total costs

The bottom of Table 2 displays the total costs, the number of participants in each school, and the costs per participant. The total societal cost of Project CHOICE implemented in eight schools ranged from $14,693.73 to $26,813.25, with the median value equal to $20,823.10 per school. The share of the student population that participated in at least one session of Project CHOICE ranged from 6.2% to 13.5%, with the median equal to 9.3%. The cost per participant ranged from $121.44 to $304.70, with the median equal to $238.38.

Labor costs for Project CHOICE and school staff members were the main drivers of implementation costs in this study of Project CHOICE, accounting for the majority of costs at all but two schools (median: 57.9%; range: 35.5%–69.2%). The remaining costs comprised displaced class time (median: 20.0%; range: 10.6%–40.6%); advertising and incentives (median: 13.7%; range: 10.4%–20.3%); facilitator, fidelity monitor, and student travel (median: 7.6%; range: 3.2%–11.0%); and "other" items (including consent forms, general supplies, and the value of school space; median: 0.7%; range: 0.6%–1.0%).

Sensitivity analyses

This section describes how the median societal cost per participant changes under alternative assumptions or perspectives. Results are summarized in Figure 1.

Figure 1.

Percentage change in the median cost per participant under alternative scenarios. The baseline estimate is the median cost per participant from the societal perspective: $238.38. S = Scenario; hr = hour.

Scenario 1: Use nationally representative cost estimates instead of southern California-specific costs for labor, space, and displaced class time.

Project CHOICE was implemented in an expensive part of the country, and our results are sensitive to variation in facilitator labor costs (i.e., contractors versus research institution staff members with benefits). For this scenario, we applied nationally representative wage figures from the Bureau of Labor Statistics (2009) to both Project CHOICE and school staff members (as opposed to actual compensation figures) and used national per capita income (as opposed to local per capita income) to impute drivers' cost of time. Nationally representative cost of space figures were obtained from the same source used for our local cost of space estimates (Cassidy Turley Commercial Real Estate Services, 2010). We also used data from the National Center for Education Statistics (Snyder and Dillow, 2010) to generate the opportunity cost of displaced class time, dividing total spending on public elementary and secondary schools by an estimated number of classroom hours. Making these changes reduced the median societal cost per participant to $182.06 (-23.6%).

Scenario 2: Still using national cost estimates, consider the school district perspective instead of the societal perspective.

The school district costs included the accounting costs that school administrators would face, assuming that there was a pool of trained facilitators available to come into the school and implement Project CHOICE. Thus, this scenario included only the facilitator costs, costs associated with purchasing materials to advertise and incentivize participation, and general supplies. Baseline cost estimates were adjusted to use nationally representative wage figures, and costs related to the Project CHOICE supervisor, fidelity monitoring, displaced class time, travel, space, donated labor, and school staff labor were excluded. These exclusions reduced the median cost per participant to $114.32 (-52.0%).

Scenario 3: Assume that the opportunity cost of displaced class time is $0.

Project CHOICE-related promotional efforts and paperwork did displace some classroom instruction time. In our baseline scenario, we followed Caulkins et al. (1999) and estimated the value of classroom time by dividing an annual school district budget over annual class time totals in that district. Yet, not all studies considered the value of such time. Therefore, as an alternative, we tested our results with a value of $0 for class time. This alternative caused the median societal cost per participant to drop to $184.53 (-22.6%).

Scenario 4: Assume that the opportunity cost for students is $8 per hour.

In our baseline scenario, we valued student time spent in Project CHOICE and in travel at $0 per hour. However, our survey results suggest that a small percentage of students did earn money by doing errands or working for their families. If we valued this time at the minimum wage in California for 2009, $8/hour (Department of Labor, 2011), it would increase the median societal cost per participant to $252.19 (+5.8%). Because the mean age for Project CHOICE participants was 12, and those younger than 14 could work only under certain circumstances (State of California Department of Industrial Relations, 2000), $8 per hour likely represents an upper bound on the value of time for these students.

Scenario 5: Use school district rental prices for calculating space costs.

Although the space used in Project CHOICE was donated by the school districts and was valued at the market rate in our baseline analysis, we obtained from each district hourly classroom rental fees that they charged to for-profit groups not focused on students ($51, $48, and $30 in Districts 1, 2, and 3, respectively). Use of these classroom fees instead of the commercial rates raised the estimated median societal cost per participant to $247.15 (+3.7%).

Scenario 6: Exclude all costs related to fidelity monitoring.

Program fidelity monitors helped to ensure that facilitators kept to protocol, which presumably conferred some benefit on attendees. Thus, we included the cost of fidelity monitors in our baseline scenario. Yet, if Project CHOICE were to be transplanted to a new location, the fidelity monitors used in this study might not be included as part of the program. Therefore, in Scenario 6, we excluded the costs of training the monitors, the costs of monitoring, and the cost of fidelity monitor travel. Doing so decreased the median societal cost per participant to $233.45 (-2.1%).

Scenario 7: Include nonsignificant changes in adult travel burden.

In our baseline analysis, we ignored additional adult travel costs (or savings) for those schools where the difference in the percentage of would-be burdened adults was not significant (based on a two-sided t test, p ≤ .05). In Scenario 7, we included nonsignificant changes in the adult travel burden brought on by Project CHOICE. This inclusion reduced the median societal cost per participant slightly, to $238.18 (-0.1%).

Scenario 8: Exclude all costs related to consent forms.

In our baseline scenario, we included the cost of printing Project CHOICE consent forms, the cost of teachers dealing with consent forms outside of class time, and the cost of class time that was displaced by consent form distribution and collection. In Scenario 8 we disregarded these costs, and doing so reduced the median societal cost per participant to $192.83 (-19.1%). Most of the decrease came from the reduction in the amount of displaced class time.

Discussion

Estimating the cost of implementing school-based prevention programs helps policy makers and school administrators budget for AOD prevention. This information also is useful for making sure that resources devoted to reducing AOD use are allocated efficiently. This article contributes to this literature by calculating the costs of Project CHOICE, a voluntary after-school AOD use prevention program that was implemented in eight middle schools in 2008–2009.

The low and high cost estimates per Project CHOICE participant were $121.44 and $304.70, with a median value of $238.38 per participant. Variation in this figure was largely driven by the wide range in the number of participants, with participation in each school ranging from 68 to 121 students (Mdn = 87). Because the largest cost component (costs related to the facilitators) was independent of the number of participants, it is not surprising that the school with the lowest cost per participant had the largest number of participants. Replacing this school's large attendance figure with the median attendance value for all eight schools would bring its per-participant costs up from $121.44 to $168.89. Other major sources of variation in per-attendee costs included the large ranges in the values of primary facilitator costs, displaced class time, and school staff labor.

Substituting nationally representative cost information for wages and space reduced the range for Project CHOICE to $100.30–$205.61 per participant, which was lower than SAMHSA's (2009) estimate that the "average effective school-based program in 2002" costs $262.36 per pupil (in 2009 dollars). It is important to note that the denominators for these groups are not the same. The Project CHOICE denominator is based on the number of students who participated in at least one session, whereas SAMHSA's denominator is based on all students in the school who received the prevention program. Whereas one could argue that this is an appropriate comparison because school costs should be denominated by those participating in the programs, others may want to know how Project CHOICE compares when denominated by the entire student body. Denominating national Project CHOICE costs by enrolled students instead of participants generates a median per-pupil cost of $21.28 (range: $14.32–$28.05).

The societal costs associated with operating a prevention program during or after school are different, especially when nonteachers are traveling to the school to lead the after-school sessions. There also is the issue of lost class time, which is not easy to monetize. These costs should be included in studies of school-based prevention programs conducted from the societal perspective. However, they should be itemized so that those who disagree with the approach can easily recalculate using their own figures and assumptions. Indeed, these costs would generally be ignored if the analysis were conducted from the school district's perspective instead of the societal perspective. For Project CHOICE, we estimated that costs would decrease by 52.0% if we considered the school district perspective and assumed that a pool of trained Project CHOICE facilitators was available.

Even though Project CHOICE is an after-school program, it did displace some class time because physical education teachers had to pass out and collect consent forms, and facilitators did interrupt some classes to make announcements about the program. This was not a negligible amount; the median share of total costs attributed to displaced class time was 20.0% (range: 10.6%–40.6%). Whether it is possible to reduce these costs and still generate a critical number of participants is an empirical question. One strategy that could be tested in future Project CHOICE sites would be to randomly assign some schools to classroom visits by facilitators and other schools to other forms of advertising.

Furthermore, when decision makers are thinking about the costs of adopting Project CHOICE and/or other school-based programs, it is important to think about the training costs involved. For example, Project CHOICE took place over the course of only 1 year and entailed considerable training costs that would become less important if divided over multiple subsequent program years. Although many annual costs would remain largely unchanged after the first year (e.g., space, supplies, incentives, administrative burden, and time spent by facilitators leading sessions), other costs could be expected to decrease over the course of a more lengthy implementation period.

One limitation to the current analysis is that the cost information had to be collected retrospectively; thus, some of the results may be influenced by measurement and/or recall bias. Although measurement bias also is a concern for studies conducted simultaneously with implementation, one would expect responses to be more accurate when inquiring about what happened last week versus several months ago. The direction of this bias, however, is unknown, and it is unclear whether it would be the same for each school.

This study is unique in that it uses a microcosting approach to assess the costs of an afterschool AOD program (Project CHOICE) from both the societal and school perspectives. The findings make an important contribution to the AOD prevention literature because they delineate the specific costs and detail how the costs vary depending on location and alternative assumptions. However, we remind readers that cost is only one piece of information needed for making decisions about AOD use prevention programming; outcome information also is necessary. Preliminary results from the experimental evaluation of Project CHOICE suggest the program reduces the probability of alcohol initiation (D'Amico et al., 2011), and data about the long-run effects of the program are currently being collected. Combining cost data with forthcoming outcome data to conduct cost-effectiveness, cost-utility, and/or cost-benefit analyses should be useful to policy makers and school administrators working to prevent substance use with scarce resources.