The Cost-Effectiveness of New York City’s Safe Routes to School Program

Abstract

Objective. We evaluated the cost-effectiveness of a package of roadway modifications in New York City funded under the Safe Routes to School (SRTS) program.

Methods. We used a Markov model to estimate long-term impacts of SRTS on injury reduction and the associated savings in medical costs, lifelong disability, and death. Model inputs included societal costs (in 2013 US dollars) and observed spatiotemporal changes in injury rates associated with New York City’s implementation of SRTS relative to control intersections. Structural changes to roadways were assumed to last 50 years before further investment is required. Therefore, costs were discounted over 50 consecutive cohorts of modified roadway users under SRTS.

Results. SRTS was associated with an overall net societal benefit of $230 million and 2055 quality-adjusted life years gained in New York City.

Conclusions. SRTS reduces injuries and saves money over the long run.

In the United States, motor vehicle crashes are the leading cause of death for children aged 8 to 19 years, and the second leading cause of death for children aged 4 to 7 years.^1–3 Virtually all motor vehicle-related pedestrian injuries are preventable. One way of reducing childhood motor vehicle injury is to improve roadway safety.^4–7 However, roadway improvements require a large up-front investment, and the chance of a child being severely injured in any given intersection is relatively small. Nevertheless, such improvements remain in place for decades, producing long-term benefits on a single investment. Given the high cost but the long-term benefits, there is uncertainty surrounding the cost-effectiveness of roadway improvements.

One roadway improvement program is Safe Routes to School (SRTS).^8–10 SRTS was enacted under the federal Safe, Accountable, Flexible and Efficient Transportation Equity (SAFETEA) Act in 2005 (Pub L No. 109-59). SRTS was a $612 million dollar program that funded state departments of transportation to build new sidewalks and bicycle lanes, improve safety at crossings (e.g., via traffic calming measures), upgrade signage, and enhance pedestrian education.^8,12 The intent of SRTS was to encourage children to walk and bike to school by making school travel safer, thereby reducing an important barrier to commuting outside of a vehicle. Capital improvement projects were funded at 10 400 schools in the United States. In 2012, Congress enacted Moving Ahead for Progress in the 21st Century (MAP-21; Pub L No. 112-141) to replace the SAFETEA. As a result, dedicated funding for SRTS ended, though the program may continue under discretionary funding at the state and local level.

Approximately $10 million in the initial wave of funding went to the New York City Department of Transportation, which introduced safety improvements at 124 schools with the highest injury rates.¹¹ SRTS has been evaluated using a variety of techniques, including a difference in difference design and a Bayesian changepoint analysis.⁵ SRTS has been found to produce an 11% increase in active transport to school (walking or biking) while simultaneously leading to a 33% to 44% reduction in school-age injury rates in high-risk intersections within New York City.^5,12,13 Among all users of these intersections, injury reduction rates were more modest, with about a 14% decline.⁵

This analysis assesses the cost-effectiveness of SRTS in preventing pedestrian injuries among school-aged children traveling to and from school (the intent of the program) relative to the status quo. We examined this subpopulation because it is the target of the SRTS legislation. We also examined the cost-effectiveness of SRTS for all users at all times. This latter analysis provides the overall societal benefits of the program, whether intended or not.

METHODS

We set out to determine the cost-effectiveness of the SRTS program in New York City for both school-aged children (the target of the legislation) and adult users of the improved intersections. When both estimates are summed, we obtain the actual societal cost of the program. In New York City, SRTS was targeted toward the highest-risk intersections, so our research question focused specifically on intersections with a history of a high rate of injury. Our models estimated fluctuation in costs associated with childhood pedestrian injury and changes in quality-adjusted life expectancy (QALE) that occurs when high-risk intersections are modified. These models, built using TreeAge Pro 2013 (TreeAge software, Williamstown, MA), are designed to estimate outcomes for the average school-aged pedestrian over his or her lifetime or the average incidental adult SRTS user over his or her lifetime. Thus, these models present the net present value of a single year of SRTS intersection use. To estimate total annual costs, we multiplied these values by the estimated number of users.

We only included those medical costs associated with the initial injury, changes in QALE, changes in bussing costs, and changes in burial costs. All outcomes were adjusted to 2013 constant dollars and discounted at a rate of 3% per year.

The models are designed such that, in the first year, the intersection user has a very small risk of injury and a significant chance of remaining healthy. If the user remains healthy, then we assign no costs other than SRTS capital costs and, in the child model, changes in bussing use. If the user is injured, we assign the user the initial medical costs associated with an injury and a very small risk of death. If the user survives, we assume that all future societal costs will be subsumed by the change in health-related quality of life (HRQL) score.¹⁴ Both the SRTS arm and the arm modeling the unmodified intersections are identical except that the SRTS arm contains a reduced risk of injury, reduced costs associated with active transport, and upfront costs associated with SRTS. To estimate the total societal costs, these lifelong costs (for a given annual cohort of users) are then multiplied over the projected useful 50-year time horizon of an intersection modification and discounted at 3% in a separate model. Both models were validated using a spreadsheet, but we used TreeAge to allow for sophisticated sensitivity analyses.

Costs

Monetary costs were adjusted to constant 2012 US dollars and are listed in Table 1.^15,16 Costs associated with pedestrian injuries were obtained using the Center for Disease Control and Prevention’s Web-based Injury Statistics Query And Reporting System (CDC WISQARS) and a comprehensive analysis of multiple sources of 2000 data.¹⁷

TABLE 1—

Values Used in the Markov Model Evaluating Safe Routes to School (SRTS) in New York City vs the Status Quo

Variable	Base	High	Low
Injury risks
Increase in active transport, SRTS, %	10	12	8
Total pedestrians at risk, no.
School-aged children (5-19 y)	40 525	. . .	. . .
Adults using intersection	181 148	. . .	. . .
Annual probability of pedestrian injury, SRTS intersection
School-aged children (5-19 y)	0.0008	0.023	0.0004
Adults using intersection	0.002	0.014	0.0003
Risk ratio, injury, SRTS intersection
School-aged children (5-19 y)	0.67	0.83	0.35
Adults using intersection	0.86	0.88	0.84
Probability of hospitalization	0.12	0.16	0.08
Case fatality ratio	0.001	0.006	0.0007
Health-related quality of life,a injured	0.95	1	0.9
Cost inputs
Total program cost (NYC), $	10 298 000	. . .	. . .
Per capita program cost (NYC), $
School-aged children (5-19 y)	254	. . .	. . .
Adults using intersection	57	. . .	. . .
Injury cost, $
Hospitalized	50 832	60 998	40 666
Not hospitalized	1170	1404	936
Totalb	7129	11 139	4114
Death cost, $
Project year 1	6351	7621	5081
At end of life/school-aged children	930	1116	744
3 y of bus transit cost, $	2016	. . .	. . .
Mean user age
School-aged children, y	10	. . .	. . .
Adults using intersection, y	35	. . .	. . .

Note. NYC = New York City.

^aThe health-related quality of life score is scaled 0 to 1, with 1 representing perfect health.

^bDoes not include lost productivity and leisure time, which is assumed to be captured in the health-related quality of life score.

The average annual cost of transporting a child to school, $692, was obtained from the US Department of Education.¹⁸ Since the vast majority of children under SRTS commute on foot rather than bike,¹² we assumed that active transport carried no costs. Costs associated with death and memorial preferences were derived from the 2009 National Funeral Director’s survey, then inflated using the general CPI.¹⁶ Our estimated cost of death considered the average cost of burial versus cremation and their associated frequencies. The total cost of the SRTS program in New York City was $10 298 000.

Health-Related Quality of Life

QALE is the product of the cohort’s mean health-related quality of life (HRQL) and its life expectancy. The impact of childhood pedestrian injury and death on victim’s HRQL was assessed using the EQ5D-5L,¹⁹ which was distributed to 2 senior pediatric orthopedic surgeons at Columbia University Medical Center in New York City; they have extensive experience working with adults who were struck by motor vehicles as children. These senior physicians were asked to provide subjective assessments of the impact of the “average” child hospitalized for vehicular injury with respect to that individual’s lifelong mobility, self-care, usual activities, pain/discomfort, and anxiety/depression.

The 2 EQ5D-5L instruments yielded scores of 0.64 and 0.47, for an average value of 0.55 for injury victims who required hospitalization (a value of 0 is equated with death, and a value of 1 indicates a year lived in perfect health). Based upon this score for hospitalized school-aged pedestrians and a value of 1.0 for nonhospitalized victims, we computed a weighted average HRQL score of 0.95 for all individuals injured in crashes.

Probabilities

Additional inputs into the first model included pedestrian-injury incidence rates and pedestrian injury case fatality rates for both SRTS and no SRTS arms. Model inputs are listed in Table 1.^4,5,12,17 We did not include estimates of those injured who did not seek medical care. It was conservatively estimated that any given school-aged child (aged 5–19 years) would only use an SRTS intersection for 3 years before moving to another school or graduating.

Sensitivity Analyses

Before running sensitivity analyses, we found that SRTS was cost saving even in the annual cohort model. This model is relevant in the event that all of the observed benefits of SRTS are transient (e.g., if they all occurred through the education component). Clearly, multiplying these benefits by many years will only amplify savings. Therefore, we only conducted sensitivity analyses on the annual model. We conduced a series of one-way sensitivity analyses along with a Monte Carlo simulation. In the Monte Carlo simulation, we either included what we recognized as plausible boundaries for the values or we included the known random error associated with an estimate. For example, injury risk reductions were varied by the standard errors. The amount of time a child spends at a school, on the other hand, must be bounded by guesswork because we do not have adequate data on this for our SRTS neighborhoods.

RESULTS

The model inputs are listed in Table 1, and the underlying assumptions of the modeling approach are listed in Table 2. For the first cohort of intersection users, school-aged SRTS users produce a net societal savings of $224 and an incremental gain of 0.0004 QALYs over their lifetimes (Table 3). For all pedestrians (societal costs), the figures are similar, with $226 in savings and 0.0008 QALYs gained. This single cohort analysis was robust to all sensitivity analyses using a willingness-to-pay threshold of $100 000 per life year gained (a predetermined threshold for the purposes of our sensitivity analyses only). An influence analysis (“tornado” diagram) suggests that the most important variables in the analysis were the probability of injury, transportation costs, the risk reduction associated with SRTS, and the discount rate. The impact on the incremental costs and effectiveness of the model of variations in these variables can be found in Table 4. These figures reflect the returns that might be expected over a single cohort of intersection users.

TABLE 2—

Assumptions Used in the Markov Model Evaluating Safe Routes to School (SRTS) in New York City vs the Status Quo

Assumption	Rationale (Impact on Estimates)
Future lost productivity and leisure time costs of injury are included within the health-related quality of life score.	EQ5D scores may implicitly include lost productivity and leisure time, however this has been debated.14 (Favors status quo.)
SRTS benefits were limited to roadway changes.	Conventional wisdom holds that safety education programs are less important than roadway modifications. (Favors SRTS.)
We assumed that exercise, reduced pollution, and reduced noise associated with active transport to school would have no impact on future costs.	If active transport does produce lifelong health benefits, this assumption favors status quo.
Mortality risk occurs only at the time of injury.	Injury victims may be at higher risk of future death both from physical limitations and the economic impact of the injury on the victim’s life. (Favors status quo.)

TABLE 3—

Costs and Quality-Adjusted Life Expectancy (QALE) of Safe Routes to School (SRTS) in New York City vs the Status Quo

	School-Aged Pedestrians				All Pedestrians
Variable	Cost, $	Incremental Cost, $	QALE	Incremental QALY	Cost, $	Incremental Cost, $	QALE	Incremental QALY
Per user per y
SRTS	4	–224	28.3009	0.0004	16	−226	51.7602	0.0008
Status quo	227		28.3005		242		51.7595
Per user older than 50 y
SRTS	352	–5449	721.7209	0.0103	664	−5500	1319.9715	0.0193
Status quo	5801		721.7106		6164		1319.9522
Total, NYCa
SRTS	14 246 784	–220 826 117	29 247 740	417	70 891 435	−230 047 354	137 619 641	2055
Status quo	235 072 901		29 247 323		300 938 789		137 617 585

Note. NYC = New York City; QALY = quality-adjuste life years.

^aTotals tend to be much higher for all users only because many more adults tend to use these intersections than do school-aged pedestrians.

TABLE 4—

One-Way Sensitivity Analyses of Variables Included in the Annual Model

	Incremental Cost		Incremental Effectivenessa
Variable	High	Low	High	Low
Probability of injury	−222	−223	0.0003	0.00005
Bussing costs	−1366	−2651	. . .b	. . .b
Risk reduction	−223	−227	0.0008	0.00002
HRQL	. . .b	. . .b	0.0008	0
Discount rate	−223	−224	0.001	0.0003

Note. HRQL = health-related quality of life. Variables are based on those used in the Markov model evaluating safe routes to school.

^aIncremental effectiveness was determined by quality-adjusted life years gained.

^bChanges in the variable had no impact on this outcome measure.

These single cohort costs, however, do not reflect the likely true cost-effectiveness of SRTS because intersection modifications tend to last for decades, and most benefits are probably realized through such modifications. When discounted over the assumed 50-year useful life of an intersection at a 3% rate, SRTS saves $5500 per user and results in a net effectiveness gain of nearly 0.02 QALYs per user. In the parlance of cost-effectiveness analysis, SRTS is said to “dominate” a no investment strategy (the status quo), indicating that every dollar spent in SRTS saves both money and lives. This dominance is not impacted by variance in the plausible ranges of the variables represented in Table 1, including the overall Monte Carlo simulation, which predicts that SRTS would save money and lives in virtually 100% of the simulations. The figures represent savings per user. When all users of SRTS intersections over 50 years are considered, the overall discounted cost and effectiveness of the policy is more apparent, reaching $230 million saved and 2055 QALYs gained.

When restricted to school-aged pedestrians, the overall program cost remains the same ($10 million), but the number of users drops substantially (from 181 000 to 41 000). However, these drops are offset by a much longer time horizon for which QALYs are gained (from age 10 years rather than from age 35 years), by reductions in bussing costs (which are a major component of the model) and by a much larger protection associated with SRTS (a risk reduction of 33% rather than just a 14% average risk reduction). The net result of targeting children produces cost reductions of $221 million. However, because adult users are much more likely to be injured on average (possibly because drivers are more careful during school hours and crossing guards are present), most QALYs are gained among adult users of SRTS intersections.

DISCUSSION

Whether evaluated using difference in difference or Bayesian changepoint analysis, SRTS has been shown to reduce both childhood and overall injury rates in New York City. Taken together, these 2 methodological approaches produce reasonable confidence that the observed effects are not confounded by exogenous variables. To the extent that these results are valid, SRTS appears to not only improve health and save lives, but to also save money. Government agencies that chose to invest in SRTS may therefore receive returns on their investment not only with respect to money but also with respect to the health of their constituents. SRTS is, by its nature, a program targeted toward school-aged pedestrians rather than all intersection users. But even when the analysis is limited to the benefits realized by children (the policy target), the program remains cost saving. When all users are considered (the societal benefits), the cost-effectiveness improves further.

It should be noted, though, that school zones are likely already safer than other areas of most cities. For example, speed limits are much lower in school zones, and crossing guards are often present during school hours. Much greater benefits may be realized by improvements to high-risk intersections outside of school zones, including larger reductions in childhood fatalities.

With respect to other benefits, SRTS appears to increase the number of children who bike or walk (relative to driving) to school.^9,10,12,20 Obesity rates in New York City are dropping simultaneously as street modifications are implemented. However, although strongly suggestive, there is no conclusive evidence that active transport reduces the incidence of childhood overweight or obesity.^21,22 The incidence of childhood overweight and obesity is about 34% to 43% in New York City school-aged children.^23–25 The annual medical cost of childhood obesity is $220 per case among school-aged children.²⁶ However, about one third of obese children become obese adults, and obese adults accrue many additional costs, such as lost productivity costs, and obesity accounts for upwards of 10% of all medical costs.^27,28 Moreover, the rise in obesity expenditures outstrips the medical portion of the consumer price index in part because obesity rates are increasing and in part because, as the population ages, the obese incur disproportionate costs.²⁹ Were SRTS to prevent a small number of cases of obesity in each cohort, this benefit alone would likely pay for the entire program over time.

This study was prone to a number of important limitations. First, although upwards of 90% of SRTS funds were used for engineering and infrastructure projects (e.g., sidewalk construction),¹¹ SRTS funded some education programs as well as roadway improvements. We assumed that the effects of SRTS would be limited primarily to roadway improvements and therefore modeled these benefits over 50 years. Education benefits, however, tend to be more transient. If the benefits of SRTS were limited to education campaigns rather than roadway improvements, then the expected benefits we observed would therefore be greatly reduced. On the other hand, the cost estimate that we used included both infrastructure programs and roadway improvements. Therefore, if roadway improvements are much more effective than education campaigns, we would have underestimated the cost-effectiveness of SRTS. Even when analyzed over just 1 cohort, however, SRTS produces a reasonable cost-effectiveness estimate.

Another limitation is that our estimates exclude any social or health benefits associated with increased exercise, reduced neighborhood traffic congestion, reduced pollution, and perhaps increased neighborhood cohesion, all of which may reduce excess morbidity and mortality and improve quality of life in New York City.³⁰ These data are difficult to ascertain, however. Inclusion of such costs could further increase the value of SRTS. A final important limitation is that our data were not derived from multicenter randomized controlled trials. The estimates upon which we base the effectiveness of SRTS are therefore susceptible to unforeseen threats to internal validity.

Finally, given its overall benefits, it would be logical to expand the program to high-risk intersections that fall outside of school zones. However, doing so assumes that the results of the present study are generalizable to high-risk census tracts outside of those studied here. Given that some questions remain surrounding the internal validity and generalizability of SRTS, randomized implementation of SRTS across neighborhoods in the United States would simultaneously allow for investments in a program with social benefits and conclusively demonstrate whether these benefits are tenable.

Public health practitioners have moved beyond initiatives that require actions on the part of the public, such as health education campaigns, and on to innovative approaches to improve population health.^31,32 These include initiatives that reengineer the environments of the public such that they produce health.^33–35 Such initiatives remove the requirement for action on the part of the user, either protecting the user from harm or “nudging” him or her into safer behaviors. A particular approach has entailed roadway modifications that make active transport safer, simultaneously reducing injuries and thereby reducing barriers to walking outside.^9,10 In New York City, SRTS appears a good investment from both the standpoint of the policy’s intended target (school-aged children) and overall (from a societal perspective). A large, nationally representative trail of roadway modifications would greatly improve our understanding of the overall utility and cost-effectiveness of such investments. Such a study would be especially timely given that this cost-saving program has recently been defunded.