Testing of the Storm Water Management Model Low Impact Development Modules

Abstract

Stormwater infrastructure designers and operators rely heavily on the United States Environmental Protection Agency’s Storm Water Management Model (SWMM) to simulate stormwater and wastewater infrastructure performance. Since its inception in the late 1970s, improvements and extensions have been tested and evaluated rigorously to verify the accuracy of the model. As a continuation of this progress, the main objective of this study was to quantify how accurately SWMM simulates the hydrologic activity of low impact development (LID) storm control measures. Model performance was evaluated by quantitatively comparing empirical data to model results using a multievent, multiobjective calibration method. The calibration methodology utilized the PEST software, a Parameter ESTimation tool, to determine unmeasured hydrologic parameters for SWMM’s LID modules. The calibrated LID modules’ Nash–Sutcliffe efficiencies averaged 0.81; average percent bias (PBIAS) −9%; average ratio of root mean square error to standard deviation of measured values 0.485; average index of agreement 0.94; and the average volume error, simulated vs. observed, was +9%. SWMM accurately predicted the timing of peak flows, but usually underestimated their magnitudes by 10%. The average volume reduction, measured outflow volume divided by inflow volume, was 48%. We had more difficulty in calibrating one study, an infiltration trench, which identified a significant limitation of the current version of the SWMM LID module; it cannot simulate lateral exfiltration of water out of the storage layers of a LID storm control measure. This limitation is especially severe for a deep LIDs, such as infiltration trenches. Nevertheless, SWMM satisfactorily simulated the hydrologic performance of eight of the nine LID practices.

Introduction

The United States Environmental Protection Agency (USEPA) first commissioned the development of the Storm Water Management Model (SWMM) in the 1970s to provide tools for assessing and managing stormwater runoff, especially when mixed with sanitary waste in combined sewer overflow (CSO) settings. In a combined sewer system, wastewater and stormwater transport into the same conveyance. During significant rainfall events, both conveyance and treatment capacity can be exceeded, resulting in the discharge of untreated wastewater into surface water bodies. While these practices may have seemed harmless and cost-effective when cities and towns constructed their sewer systems many decades ago, they often did not meet the Clean Water Act of 1971 standards. USEPA has consent decrees with 45 major municipalities and other entities (USEPA 2017a) mandating regulated entities to reduce CSO by significant quantities (e.g., an 85% reduction in volume and large reductions in the number of overflows occurring each year). Complying with the consent decrees often requires multibillion-dollar upgrades to sewer systems that may take decades to accomplish. In 2012, the USEPA estimated that the United States would need $271 billion of capital investment to upgrade its stormwater and wastewater infrastructure systems during the subsequent 20 years (USEPA 2012).

Traditional designs upgrading CSO’s have used gray infrastructure, including tanks, pumps, and pipes, to accomplish the needed improvements. Typically, “gray” infrastructure conveys stormwater and wastewater to a wastewater treatment facility where it receives treatment and discharges to receiving waters. It can be characterized by simplistically being “big enough” to accommodate the maximum combined flow. In contrast with gray infrastructure, green infrastructure (GI) reduces the volume of combined wastewater needing treatment by increasing the stormwater infiltration into the subsurface while increasing the evapotranspiration. Also, GI reduces stormwater peak flow, improves runoff water quality, and can restore watershed ecoservices (Hunt et al. 2012). GI is often considered in the upgrades of stormwater and wastewater infrastructure as it could reduce costs and lower CSO.

Fletcher et al. (2015) describe GI terminology for a variety of stormwater control measures (SCMs), also known as best management practices (BMPs), and low impact development (LID). GI refers to these practices that are linked to function as a system. LID is a holistic water management technique that attempts to restore the natural hydroperiod to predeveloped conditions. This technique uses a variety of processes, including infiltration and evapotranspiration. Common SCMs using LID principles include green roofs, rain gardens, infiltration trenches, and permeable pavements (USEPA 2017b). GI can combine these SCMs in aggregrate (Ahiablame et al. 2012). USEPA has published guidance on performing system upgrades, modeling these upgrades, and using GI to address CSO consent decrees, Municipal Separate Storm Sewer System, and other permits (USEPA 2017c).

Physical Characteristics of LID Units

LID units such as permeable pavements, infiltration trench, green roofs, vegetative swales, and bioretention units, can replace parking lots and other typically impervious surfaces. Permeable pavement can be porous asphalt, porous concrete, or interlocking pavers that allow for more infiltration (Drake et al. 2013; Imran et al. 2013; Mullaney and Lucke 2014). Infiltration trenches are narrow depressions that provide storage volume to collect and infiltrate runoff from upslope impervious areas (Dietz 2007). Rain barrels collect stormwater from roofs and are inexpensive and easy to implement (Jones and Hunt 2010; Steffen et al. 2013).

Many LID practices couple biological processes with hydrological functions. Vegetative swales are infiltration trenches with plants (Davis, Stagge, et al. 2012; Davis, Traver, et al. 2012). Bioretention cells are depressions in the ground containing vegetation grown in an engineered soil mixture and can have a gravel storage bed and sometimes a drainage pipe from one of the layers (USEPA 2019). The layers in a bioretention cell allow storage, infiltration, and evaporation of direct rainfall and runoff from upstream areas. Davis et al. (2009), Roy-Poirier et al. (2010), and Hunt et al. (2012) are compilations of the results of bioretention projects. Rain gardens are a type of bioretention cell, and Selbig and Balster (2010) investigated the rain gardens with different species of vegetation (turf grass vs. native prairie species) and soil types. Green roofs can be modeled similarly to a bioretention cell, but with a thin layer of a synthetic drainage mat material or a highly porous layer to convey water off the roof. Green roofs may be especially attractive in dense urban areas where open space is limited. Green roofs increase contiguous green space for migrating fowl and reduce building energy requirements (Fernandez-Canero and Gonzalez-Redondo 2010; Thuring and Grant 2016; Besir and Cuce 2018).

Performance of Models for LID Units

SWMM was originally developed to assist practitioners in sizing stormwater and wastewater gray infrastructure. The original edition of SWMM did not include explicit provisions to consider GI. Since the 1970s, SWMM has been upgraded to incorporate various features, including robust hydraulic formulations, more powerful solution engines, interactive interfaces, and more. SWMM Version 5, introduced in 2010 (Rossman 2010) included GI. Calibration of SWMM and its earlier advancements uses comparisons of predictions to the performance of multiple real systems, including GI systems.

The International BMP Database (2019) (http://www.bmpdatabase.org) contains many empirical LID studies, but there also have been several articles compiling these studies. Two comparisons of SWMM LID modules to empirical studies for green roofs have been published recently. Peng and Stovin (2017) modeled green roofs and found that the uncalibrated model provided reasonable estimates of total annual retention. Still, the modeled runoff deviated significantly from the measured data during dry summer months. The authors stated that the failure to account for the influence of initial soil moisture ultimately limits the model’s predictive results. Cipolla et al. (2016) found that SWMM accurately predicted the runoff from green roofs throughout the year, as evidenced by Nash–Sutcliffe efficiencies (NSE) approaching unity. These specific SWMM LID testing studies focused on hydrology rather than water quality.

Zhang and Guo (2015) calculated infiltration through pavement layers or permeable pavement systems with SWMM version 5.1.006. They found its results to be inadequate when depths of permeable pavement layers are <120 mm, and computational times are longer than 30 min. They proposed an alternative method for representing equivalent subcatchments. This proposal is consistent with Rossman and Huber’s (2016a, 2016b) guidance for SWMM time steps.

On the urban watershed scale, some models of GI impacted watersheds were calibrated with field data: (cf. Rosa et al. 2015). Still, most watershed stormwater studies were hypothetical (i.e., without comparison to empirical data) with authors using SWMM to calculate the benefits of future GI scenarios: (Luan et al. 2017; Tao et al. 2017). Krebs et al. (2013) presented parameter sensitivity analysis for a high-resolution, highly urbanized small subcatchment in Finland. They identified two critical parameters, depression storage and Manning’s roughness (n) for conduit flow that led to the proper calibration and validation of the model.

The objective of this study assesses how accurately SWMM v5.1.10 models the hydrologic performance of LIDs, by comparing empirical data from specific LID monitoring studies using a multievent, multiobjective calibration method. We compared observed vs. simulated outflow magnitudes and intensities, the timing and magnitude of peak flows, and parameter sensitivities. Calibration was based on maximizing the model performance evaluation criteria, such as the NSE, the index of agreement (d), the ratio of root mean square error and standard deviation of measured data (RSR), and minimizing the PBIAS by comparing measured and simulated outflow datasets. A sensitivity analysis was performed to determine which parameters have the most significant impact on the modeled output, highlighting which parameters should be most closely measured for accurate simulation results.

SWMM LID Equations and Parameters

SWMM simulates runoff quantity and quality by modeling a watershed as a collection of subcatchment areas that receive precipitation. SWMM tracks runoff generated from a subcatchment during a simulation period comprised of multiple time steps. LID practices are a type of subcatchment (Rossman and Huber 2016a, 2016b). The core processes of SWMM contain the following components: (1) precipitation and evaporation; (2) land surface runoff; (3) subsurface groundwater; (4) conveyance system of pipes, channels, pumps, flow regulators, and storage units; and (5) rain gages, weirs, and other such monitoring equipment. SWMM considers heat and mass transport processes and can model snowmelt and evaporation, as well as contaminant buildup, wash-off, and simple treatment. Please see Figure 1 for the SWMM conceptual model. Rainfall as precipitation is introduced as a time series function by a rain gage. It falls onto the watershed and either infiltrates into the subsurface, evaporates, or runs off the subcatchment into a receiving unit, such as a culvert, sewer, or natural water body. Rainfall should be even over the subcatchment and even over the rainfall time step. From the conservation of mass, the net change in depth d (Length, L) per unit of time t is simply the difference between inflow and outflow rates over the subcatchment (Rossman and Huber 2016a):

$$\frac{\partial d}{\partial t}=r+i-e-f-q,$$ (1)

where r = run on rate from upstream area, if present (L/t); i = rainfall + snowmelt rate (L/t); f = infiltration rate (L/t); q = runoff rate (L/t).

Figure 1.

SWMM conceptual model (for a hypothetical watershed) Rossman and Huber (2016a). SWMM, Storm Water Management Model.

Note that the fluxes i, e, f, and q are expressed as flow rates per unit area, volume/area/time (L³/L²/t). SWMM can report total volumes or normalized flux.

The generic SWMM LID module is derived from the standard designs of a prototypical bioretention cell and is depicted in Figure 2.

Figure 2.

Typical bioretention cell cross section (Rossman and Huber 2016b).

Several simplifying assumptions are made within SWMM when modeling LID practice hydrologic performance for this study (Rossman and Huber 2016a, 2016b; Rossman 2017):

1. The cross-sectional area of the unit remains constant throughout its depth.

2. Flow, through the unit, is one-dimensional in the vertical direction; there is no lateral exfiltration from the soil, sand, or storage layers.

3. Inflow to the unit is distributed uniformly over the top surface.

4. Initial moisture content uniformly distributes throughout the soil layer.

5. Matric forces within the storage layer are negligible so that it acts as a simple reservoir that stores water from the bottom up.

SWMM models vertical water flow across areas in a layer. For permeable pavements, the surface layer is the permeable material see Figure 3. Otherwise, the surface layer is vegetated or soil and receives direct precipitation and inflow routed from other impervious areas. Water leaves the surface layer via evaporation to the atmosphere, as surface runoff, or through infiltration to the soil layer. The intermediate soil layer can be an engineered soil mixture or native soil used to support plant growth and loses water via evapotranspiration to the atmosphere and percolation to the drainage layer. The storage layer allows infiltration into native soils and may have water removal via a drainage pipe. A green roof does not drain into a native formation but has a drainage mat. A permeable pavement LID has an additional pavement layer and a sand layer.

Figure 3.

Permeable pavement cell cross section (Rossman and Huber 2016b).

We used SWMM version 5.1.010 for all results presented in this work. SWMM comprises the hydrology equations that govern water storage and transport in at least three layers: the surface layer, the soil layer, and an underlying storage layer. The equations unite the concepts of precipitation, infiltration, evapotranspiration, percolation, and exfiltration. The equations also include definitions of bounding features such as permeable pavers to moderate water transport at the surface and drainage mats to interact with discharge at the bottom. A LID, as defined within SWMM, is a GI subcatchment that allows water to evapotransporate, infiltrate into various layers, or runoff from the surface or subsurface drainage pipe. SWMM LID uses the Green-Ampt equation.

The LID infiltrates, evaporates, or stores precipitation and runon until the flux of water coming flux of water into the LID is greater than what it can infiltrate, evaporate, and store. The downward vertical water goes into the native soil. The infiltration rates of the surface, LID soil, storage, drainage layers, and native soil layer are functions of their hydraulic conductivity, moisture content, suction head, and wilting points.

Methods

The objective of our study compares SWMM predictions to empirical data. We located several well-documented studies and contacted the authors for measured data: LID dimensions, inflow volume, outflow volume, precipitation, and any other measured parameters. Precipitation was measured via tipping bucket rain gage, inflows and outflows were measured by weirs or flow meters. Details of each specific study can be found in https://edg.epa.gov/metadata/catalog/main/home.page. Many of the references used in this study were municipal reports or graduate student theses and are listed in Tables 2–6. We created SWMM input files, based on these data, and compared SWMM predicted outflow to observed outflow at regular time intervals. Underdrain runoff, if present, or weir captured surface overflow, was routed to the LID outfall. We compared the observed vs. calculated hydrograph, peak flow, the timing of the peak flow, and outflow volumes. We did not consider evaporation in any of these study evaluations.

Table 2.

Multi-objective SWMM LID module performance evaluation criteria data Bioretention.

Date	NSE	PBIAS, %	RSR	d	Observed inflow, mm	Observed outflow, mm	Simulated outflow, mm	Simulated to observed volume error percent	Simulated to observed peak flow percent	Observed volume reduction percent
Graham Bioretention, (Passeport et al. 2009)
March 1, 2007	0.70	2.59	0.29	0.98	611.09	607.34	591.63	−2.59	−22.39	0.61
March 16, 2007	0.57	−3.76	0.42	0.94	760.12	722.87	777.26	7.52	−39.81	4.90
April 1, 2007	0.55	18.38	0.30	1.00	98.36	28.61	23.35	−18.38	−8.02	70.92
May 9, 2007	0.88	18.11	0.38	0.97	79.04	19.03	15.59	−18.11	15.17	75.92
Average	0.67	8.83	0.34	0.97	387.15	344.46	351.96	−7.89	−13.76	38.09
Min	0.55	−3.76	0.29	0.94	79.04	19.03	15.59	−18.38	−39.81	0.61
Max	0.88	18.38	0.42	1.00	760.12	722.87	777.26	7.52	15.17	75.92
Villanova BTI (Emerson and Traver 2008; Lord 2013)
June 15, 2004	0.74	−73.92	0.46	0.96	355.87	103.87	180.66	73.92	8.88	70.81
June 3, 2005	0.91	−42.17	0.29	0.98	566.05	194.04	275.88	42.17	92.21	65.72
August 21, 2009	0.74	−25.35	0.44	0.94	1,316.20	936.91	1,174.45	25.35	−45.99	28.82
October 29, 2012	0.97	−8.86	0.19	0.99	2,531.85	2,150.32	2,531.85	17.74	−7.06	15.07
Average	0.84	−37.58	0.35	0.97	1,192.49	846.29	1,040.71	39.80	12.01	45.10
Min	0.74	−73.92	0.19	0.94	355.87	103.87	180.66	17.74	−45.99	15.07
Max	0.97	−8.86	0.46	0.99	2,531.85	2,150.32	2,531.85	73.92	92.21	70.81
Bioretention
Average	0.76	−14.37	0.35	0.97	789.82	595.37	696.33	15.95	−0.87	41.60
Min	0.55	−73.92	0.19	0.94	79.04	19.03	15.59	−18.38	−45.99	0.61
Max	0.97	18.38	0.46	1.00	2,531.85	2,150.32	2,531.85	73.92	92.21	75.92

Note: Bold values indicate satisfactory agreement between predicted results and observed data as per Moriasi et al. (2015).

Table 6.

Multi-objective SWMM LID module performance evaluation criteria data permeable pavement.

Date	NSE	PBIAS, %	RSR	d	Observed inflow, mm	Observed outflow, mm	Simulated outflow, mm	Simulated to observed volume error percent	Simulated to observed peak flow percent	Observed volume reduction percent
Boone Permeable Pavement (Wardynski et al. 2013)
April 27, 2011	0.85	20.93	0.41	0.94	31.57	0.53	0.42	−20.93	−33.09	98.31
June 8, 2011	0.64	−26.97	1.01	0.90	34.44	0.38	0.49	26.97	−38.20	98.89
July 4, 2011	0.77	−0.64	0.45	0.93	41.83	1.21	1.21	0.64	−20.53	97.12
September 5, 2011	0.70	0.50	0.67	0.90	54.61	2.26	1.66	−26.63	−47.99	95.86
Average	0.74	−1.54	0.63	0.92	40.61	1.10	0.95	−4.99	−34.95	97.54
Min	0.64	−26.97	0.41	0.90	31.57	0.38	0.42	−26.63	−47.99	95.86
Max	0.85	20.93	1.01	0.94	54.61	2.26	1.66	26.97	−20.53	98.89

Note:Bold values indicate satisfactory agreement between predicted results and observed data as per Moriasi et al. (2015).

Our analysis was based on representing each site as a LID practice, which occupies the entire subcatchment so that we could focus on the performance of each LID. The SWMM LID configuration for Graham Bioretention Cell, Villanova BioInfiltration Traffic Island (BTI), Villanova Infiltration Trench, and Washington Department of Transportation (Washington DOT) is presented in Figure 4. One study, the University of Maryland (UMD) Bioswale, presented in Figure 5, had SWMM calculated the runoff from the upstream impermeable highway, which drained onto the bioswale. The LID configuration for the Boone Permeable Pavement and the three green roof studies, Hamilton EcoRoof, Fire Station 10 (FS10), and the Emergency Operations Center (EOC) Green Roof, is presented in Figure 6.

Figure 4.

Most common SWMM5 LID conceptual configuration used in this study. LID, low impact development.

Figure 5.

M5 LID configuration used for UMD BioSwale. UMD, University of Maryland.

Figure 6.

SWMM5 model permeable pavement and green roof configuration.

Parameter Estimation Approach

Usually, the original work contained several storms, and we randomly selected two to four storms that experienced runoff, but were representative of the range of inflow/outflow performance of the LID in terms of intensity, inflow volume, outflow volume, and season. For one study, FS10 Green Roof, we modeled continuous storm information for two quarters. The lead author of another study, the Graham Bioretention, speculated that the forebay located immediately upstream of the unit became flooded, resulting in an underestimate of the inflow into the BMP, so we selected smaller storms for this test. We selected the first chronological storm to calibrate and optimized unmeasured parameters using PEST, a nonlinear Parameter ESTimation (PEST) tool. PEST uses the Gauss–Marquardt–Levenberg method to optimize parameters of a given model (Doherty 2005). First initial values were assumed for those that were not measured for each study. PEST then applied a Taylor series expansion to generate a linear function to compare model parameters to the modeled results. This linear function is used to generate new parameters whose model results can be compared to observed results. The process continues until the parameter changes do not seem to improve the results. The PEST program determines parameter sensitivity by calculating the change in the model result due to a numerical change was in parameter value.

SWMM was run using the PEST optimized calibration parameters for the remaining storms, and we calculated the NSE, the index of agreement (d), the PBIAS, and the RSR as was recommended by Moriasi et al. (2015). The NSE is a quantitative measure for comparing simulated to measured results and varies from negative infinity to unity. The index of agreement, d, detects additive and proportional differences in the observed and simulated means and variances, can be sensitive to extreme values, and ranges from 0.0 to 1.0. PBIAS is used to determine how closely the simulated results match the average magnitudes for the measured values and ranges from negative infinity to 0.0 percent. The ratio of the root mean square error to the standard deviation of measured data (RSR) has the benefits of error statistics and a scaling normalization factor. It can vary from 0 to positive infinity with the smaller RSR, the closer the simulated result is to the observed.

Moriasi et al. (2015) state that model simulation can be used satisfactorily if NSE > 0.5, d > 0.75, PBIAS ± 15%, and RSR ≤ 0.7. Moriasi et al. (2007, 2015) described model “calibration” as the process of estimating model parameters by comparing model predictions for a given set of conditions to observed data for the conditions. They describe model “validation” as running the model using the parameters from the calibration process. Rossman (1997) recommended avoiding the terms “validation” and “verification” because it is debatable whether a model can attain these states and recommended the term “testing” in the title of this paper.

Results and Discussion

We compared the observed vs. simulated outflow intensities, total outflow volumes, the timing and magnitudes of the peak flows and investigated parameter sensitivity. We also calculated the reduction in inflow vs. outflow volumes for each study. The SWMM result analyses for all calibrated studies are presented in Table 1. Individual LID results are presented in Tables 2–6 and Figures 7–14 are representative hydrographs from each study. The references for the original studies are given in the Tables.

Table 1.

Multi-objective SWMM LID model performance evaluation criteria data all studies except Villanova Infiltration Trench.

Date	NSE	PBIAS, %	RSR	d	Observed inflow, mm	Observed outflow, mm	Simulated outflow, mm	Simulated to observed volume error percent	Simulated to observed peak flow percent	Observed volume reduction percent
Average	0.83	−8.56	0.48	0.94	237.76	174.56	203.29	8.35	−8.52	46.10
Min	0.55	−73.92	0.18	0.85	0.76	0.21	0.21	−26.63	−47.99	0.61
Max	0.99	20.93	1.11	1.00	2,531.85	2,150.32	2,531.85	73.92	92.21	98.89

Notes: Simulated to Observed Volume Error = 100 × ((Simulated Volume – Observed Volume)/Observed Volume). Observed Volume Reduction = 100 × (Outflow Volume/Inflow Volume).

NSE, Nash–Sutcliffe efficiency; PBIAS, percent bias; RSR, ratio of root mean square error and standard deviation of measured data; d, index of agreement.

Bold values indicate satisfactory agreement between predicted results and observed data as per Moriasi et al. (2015).

Figure 7.

Graham Bioretention representative hydrograph.

Figure 14.

Boone Permeable Pavement representative hydrograph.

Bioretention

The SWMM result analyses of the two bioretention studies, Graham and the Villanova BioInfiltration Traffic Island (BTI), are presented in Table 2 and Figures 7 and 8. Graham was modeled via the SWMM bioretention LID with a surface, soil, storage, and drain layers. Villanova BTI was modeled as a SWMM rain garden LID two layers: surface and soil. SWMM satisfactorily predicted the outflow hydrograph for most criteria, with NSE [average 0.76, range (0.55 to 0.97)]; RSR [average 0.35 (range 0.19 to 0.46)]; and d [average 0.97 (range 0.94 to 1.00)]. The only satisfactory performance criteria not consistently met was PBIAS, [average 14%, range (−74% to 18%)]. SWMM accurately predicted the timing of peak flows both studies and had a close average of their magnitudes with a wide range [average −0.87%, range (−46% to +92%)]. Similarly, the average error for simulated vs. observed total outflow volume was 16%, but ranged from −18% to +74%. The volume reduction percent, the volume of outflow divided by volume of inflow average was 41%, (0.61% to 76%) for the two bioretention studies.

Figure 8.

Villanova BTI representative hydrograph.

Bioswale

The UMD and the Washington DOT Bioswale studies were investigated, see Table 3 and Figures 9 and 10. These studies’ hydrographs generally met the performance evaluation criteria NSE [average 0.82, range (0.74 to 0.94)] and d [average 0.93, (0.91 to 0.9)], but RSR was slightly higher than recommended [average 0.59, (0.60 to 1.11)]. The PBIAS (averaged −5.35% (−48% to +20%) varied widely for the Washington DOT study. UMD was unique for our evaluation in that we had to calculate the runoff from the upgradient Highway Drainage Area via SWMM (see Figure 5), whereas the bioretention, bioswale, infiltration, and trench studies had directedly measured inflows (see Figure 4) and the permeable pavement and green roof studies did not have inflows. While all studies directly measured precipitation, most had other inflows were also directly measured and the LID was calibrated with the directly measured inflow. We used PEST to estimate the UMD Highway Drainage parameters as well as estimating the LID parameters.

Table 3.

Multi-objective SWMM LID module performance evaluation criteria data Bioswale.

Date	NSE	PBIAS, %	RSR	d	Observed inflow, mm	Observed outflow, mm	Simulated outflow, mm	Simulated to observed volume error percent	Simulated to observed peak flow percent	Observed volume reduction percent
UMD Bioswale (Davis, Stagge, et al. 2012)
November 4, 2004	0.74	−6.97	0.54	0.91	30.78	28.76	30.76	6.97	−44.28	6.57
June 3, 2005	0.76	7.47	1.11	0.93	14.78	8.20	7.59	−7.47	−15.70	44.51
November 16, 2005	0.81	3.42	0.47	0.93	17.84	14.21	13.72	−3.42	−22.07	20.37
January 11, 2006	0.79	16.16	0.47	0.92	5.69	5.48	5.69	3.78	−37.46	3.64
Average	0.78	5.02	0.65	0.92	17.27	14.16	14.44	−0.04	−29.88	18.78
Min	0.74	−6.97	0.47	0.91	5.69	5.48	5.69	−7.47	−44.28	3.64
Max	0.81	16.16	1.11	0.93	30.78	28.76	30.76	6.97	−15.70	44.51
Washington DOT (Maurer 2009)
September 19, 2009	0.76	20.19	0.49	0.93	0.76	0.27	0.21	−20.19	−17.81	65.15
October 13, 2009	0.83	−5.47	0.41	0.95	11.10	6.56	6.92	5.47	−39.19	40.95
November 5, 2009	0.89	−47.83	0.58	0.92	12.22	5.83	8.62	47.83	−21.63	52.28
February 16, 2010	0.94	−29.75	0.64	0.93	1.45	0.21	0.28	29.75	54.71	85.18
Average	0.86	−15.72	0.53	0.93	6.38	3.22	4.01	15.72	−5.98	60.89
Min	0.76	−47.83	0.41	0.92	0.76	0.21	0.21	−20.19	−39.19	40.95
Max	0.94	20.19	0.64	0.95	12.22	6.56	8.62	47.83	54.71	85.18
Bioswale
Average	0.82	−5.35	0.59	0.93	11.83	8.69	9.22	7.84	−17.93	39.83
Min	0.74	−47.83	0.41	0.91	0.76	0.21	0.21	−20.19	−44.28	3.64
Max	0.94	20.19	1.11	0.95	30.78	28.76	30.76	47.83	54.71	85.18

Note: Bold values indicate satisfactory agreement between predicted results and observed data as per Moriasi et al. (2015).

Figure 9.

The UMD Bioswale representative hydrograph.

Figure 10.

Washington State Bioswale representative hydrograph.

Additionally, the UMD project had different slopes on either side, which could not be put into SWMM. We put the average of the two into the SWMM input file. SWMM accurately predicted the timing of the peak flow, but usually underpredicted its magnitude [average −18%, (−44% to +55%)]. SWMM averaged predicted total outflow volume compared to observed was +7.8% (−20% to +48%). The average volume reduction, outflow divided by inflow, was 40% (4% to 85%).

Infiltration Trench

The Villanova Infiltration Trench study was the least successful calibration study. Initially, the NSE values ranged from [−3.87 to 0.65], see Figure 11. We have severely underestimated the outflow and attributed this phenomenon to SWMM’s inability to model lateral flow from the soil storage layer into the native soil vertically surrounding the LID. An underdrain was used to attempt to account for this lateral exfiltration. Rather than placing the drain at 1.58 m as occurred in the physical configuration, placing the drain at the bottom allowed water from both lateral exfiltration and drain flow to exit the trench. This inverse modeling indicated that 60% of the outflow was not measured, but we were unable to test this result and did not include them in the statistics presented in Table 1.

Figure 11.

Villanova Infiltration Trench representative hydrograph — inverse modeling results based on storage heights.

Green Roof

The three green roof studies, Hamilton, EOC, and FS10, all met the satisfactory performance criteria: NSE [average 0.93, (0.85 to 0.99)], RSR (average 0.45, (0.18 to 0.77)], d (average 0.94 (0.85 to 0.99) with the PBIAS averaged −9%, (−28% to +0.18%). See Table 5 and Figures 12–14. Individual storms were calibrated for the Hamilton and EOC studies, whereas the FS10 study was calibrated over a winter quarter (January to March 2009) and a summer quarter (July to September 2010). The timing for the peak volume was accurate, and the average predicted peak volume was 3.47, (−18% to +47%) simulated vs. observed. The average predicted outflow volume was 8% higher than observed (−13% to +28%). The average LID performance was 34%, (10% to 66%) volume reduction, outflow volume divided by inflow. Every LID studied reduced the amount of outflow vs. inflow, both as simulated and observed.

Table 5.

Multi-objective SWMM LID module performance evaluation criteria data Green Roof.

Date	NSE	PBIAS, %	RSR	d	Observed inflow, mm	Observed outflow, mm	Simulated outflow, mm	Simulated to observed volume error percent	Simulated to observed peak flow percent	Observed volume reduction percent
Hamilton Green Roof (Portland Bureau of Environmental Services, 2010; Hutchison et al. 2003; She and Pang 2010)
October 16, 2004	0.94	−6.91	0.62	0.92	22.11	10.81	11.55	6.91	51.11	15.84
December 25, 2004	0.91	−7.98	0.45	0.94	10.07	7.39	7.97	7.89	26.57	−17.94
January 16, 2005	0.96	0.18	0.34	0.97	21.37	19.18	16.78	−12.51	10.25	−14.85
February 26, 2006	0.87	−5.61	0.50	0.94	21.56	7.44	7.85	5.58	65.51	13.20
Average	0.92	−5.08	0.48	0.94	18.78	11.20	11.04	1.97	38.36	−0.94
Min	0.87	−7.98	0.34	0.92	10.07	7.39	7.85	−12.51	10.25	−17.94
Max	0.96	0.18	0.62	0.97	22.11	19.18	16.78	7.89	65.51	15.84
EOC Green Roof (Taylor 2008; Cardno TEC 2012; Taylor Associates 2012)
April 12, 2009	0.99	−0.11	0.18	0.99	19.05	9.31	9.32	0.06	51.11	−5.68
May 5, 2009	0.97	−1.82	0.21	0.99	21.38	16.95	17.24	1.74	20.73	−2.51
October 9, 2010	0.94	−10.02	0.44	0.97	43.14	27.63	30.40	10.02	35.95	−1.26
December 11, 2010	0.85	−26.58	0.60	0.94	91.52	64.37	81.48	26.58	29.67	46.77
Average	0.94	−9.63	0.36	0.97	43.77	29.57	34.61	9.60	34.36	9.33
Min	0.85	−26.58	0.18	0.94	19.05	9.31	9.32	0.06	20.73	−5.68
Max	0.99	−0.11	0.60	0.99	91.52	64.37	81.48	26.58	51.11	46.77
FS10 Green Roof (Taylor 2008; Cardno TEC 2012; Taylor Associates 2012)
Q1/2009	0.97	−6.16	0.77	0.85	185.67	160.63	170.53	6.16	13.49	15.38
Q3/2010	0.90	−27.70	0.38	0.88	121.16	76.27	97.40	27.70	37.05	−14.23
Average	0.93	−16.93	0.57	0.87	153.42	118.45	133.97	16.93	25.27	0.57
Min	0.90	−27.70	0.38	0.85	121.16	76.27	97.40	6.16	13.49	−14.23
Max	0.97	−6.16	0.77	0.88	185.67	160.63	170.53	27.70	37.05	15.38
Green Roof
Average	0.93	−9.27	0.45	0.94	55.70	40.00	45.05	8.01	34.14	3.47
Min	0.85	−27.70	0.18	0.85	10.07	7.39	7.85	−12.51	10.25	−17.94
Max	0.99	0.18	0.77	0.99	185.67	160.63	170.53	27.70	65.51	46.77

Note: Bold values indicate satisfactory agreement between predicted results and observed data as per Moriasi et al. (2015).

Figure 12.

Hamilton Green Roof representative hydrograph.

Permeable Pavement

SWMM satisfactorily modeled the Boone study for NSE [average 0.74 (0.64 to 0.85)], d (average 0.92 (0.90 to 0.94)], but slightly did not meet it for RSR [average 0.63, (0.41 to 1.01)] and PBIAS [average −1.54%, (−27% to +21%)] (see Table 6 and Figure 14). The timing for the peak volume was accurate, and the average predicted peak volume was −35%, (−48% to −21%) simulated vs. observed. The predicted outflow volume was 4% lower than observed, (range −27% to +27%). The average LID performance was 98%, (96% to 99 66%) volume reduction, outflow volume divided by inflow.

Parameter Sensitivity

Most of the studies measured the precipitation, inflow intensity, outflow intensity (surface runoff and drainage), area, width, slope, and the thickness of the layers of the LIDs. Usually, surface conditions such as roughness, depression storage, and vegetation volume had to be estimated via PEST, as did LID layer conductivity, porosity, wilting point, field capacities, and suction head. Conditions such as initial moisture deficit and clogging could change over time per LID. Measuring these parameters would greatly improve SWMM’s ability to simulate LID performance. The LID performance is dependent on the volume, intensity of rainfall, and stormwater inflow from upstream drainage areas. Its runoff is determined by the amount of water it can hold and its infiltration rate. The LID infiltration rate is limited by one of its layers — either surface layer, storage layer, drainpipe, and the infiltration into native soils. The LID has a maximum infiltration rate and if precipitation and inflow overwhelm this rate, it will overflow. SWMM results will be the most sensitive to the parameters of the LID’s limiting layer.

Conclusions

SWMM simulates flow vertically into the native soil; it does not model lateral flow into the native soil sides of the LID. We were unsuccessful in modeling a deep LID, such as the Villanova infiltration Trench, because we did not have accurate measurements of its lateral exfiltration. We are planning to adjust the SWMM LID modules to account for lateral exfiltration. Moriasi et al. (2015) state that model simulation can be used satisfactorily if NSE > 0.5, d > 0.75, PBIAS ± 15%, and RSR ≤ 0.7. Other than the Villanova infiltration Trench, the average NSE, d, PBIAS, and RSR meet these criteria for the bioretention, bioswale, green roof, and permeable pavement studies.

Figure 13.

Fire Station 10 Green Roof representative hydrograph.

Table 4.

Multi-objective SWMM LID module performance evaluation criteria data Villanova Infiltration Trench — inverse modeling.

Date	NSE	PBIAS, %	RSR	d	Observed runon, mm	Simulated runoff, mm	Simulated volume reduction percent
Villanova Infiltration Trench (Emerson 2008)
July 7, 2004	0.992	−0.134	0.091	0.998	1,318.3	1,279.1	97.0
September 8, 2004	0.965	1.255	0.186	0.991	485.1	464.6	95.8
December 9, 2004	0.489	12.194	0.701	0.818	800.4	736.1	92.0
Average	0.815	4.438	0.326	0.936	867.9	826.6	94.9
Min	0.489	−0.134	0.091	0.818	485.1	464.6	92.0
Max	0.992	12.194	0.701	0.998	1,318.3	1,279.1	97.0

Note: Bold values indicate satisfactory agreement between predicted results and observed data as per Moriasi et al. (2015).