Statistical models of fecal coliform levels in Pacific Northwest estuaries for improved shellfish harvest area closure decision making

Abstract

There is a substantial need for tools that effectively predict spatial and temporal fecal pollution patterns in estuarine waters. In this study, statistical models of exceedances of shellfish fecal coliform (FC) water quality criteria were developed using a 10-year dataset of FC levels and environmental data. Performance (sensitivity, specificity, and predictive capacity) of five different types of models was tested (MLR regression, Tobit (censored) regression, Firth’s binary logistic regression (BLR), classification trees, and mixed-effects regression) for each of three conditionally managed shellfish-harvesting areas in Tillamook Bay, Oregon (USA). The most influential variables were related to precipitation and river stage height in the wet season and wind and tidal-stage in the dry season. Classification tree and Firth’s BLR approaches better predicted exceedances of shellfish water quality standards than the current closure thresholds. Findings demonstrate the utility of statistical modeling approaches for improved management of shellfish harvesting waters.

Graphical Abstract

Introduction

Shellfish are primarily grown and harvested in sheltered estuarine waters subject to microbial pollution from multiple potential sources including run-off from agricultural operations (Shanks et al., 2006), as well as human waste inputs associated with septic systems and wastewater treatment facilities (Campos and Lees, 2014; Glasoe and Christy, 2004). Pathogens can become concentrated in the somatic tissues of filter-feeding shellfish, which are often eaten raw or undercooked, resulting in risk of food-borne illness to consumers (Oliveira et al., 2011). Fecal bacteria indicators (FIB) are routinely monitored in shellfish harvesting waters as proxies for pathogens, and along with sanitary surveys, are used to establish areas where harvesting is prohibited, allowed, or allowed to occur conditionally, as specified by the National Shellfish Sanitation Program (FDA, 2015). In order for harvest areas to remain open, water quality thresholds are selected that link to specific pollution sources including sewage treatment (or other point source) effluent bacteriological quality, precipitation levels, and/or river stage height (FDA, 2015). When established thresholds are exceeded, shellfish are deemed unsafe for human consumption and the area is closed to harvest. While necessary for the protection of human health, closures of shellfish harvesting waters have significant economic consequences felt by local communities including loss of employment opportunities and revenue (Anderson et al., 2016; Evans et al., 2016). Thus, there is a substantial need for accurate identification of spatial and temporal fecal pollution patterns in shellfish harvest areas.

Empirical statistical modeling efforts can aid in management and prediction of water quality (WQ) as well as help to identify areas or times where more sampling is necessary. Statistical modeling efforts have been applied as effective management tools for monitoring FIB in recreational waters. Models generally result in increased predictive power when compared to existing beach management methods, which depend on previous-day FIB measurements. Multiple linear regression (MLR)-based approaches are the most common and have been developed for marine (Frick et al. 2008) and freshwater beaches (Liu et al., 2006), and rivers (Herrig et al., 2015; Pandey et al., 2012). Alternative modeling strategies have also been tested to a lesser extent, including binary logistic regression (Brooks et al., 2016; Thoe et al., 2014), classification tree (Bae et al., 2010; Thoe et al., 2014), partial least squares (Brooks et al., 2016; Thoe et al., 2014), artificial neural network modeling (Zhang et al., 2015; Vijayashanthar et al. 218), and Bayesian model approaches (Farnham and Lall, 2015). Statistical modeling approaches applied in U.S. shellfish harvesting areas and estuarine systems are limited in scope. MLR-based models developed by Gonzalez et al. (2012, 2014) were able to successfully predict E. coli and enterococci levels over a range of environmental conditions in North Carolina estuarine waters. As promising as these studies were, questions remain about utility of statistical modeling for management of shellfish harvesting waters on a broader geographical scale and in terms of the environmental parameters and regression models tested.

Shellfish are filter feeders and can bioaccumulate bacteria and viruses found in the surrounding water column (Oliveira et al., 2011). As a result, WQ standards for shellfish harvesting areas are well below standards required for recreational waters. Moreover, dilution in receiving waters and periods of low-flow reducing inputs from over-land sources often lead to low FIB concentrations in shellfish-harvesting waters, particularly in the dry season. This can lead to highly skewed or censored data sets, which can result in traditional MLR-based approaches utilizing ordinary least squares (OLS) estimation producing biased parameter estimates. Previous statistical modeling efforts have generally been applied to areas with higher levels of FIB, with those sites with persistently lower bacteria levels less critical to understand. Estuaries are also highly dynamic in space, time and season, and environment with large tidal swings, and wind and salinity gradients (Jovanovic et al., 2017; Quilliam et al., 2011; Wiegner et al., 2017), further complicating management efforts. Current management models don’t often address the complexity of these systems. In addition, sampling is often constrained due to logistical (e.g. storm events and high wind or surf advisories) and financial limitations, making it imperative to maximize sampling efforts.

In this study, statistical, data-driven models were applied to past measurements of FC levels in Tillamook Bay, OR. This estuary provides valuable shellfish habitat in the Pacific Northwest (PNW); however, harvesting opportunities are regularly limited by health-related closures. Current closures of shellfish harvesting areas within Tillamook Bay are triggered by precipitation and river stage height levels as outlined in the commercial shellfish management plan (ODA, 2015). To our knowledge, this is the first review of multiple statistical model types applied to shellfish harvesting areas. The current study evaluates a different suite of regression methods and environmental variables, chosen to address the dynamics and constraints of predicting water quality in the PNW. Twenty-three environmental variables were chosen that describe tidal forcing, meteorological, and runoff-based factors. Five different model types were developed and evaluated (OLS-based MLR, tobit regression, Firth’s logistic regression, classification tree analysis (CT), and generalized linear mixed-effects regression models).

Models were selected based on three criteria. 1) Model approaches needed to produce easily interpretable parameter estimates and describe relationships between observations and environmental drivers. PLS and ANN methods were not utilized, as the emphasis of these methods is generally on prediction and not on understanding the relationship between the variables. Shellfish management plans incorporate knowledge of local drivers into closure criteria, thus models that can improve managers’ understanding of relationships between environmental drivers and FC levels were prioritized. 2) Approaches used had to account for the significant seasonality observed in fecal coliform levels in this region. Unlike recreational waters, where the majority of use is concentrated during a few months, shellfish are harvested year-round in the Pacific Northwest (PNW). 3) Models were considered that address challenges associated with high levels of left-censored values. A high number of non-detects is consistent with data utilized in this study as well as FIB data collected throughout shellfish-harvesting waters in the PNW, including multiple estuarine watersheds used for commercial shellfish harvesting in Washington State (DOE, 2014) and Oregon (ODA, unpublished data).

This study aims to provide managers with optimum and easily replicated statistical modeling approaches providing specific guidance regarding environmental conditions that are most closely related to exceedances of shellfish water quality standards. Critical factors affecting shellfish water quality were identified and model performance was compared to current WQ closure thresholds. Study results can aid in the optimization of shellfish management and sampling plans.

Methods

2.1. Study Site

Tillamook Bay (latitude 45.5136 deg N, longitude 123.9189 deg W), a National Estuary Program site, is located on the coast of Oregon. It is a shallow estuary (average depth of 6.6 feet), approximately 3.2 km wide and 11.3 km long, and drains five major river systems. The watershed is mixed use, with potential sources of fecal bacteria to receiving waters including animal (extensive dairy operations in lowlands), wildlife, and human (five wastewater treatment facilities (WWTFs) and non-point inputs such as those associated with residential on-site septics). Land use is segmented with forested uplands being managed for timber production, lowlands for agricultural, mainly dairy farms, and rural residential housing scattered throughout (Shanks et al., 2006). This region has relatively dry summers, with increasing rainfall in the Fall, and the majority of the precipitation occurring between October and April (rainfall data obtained from the Western Regional Climate Center (https://wrcc.dri.edu/)).

Tillamook Bay is one of Oregon’s leading producers of shellfish for human consumption. However, a long history of elevated fecal indicator bacteria levels has led to exceedances of water quality standards in the tributaries and closures of the Bay for commercial shellfish harvest (Sullivan et al., 2005). Currently, there are three regions in Tillamook Bay where harvesting is conditionally allowed. The Flower Pot (FP) management area is located farthest from the mouth of the estuary and has the most restrictive closure guidelines; the Upper Bay (UB) management area is located in the middle of the Bay, and the Lower Bay (LB) management area is located closest to the mouth of the estuary and has the least restrictive closure guidelines (Figure S1). The FP region is closed for shellfish harvesting November through January and conditionally closed between February and October when there is more than 1 inch of rain within a 24-hour period or when there is more than 2 inches of rain accumulated in three days. Shellfish harvesting is conditionally allowed year-round in both the UB and LB areas. Closures occur in the UB area whenever there is more than 1 inch of rain in a 24-hour period or more than 1 inch (November through January) or 2 inches (February through October) of rain accumulated in three days. Closures occur in the LB area whenever the Wilson River gauge rises above seven feet and/or when dry conditions are followed by more than 2 inches of rain within a 24-hour period or more than 2.5 inches in a 48-hour period. Management guidelines in Tillamook Bay are consistent with approaches taken throughout the PNW. Eight growing areas in Oregon are approved for commercial harvest of oysters for human consumption. Rainfall events/stream gauge height trigger harvest closures in four of the eight management areas, while the other four harvest areas do not have any conditions attached to their management.

2.2. Data for Predictive Modeling Effort

FC data were obtained from the Oregon Department of Agriculture (ODA) for a 10-year time period from January 2006 to December 2015. New and updated predictive models are often generated annually for recreational water quality (Francy et al. 2013; Thoe et al., 2015). However, shellfish harvesting waters are typically sampled monthly, making it difficult to derive new models each year. This study was interested in applicability of models generated from data collected over a longer time period, which is representative of data available for shellfish harvesting waters.

Grab water samples were collected by boat approximately monthly from 13 sites within Tillamook Bay. Eight of the thirteen sites were chosen for inclusion in the modeling effort that were sampled regularly over the 10-year time period and representative of the three management areas (Figure 1). FC levels were measured using the standard 5-tube multiple tube fermentation (MTF) method (FDA, 2015). Data contained multiple lower detection limits (LDL) ranging between 0 and 2 most probable number (MPN)/100 mL. For consistency, the highest reported lower detection limit (LDL) (2 MPN/100 mL) was applied to all data. Low FC levels were a common occurrence with 55% of the observations having concentrations lower than the LDL (left-censored). This was particularly true during the dry season (75% below LDL) as compared to the wet season (36%) with dry season defined as May-October and wet season defined as November-April, based on historical precipitation trends. Left-censored levels differed by management area with 40% of data below the LDL in FP (60% dry season and 15% wet season); 49% in the UB (75% dry season and 27% wet season); and 68% in the LB (84% dry season and 27% wet season) management areas. All FC levels were log-transformed to reduce skewness of the data. Temporal and seasonal trends in FC exceedance frequencies over the 10-year time period were explored, utilizing Firth’s binary logistic regression (BLR) approach. Firth’s BLR model is appropriate when events are rare. Previous analyses have been completed analyzing temporal trends in water quality in the five major river systems in the Tillamook Bay watershed. There is evidence of improvements in tributary water quality (TEP 2015) but to date there has not been a formal assessment of trends in bay data.

Figure 1.

Location of Tillamook Bay sampling stations and the corresponding hydro-meteorological stations (NOAA Buoy, WRCC Weather Station, and USGS stream gauge).

Meteorological, hydrologic, and tidal data were retrieved and used in the statistical modeling effort (Table 1). All parameters listed in Table 1 were initially included in each model and subjected to variable selection criteria described below. Meteorological data (including precipitation, wind speed and direction, air temperature) were obtained from the WRCC Tillamook Airport Climate Center Airport Weather Station (https://wrcc.dri.edu/). Water temperature data were obtained from NOAA’s National Data Buoy Center at the Garibaldi Station (Station: TLBO3; http://ndbc.noaa.gov/). Water temperature and air temperature were both included; water temperature is measured near the entrance of the bay, representing ocean conditions. Air temperature, collected at the WRCC Weather Station, is more representative of overall climatic conditions.

Table 1.

Independent variables used in this study with their respective data sources.

Var Type	Var Name	Description	Source
Hydrologic	TR_DY	Daily mean discharge (100 ft3/sec)- Trask River	USGS
	ANT48	Cumulative 48 hr rainfall (in)
	ANT72	Cumulative 72 hr rainfall (in)
	ANT96	Cumulative 96 hr rainfall (in)
	ANT120	Cumulative 120 hr rainfall (in)	Western Regional Climate Center (WRCC) Airport Weather Station
	PRECIP	Daily rainfall −24 hours (in)
	PRECIP24	Rainfall- past 24 hours (in)
	PRECIP48	Rainfall-past 48 hours (in)
	PRECIP72	Rainfall-past 72 hours(in)
Climatic	S_RAD	Total Solar Radiation (Ly))
	GUST	Max daily wind gust (mph)
	WTMP	Water temp (deg C)
	ATMP	daily ave air temp (deg F)
	WINDA	Wind Alongshore (mph)	Calculated from WRCC
	WINDO	Wind Offshore (mph)
Seasonal	MONTH	Month
Tidal	TDV_TDP	Diff verified/observed-predicted wat lev (m)	National Data Buoy Center
	TMAX	Daily max high tide (m)
	HL_DL	Diff nearest high-low tide (m)
	HL_DL_Y	Diff daily high high- daily Low Low (m)
	HGH_HR	Hrs since last high tide (hrs)
	TIDE_STG	Tidal stage: 0 if Ebb, 1 if Flood (derived)
	TMIN	Daily low low tide (m)

The alongshore (Eqn.1) and offshore (Eqn.2) wind components were derived from wind direction ( ° ) and wind speed (mph), where the beach orientation angle (ϕ) referred to the outercoast angle:

$$\mathrm{wind}\mathrm{alongshore}=\mathrm{-windspeed*}\left[\mathrm{cosine}\right(\mathrm{wind}\mathrm{direction-\varphi })\mathrm{*}\mathrm{\pi }/180];$$ Eqn 1.

$$\mathrm{wind}\mathrm{offshore}=\mathrm{windspeed*}\left[\mathrm{sine}\right(\mathrm{wind}\mathrm{direction-\varphi })\mathrm{*\pi }/180,$$ Eqn 2.

(Cyterski et al., 2013).

Precipitation as a driver was explored using different metrics including average daily precipitation on day-of sampling (PRECIP) and one to three days preceding sampling (PRECIP24, PRECIP48, RPECIP72) as well as cumulative precipitation occurring in the past two to five days (ANT48, ANT72, ANT96, ANT120). River stage height was retrieved from USGS’s National Water Information System website (https://waterdata.usgs.gov/). Tidal data were retrieved from NOAA’s National Data Buoy Center at the Garibaldi Station (Station: TLBO3; http://ndbc.noaa.gov/). Various metrics for tidal forcing were explored: daily maximum high and daily minimum low tides (TMAX and TMIN), hours since last high tide (HGH_HR), daily tidal range (HL_DL_Y), difference between adjacent high and low tides (HL_DL), tidal stage (Flood or Ebb), and the difference between verified observed water level and predicted water level (TDV_TDP). Local weather conditions (e.g., wind conditions and atmospheric pressure) and river levels can effect water levels, these factors are encompassed in the TDV_TDP metric. All derived tidal variables were computed in R version 3.0.3 (R, Vienna, Austria).

2.3. Predictive Models Selected

Ordinary least squares (OLS) Multiple linear regression (MLR), Tobit (censored regression), Firth’s binomial logistic regression (BLR), classification trees (CT), and mixed-effects regression models were developed in R version 3.3.0 (R, Vienna, Austria); model details are described in Table 2. MLR, Tobit, Firth’s BLR, and CT models were developed separately for dry and wet weather; the mixed-effects regression model utilized all data. These five model types were developed and tested individually for the three shellfish management areas within Tillamook Bay (FP, UB, and LB) in order to evaluate if there were systematic differences in drivers between the different management areas. Data collected between 2006–2015 were used for testing and validation, with 75% of data randomly selected for model training and 25% used for model validation. Data was subset in R version 3.3.0 using indexing. The training and validation datasets were compared to ensure each dataset reflected typical and comparable environmental conditions in the bay.

Table 2.

Description of models tested.

Model	Season	Outcome Variable	Data Reported		R package	Reference
			Output: parameter scores	Goodness of fit statistic
MLR	Wet/Dry	LogFC	Standardized coef	R2	stats
Tobit (censored MLR)	Wet/Dry	LogFC	Coef	McFadden R2	VGAM	Yee, 2017
Firth’s BLR	Wet/Dry	Exceedance [0,1]	Odds ratio	Cox and Snell R2	logistf	Heinze and Ploner, 2016
Classification Tree (CT)	Wet/Dry	Exceedance [0,1]	Var importance*	R2	rpart	Therneau et al. 2018
Mixed-effects	All	LogFC	Standardized coef	Marginal R2/Conditional R2**	lme4	Bates et al. 2015

^*Variable importance is calculated based on reductions in model accuracy when that variable is removed (Song and Lu 2015).

^**Marginal R² represents variance explained by the fixed factors/Conditional R² represents variance explained by the fixed and random factors.

OLS-based MLR models were utilized for comparison to other published studies and to evaluate impacts of censoring on parameter estimates and model performance. In OLS regression estimation, estimated values are often substituted for censored values, leading to potentially biased parameter estimates in the case of highly censored datasets. In contrast, distributional methods, such as maximum likelihood estimation, fit a distribution to the uncensored data and use the distribution to estimate summary statistics (Osgood et al., 2002). As an alternative to ordinary least squares (OLS) regression, Tobit models, which utilize maximum likelihood estimation, were also developed.

Two additional models were considered with binary outcome variables: Firth’s BLR models and a CT approach. Models with binary outcome variables are more robust to highly censored data and datasets with few events observed. The FC data were reclassified as a binary variable: whether bay waters were below or above shellfish water quality standards of FC 14 MPN per 100 mL. For a growing area to be classified as conditionally open, FC levels in water samples (geometric mean) are not to exceed 14 MPN (define MPN) per 100 ml and less than 10% of samples collected are allowed to be at or above 43 MPN per 100 ml (FDA, 2015). If probability exceeded 50%, the predicted outcome was classified as an exceedance. Although WQ criteria are based on geomean calculations, models were derived from single-day/event samples. It is not uncommon for shellfish closure to be based on single samples, based on sampling limitations. Firth’s BLR models use a penalized logistic regression, applying a Bayesian prior for coefficient estimation. Penalized likelihood reduces small-sample bias in maximum likelihood estimation, allowing for consistent parameter estimates in the case of data sets with few events and complete or quasi-separation of the data (Pajouheshnia et al., 2016; Riedel et al., 2014). CT models use a recursive partitioning approach to develop criteria for each dependent variable where each split (above/below each criteria) contains observations with similar response values (Strobl et al., 2009). WQ exceedance or compliance is then predicted via a decision tree approach. CT models can be advantageous in that they are straight forward and easy to interpret, automatically handle variable interactions, missing values, highly skewed data and nonlinear relationships, and don’t make any distributional assumptions (Song and Lu, 2015).

In order to effectively take into account the significant seasonal variability observed in the data, separate wet and dry season models were developed for each management area as described above. As an alternative to separate models, mixed-effects regression models were also tested which utilized all data. Mixed models can be very flexible when dealing with repeated measurements in time and/or space (Minalu et al., 2011). In this study, mixed effects models were used to take into monthly seasonal variability. Mixed models were run on combined wet and dry season data with month included as a random effects term, assuming observations collected in the same month to be correlated.

2.4. Variable Selection

In order to reduce multi-collinearity for the MLR, Tobit, Firth’s BLR, and mixed models, correlation coefficients among all independent variables were calculated. Variables that were highly correlated to each other (correlation coefficient greater than 0.6) were identified and the variable with the lower correlation to log-normalized FC levels was removed (a similar approach to Thoe et al. 2015). For the CT models, all independent variables were initially included, with only those that best describe the data selected by the algorithm and included in the final model.

For the MLR and Tobit models, model parameters were selected by exact Akaike information criterion (AIC) conducted in both directions (Forwards and Backwards). In addition, the variance inflation factor (VIF) was calculated and variables with square root of VIF greater than 2 were removed from the model selection process (Fox and Weisberg, 2011). For the Firth’s BLR models, an automated backward elimination function using penalized likelihood ratio tests was used for variable selection. For the mixed-effects model, parameter selection was conducted utilizing AIC and backwards elimination of non-significant effects. Parameter selection techniques, described above, were run 100 times for each model and variables that were selected in more than 50% of the models were utilized. CT models were run initially 100 times and models selected most often were chosen as the final CT output.

2.5. Model Performance

Overall model performance was described by sensitivity, specificity, and predictive accuracy. Sensitivity is the ratio of the number of times the model correctly predicts an exceedance to total number of observed exceedances, which describes the percentage of days correctly classified as exceedance of shellfish aquaculture WQ standards (> FC 14 MPN per 100 ml). Specificity is the ratio of the number of times the model correctly predicts when water quality is in compliance to the total number of compliance days, which describes the percentage of days correctly classified as compliance of shellfish WQ standards (< FC 14 MPN per 100 ml). Models were compared to current WQ closure thresholds based on predictive accuracy, sensitivity + specificity, and sensitivity metrics (similar metrics applied in Thoe et al. 2015). Model goodness of fit for the best performing models are reported as R² or pseudo R² (Table 2).

3. Results

3.1. Fecal Coliform Levels 2006–2015

During the 10-year time period selected, FC levels exceeded shellfish WQ standards (> 14 MPN/100 mL) 27%, 23%, and 11% of the time in the FP, UB, and LB management areas, respectively. There were strong seasonal variations in frequency of exceedances. Exceedances were far less frequent in the dry versus wet season: frequencies of exceedances were 5%, 4%, and 1% in the FP, UB, LB areas in the dry season compared to exceedance frequencies of 52%, 40%, 18% in the wet season, respectively.

Firth’s BLR models were used to identify spatial and temporal trends in historical FC levels. After accounting for differences between the wet and dry seasons, there was not a significant temporal trend in the probability of an exceedance of shellfish WQ standards for any of the three management areas over the 10-year time period analyzed (p>0.05; Figure 2). There were, however, significant differences in probability of an exceedance between the wet and dry seasons for all three management areas: FP (Odds Ratio (OR): 16.33, 95% CI [6.67, 16.33], p<0.01), UB (OR: 15.76, 95% CI [7.31, 15.76], p<0.01), and LB management areas (OR: 13.02, 95% CI [4.24, 13.02], p<0.01). Exceedances were between 13 and 16 times more likely to occur in the wet versus dry season, dependent on management area.

Figure 2.

Predicted probability of an exceedance of FC 14 MPN per 100 mL shellfish criteria for the three management areas (Flower Pot (FP), Upper Bay (UB), and Lower Bay (LB)) over a 10-year time period for the wet (blue line) and dry seasons (red). 95% CI intervals denoted by blue and red shading around line.

3.2. Critical Factors

A combination of precipitation and river stage height, climatic (water temperature wind, and solar radiation), and tidal variables contributed to the observed variability in FC levels. Variables included in the final models differed seasonally (wet versus dry season) and spatially (by management area). Figure 3 describes statistically significant variables retained in the MLR (wet and dry seasons) and mixed-effects models (all data) and corresponding standardized coefficients for the three management areas. The same variables were included in the final MLR and Tobit models. Tobit and MLR predictor coefficients and R² values were compared and results are included in the Supplemental Information (Table S1). Figure 4 describes statistically significant variables retained in the Firth’s BLR models and corresponding odds ratios (OR). An OR greater than one indicates that the probability of an exceedance increases as that variable increases. Figure 5 describes variables selected by the final CT analysis output and relative importance of each variable selected. The relative importance score is based on improvements in model prediction attributable to that variable (Song and Lu 2015).

Figure 3.

Independent variables included in the MLR (Dry and Wet seasons) and mixed-effects models (All) and their corresponding standardized coefficients for each of the three management areas (FP, LB, UB). Color of bubble represents parameter type (blue=hydrologic, green=tidal, pink=climatic), with darker colors indicating an inverse relationship between FC levels and that variable. Size of bubble represents that variables’ importance (size increases as magnitude of standardized coefficient increases). Only predictor variables with significant parameter estimates (p≤0.1) have corresponding colored circles.

Figure 4.

Independent variables in the Firth’s LR models and their corresponding odds ratios (OR). Model variables included for each of the three management areas (FP, LB, UB) and for the wet and dry seasons. Color of bubble represents parameter type (blue=hydrologic, green=tidal, pink=climatic), with darker colors indicating an inverse relationship between probability of an exceedance and that variable. and size of bubble represents corresponding odds ratio. OR >1: probability of an exceedance increases as that variable increases; OR<1: probability of an exceedance decreases as that variable decreases. Only predictor variables with significant parameter estimates (p≤0.1) have corresponding colored circles. Units for TR_DY were converted to per 100 ft³/sec.

Figure 5.

Independent variables selected by the CT models. Model variables included for each of the three management areas (FP, LB, UB) and for the wet and dry seasons. Color of bubble represents parameter type (blue=hydrologic, green=tidal, pink=climatic), with darker colors indicating an inverse relationship between probability of an exceedance and that variable. Size of bubble represents that variable’s relative importance score.

3.2.1. Flower Pot

The regressors most often incorporated into the individual FP models included river discharge, water temperature, tidal stage, solar radiation, 48-hour precipitation, and difference between verified observed water levels and predicted water levels (which is an indicator of non-tidal factors that influence water level including river flow, wind conditions and atmospheric pressure). When either seasonal data were combined (mixed model) or when data from the wet season only were considered, runoff-based variables explained much of the variability in FC levels. In addition, solar radiation was inversely related to FC levels in the wet season; with decreasing solar radiation levels associated with increasing FC levels. Solar radiation in the wet season was negatively correlated with precipitation; increasing precipitation levels associated with decreasing solar radiation. Wet season solar radiation is likely serving as an indirect measure of precipitation. In the dry season, tidal variables were equally important to runoff-based variables, in most cases. The primary predictor variable reported for the CT and Firth’s BLR models (dry season) was difference between verified and predicted water levels (Figure 5) with larger, positive differences between verified and predicted water levels (elevated water levels) linked to higher FC levels.

3.2.2. Upper Bay

Variables selected included river discharge, tidal stage, hours since last high tide, daily minimum low tide, and offshore wind. When seasonal data were combined (mixed model) and when data from the wet season only were considered, runoff-based variables (river discharge and 48-hour precipitation) explained much of the variability in FC levels, consistent with trends observed in the FP management area. In the dry season, tidal variables and wind were the most important variables (Figures 3 -5). The primary predictor variable for the CT models was wind (wind offshore), with wind blowing from land to the water (more negative WindO result) resulting in an increased probability of an exceedance.

3.2.3. Lower Bay

Variables selected included river discharge, hours since last high tide, precipitation, and wind (wind offshore). In the wet season, stream gauge height was the primary predictor variable (Figures 4 and 5), with the exception of the MLR model, where 24- hour precipitation explained the most variation in FC levels (Figure 3). In the dry season, in addition to stream gauge height and 24-hour precipitation, hours since last high tide and wind offshore (WindO) were important predictor variables. Lower tides (increase in number of hours since last high tide) and increasing offshore wind led to increased FC levels.

3.3. Overall Model Performance

Models were evaluated by sensitivity, specificity, and predictive accuracy criteria during the validation phase (Figure 6). The best performing model for each management area was defined based on sensitivity and sensitivity + specificity and results are presented in Table 2. Overall, models achieved higher predictive power in the dry versus wet season, for all three management areas. However, in the dry season, exceedance events were less frequent and harder to predict, leading to lower sensitivity.

Figure 6.

Sensitivity, specificity, and total correct prediction obtained by the ifferent models for the wet and dry season and for combined wet/dry seasons (current WQ closure thresholds and mixed-effects model only). Each panel representes a different management area; upper panel- FP, middle panel- UB, lower panel- LB.

3.3.1. Flower Pot

In the FP management area, the five models tested achieved similar overall predictive power when compared to current WQ closure thresholds (approximentely 90% dry season, 60% wet season). However, updated models resulted in an improved ability to predict exceedance events. In the dry season, sensitivity was improved with application of the Firth’s BLR (50%), Tobit (50%), or CT (50%) models when compared to current WQ closure thresholds (33%). In the wet season, the Firth’s BLR (80%), CT model (100%), and the Tobit models (62%) all led to improvements in sensitivity when compared to current WQ closure thresholds (49%).

3.3.2. Upper Bay

In the UB management area, the CT model was the best performing model in both the wet and dry seasons. During the dry season, the four models tested all achieved similar overall prediction (on average 97% predictive power, when compared to current WQ closure thresholds (93%)) and specificity (on average 98% specificity, when compared to current WQ closure thresholds (95%)). However, sensitivity was increased when any of the four updated models (CT: 100%, Firth’s BLR: 50%, MLR: 50%, Tobit: 50%) were compared to the current WQ closure thresholds (33%). In the wet season, three of the four models led to improvements in overall predictive power (CT: 85%, MLR: 80%, Tobit: 84%) when compared to current WQ closure thresholds (70%). The Tobit and CT models also led to increased sensitivity (82% and 84%, respectively) versus current WQ closure thresholds (60%), with all three methods returning similar specificities (82% and 81% versus 80%).

3.3.3. Lower Bay

In the LB management area, the best performing models in both the wet and dry seasons had a binary outcome variable (Firth’s BLR in the wet season and the CT model in the dry season). In the dry season, the four models tested achieved similar overall prediction, on average 97% predictive power, and overall predictive power was similar to current WQ closure thresholds (95%). The CT model was the only model tested that led to improvements in sensitivity over current WQ closure thresholds; sensitivity reported for the CT was 100% versus 25% for current WQ closure thresholds. In the wet season, all four updated models resulted in increased overall prediction when compared to the current WQ closure thresholds (67%), with Firth’s BLR and the MLR achieving the best results (86% and 89%, respectively). In the wet season, Firth’s BLR model achieved the highest sensitivity (80% predicting exceedances occurring on closure days) which was similar to sensitivity achieved by current WQ closure thresholds (65%).

4. Discussion

Both more targeted shellfish management plans and water quality improvement actions are needed in order to reduce costly closures of important shellfish harvesting waters. The models tested here more accurately described exceedances of shellfish water quality standards than current WQ closure thresholds (Figure 6) and offer important insights into factors responsible for observed variability in FC levels, which can be used to help direct sampling efforts. While the majority of previous statistical modeling efforts have been conducted at urban, recreational locations, this study was unique in that it evaluated model performance for explaining exceedances of shellfish harvesting water quality criteria in PNW estuarine waters.

4.1. Important factors influencing FC levels in Tillamook Bay

Significant factors affecting water quality in Tillamook Bay varied seasonally and spatially (Figure 6). In the wet season, runoff-based variables, including precipitation and river stream height, explained the most variance in FC levels with wet season models leading to improvements over current WQ closure thresholds, in most cases. Other studies have observed similar increases in estuarine FIB levels during periods of high riverine discharge following precipitation events (Coulltier et al., 2010; Lewis et al. 2013; Riou et al., 2007; Wiegner et al., 2017). During the wet season, runoff-based processes connect upland fecal pollution sources to the estuary; which includes stimulating transport of FIB accumulated on the land from extensive agricultural activities in the Tillamook Bay watershed (Shanks et al., 2006). In this study, precipitation and stream gauge height thresholds triggering an exceedance event were highly dependent on management area within the Bay, with areas located furthest from the mouth of the estuary the most responsive to increases in precipitation and stream gauge height. These sites are also located closest to the confluence of the Trask and Tillamook Rivers. The CT models provided an easy way to interpret differences in precipitation/stream gauge height needed to trigger an exceedance event (see Supplemental Information: Figures S2-S4). Spatially-targeted thresholds may allow for a delay in closures in areas located in the lower bay and closest to the mouth of the estuary.

In the dry season, tidal and wind-based factors were consistently among the most important variables, along with runoff-based variables, suggesting that additional dynamics internal to the Bay play a role in dry season exceedances. Tillamook Bay is a very shallow system (a few meters) and impacted by wind and tidally-induced mixing. Tides and wind can stimulate increased interaction and connectivity with adjacent low-lying areas and marsh habitat, associated with higher water levels and tides (Koch and Gobler, 2009), as well as resuspension from estuarine sediments, associated with lower tides and water levels (Lewis et al. 2013; Jovanovic et al. 2017). In the dry season, these forces are coupled with reduced additional freshwater inputs (decreased water depth and less water available for dilution of sediment-borne microbes.

In this study, impact of tidal processes varied by management area. In the Flower Pot management area (located in the upper bay, closest to freshwater inputs), the difference between verified and predicted tidal levels was the most important variable for the CT and Firth’s BLR models, with higher than predicted water levels associated with increased FC levels. Higher than predicted water levels are associated with high wave conditions and potential flooding of surrounding low-lying areas. It is hypothesized that increased storm or wave action (associated with larger values for the TDV_TDP variable) leads to increased connectivity with surrounding low-lying marsh habitat. Previous studies have observed higher concentrations of total coliform bacteria in salt marsh ditches relative to adjacent estuarine waters (Koch and Gobler, 2009).

In the Upper Bay, flood tides, hours since last high tide, and daily minimum low tide (TIDE_STG, HGH_HR, TMIN), were all positively associated with increased FC levels. This suggests that fecal coliform levels are elevated immediately following outgoing tides (during flood tide events; furthest away from nearest hightide), and especially following low tide events of increased magnitude (likely during spring tide events). Tidally-induced re-suspension of sediment-associated FIBs has been shown previously to impact water column water quality in estuarine systems (Davies et al., 1995; Jovanovic et al., 2017; Koch and Gobler, 2009), with decreasing water levels hypothesized to result in increasing resuspension of sediment-borne E. coli (Jovanovic et al., 2017). Moreover, sediment samples collected in Tillamook Bay between 2016–2017 had average concentrations of FC 136 MPN/ 100 g (unpublished data), these results taken in parallel with model results are suggestive of potential for resuspension of fecal microbes from sediments to impact dry season water quality. Wind was also an important parameter in both the Upper and Lower Bay management areas, with increasing winds blowing from the land to the ocean associated with increasing FC levels. Lewis et al. (2013) observed a similar correlative relationship between increasing winds and E. coli levels in California estuaries, which was hypothesized to be a result of increasing resuspension of sediment-borne bacteria.

4.2. Model Performance

Overall, the Tobit, Firth’s BLR and CT models were the best performing models, independent of season. These models resulted in increases in model sensitivity when compared to OLS-based MLR models and current WQ closure thresholds. For the UB and LB management areas, where fewer exceedances were observed, Firth’s BLR and CT models consistently performed the best based on sensitivity and sensitivity + specificity metrics. These areas are also where the majority of the ongoing shellfish harvesting is occurring in Tillamook Bay and are representative of conditionally approved shellfish harvesting areas in the region. Firth’s BLR and CT models have a binary outcome variable and are advantageous when there are a low number of events occurring or when data are heavily censored, such as the case in the Upper and Lower bay management areas. Such highly left-censored datasets are typical for shellfish harvesting areas in this region. In this study, on average 55% of data were left-censored. Similarly, in the State of Washington, water samples were collected from four different shellfish harvesting areas and 33% of samples were left-censored using the same methods as employed in this study (5-tube MTF) (DOE, 2014).

In addition to predictive accuracy, CT models also used the fewest variables and were easy to interpret. CT model application in predictive modeling of surface water quality is limited in scope. However, CT approaches have been previously reported to perform wellThoe et al. (2014) found that CT models foremost accurately predicted exceedance events at beaches in Santa Monica, CA. Although CT approaches also performed well in this study, it is important to take into account potential for instability of the output. Model output can vary drastically with slight changes in the data, making the variable selection process and position of the cutpoints highly unstable (Strobl et al., 2009). In this study in order to minimize variability, data were randomly resampled 100 times, and the most common CT output was chosen for further evaluation. Care should be taken before utilizing CT output as the final and only way to describe the data.

In addition to models utilizing a binary response variable, the Tobit model was also evaluated as an alternative to OLS-based MLR approaches. In this study, the Tobit models generally had slightly shifted parameter estimates. The modified models resulted in improved performance in most cases, and Tobit models reported higher sensitivity when compared to OLS-based MLR models in all cases. This approach should be considered in water quality statistical model development when data sets are highly censored and a quantitative response variable is favored.

Model performance was highly seasonally dependent. In the dry season, fewer exceedances were observed and waters generally met NSSP shellfish harvesting requirements. Runoff-based factors were secondary to tidal and wind-based parameters which influence water quality in a less direct way; as a result, exceedances were harder to predict. Oysters are harvested year-round in the PNW, however production is approximately 50% higher in the summer months (ODA, unpublished data), likely due to combination of increased demand and fewer closures, making prediction of elevated dry season FC levels important. In this study, BLR models were interpreted as predicting an exceedance at >50% threshold. Previous studies have investigated how using alternative thresholds can improve performance, with lower exceedance thresholds (~30% reported by Thoe et al. 2014) resulting in improved sensitivity. Such an approach was not evaluated in this study but could result in increased sensitivity in the dry season models. In the wet season, exceedances of shellfish WQ standards were more frequent, resulting in an increased ability to predict those events, when compared to dry season exceedance events.

As an alternative to separate wet and dry season models, the mixed-effects model, which has not been tested previously for water quality predictive modeling efforts, was evaluated. This approach allowed for variability associated with season to be accounted for, which is useful when less data are available or when managers want one overarching model that is not dependent on seasonality. The mixed-effects models offered similar overall predictive power, but lower sensitivity than wet season models and these models were not able to as effectively inform seasonal differences in drivers of water quality.

Other analytical approaches have been explored for use in beach monitoring that were not included in this study, such as artificial neural networks (ANN) (Zhang et al., 2015), that may have resulted in even greater improvements in predictive capacity. ANN models were initially applied (data not shown) in this study but were not included in final comparisons. ANN models offer more of a black box approach, making generalization and application to shellfish management plans, which require tangible connections between drivers and FC levels, challenging. The models utilized in this study offer easy to interpret parameter estimates that can be utilized by shellfish managers to apply more targeted sampling and/or to update closure criteria. Moving forward, further validation of similar modeling approaches with independently collected datasets in preceding years, would confirm predicative capabilities of these approaches.

4.3. Conclusions

This study suggests that Firth’s BLR models and CT approaches can serve as valuable management tools to assess shellfish-harvesting water quality in the PNW. Both of these approaches have binary outcome variables making them more robust to highly censored data and datasets with few events observed. These models also outperformed the other models tested, when comparing model sensitivity (ability to predict exceedances of closure criteria) and overall predictive accuracy. CT models also used the fewest number of variables overall (they were the simplest models) and provided criteria for predicting exceedances that could be directly applied to shellfish management plans. CT thresholds for explaining closure events differed spatially. Shellfish management plans that better take into account spatial responses to riverine discharge may allow for extended times when areas can remain open (especially in the lower part of the Bay). Differences were noted between the wet and dry season models as a result not only in differences in FC levels observed, with highest levels observed in the wet season, but also in terms of drivers. It would be advantageous for managers to design future sampling efforts based separately on wet and dry weather regimes, including timing of sanitary surveys. For instance, tidal stage was especially important during dry weather, whereas wet weather exceedances were driven primarily by precipitation events. Managers may want to design sampling accordingly, sampling over the tidal cycle, or targeting certain tidal events (e.g. Spring tides) that may impact water quality during the dry season. When data are combined, wet season trends drove patterns and resulted in decreased ability of the models to correctly explain variability in exceedance events. PNW estuarine systems receive multiple sources of fecal contamination and delivery of fecal contamination is affected by various physical, seasonal, and climatic forces. Statistical modeling efforts applied in this work improved water quality attainment estimations and our understanding of drivers of water quality. This approach provides a framework for how shellfish managers can utilize historical data for improved management of PNW shellfish-harvesting waters.

Table 3.

Best performing model statistics (based on sensitivity and sensitivity + specificity metrics) for each management area and the wet and dry seasons.

Management Area	Season	Model Performance					Metric
		Model	OP	Sens	Spec	R2
Flower Pot	Wet	CT	0.75	1	0.5	0.5	Sens
		Tobit	0.75	0.62	1	0.23*	Sens+Spec
	Dry	Firth	0.96	0.5	1	0.01**	Sens; Sens + Spec
		Tobit	0.96	0.5	1	0.53*	Sens; Sens + Spec
		CT	0.96	0.5	1	0.67	Sens; Sens + Spec
Upper Bay	Wet	CT	0.85	0.84	0.84	0.65	Sens; Sens + Spec
	Dry	CT	0.98	1	0.97	0.5	Sens; Sens + Spec
Lower Bay	Wet	Firth	0.89	0.8	0.9	0.02**	Sens; Sens + Spec
	Dry	CT	0.97	1	0.97	0.5	Sens; Sens + Spec

^*McFadden’s Psuedo R² reported for Tobit regression models

^**Cox and Snell Psuedo R² reported for Firth’s logistic regression models

Highlights (3–5 bullet points, max 85 characters per bullet point).

• Historical data can be utilized to understand drivers of fecal coliform (FC) levels.

• Predictive capability improved when wet and dry seasons were separated.

• FC levels during wet weather were closely linked to runoff-related factors.

• FC levels during dry weather were closely linked to tidal and wind related factors.

• Regression models using a binary outcome variable are useful management tools in shellfish-harvesting waters.