Skip to main content
NCBI home page
EPA Author Manuscripts logoLink to EPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Feb 10.
Published in final edited form as: Atmos Chem Phys. 2020 Feb 10;20(3):1627–1639. doi: 10.5194/acp-20-1627-2020

On the Limit to the Accuracy of Regional-Scale Air Quality Models

S Trivikrama Rao a,b, Huiying Luo b, Marina Astitha b, Christian Hogrefe c, Valerie Garcia c, Rohit Mathur c
PMCID: PMC7048235  NIHMSID: NIHMS1559398  PMID: 32117469

Abstract

Regional-scale air pollution models are routinely being used world-wide for research, forecasting air quality, and regulatory purposes. It is well recognized that there are both reducible (systematic) and irreducible (unsystematic) errors in the meteorology-atmospheric chemistry modeling systems. The inherent (random) uncertainty stems from our inability to properly characterize stochastic variations in atmospheric dynamics and chemistry, and from the incommensurability associated with comparisons of the volume-averaged model estimates with point measurements. Because these stochastic variations are not being explicitly simulated in the current generation of regional-scale meteorology-air quality models, one should expect to find differences between the model estimates and corresponding observations. This paper presents an observation-based methodology to determine the expected errors from current generation regional air quality models even when the model design, physics, chemistry, and numerical analysis, as well as its input data, were “perfect”. To this end, the short-term synoptic-scale fluctuations embedded in the daily maximum 8-hr ozone time series are separated from the longer-term forcing using a simple recursive moving average filter. The inherent uncertainty attributable to the stochastic nature of the atmosphere is determined based on 30+ years of historical ozone time series data measured at various monitoring sites in the contiguous United States. The results reveal that the expected root mean square error at the median and 95th percentile is about 2 ppb and 5 ppb, respectively, even for “perfect” air quality models driven with “perfect” input data. Quantitative estimation of the limit to the model’s accuracy will help in objectively assessing the current state-of-the-science in regional air pollution models, measuring progress in their evolution, and providing meaningful and firm targets for improvements in their accuracy relative to ambient measurements.

1. Introduction

Confidence in model estimates of pollutant distributions is established through direct comparisons of modeled concentrations with corresponding observations made at discrete locations for retrospective cases. Pinder et al. (2008) discussed the reducible (i.e., structural and parametric) uncertainties that are attributable to the errors in model input data (e.g., meteorology, emissions, initial and boundary conditions) as well as our incomplete or inadequate understanding of the relevant atmospheric processes (e.g. chemical transformation, planetary boundary layer evolution, transport and dispersion, modeling domain, grid resolution, deposition, rain, clouds). Inherent or irreducible (random or unsystematic) uncertainties stem from our inability to properly characterize the stochastic nature of the atmosphere (Wilmott, 1981, 1985: Fox, 1984; Rao et al., 1985; Dennis et al., 2010; Rao et al., 2011) and from the incommensurability associated with comparing the volume-averaged model estimates with point measurements (e.g., McNair et al., 1996; Swall and Foley, 2009). Also, without completely knowing the 3-dimensional initial physical and chemical state of the atmosphere, its future state cannot be simulated accurately (Lamb, 1984; Lamb and Hati, 1987; Lewellen and Sykes, 1989; Pielke, 1998; Gilliam et al., 2015). Given the presence of the irreducible uncertainties, precise replication of observed concentrations or their changes by the models cannot be expected (Dennis et al., 2010; Rao et al., 2011; Porter et al., 2015; Astitha, 2017)).

Whereas an air quality model’s prediction represents some time/space-averaged concentrations, an observation at any given time at a monitoring location reflects an individual event or specific realization out of a population that will almost always differ from the model estimate even if the model and its input data were perfect (Rao et al., 1985). Consequently, comparisons of modeled and observed concentrations paired in space and time indicate biases and errors in simulating absolute levels of pollutant concentrations at individual monitoring sites (Porter et al., 2015). The scientific discussion on modeling uncertainty goes back more than three decades with the current practice including data assimilation, ensemble modeling, and model performance evaluation (e.g., Fox, 1981, 1984; Lamb, 1984; Demerjian, 1985; Oreskes et al., 1994; Pielke, 1998; Lewellen and Sykes, 1989; Lee et al., 1997; Carmichael et al., 2008; Hogrefe et al., 2001a, 2001b; Biswas and Rao, 2001; Grell and Baklanov, 2011; Gilliam et al., 2006; Herwehe et al., 2011; Baklanov et al., 2014; Bocquet et al., 2015; Solazzo and Galmarini, 2015; Ying and Zhang, 2018; McNider and Pour-Biazar, 2020; Stockwell et al., 2020). While ever-improving process knowledge and increasing computational power will continue to help reduce the structural and parametric uncertainties in air quality models, the inherent uncertainty associated with our inability to properly characterize the stochastic nature of the atmosphere will always result in some mismatch between the model results and measurements; this could lead to speculation on the inferred accuracy of the future states simulated by the regional-scale air quality models (Dennis et al., 2011; Rao et al., 2011; Porter et al., 2015; Astitha et al., 2017; Luo et al., 2019).

The sensitivity of model results to meteorology, chemical mechanisms, and emissions has been examined in numerous studies (e.g., Vautard et al., 2012; Sarwar et al., 2013; Pierce et al., 2010; Napelenok et al., 2011; Kang et al.,2013). Herwehe et al. (2011) attributed the differences in ground-level ozone predictions between WRF-Chem and WRF-CMAQ models to the way meteorology and chemistry interactions are handled within these two modeling systems. Thomas et al. (2019) examined the ozone predictions in the Mid‐Atlantic region of the United States during June 2016 through a series of simulations with WRF‐Chem, focusing on the sensitivity to the meteorological initial/boundary conditions (IC/BCs), emissions inventory (EI), and planetary boundary layer (PBL) scheme. Ying and Zhang (2018) discussed the use of satellite-based observations for improving the predictability of multiscale tropical weather and equatorial waves. Ensemble modeling is being advocated for quantifying the uncertainty in model predictions; however, the spread in the model estimates for the variable of interest reflects the impact of our incomplete or inadequate knowledge of the physical and chemical processes (i.e., the reducible errors stemming from structural and parametric uncertainty) occurring in the atmosphere (Solazzo et al., 2015; Thomas, et al., 2019; Stockwell et al., 2020). McNider and Pour-Biazar (2020) reviewed the many issues in predicting the prevailing meteorology for regional air quality simulations and indicated that errors in the specification of the physical atmosphere such as temperature, winds, and mixing heights can affect the air quality predictions. Stockwell et al. (2020) discussed the problems relating to the atmospheric chemical mechanisms currently being used for simulating air quality. The current generation of regional models consider only the mean values of a meteorological variable for a given timescale and the average rate constant derived from gas chamber experiments for chemical reactions and does not include their fluctuations in solving the equations of motion for each time step. Further, the current operational regional-scale meteorological and air quality models do not explicitly simulate the stochastic nature of the atmosphere and, as such, typically miss the extreme values at both the low and high ends of the concentration distribution function.

In most applications of regional-scale air quality models, statistical metrics such as bias, root mean square error (RMSE), correlation, and index of agreement are being used to judge the quality of model predictions and determine if the model is suitable for forecasting or regulatory purposes (e.g., Fox, 1981, 1984; Solazzo et al., 2011; Appel et al., 2012; Simon et al., 2012; Foley et al., 2014; Ryan et al., 2016; Emery et al. 2016; Zhang, 2016; U.S. EPA, 2018). While significant improvements in the formulation, physical and chemical parameterizations, and numerical techniques have been implemented in atmospheric models over the past three-decades, it is not clear if the improvement claimed in the model’s performance relative to the routine network measurements is statistically significant based on these metrics (Hogrefe et al., 2008). Also, no assessments have been made to date on the errors that are to be expected even from “perfect” regional-scale air quality modeling systems. To estimate such irreducible model errors due to atmospheric stochasticity (which we consider to be the errors that are expected even from a “perfect” model (devoid of structural and parametric uncertainties) with “perfect” (error-free) inputs), we analyzed the observed daily maximum 8-hr (DM8HR) ozone time series data at monitoring locations across the contiguous United States (CONUS) during the 1981–2014 time period and present the results of this analysis in Section 3.1. In Section 3.2, we illustrate how this information could be used in guiding model development specifically aimed at addressing reducible errors in the synoptic component by contrasting the results from Section 3.1 with analysis using the synoptic component from a 21-year simulation performed with the fully coupled WRF-CMAQ simulations covering the 1990–2010 period. Since we relied on multi-decadal historical ozone observations to assess the impact of the stochastic nature of the atmosphere, the results presented here are applicable to both forecasting and retrospective applications of current regional-scale air quality models.

2. Data and Methods

Ground-level DM8HR ozone data covering the CONUS during May to September in each year were obtained from the U.S. Environmental Protection Agency’s (EPA) Air Quality System (AQS) (see https://www.epa.gov/aqs). A valid ozone season consists of at least 80% data coverage during May to September at each station. A total 185 monitoring stations with at least 30 valid years (to provide enough variety of synoptic conditions, denoted hereafter as 30+ in this paper) from the year 1981 to 2014 are analyzed. Also, fully coupled WRF-CMAQ model simulations over the CONUS for the 1990–2010 period were utilized in this study to demonstrate a new perspective on model performance evaluation. To ensure better characterization of the prevailing meteorology (i.e., synoptic forcing) in the retrospective 21-year WRF-CMAQ simulations, four-dimensional data assimilation (FDDA) was utilized following the methodology suggested by Gilliam et al. (2012) and modified for fully-coupled meteorology-chemistry model applications as described in Hogrefe et al. (2015). The model set-up and performance evaluation of these historical multiyear WRF-CMAQ simulations have been published by Xing et al. (2015), Gan et al. (2015), and Astitha et al. (2017). Time-varying chemical lateral boundary conditions are nested from the 108 km hemispheric WRF-CMAQ simulation from 1990 to 2010 (Xing et al., 2015).

It has been shown that time series of the daily maximum 8-hr ozone concentrations contain fluctuations operating on different time scales (e.g., intra-day forcing induced by the fast-changing emissions and atmospheric boundary layer evolution; diurnal forcing induced by the day and night differences; synoptic forcing induced by the passage of weather systems across the country, sub-seasonal forcing due to Madden-Julian Oscillation (MJO), and long-term forcing induced by emissions, El-Nino-Southern Oscillation (ENSO), climate change, and other slow-varying processes such as seasonal and sub-seasonal variations in the atmospheric deposition and stratosphere-troposphere exchange processes) as noted by Rao et al. (1997), Vukovich, (1997), Hogrefe et al. (2000), Porter et al. (2015), Astitha et al. (2017), Xing et al. (2016), and Mathur et al. (2017)). Variations in the 8-hour ozone can be thought of comprising of the baseline of pollution that is created by various emitting sources and modulated by the prevailing synoptic weather conditions (Rao et al., 1996 and 2011). Thus, the magnitude of the baseline (BL) concentration and the strength of the synoptic component (SY) should be viewed as the necessary and sufficient conditions for how high ozone levels can reach on a given day (Astitha et al., 2017). Scale separation can be achieved by applying filtering methods such as the Empirical Mode Decomposition (Huang et al., 1998), Elliptic filter (Poularika, 1998), Kolmogorov-Zurbenko (KZ) filter (Rao and Zurbenko, 1994), Adaptive Filter Technique (Zurbenko, et al., 1996), and Wavelet (Lau and Weng, 1995). Because Improved Complete Ensemble Empirical Mode Decomposition with Adaptive Noise (Improved CEEMDAN, a version of the Empirical Mode Decomposition method) and KZ filter yielded similar results for the DM8HR time series data as shown in Figs. 1–2 discussed in the next section, only the results from the KZ filter are presented in the subsequent analysis for quantifying the impact of the stochastic nature of the atmosphere on observed and simulated ozone concentrations. Furthermore, the KZ filtering is a simple method and works well even in the presence of missing data (Hogrefe et al., 2003). In this study, we used the KZ5,5 with a window size of 5 days and 5 iterations on raw ozone time series [O3 (t)] in the same manner as in Luo et al. (2019), Porter et al. (2015), and Rao et al. (2011). The size of the window and the number of iterations determine the desired scale separation. The KZ5,5 filtering process helps separate the synoptic-scale weather-induced variations embedded in the May-September DM8HR time series data (short-term component, noted as SY) from the long-term baseline component (denoted as BL).

BL(t)=KZ5,5(O3(t)) (1)
SY(t)=O3(t)−KZ5,5(O3(t)) (2)
O3(t)=SY(t)+BL(t) (3)

Figure 1.

Figure 1.

Open in a new tab

Results of the application of the Improved CEEMDAN technique (a modified version of EMD) technique, which is designed for analyzing non-stationary and non-linear time series data, to the daily maximum 8-hour ozone time series data at the Altoona, PA site. The numbers on the right side represent the time scale (in days) associated with each IMF. Note, the power spectrum of raw ozone time series (upper right panel) shows that the energy in the 1–10 days (synoptic) time scale is an order of magnitude less than that in the longer (baseline) time scale.

Figure 2.

Figure 2

Open in a new tab

a. Raw observed DM8HR ozone time series (black) and the embedded baseline (red for EMD and blue for KZ) at Altoona, PA in 2010; Figure 2b. Time series of synoptic forcing (red for EMD and blue for KZ); Figure 2c and 2d are their corresponding power spectra. The bottom two panels compare the power spectra of the baseline forcing (left) and the synoptic forcing (right) derived from KZ filtering and EMD (sum of IMF 1 and IMF2). Notice that most of the energy in the baseline time series is in the longer time scale while most of the energy of the short-term component is in the high-frequency range. The similarity of results from both scale separation techniques demonstrates that the two scales of interest (i.e., baseline and synoptic forcing) have been extracted reasonably well by these two methods.

Because we are working with the daily maximum 8-hr ozone data, the Nyquist interval is 2-days, indicating that the dynamical features having time scales less than 2 days (e.g., intra-day forcing from fast changing emissions and chemical transformations, boundary layer evolution, diurnal forcing due to night vs. day differences) are not resolvable in this analysis (see Fig. 2 in Dennis et al., 2010). The 50% cut-off frequency for the KZ5,5 is ~24 days, and, hence, time scales less than those associated with synoptic-scale weather fluctuations are embedded in the short-term or SY forcing. The KZ filtering is applied to both DM8HR observations and modeled DM8HR time series. Once the baseline is separated from the original DM8HR time series from all monitoring stations, then the synoptic forcing in the historical ozone time series data is used to estimate the variability in ozone concentrations that can be expected because of the chaotic/stochastic nature of the atmosphere by taking into account the relationship between the strength of synoptic forcing and mean of baseline ozone at each location over CONUS. This methodology was applied to both measured and modeled ozone concentrations (see details in Luo et al., 2019). Whereas the focus of Luo et al. (2019) was on transforming the deterministic modeling results into a probabilistic framework for assessing the efficacy of different emission control strategies in achieving compliance with the ozone standard, this paper is aimed at quantifying the model performance errors to be expected at each monitoring site over CONUS even from “perfect” regional-scale ozone models driven with “perfect” input data from the ever-present stochastic nature of the atmosphere.

3. Results and Discussion

3.1. Analysis of ambient ozone data

Using both Improved CEEMDAN and KZ filtering methods, we separated the synoptic forcing (time scale < 24 days) and baseline (time scale > 1 month) forcing embedded in the time series of observed and modeled daily maximum 8-hour ozone concentrations. To illustrate, the results from the application of Improved CEEMDAN to the daily maximum 8-hr ozone time series data measured at Altoona, PA are presented in Fig. 1. The top left panel displays the raw ozone time series while the top of the right panel shows its power spectrum. The 7 intrinsic mode functions (IMFs) and the residual on the left side, and their corresponding power spectra on the right reveal that most of the synoptic-scale features in ozone data are imbedded in IMFs 1 and 2. The baseline ozone is extracted by removing the first two IMFs from the raw ozone time series. To illustrate the concept of the ozone baseline, DM8HR time series measured in 2010 at Altoona, PA is presented in Fig. 2a together with the embedded baseline concentration as extracted by the KZ5,5 and Improved CEEMDAN. It is evident that high ozone levels are always associated with the elevated baseline. The difference between the raw ozone time series and baseline, denoted as the short-term or synoptic forcing (SY), is displayed in Fig. 2b. The power spectra, displayed in Figs. 2c and d, reveal both methods yielded good scale separation. Due to the good agreement between both scale separation techniques, only the results from the KZ filter are presented for the remainder of the manuscript.

Once the scale separation is achieved with the KZ5, 5, we superimposed the SY forcing imbedded in 30+ years of historical DM8HR ozone time series measured at a given location on the baseline component of the ozone time series at that location to generate 30+ reconstructed or pseudo ozone distributions. Illustrative results using eq. (3) at a suburban location in Altoona, PA are presented for 2010 base year in Fig. 3a; it should be noted that the linear relationship between the strength of SY (defined as the standard deviation of the data in the synoptic component) and the magnitude of the BL (defined as the mean of the data in the baseline component) has been taken into account in generating 30+ years of adjusted SY forcing as illustrated in Luo et al. (2019). As expected, there is excellent agreement between the average of 30+ values (solid blue line) and observed ozone in 2010 at each percentile of the concentration distribution function (red line). Also, the original cumulative distribution function (CDF) in 2010 (red line) is constrained within the 30+ CDFs of pseudo distributions (Fig. 3a); note, it is equally likely for any of these 30+ CDFs to occur because of the stochastic nature of the atmosphere even though the individual event in 2010 yielded the CDF shown in red. As mentioned before, ozone mixing ratio at any given probability point on the red line in Fig. 3a reflects an individual event while ozone values at the same probability in different CDFs (gray lines) reflect the population stemming from the stochastic nature of the atmosphere. In other words, there are 30+ dynamically consistent ozone time series attributable to the 2010 baseline (given 2010 emissions) for examining the inherent variability due to atmospheric stochasticity. It is evident in Fig. 3a that there is larger variability at the lower and upper percentiles than that in inter-quartile range, revealing that the tails of the concentration distribution function are subject to large inherent uncertainty. Using these 30+ pseudo-observation ozone mixing ratios and the actual observed ozone values at each percentile, statistical metrics such as Bias, RMSE, coefficient of variation (CV=standard deviation/mean), normalized mean error (NME) and normalized mean bias (NMB) are presented in Fig. 3b and c (see Emery et al. (2016) for the description of the statistical metrics considered here). As expected, the lower and upper tails of the distribution are prone to large errors. These results demonstrate the presence of substantial natural variability at the upper 95th percentile, which is of primary interest in regulatory analyses. The extreme values are better described in statistical terms rather than in deterministic sense (Hogrefe and Rao, 2001; Luo et al., 2019).

Figure 3.

Figure 3

Open in a new tab

a: Comparison between the observed cumulative distribution function (CDF) for 2010 shown in red with 30+ pseudo-observations CDFs generated from historical DM8HR ozone time series shown in gray at a suburban site at Altoona in PA (AQS station identifier 420130801). The blue line represents the average of the 30+ gray lines; Figure 3b: Display of various statistical metrics (standard deviation (std), root mean square error (RMSE), bias) derived by comparing the actual observed and pseudo ozone values in Fig. 3a; Figure 3c: Normalized statistical metrics of normalized mean error (NME), normalized mean bias (NMB), coefficient of variation (CV). Notice the large variability occurring at the lower and upper percentiles.

Ozone time series at 185 monitoring stations covering CONUS, having at least 80% data completeness, are analyzed in the above manner and the results are displayed as box plots in Fig. 4. Note the presence of large variability in the CV, NME, and NMB, and Bias at the lower and upper percentiles (Fig. 4). The RMSE expected for the ozone mixing ratios in the interquartile range is ~1.5 ppb, but it is >5 ppb for the upper 95th percentile (Fig. 4b). The spatial distribution of RMSE at the 50th and 95th percentiles is displayed in Figures 5a and 5b, respectively. The RMSE at the upper 95th percentile is very high at some monitoring sites in California and Michigan (Fig. 5b). Monitoring stations situated in the urban areas, near large water bodies, and in regions of complex terrain influenced predominantly by local conditions tend to exhibit higher RMSE. The elevation of the monitoring sites is displayed in Fig. 5c.

Figure 4.

Figure 4.

Open in a new tab

Box plots of statistical metrics based on the results from the analysis of DM8HR data at 185 monitoring sites: (a) Standard deviation, (b) Root mean square error, (c) Mean bias, (d) Coefficient of variation, (e) Normalized mean error, and (f) Normalized mean bias. The lower and upper edges of the boxes represent the 25th and 75th percentile values while the whiskers represent the 5th and 95th percentiles. See data analysis procedures using the ozone baseline observed in the year 2010 as the target BL in equations 7 and 8 of Luo et al. (2019).

Figure 5.

Figure 5.

Open in a new tab

Spatial distribution of the lower bound for the RMSE or expected RMSE at each monitoring site over CONUS (a) at the median and (b) at the 95th percentile; (c) elevation (km) above the mean sea level of each monitoring site.

3.2. Analysis of modeled ozone concentrations

The analysis in the previous section quantified the inherent stochastic variability that is present in the SY component using long-term records of ozone observations. In this section, we analyze long-term records of model simulations in an attempt to quantify the error associated with the modeled SY component that results both from not explicitly representing stochastic variations in atmospheric dynamics in the current generation regional air quality models and from other reducible sources of model error. The model simulations were performed with the fully coupled WRF-CMAQ system with a 36-km horizontal grid cell size and covered the 21-year period from 1990 to 2010 (Gan et al., 2015). In this section, we examine the impact of superimposing different SY forcings embedded in ozone observations vs. those in the WRF-CMAQ model on the observed baseline concentration. To provide an illustration of the differences between observed and modeled time series over this period, Figure 6a displays a scatter plot of the strength of the SY component (standard deviation of data in the SY component) vs. the mean of the baseline (BL) component for both observations and model simulations at the Altoona, PA site. While both observations and WRF-CMAQ simulations show a strong correlation between these two variables, it is evident that at this monitoring location the standard deviation (i.e., strength) of the SY component is substantially lower for the WRF-CMAQ simulations for a given mean of the BL component (i.e., for any given year). The year-to-year variation in the observed and modeled mean of BL and strength of SY forcing, displayed in Fig. 6b, reveals that the model overestimated BL and underestimated the strength of SY forcing. The 36-km grid may be better representing the large-scale synoptic forcing associated with the translation of weather systems than the meso-scale weather and urban influences (both dynamics and chemistry) that are embedded in the observed SY component. Meteorological modeling with higher horizontal grid resolution might be able to capture the land-sea breeze, lake-sea breeze, and terrain influences that observations are seeing at certain monitoring locations.

Figure 6.

Figure 6.

Open in a new tab

(a) Scatter plot of the standard deviation (i.e., strength) of the SY component vs. the mean of the baseline (BL) component for each of the 21 years from 1990 to 2010 at the Altoona, PA monitoring site. Observations are shown in red while WRF-CMAQ results are shown in blue. (b) Inter-annual variability in the mean of the baseline component and standard deviation of the synoptic component in the WRF-CMAQ model and observations at the Altoona, PA site. Although year-to-year variation is captured, the model has overestimated the baseline forcing and underestimated the synoptic forcing.

To isolate the impact of model imperfections on only the SY time scale on errors across the ozone distribution, we assume that the model perfectly reproduces the ‘true’ BL depicted by the observed 2010 BL. We then use this ‘perfect’ modeled BL and reconstruct ‘pseudo-simulated’ ozone time series, like what was done in Fig. 3, except for using the SY component embedded in the 21 years of coupled WRF-CMAQ simulations. The rationale for this analysis is to quantify the amount of model error present in the current simulations that could conceivably be reduced through improving the representation of synoptic and mesoscale processes and/or increased horizontal resolution with appropriate data assimilation techniques. Fig. 7a displays the CDF of actual observed ozone (red line) overlaid on 21 pseudo-simulated ozone CDFs (gray lines, with averages of all 21 pseudo-simulated ozone percentiles shown in blue) at the Altoona, PA site while Figs. 7b and 7c display absolute and normalized performance metrics. Figure 7a confirms that the coupled WRF-CMAQ SY components have less intra-annual variability than observed SY components, causing overestimation at the low end and underestimation at the high end of the observed CDF for all 21 years of reconstruction; these results imply that the model’s results at the upper and lower percentiles will always tend to be unreliable or prone to large errors even when the baseline concentration is predicted perfectly. The U-shape of the absolute and relative error curves in Figures 7b and c is similar to the corresponding curves in Figure 3, but the larger magnitude at the high and low end of the distribution indicates that the effects of the underestimated intra-annual SY variability (note that the distribution of modeled values in Fig. 7a is much flatter (i.e., having higher Kurtosis) than that of the observations) outweigh those errors attributable to the stochastic variability presented in Figure 3. The shape of the absolute and normalized bias curves deviates from those shown for the pseudo-observations in Figures 3b–c and, thus, also reveals the effect of the underestimation of the intra-annual SY variability. Figures 7d–f present differences between the curves shown in Figures 7a–c and a version of Figure 3a–c computed from the 1990–2010 data instead of 30+ years of historical ozone observations. Panels e and f show that at the 50th percentile, the differences in the error curves are close to zero since both the pseudo-simulations and pseudo-observations used the same observed BL component. At the upper percentiles, the differences reach 3 – 5 ppb, providing an estimate of the reducible error in simulating the extreme values at this location because of the differences in the observed SY and WRF-CMAQ SY components at this location; high-resolution meteorological modeling may help address these reducible errors.

Figure 7.

Figure 7.

Open in a new tab

a) Comparison between the observed CDF overlaid on 21 ‘pseudo-simulated’ or reconstructed ozone CDFs with SY generated from modeled DM8HR ozone time series at a suburban site at Altoona in PA (AQS station identifier 420130801); b) Display of various statistical metrics derived by comparing the actual observed and pseudo-simulated ozone values in Fig. 7a; c) Normalized statistical metrics; d).Difference between the pseudo-simulated CDFs shown in Figure 7a and the pseudo-observed CDFs as shown in Figure 7a but calculated from 21 years (1990–2010) of observations only. The gray lines represent the differences for a specific SY year while the blue line represents the differences between the means of the 21 reconstructions; e) Difference between the absolute performance metrics for pseudo-simulations shown in Figure 7b and those calculated for pseudo-observations as shown in Figure 7b but calculated for 21 years (1990–2010) only. f) As in panel e) but for normalized performance metrics.

Figs. 8a and b display the RMSE at the median and 95th percentile for the ‘pseudo-simulated’ ozone values at each monitoring site. For the 50th percentile, the RMSE values range from 0.2 ppb to 3.2 ppb over CONUS with a median value of 1 ppb while at the 95th percentile, the RMSE values range from 1 ppb to 15 ppb with a median value of 4 ppb across all sites over CONUS. The values are highest along the California coast and near Great Lakes, possibly due to inadequacies in simulating the land-sea breeze and land-lake breeze regimes, respectively, with modeling at 36 km grid cells. Air quality modeling uncertainty even for the retrospective modeling cases, outside of the chemistry formulation and boundary conditions, is attributed primarily to meteorology and emissions inputs. Vautard et al. (2012) and McNider and Pour-Biazar (2020) concluded that major challenges remain in the simulation of prevailing meteorology (e.g., errors in wind speed, PBL, night-time meteorology, nocturnal transport aloft, clouds) in retrospective air quality modeling. Based on the retrospective ozone episodic modeling with the WRF-CMAQ model using various sets of equally likely initial conditions for meteorology along with FDDA, Gilliam et al. (2015) confirmed the presence of sizable spread in WRF solutions, including common weather variables of temperature, wind, boundary layer depth, clouds, and radiation, thereby causing a relatively large range of ozone concentrations. Also, pollutant transport is altered by hundreds of kilometers over several days. Ozone concentrations of the ensemble varied as much as 10–20 ppb (or 20–30%) in areas that typically have higher pollution levels. As model improvements are made, one can quantitatively assess how close the predictions of the improved model are for each percentile for the given base year simulation to the expected errors from a “perfect” model with “perfect” input, i.e. the target RMSE shown in Fig.5a and b. Perhaps, the next generation of regional-scale meteorological and air quality models might be capable of explicitly simultaneously treating the mean and fluctuation components for all variables within the deterministic-stochastic modeling framework to properly account for the stochastic nature of the atmosphere.

Figure 8.

Figure 8.

Open in a new tab

Errors in the 21 ‘pseudo-simulated’ or reconstructed ozone time series with SY generated from modeled DM8HR ozone time series using BL obtained from observations at (a) the median and (b) 95th percentile.

4. Conclusions

Regardless of how accurate the regional air quality model is, the stochastic variations in the atmosphere cannot be consistently reproduced by the deterministic numerical models. In this study, we demonstrate how to quantify this irreproducible stochastic component by isolating the synoptic forcing imbedded in 30+ years of historical observations and assess the performance of the 36 km fully coupled WRF-CMAQ model in simulating 21 years of ozone concentrations over the contiguous U.S. Observation-based analysis reveals that on average, the irreducible error attributable to the stochastic nature of the atmosphere ranges from ~2 ppb at the 50th percentile to ~ 5 ppb at the 95th percentile. To improve regional-scale ozone air quality models, attention should be paid to accurately simulate the baseline concentration by focusing on the quality of the emission inventory and the model’s treatment for the boundary conditions and slow-changing (operating on sub-seasonal, seasonal, and longer-term time scales) atmospheric processes. Also, errors in reproducing the synoptic forcing can possibly be reduced with high-resolution meteorological modeling using appropriate data assimilation techniques. Nonetheless, these results demonstrate the presence of large variability in the upper tail of the DM8HR O3 concentration cumulative distribution even with “perfect” models using “perfect” input data. Having this quantitative estimation of practical limits for model’s accuracy helps in objectively assessing the current state of regional-scale air quality models, measuring progress in their evolution, and providing meaningful and firm targets for improvements in their accuracy relative to measurements from routine networks.

Footnotes

Code availability: Source code for version 5.0.2 of the Community Multiscale Air Quality (CMAQ) modeling system can be downloaded from https://github.com/USEPA/CMAQ/tree/5.0.2. For further information, please visit the U.S. Environmental Protection Agency website for the CMAQ system: https://www.epa.gov/cmaq.

Data availability: All ozone observations used in this article are available from https://aqs.epa.gov/aqsweb/airdata/download_files.html (AQS). Paired ozone observation and CMAQ model data used in the analysis will be made available at https://edg.epa.gov/metadata/catalog/main/home.page. Raw CMAQ model outputs are available on request from the U.S EPA authors.

Competing interests: The authors declare that they have no conflict of interest.

Publisher's Disclaimer: Disclaimer: The views expressed in this paper are those of the authors and do not necessarily represent the view or policies of the U.S. Environmental Protection Agency.

References

  1. Appel KW, Chemel C, Roselle SJ, Francis XV, Hu R-M, Sokhi RS, Rao ST, and Galmarini S: Examination of the Community Multiscale Air Quality (CMAQ) model performance over the North American and European domains. Atmospheric Environment, AQMEII: An International Initiative for the Evaluation of Regional-Scale Air Quality Models - Phase 1 53, 142–155. 10.1016/j.atmosenv.2011.11.016, 2012. [DOI] [Google Scholar]
  2. Astitha M, Luo H, Rao ST, Hogrefe C, Mathur R, and Kumar N: Dynamic evaluation of two decades of WRF-CMAQ ozone simulations over the contiguous United States. Atmospheric Environment 164, 102–116. 10.1016/j.atmosenv.2017.05.020, 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Baklanov A, Schlünzen K, Suppan P, Baldasano J, Brunner D, Aksoyoglu S, Carmichael G, Douros J, Flemming J, Forkel R, Galmarini S, Gauss M, Grell G, Hirtl M, Joffre S, Jorba O, Kaas E, Kaasik M, Kallos G, Kong X, Korsholm U, Kurganskiy A, Kushta J, Lohmann U, Mahura A, Manders-Groot A, Maurizi A, Moussiopoulos N, Rao ST, Savage N, Seigneur C, Sokhi RS, Solazzo E, Solomos S, Sørensen B, Tsegas G, Vignati E, Vogel B, and Zhang Y: Online coupled regional meteorology chemistry models in Europe: current status and prospects. Atmospheric Chemistry and Physics 14, 317–398. 10.5194/acp-14-317-2014, 2014. [DOI] [Google Scholar]
  4. Bocquet M, Elbern H, Eskes H, Hirtl M, Žabkar R, Carmichael GR, Flemming J, Inness A, Pagowski M, Pérez Camaño JL, Saide PE, San Jose R, Sofiev M, Vira J, Baklanov A, Carnevale C, Grell G, and Seigneur C: Data assimilation in atmospheric chemistry models: current status and future prospects for coupled chemistry meteorology models. Atmospheric Chemistry and Physics 15, 5325–5358. 10.5194/acp-15-5325-2015, 2015. [DOI] [Google Scholar]
  5. Biswas J, Rao ST: Uncertainties in episodic ozone modeling stemming from uncertainties in the meteorological fields. J Appl Meteor., 40:117–136, 2001. [Google Scholar]
  6. Carmichael GR, Sakurai T, Streets D, Hozumi Y, Ueda H, Park SU, Fung C, Han Z, Kajino M, Engardt M, Bennet C, Hayami H, Sartelet K, Holloway T, Wang Z, Kannari A, Fu J, Matsuda K, Thongboonchoo N, and Amann M: MICS-Asia II: The model intercomparison study for Asia Phase II methodology and overview of findings. Atmospheric Environment, MICS-ASIA II 42, 3468–3490. 10.1016/j.atmosenv.2007.04.007, 2008. [DOI] [Google Scholar]
  7. Colominas MA, Schlotthauer G, Torres ME, 2014. Improved complete ensemble EMD: A suitable tool for biomedical signal processing. Biomedical Signal Processing and Control 14, 19–29. [Google Scholar]
  8. Demerjian KL: Quantifying Uncertainty in Long-Range-Transport Models: A Summary of the AMS Workshop on Sources and Evaluation of Uncertainty in Long-Range-Transport Models, Bull. Amer. Meteor. Soc, 66, 12, 1533–1540, 1985. [Google Scholar]
  9. Dennis R, Fox T, Fuentes M, Gilliland A, Hanna S, Hogrefe C, Irwin J, Rao ST, Scheffe R, Schere K, Steyn D, Venkatram A: A framework for evaluating regional-scale numerical photochemical modeling systems. Environ Fluid Mech 10, 471–489. 10.1007/s10652-009-9163-2, 2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Emery C, Liu Z, Russell AG,Talat Odman M, Yarwood G, and Kumar N: Recommendations on statistics and benchmarks to assess photochemical model performance, J. Air & Waste Manage. Assoc, 10.1080/10962247.2016.1265027, 2016. [DOI] [PubMed] [Google Scholar]
  11. Foley KM, Napelenok SL, Jang C, Phillips S, Hubbell BJ, and Fulcher CM: Two reduced form air quality modeling techniques for rapidly calculating pollutant mitigation potential across many sources, locations and precursor emission types. Atmospheric Environment 98, 283–289. 10.1016/j.atmosenv.2014.08.046, 2014. [DOI] [Google Scholar]
  12. Fox DG: Judging Air Quality Model Performance: A Summary of the AMS Workshop on Dispersion Model Performance, Douglas O. box Woods Hole, Mass., 8–11 September 1980, Bull. Amer. Met. Soc, Vol. 62, No. 5, May 1981, pp 599–609, 1981. [Google Scholar]
  13. Fox DG: Uncertainty in Air Quality Modeling A Summary of the AMS Workshop on Quantifying and Communicating Model Uncertainty, Woods Hole, Mass., September 1982, Vol. 65, No. 1, January, pp 27–36, 1984. [Google Scholar]
  14. Gan C-M, Pleim J, Mathur R, Hogrefe C, Long CN, Xing J, Wong D, Gilliam R, and Wei C: Assessment of long-term WRF–CMAQ simulations for understanding direct aerosol effects on radiation “brightening” in the United States, Atmos. Chem. Phys, 15, PP 12193–12209, doi: 10.5194/acp-15-12193-2015, 2015. [DOI] [Google Scholar]
  15. Gilliam RC, Hogrefe C, and Rao ST: New methods for evaluating meteorological models used in air quality applications, Atm. Environ, Vol. 40, Issue 26, PP 5073–5086, 2006. [Google Scholar]
  16. Gilliam RC, Godowitch J, and Rao ST: Diagnostic evaluation of ozone production and horizontal transport in a regional photochemical air quality modeling system, Atmos. Environ, 53, PP 3977–3987, 10.1016/j.atmosenv.2011.04.062, 2012. [DOI] [Google Scholar]
  17. Gilliam RC, Hogrefe C, Godowitch G, Napelenok S, Mathur R, and Rao ST: Impact of inherent meteorology uncertainty on air quality model predictions, J. Geophy. Res.: Atmospheres, Vol. 120, No. 23 10.1002/2015JD023674, 2015. [DOI] [Google Scholar]
  18. Grell G, and Baklanov A: Integrated modeling for forecasting weather and air quality: A call for fully coupled approaches. Atmospheric Environment, Modeling of Air Quality Impacts, Forecasting and Interactions with Climate. 45, 6845–6851. 10.1016/j.atmosenv.2011.01.017, 2011. [DOI] [Google Scholar]
  19. Herwehe JA, Otte TL, Mathur R, and Rao ST: Diagnostic analysis of ozone concentrations simulated by two regional-scale air quality models, Atmospheric Environment, 45, Pages 5957–5969, 2011. [Google Scholar]
  20. Hogrefe C, Rao ST, Zurbenko IG, and Porter PS: Interpreting the Information in Ozone Observations and Model Predictions Relevant to Regulatory Policies in the Eastern United States. Bull. Amer. Meteor. Soc 81, 2083–2106. , 2000. [DOI] [Google Scholar]
  21. Hogrefe C, and Rao ST: Demonstrating attainment of the air quality standards: Integration of observations and model predictions into the probabilistic framework. J. of the Air & Waste Manage. Assoc, 51:7, 1060–1072, DOI: 10.1080/10473289.2001.10464332 [DOI] [PubMed] [Google Scholar]
  22. Hogrefe C, Rao ST, Kasibhatla P, Hao W, Sistla G, Mathur R, and McHenry J: Evaluating the performance of regional-scale photochemical modeling systems: Part II—ozone predictions. Atm. Environ, 35, 4175–4188. 10.1016/S1352-2310(01)00183-2, 2001a. [DOI] [Google Scholar]
  23. Hogrefe C, Rao ST, Kasibhatla P, Kallos G, Tremback CJ, Hao W, Olerud D, Xiu A, McHenry J, and Alapaty K: Evaluating the performance of regional-scale photochemical modeling systems: Part I—meteorological predictions. Atm. Environ, 35, 4159–4174. 10.1016/S1352-2310(01)00182-0, 2001b. [DOI] [Google Scholar]
  24. Hogrefe C, Vempaty S, Rao ST, and Porter PS: A comparison of four techniques for separating different time scales in atmospheric variables. Atmos. Environ, Vol. 37, Issue 3, 313–325, 10.1016/S1352-2310(02)00897-X, 2003. [DOI] [Google Scholar]
  25. Hogrefe C, Ku JY, Sistla G, Gilliland A, Irwin JS, Porter PS, Gégo E, and Rao ST: How has model performance for regional scale ozone simulations changed over the past decade?, Air Pollution Modeling and its Application XIX, Borrego C and Miranda AI (Eds.), Springer, Dordrecht, The Netherlands, pp. 394–403, 2008. [Google Scholar]
  26. Hogrefe C, Pouliot G, Wong D, Torian A, Roselle S, Pleim J, and Mathur R: Annual application and evaluation of the online coupled WRF–CMAQ system over North America under AQMEII phase 2, Atmos. Environ, 115, pp 683–694, 10.1016/j.atmosenv.2014.12.034, 2015. [DOI] [Google Scholar]
  27. Huang NE, Shen Z, Long SR, Wu MC, Shih HH, Zheng Q, Yen NC, Tung CC, and Liu HH The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis. Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences 454, 903–995. 10.1098/rspa.1998.0193, 1998. [DOI] [Google Scholar]
  28. Kang D, Hogrefe C, Foley K, Napelenok S, Mathur R, and Rao ST: Application of the Kolmogorov–Zurbenko filter and the decoupled direct 3D method for the dynamic evaluation of a regional air quality model, Atmos. Environ, 80, 58–69, 10.1016/j.atmosenv.2013.04.046, 2013. [DOI] [Google Scholar]
  29. Lamb RG: Air pollution models as descriptors of cause-effect relationships, Atmos. Environ, 18, 591–606, 1984. [Google Scholar]
  30. Lamb RG, and Hati SK: The representation of atmospheric motions in models of regional-scale air pollution, J. Climatology and Appl. Meteor, 26, 837–846, 1987. [Google Scholar]
  31. Lau K-M, and Weng H-Y: Climate signal detection using wavelet transform: how to make a time series sing. Bull. Amer. Meteor. Soc, 76, 2391–2402, 1995. [Google Scholar]
  32. Lee AM, Carver GD, Chipperfield MP, and Pyle PA: Three-dimensional chemical forecasting: A methodology, J. Geophys. Res, 102, 3905–3919, 1997. [Google Scholar]
  33. Lewellen WS, and Sykes RI: Meteorological data needs for modeling air quality uncertainties, J. Atmos. Oceanic Technol, 6, 759–768, 1989. [Google Scholar]
  34. Luo H, Astitha M, Hogrefe C, Mathur R, and Rao ST: A new method for assessing the efficacy of emission control strategies. Atm. Environ 199, 233–243. 10.1016/j.atmosenv.2018.11.010, 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Mathur R; Xing J; Gilliam R; Sarwar G; Hogrefe C; Pleim J; Pouliot G; Roselle S; Spero TL; Wong DC; Young J Extending the Community Multiscale Air Quality (CMAQ) modeling system to hemispheric scales: overview of process considerations and initial applications. Atmos. Chem. Phys 2017, 17, 12449–12474, 10.5194/acp-17-12449-2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. McNair LA, Hartley, and Russell AG: Spatial inhomogeneity in pollutant concentrations, and their implications for air quality model evaluation, Atm. Environ, 30, 4291–4301. 10.1016/1352-2310(96)00098-2, 1996. [DOI] [Google Scholar]
  37. McNider RT, and Pour-Bizar A: Meteorological modeling relevant to mesoscale and regional air quality applications: A Review, J. Air & Waste Manage. Assoc, 2020, 10.1080/10962247.2019.1694602, 2020. [DOI] [PubMed] [Google Scholar]
  38. Napelenok S, Foley K, Kang D, Mathur R, Pierce T, and Rao ST: Dynamic evaluation of regional air quality model’s response to emission reductions in the presence of uncertain emission inventories, Atmos. Environ, 45, 4091–4098, 10.1016/j.atmosenv.2011.03.030, 2011. [DOI] [Google Scholar]
  39. Oreskes N, Shrader-Frechette K, and Belitz K, Verification, Validation, and Confirmation of Numerical Models in the Earth Sciences, Science, 263, 641–646, 10.1126/science.263.5147.641, 1994 [DOI] [PubMed] [Google Scholar]
  40. Pielke RA: The need to assess uncertainty in air quality evaluations, Atmos. Environ, 32, 1467–1468, 1998. [Google Scholar]
  41. Pinder RW, Gilliam RC, Appel KW, Napelenok S, Foley KM, and Gilliland AB: Efficient Probabilistic Estimates of Surface Ozone Concentration Using an Ensemble of Model Configurations and Direct Sensitivity Calculations, Environ. Sci. Technol, 43 (7), pp 2388–2393, 2008. [DOI] [PubMed] [Google Scholar]
  42. Porter PS, Rao ST, Hogrefe C, Gego E, and Mathur R: Methods for reducing biases and errors in regional photochemical model outputs for use in emission reduction and exposure assessments. Atm. Environ, 112, 178–188. 10.1016/j.atmosenv.2015.04.039, 2015. [DOI] [Google Scholar]
  43. Poularika AD: The Handbook of Formulas and Tables for Signal Processing. CRC Press, Boca Raton, FL, 1998. [Google Scholar]
  44. Rao ST, Sistla G, Pagnotti V, Petersen WB, Irwin JS, and Turner DB: Resampling and Extreme Value Statistics in Air Quality Model Performance Evaluation, Atm. Environ, Vol. 19 No. 9 pp. I503–1518, 1985. [Google Scholar]
  45. Rao ST, and Zurbenko IG: Detecting and Tracking Changes in Ozone Air Quality. Air & Waste 44, 1089–1092. 10.1080/10473289.1994.10467303, 1994. [DOI] [PubMed] [Google Scholar]
  46. Rao ST, Zurbenko IG, Porter PS, Ku J-Y, and Henry RF: Dealing with the ozone non-attainment problem in the Eastern United States. Environmental Management, January 17–31, 1996. [Google Scholar]
  47. Rao ST, Zurbenko IG, Neagu R, Porter PS, Ku JY, and Henry RF: Space and Time Scales in Ambient Ozone Data. Bull. Amer. Meteor. Soc 78, 2153–2166. , 1997. [DOI] [Google Scholar]
  48. Rao ST, Galmarini S, and Pucket K: Air Quality Model Evaluation International Initiative (AQMEII)-Advancing the State of the Science in Regional Photochemical Modeling and Its Applications, Bull. Amer. Meteor. Soc, 23–30, DOI: 10.1175/2010BAMS3069.1, 2011. [DOI] [Google Scholar]
  49. Rao ST, Porter PS, Mobley JD, and Hurley F: Understanding the spatio-temporal variability in air pollution concentrations. Environmental Management, November 42–48, 2011. [Google Scholar]
  50. Ryan WF: The air quality forecast rote: Recent changes and future challenges, Journal of the Air & Waste Manage. Assoc, 66, 576–596. 10.1080/10962247.2016.1151469, 2016. [DOI] [PubMed] [Google Scholar]
  51. Sarwar G, Godowitch, Henderson B, Fahey K, Pouliot G, Hutzell W, Mathur R, Kang D, Goliff W, and Stockwell W: A comparison of atmospheric composition using the Carbon Bond and Regional Atmospheric Chemistry Mechanisms, Atmos. Chem. Phys, 13, 9695–9712, doi: 10.5194/acp-13-9695-2013. 2013. [DOI] [Google Scholar]
  52. Solazzo E, and Galmarini S: Comparing apples with apples: Using spatially distributed time series of monitoring data for model evaluation, Atmos. Environ 112, 234–245. 10.1016/j.atmosenv.2015.04.037, 2015. [DOI] [Google Scholar]
  53. Solazzo E, and Galmarini S: A science-based use of ensembles of opportunities for assessment and scenario studies, Atmos. Chem. Phys 15, 2535–2544, 2015. [Google Scholar]
  54. Stockwell WR, Saunders E, Goliff WS, and Fitzgerald RM: A perspective on the development of gas-phase chemical mechanisms for Eulerian air quality models, J. Air & Waste Manage. Assoc, DOI: 10.1080/10962247.2019.1694605, 2020. [DOI] [PubMed] [Google Scholar]
  55. Swall JL, and Foley KM: The impact of spatial correlation and incommensurability on model evaluation, Atmospheric Environment, 43, 1204–1217, 10.1016/j.atmosenv.2008.10.057, 2009. [DOI] [Google Scholar]
  56. Thomas A, Huff AK, Hu X-M, and Zhang F: Quantifying uncertainties of ground‐level ozone within WRF‐Chem simulations in the mid‐Atlantic region of the United States as a response to variability, J. of Advances in Modeling Earth Systems, 1100–1116, 10.1029/2018MS001457, 2019. [DOI] [Google Scholar]
  57. U.S. Environmental Protection Agency: Modeling Guidance for Demonstrating Air Quality Goals for Ozone, PM2.5, and Regional Haze, EPA 454/R-18–009, 203 pp., available online at https://www3.epa.gov/ttn/scram/guidance/guide/O3-PM-RH-Modeling_Guidance-2018.pdf, 2018. [Google Scholar]
  58. Vautard R, Moran MD, Solazzo E, Gilliam R, Mathias V, Bianconi R, Chemel C, Ferreira J, Geyer B, Hansen AB, Jericevic A, Prank M, Segers A, Silver JD, Werhahn J, Wolke, Rao ST, and Galmarini S: Evaluation of the meteorological forcing used for the Air Quality Model Evaluation International Initiative (AQMEII) air quality simulations, Atmos. Environ, Vol. 53, 15–37, 10.1016/j.atmosenv.2011.10.065, 2012. [DOI] [Google Scholar]
  59. Vukovich FM: Time Scales of Surface Ozone Variations in the Regional, Non-URBAN Environment, Atm. Environ, Vol. 31, No. 10, pp. 1513–1530, 1997. [Google Scholar]
  60. Willmott CJ: On the Validation of Models. Phys. Geography 2: 184–194. 10.1080/02723646.1981.10642213, 1981. [DOI] [Google Scholar]
  61. Wilmott C, Ackleson S, Davis R, Feddema J, Klink K, Legates R, O’Donnell R, Rowe C: Statistics for the Evaluation and Comparison of Models, J. Geophy. Res, 90, C5, pp 8995–9005, 1985. [Google Scholar]
  62. Xing J, Mathur R, Pleim J, Hogrefe C, Gan C-M, Wong DC, Wei C, Gilliam R, and Pouliot G: Observations and modeling of air quality trends over 1990–2010 across the Northern Hemisphere: China, the United States and Europe Atmospheric Chemistry and Physics 15, 2723–2747. 10.5194/acp-15-2723-2015, 2015. [DOI] [Google Scholar]
  63. Xing J, Mathur R, Pleim J, Hogrefe C, Wang J, Gan C-M, Sarwar G, Wong D, and McKeen S, Representing the effects of stratosphere-troposphere exchange on 3D O3 distributions in chemistry transport models using a potential vorticity based parameterization, Atmos. Chem. Phys, 16, 10865–10877, doi: 10.5194/acp-16-10865-2016, 2016 [DOI] [Google Scholar]
  64. Ying Y, and Zhang F: Potentials in improving predictability of multiscale tropical weather systems evaluated through ensemble assimilation of simulated satellite-based observations, J. Atmos. Sci, 75, 1675–1697, DOI: 10.1175/JAS-D-17-0245.1, 2018. [DOI] [Google Scholar]
  65. Zhang Y, Hong CP, Yahya K, Li Q, Zhang Q, and He K-B: Comprehensive evaluation of multi-year real-time air quality forecasting using an online-coupled meteorology-chemistry model over southeastern United States, Atmos. Environ, 138, 162–182, doi: 10.1016/j.atmosenv.2016.05.006, 2016. [DOI] [Google Scholar]
  66. Zurbenko IG, Porter PS, Gui R, Rao ST, Ku JY, and Eskridge RE: Detecting discontinuities in time series of upper-air data: Development and demonstration of an adaptive filter technique, J. of Climate, Vol. 9, PP 3548–3560, , 1996. [DOI] [Google Scholar]

RESOURCES