Evaluation of the Multi-Angle Implementation of Atmospheric Correction (MAIAC) Aerosol Algorithm through Intercomparison with VIIRS Aerosol Products and AERONET

Abstract

The Multi-Angle Implementation of Atmospheric Correction (MAIAC) algorithm is under evaluation for use in conjunction with the Geostationary Coastal and Air Pollution Events (GEO-CAPE) mission. Column aerosol optical thickness (AOT) data from MAIAC are compared against corresponding data from the Visible Infrared Imaging Radiometer Suite (VIIRS) instrument over North America during 2013. Product coverage and retrieval strategy, along with regional variations in AOT through comparison of both matched and un-matched seasonally gridded data are reviewed. MAIAC shows extended coverage over parts of the continent when compared to VIIRS, owing to its pixel selection process and ability to retrieve aerosol information over brighter surfaces. To estimate data accuracy, both products are compared with AERONET Level 2 measurements to determine the amount of error present and discover if there is any dependency on viewing geometry and/or surface characteristics. Results suggest that MAIAC performs well over this region with a relatively small bias of −0.01; however there is a tendency for greater negative biases over bright surfaces and at larger scattering angles. Additional analysis over an expanded area and longer time period are likely needed to determine a comprehensive assessment of the products capability over the Western Hemisphere.

1. Introduction

Aerosols are a key component of the Earth’s climate and environmental system due to their impact on the radiative budget of the planet and influence on air quality events [Ramanathan et al., 2001]. Information on the amount and composition of the aerosol particles suspended in the atmosphere is required to understand their role as both direct contaminantes and precursors to air pollution [Wang and Christopher, 2003; Al-Saadi et al., 2005]. The GEO-CAPE mission was recommended by the National Research Council’s 2007 Decadal Survey in order to provide multiple observations per day in support of the atmospheric composition and coastal biophysics disciplines [NRC, 2007]. Many current sensors dedicated toward atmospheric composition sit in Low Earth Orbit (LEO) and have only one daytime and one nighttime overpass for a given location when more frequent measurements are needed to fully monitor the emission of pollutants and their transport. A geostationary platform provides both the temporal and spatial resolution needed to understand the conditions and processes leading to poor air quality events and the necessary response [Lahotz et al., 2012].

Originally planned as a large satellite carrying multiple instruments, GEO-CAPE has shifted toward a phased implementation making use of available space on commercial geostationary satellites. This utilization of hosted payloads should help to reduce risk and costs, and has been supported by both science working groups [Fishman et al., 2012]. The atmospheric science working group is tasked with developing a strategy which allows for the observation of aerosols and trace gases for use in air quality studies. The MAIAC algorithm is the current candidate to provide information on aerosols from this geostationary satellite.

The MAIAC algorithm provides simultaneous retrievals of surface bidirectional reflectance distribution function (BRDF), bidirectional reflectance factor (BRF) commonly called surface reflectance, and AOT at 466 nm over clear sky and snow-free scenes using a time series of MODerate Imaging Spectroradiometer (MODIS) observations. This BRDF characterization over time for varying geometries is used, along with the spectral regression coefficient (SRC), to help the MAIAC algorithm retrieve AOT over bright surfaces with improved accuracy [Lyapustin et al., 2011].

Here this new generic algorithm is assessed through a comparison with the operational VIIRS aerosol algorithm which uses an atmospheric correction approach. VIIRS was chosen for this comparison due to the improvements over its predecessors in terms of resolution, pixel aggregation, and swath width. For instance, MODIS has a long history of providing aerosol retrievals with high accuracy, but it currently only produces AOT at a maximum resolution of 3 km, and has greater distortion at the swath edge when compared to VIIRS. The Multi-angle Imaging Spectroradiometer (MISR) uses nine fixed-angle cameras to view each location at a variety of viewing angle which allows it to also retrieve AOT over brighter surfaces; however its limited swath width (400 km) and coarse resolution (17.6 km) are prohibitive to its inclusion in this analysis. Ultimately, the sensor characteristics and availability of .75 km AOT retrievals make it ideal for a comparison with MAIAC. In this study, a years’ worth of AOT from both MAIAC and VIIRS over the North American continent is analyzed to look at differences in cloud screening, bias dependence and overall accuracy.

2. Data

2.1 MAIAC AOT

The MAIAC algorithm retrieves surface reflectance and AOT using MODIS L1B reflectances which have been gridded at a 1 km resolution. It utilizes a 4–16 day time series of clear MODIS scenes to retrieve BRDF and Spectral Regression Coefficients (SRC), which relates surface reflectance at 0.466μm and 2.13μm (MODIS bands 3 and 7) [Lyapustin et al., 2012]. Unlike MISR, which collects nearly-simultaneous observations of each pixel from various angles, the MAIAC algorithm uses consecutive overpasses from a single-look instrument like MODIS to acquire multi-angle sets of observations for each location. The use of a time-series of gridded MODIS observations also has the advantage of being able to simulate geostationary satellite observations, albeit with a significantly larger time difference between images. MAIAC relies on the assumption that surface reflectance changes rapidly in space but slowly in time, and therefore can be assumed constant over limited time scales. By contrast, the extent of clouds and aerosols can change greatly between MODIS overpasses.

The following is a brief overview of the MAIAC aerosol algorithm, a more detailed description of the MAIAC theoretical background and processing steps can be found in Lyapustin et al., (2011). Once the MODIS reflectance is gridded and split into both 600 × 600 km tiles and 25 × 25 km blocks, they are placed in a queue of 4–16 days. Water vapor is first derived from MODIS near-IR bands [Lyapustin et al., 2014] using a modification of the algorithm described in Gao and Kauffman (2003). An internal cloud mask uses spectral reflectance and brightness temperature tests similar to the operational MODIS cloud mask algorithm [Frey et al., 2008], along with the reference clear-sky image developed using a covariance based algorithm. Clouds can be detected since the spatial pattern of the surface often doesn’t change noticeably from day to day, while cloud residency is relatively short. Scenes are compared at both the block and pixel level against a clear-sky reference image built using the data queue [Lyapustin et al., 2008]. The BRDF is then retrieved at MODIS band 7 (2.1 μm) for clear pixels, followed by retrieval of SRC in MODIS band 3 (0.466um). This retrieval of SRC gives an assessment of surface BRDF (0.466um) at pixel level, which allows MAIAC to retrieve AOT at high 1km resolution.

The MAIAC algorithm provides AOT at 466 nm, however in order to compare directly with VIIRS, it must be converted to AOT at 550 nm. To do this, a set of ratios representing the spectral slope of a given AOT are used. These ratios, which are taken directly from the aerosol background model, are part of the MAIAC look-up tables [Lyapustin et al., 2011]. MODIS-based MAIAC aerosol products were produced over North America for the entire MODIS record up until July 2014. MAIAC is currently at version 1, and data used for this analysis was obtained from NASA on November 17, 2014.

2.2 VIIRS AOT

The Visible and Infrared Imaging Radiometer Suite (VIIRS) is a scanning radiometer carried on board the Suomi-NPP (National Polar-orbiting Partnership) satellite; a joint venture between NOAA and NASA meant to help transition to the Joint Polar Satellite System (JPSS), the next generation in U.S. polar-orbiting satellites. The operational VIIRS AOT product is produced by the Interface Data Processing System (IDPS), which takes raw instrument data from S-NPP and processes them into the Sensor Data Records (SDRs) that are used as inputs for the Environmental Data Records (EDRs), including AOT. The aerosol algorithm uses the dark-target approach to retrieve AOT. This method is built upon the legacy of retrieving aerosol properties from previous earth sensing satellite missions [Holben et al., 1992; Kaufman et al., 1997]. The algorithm is comprised of two distinct parts which are applied based on the surface type. Over ocean, the VIIRS algorithm is nearly identical to the MODIS ocean algorithm [Tanre et al., 1997], which uses a combination of fine and coarse mode aerosol models in attempt to replicate the top-of-atmosphere (TOA) reflectance. Over land, the VIIRS aerosol algorithm is based on the MODIS Atmospheric Correction algorithm for determining surface reflectance [Vermote and Kotchenova, 2008]. Aerosol information is retrieved by comparing the derived spectral surface reflectance ratios to prescribed ratios of those reflectances, and chooses the aerosol model and AOT that minimizes the residual. The VIIRS aerosol algorithm operates under the assumption of a Lambertian surface when retrieving the surface reflectance. An overview of the VIIRS sensor and an in-depth explanation of the scientific background and flow of the VIIRS aerosol algorithm are presented in Jackson et al., (2013).

The aerosol retrieval for both ocean and land is performed at the pixel resolution (750 m). This pixel level product is known as the Intermediate Product (IP) as it is used to create the aggregated AOT EDR, along with acting as an input for other VIIRS products. The VIIRS algorithm aggregates 8×8 arrays of IP AOT pixels into a single EDR pixel with a resolution of 6 km at nadir. At the IP level, the VIIRS Cloud Mask (VCM) and a series of internal checks are applied to the aerosol product, resulting in each pixel being given one of four quality designations. AOT is reported only for pixels in the two best quality levels (good and degraded) and therefore these are the only pixels included in the aggregation process, which also incorporates additional filtering and internal checks, producing a higher quality product.

A full year of VIIRS IP AOT spanning the time from February 1, 2013 to February 1, 2014 was used to compare against the MAIAC product. The selection of this time period was predicated by data availability and maturity. The VIIRS Aerosol algorithm has undergone multiple upgrades since launch to improve the accuracy and precision of its retrievals. One significant upgrade was a change to the spectral reflectance ratios used in the land inversion which took place in January 2013 [Hongqing et al., 2013]. This greatly reduced the bias in the aerosol products over land and allowed the product to reach ‘validated’ status. Because data prior to this change becoming operational are still considered ‘provisional’, they were not included in this analysis. Officially, the version of the product used in this study was given a maturity level of Validated Stage II in August 2014, meaning that it has been shown to meet the performance thresholds [NOAA-NESDIS, 2014] using a moderate set of test data. There are no such standards for the IP product; however it also meets the EDR requirements, making it suitable for quantitative analysis.

Other significant changes have occurred to the AOT product after the time period used in this study which had impacts on retrieval accuracy and to a lesser extent, spatial coverage. These include an improvement in snow screening, spatial homogeneity tests, and the removal of the ephemeral water test which often incorrectly screened out portions of heavy smoke plumes. Unfortunately due to the MAIAC data record ending in mid-2014, data containing these fixes were not included in this analysis.

2.3 AERONET

AERONET is a global network of ground-based, automatic sky-scanning spectral radiometers used to measure aerosol optical properties [Holben et al., 1998]. Developed and maintained by NASA, these weather resistant sun photometers are a vital source of information for aerosol research and the validation of satellite derived aerosol properties. The direct-sun measurements are used to compute the column AOT at a variety of wavelengths from 340–1020 nm, spanning a majority of the visible and Near-IR spectrum. Angstrom Exponent (AE) is also retrieved using wavelength pairs in the aforementioned range, along with the column water vapor. Level 2.0 AOT from AERONET sites in North America are used to compare against both the MAIAC and VIIRS AOT to determine accuracy and uncover any bias dependencies. Level 2 data has the highest quality assurance of all AEROENT data and is cloud-cleared and fully calibrated [Smirnov et al., 2000]. The “ground truth” AOT at the VIIRS and MAIAC wavelengths are computed using the AERONET AOT at 500 and 440 nm respectively, using the AE retrieved in the 440–675 nm range.

2.4 CALIPSO

The Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) is an active lidar instrument aboard the CALIPSO satellite. It provides vertically resolved information on clouds and aerosols using profiles of attenuated backscatter at 532 and 1064 nm at an along track resolution of 333 meters and a vertical resolution of 30 meters [Winker et al., 2009]. CALIOP is able to detect the number and extent of features such as aerosol or cloud layers using the backscatter profiles [Vaughan et al., 2004]. The level 2 products are produced at the nominal resolution of 333 m as well as 1 and 5 km by aggregating consecutive observations. For this study, the 1 km cloud layer products are used to verify the accuracy of the MAIAC and VIIRS cloud masks and determine if any issues related to cloud screening are influencing the analysis. A binary cloud mask is constructed from the ‘Number of Layers Found’ dataset, which simply gives the number of cloud layers found within that 1 km profile.

3. Results and Discussion

3.1 Daily gridding of VIIRS and MAIAC

Before assessing the MAIAC algorithm and how it compares to VIIRS, the datasets were gridded to directly compare their spatial extent and the quality of AOT retrievals. A grid was constructed with a 0.25° resolution in order to capture as much of the AOT spatial variability while limiting computational cost. The shaded domain outlined in Figure 1 shows the extent of the grid whose domain is limited by the MAIAC coverage over North America, which is largely confined to the Continental U.S. and Mexico. The result is a grid with dimensions of 256 × 116, or a total of 29,696 grid boxes.

Figure 1.

Map of the domain area used to grid the MAIAC and VIIRS AOT datasets. Domain was chosen based on the extent of MAIAC data currently available over North America. Coordinates of the upper left corner are (51° N, 129° W) and the lower right coordinates are (22° N, 65° W). Map data courtesy of Google Earth Pro (V 7.1.2.2041), Landsat.

In order to compare the best retrievals from both algorithms, a set of quality checks were applied during the gridding process. To start, data from both algorithms are restricted to the highest quality retrievals over land. To avoid any possible cloud leakage, the candidate pixel was required to be confidently clear and not be adjacent to a cloudy pixel in order to be used for gridding. Both MAIAC and VIIRS AOT have an associated geolocation file which gives the center coordinates of each pixel. The gridding process averages any valid pixels whose center lat/lon falls within the same grid box, and the number of observations included in that average is recorded. These daily gridded datasets were then averaged to look at statistics on the monthly to seasonal scale.

3.2 Direct Comparison

Once gridding of the data was completed, the datasets were directly compared through analysis of unpaired seasonal AOT and looking at the differences in retrieval numbers. Due to the ability of MAIAC to retrieve AOT over brighter surfaces, it was expected that it would have greater spatial coverage than the operational VIIRS product, particularly in areas of sparse vegetation.

3.2.1 Data coverage

Seasonal averages of AOT from MAIAC and VIIRS and the total number of retrievals per grid were analyzed in order to get a sense of the differences in coverage, and gain insight into the retrieval strategy and cloud screening of each algorithm. Figure 2 provides a look at the average of AOT (top) and number of retrievals per grid (bottom) per season for each dataset. MAIAC has greater coverage and more retrievals than VIIRS particularly across the western half of the CONUS. MAIAC coverage is nearly complete during the summer and fall seasons, save for some inland water bodies and regions such as Great Salt Flats (UT) and White Sands (NM), while VIIRS is not able to retrieve over the bright surfaces that make up a large portion of the western U.S. This disparity in coverage is seen across all seasons with the differences being greater during winter and spring due to seasonal phenology. There are some similarities however; for instance during winter when neither MAIAC or VIIRS retrieve enough to populate grids over the northernmost sections of the U.S. or the high altitude regions of the inter-mountain west. The reason for this is likely a combination of the solar zenith angle limits placed on good quality data and near-constant snow cover in these regions during the cold season.

Figure 2.

Maps of gridded AOT at 550 nm (top) and retrieval count (bottom) from VIIRS and MAIAC for: (a) winter; (b) spring; (c) summer; and (d) fall. Large portions of missing data in MAIAC maps over southern Canada are caused by the geographic extent of available data in this region.

In terms of actual AOT values, Figure 2c highlights some differences between MAIAC and VIIRS. While the spatial patterns are very similar between the two, VIIRS tends to retrieve slightly higher AOT over many regions. Over urban areas or mountainous terrain, this difference can be quite large and is noticeable in many seasons. In the springtime months, VIIRS AOT is also higher in the upper Mid-west and Great Lakes region where melting snow is likely contaminating the pixels leading to a poor retrieval. These anomalies associated with sub-pixel snow have since been addressed in the operational VIIRS algorithm.

Looking collectively at the results of this comparison, there are some features present in multiple seasons which emphasize the differences between the two algorithms and their pixel selection strategy. The underlying surface reflectance plays an important role in coverage of both datasets. MAIAC has shown the ability to retrieve AOT over the bright and soil dominated surfaces that are present across much of the western U.S., while VIIRS is only able to retrieve over darker or vegetated regions. This is also a problem in regions with high agricultural activity, such as the Lower Mississippi River Basin where fallow land prevents VIIRS from consistently retrieving AOT in all seasons besides the primary growing season (JJA). However surface reflectance alone cannot account for the differences in retrievals seen in many other parts of the US throughout the year.

3.2.2 Cloud Screening

In an effort to understand the difference in coverage and to determine how the cloud masks are performing, data from MAIAC and VIIRS were collocated with the CALIOP instrument aboard the CALIPSO satellite. First, the two cloud masks are converted to a binary mask with either a ‘clear’ or ‘cloudy’ designation. All datasets are subsetted to regions of overlap, after which the closest MAIAC/VIIRS pixel to the CALIOP profile is found using a modified version of the nearest neighbor approach utilized in similar comparison studies [Heidinger et al., 2012; Kopp et al., 2014]. Here we use a time window of 10 minutes centered on the CALIOP observation time in order to avoid cases where clouds detected by CALIOP have moved out of the MAIAC/VIIRS field of view. A maximum allowed distance of one pixel width is used to ensure that the closest pixel is indeed chosen, this is particularly necessary where the CALIOP profile passes from one tile/granule to the next. Collocation results between the cloud masks and CALOP detection were compared and are presented in Table 1 as a confusion matrix.

Table 1.

Confusion matrix showing the designation of pixels from each cloud mask associated with the two algorithms compared with information on clouds from CALIPSO lidar taken as the “truth” datasets.

		VIIRS		MAIAC
		Cloudy	Clear	Cloudy	Clear
CALIPSO	Cloudy	65079 (TP)	14479 (FN)	1055111	40781
	Clear	4129 (FP)	47298 (TN)	235293	605130
Accuracy		86%		86%

The abbreviations in parenthesis note the location of the following test outcomes for both sets of data: True Positive (TP); False Positive (FP); False Negative (FN); and True Negative (TN).

Our first observation from Table 1 is that a considerably higher number of collocations for MAIAC exist than for VIIRS. This is not only due to MAIAC’s increased retrieval numbers but the use of reflectance data from MODIS, which is part of the A-train constellation [Stephens et al., 2002] and shares a similar orbit and overpass time with CALIPSO. The VIIRS instrument flies at a slightly higher altitude and therefore has a different orbital track, the consequence of which is a ground track that only coincides closely with the A-train satellites once every few days.

To help determine the performance of each set of matchups we look at overall accuracy (Equation 1) along with two additional statistical measures: the True Positive Rate (TPR), and True Negative Rate (TNR) for which the formulas are given in Equations 2 and 3, respectively. The abbreviations used in these equations are noted next to their respective statistics in Table 1. A high TPR value indicates that the cloud mask is able to limit the number of false negatives (type II error), which lead to cloud leakage in the resulting product. Conversely, TNR is a measure of how good the cloud mask is at reducing the number of false positives (type I error); these false alarms can reduce the number of high quality retrievals and introduce sampling biases.

$$\begin{gathered} \mathit{Accuracy}=\frac{TP+TN}{TP+TN+FN+FP} \\ \mathit{TPR}=\frac{TP}{TP+FN} \\ \mathit{TNR}=\frac{TN}{TN+FP} \end{gathered}$$

Overall accuracy of the both the MAIAC cloud mask (MCM) and the VCM were found to be identical (Table 1), but while the overall accuracy for the two cloud masks may be comparable, the errors observed were dissimilar. The TPR and TNR metrics highlight the different types of errors associated with each cloud mask. For instance, TPR for the MCM during this period is 96%, meaning that less than 5% of cloudy pixels were incorrectly designated as clear, while the TNR for MAIAC is only 72%, leaving over a quarter of the clear pixels as determined by CALIOP out of the AOT processing chain due to the supposition they are cloudy. Monthly statistics for MAIAC show there is some seasonality to the TNR since it does not fall below 71% for much of the year except during summer (JJA) when it is in the 63%–66% range. The VCM displays a smaller difference between its error types with a TPR of 82% and a TNR of 92%, and a more limited seasonal dependence. These results show that VIIRS is able to strike a better balance between the Type I and Type II errors, while MAIAC’s strength is its ability to greatly reduce false negatives in the AOT record, thereby reducing bias.

In terms of these Type I errors, since the MCM operates at both the block and pixel level, it is possible that diurnal convection produces sufficient cloud cover to cause the covariance between that block and the clear-sky reference image to decrease to the point that it is deemed cloudy. Likewise, cumulus cloud fields common over land during this season may be enough to trigger a cloudy designation for that pixel from MAIAC, while the very narrow field of view of the CALIOP sensor may pass between these small clouds leading to a conflicting collocation. Such instances of small clouds and sub-pixel clouds pose problems for all types of cloud masks produced by passive sensors.

Seasonal statistics (Fig. 2) showed that MAIAC has a significantly greater number of high quality retrievals than VIIRS in many U.S. regions, even those where the surface is not bright enough to keep the algorithm from performing the retrieval. This would imply that either MAIAC is opting to retrieve AOT in unfavorable conditions (presence of clouds/snow, etc.) or that VIIRS is failing to retrieve at a high quality over these areas. The results of the matchups with CALIPSO seem to suggest the later, as the MCM is being conservative in determining which pixels are cloud-free. Therefore, cloud screening is not thought to be a substantial driver behind the differences in retrieval numbers; however other limits placed on AOT retrievals within the algorithms may be playing a part in the spatial coverage.

Some recent preliminary analysis by the VIIRS Aerosol team into gaps in AOT over the CONUS has shown that the most probable cause for the reduced number of high quality IP retrievals is the limited AOT range (0 to 2); and more precisely in this case, the lower bound of zero. Unlike VIIRS, which excludes the candidate pixel if the minimum residual corresponds to an AOT less than 0, MAIAC does not reject pixels whose surface reflectance falls below the expected value when computed with an AOD equal to 0. This happens on the occasion that the surface has changed significantly, or that the previous surface characterization is not correct. In the event this situation occurs, MAIAC reports an AOT of zero and then focuses on correcting the surface characterization with the next observation.

Large areas of missing AOT in VIIRS granules can be found in regions where the atmosphere is free of clouds or visible aerosols, meaning that the AOT is too small (negative) to be given a quality level high enough to be reported by the algorithm. This phenomenon is most prevalent in winter and spring when the AOT loading is small, and tends to be enhanced when the surface is sparsely vegetated and being viewed from the backscattering direction. In the recent VIIRS aerosol validation analysis performed by Huang et al., (2016) it was shown that VIIRS is often negatively biased during the period from late fall to early spring. Additionally, Liu et al., (2013) showed that VIIRS AOT tends to underestimate AOT when the surface is soil dominated. These two conclusions from previous validation studies support the notion that VIIRS has a tendency to retrieve more negative AOT when certain seasonal, geometric, and surface conditions are present, which can lead to relatively large areas with limited to no retrievals.

3.2.3 Collocated retrievals of AOT

As noted in the previous section, VIIRS and MAIAC tend to characterize the spatial patterns of seasonal AOT in similar ways. It also appears that MAIAC is generally a bit lower when compared to VIIRS, especially in the warm season. Observations collocated in time and space are needed to make sure that these two AOT products are being compared to one another under the same conditions. Therefore, the gridded data are filtered so that only days when both algorithms have enough retrievals to populate the grid cell are used in the analysis. Figure 3 presents the results of this collocation for the spring and summer seasons when the differences between the two are greatest. While there is better aggrement between MAIAC and VIIRS across much of the domain, the same trend of elevated AOT from VIIRS over the larger urban areas persists. Summer is the season with the highest disparity between the two algorithms, when a widespread difference between VIIRS and MAIAC is seen in the eastern half of the domain. In Figure 3d, this difference is shown to be predominately ±0.1 or less; however there are small isolated pockets of larger bias up to 0.5. In other seasons, there is little systematic disagreement between the two with the exception of some high AOT from VIIRS over Montana and the Dakotas during the spring season. This discrepancy between the two could be a result of cloud contamination, or differences in surface characterization.

Figure 3.

Set of four-panel plots showing matched VIIRS (upper left) and MAIAC (upper right) AOT along with number of days with coincident observations (lower left), and AOT difference between the products (lower right) for the spring (a) and summer (b) seasons.

Those areas where VIIRS is significantly higher than MAIAC are likely caused by the underlying surface since many of these anomalies are predominately located over heavily urbanized areas and mountainous terrain. There are also smaller differences which are not as persistent but cover larger areas. An example of this can be seen in the summer season where VIIRS AOT in the eastern half of the U.S. is ubiquitously higher than MAIAC. Aerosol type and concentration can be widely different based on region, and problems characterizing these differences may be caused by certain underlying aspects of the aerosol algorithms.

One such component of the algorithms that could be responsible for the regional contrast is the different aerosol models used to retrieve AOT. MAIAC uses a dynamic model where physical parameters can change based on the magnitude of AOT. Volumetric concentrations of the fine and coarse particles can also be varied, thereby allowing for a wider range of size parameter to be simulated. In addition, MAIAC uses a background aerosol model that is tuned regionally based on AERONET optical thickness measurements. As a global product, VIIRS on the other hand uses five predefined aerosol models which have bimodal size distributions and static volumetric concentration parameters for each of the models and both particle sizes. Although not related to the aerosol models themselves, VIIRS also uses a globally constant surface reflectance ratio to compare against the retrieved reflectance. This lack of accounting for such variations in surface type was discussed by Liu et al., (2013) as a potential source of regional bias in the AOT retrievals. In that analysis it was also found that VIIRS is biased high in the Eastern U.S. when compared to both AERONET and MODIS. Together, these differences in aerosol models and surface characterization are capable of producing the regional variations in AOT retrieved from MAIAC and VIIRS.

3.3 Validation of products

3.3.1 Comparison with AERONET AOT

In general, AOT from MAIAC and VIIRS compare well to one another, however there are differences and it is difficult to get a sense of which exhibits the higher level of accuracy without an ‘unbiased’ dataset to compare against. Measurements from AERONET sun photometers have been used for this purpose for many of the satellite derived aerosol products since the network’s inception [Chu et al., 2002; Kahn et al., 2005, Liu et al., 2013]. Most recently, in a manuscript by Huang et al., (2016) it was found using AERONET level 2 data that the VIIRS IP product has a global bias of 0.04. To determine the bias of the AOT produced by the two algorithms in question over our domain, we construct a set of matchups with AERONET level 2 data using the original datasets at their nominal resolution. Petrenko et al., (2012) outlined a system for subsetting data from spaceborne sensors based on the location of ground-based sensors such as AERONET. This same process of matching our datasets with AERONET is used here, where all good quality retrievals within 27.5 km of the AERONET site are selected. As part of the matchup criteria, at least 20% of the total number of possible pixels within this circle are needed along with a minimum of 4 AERONET measurements over the time period of one hour centered on the satellite overpass time are required. All pixels found to meet these requirements are averaged together, as are all ground measurement that fall in the time window.

Figure 4 shows the scatter plots constructed using the AERONET matchups with VIIRS and MAIAC. For all data matchups, VIIRS has a noticeable high bias which is pervasive at AOT < 0.04, and a moderate correlation of 0.64 with AERONET. However, a VIIRS positive bias of 0.043 compares well with the results of the global matchups presented in Huang et al., (2016). MAIAC on the other hand is highly correlated (0.82) with AERONET and exhibits only a slight negative bias when compared with AERONET. The greater number of MAIAC matchups is further evidence of its coverage and ability to retrieve over the brighter surfaces over which many AERONET stations in the western U.S. are located. In Figure 5, we highlight the dependence of the AOT bias on the magnitude of AOT by plotting the differences between VIIRS and AERONET at 25 AOT bins of increasing size. The typical error (median of all matchup errors) is often less than ±0.05 with the exception of the strong negative bias for both products during times of high aerosol loading, with MAIAC having slightly greater bias as AOT increases. The spread of VIIRS errors however is much greater than those for MAIAC as evidenced by the larger quartile ranges in most bins and the much higher maximum errors seen at low AOT.

Figure 4.

Scatter plots showing the relationship between AERONET AOT and VIIRS (a) and MAIAC (b). The dashed line represents the 1:1 line where the two datasets would be in complete agreement, while the solid lines represent the linear regression model (chi-squared test) provided at the top of each figure. Relevant relational statistics for correlation, r; number of observation, N; and bias are also given.

Figure 5.

Box and whisker plot showing the dependence of the VIIRS (blue) and MAIAC (red) bias on the AOT as measured by AERONET. Any missing data is due to the lack of matchups (< 5) in that AOT bin.

Aerosol type is also an important consideration when evaluating the AOT retrievals since the chosen aerosol model determines the spectral dependence of AOT. This spectral AOT can act as a proxy for particle size, and the Angstrom Exponent (AE) is often used to qualitatively describe this spectral dependence [Angstrom, 1929]. AE for coarse mode particles such as dust tend to be < 1, while finer particles produced from urban pollution or biomass burning are associated with AE values > 2 [Reid et al., 1999; Schuster et al., 2006]. AERONET provides AE for multiple wavelength pairs and can be used to determine if the retrieval errors from MAIAC or VIIRS are dependent on particle size. Figure 6 provides a look at how each algorithm performs across the range of particle sizes. The color coding of the individual matchups is based on the AOT retrieved by AERONET. There is evidence of the larger positive biases present and previously discussed in the VIIRS data which is limited to low-to-moderate loading of finer particles. MAIAC meanwhile has very limited bias and dependence on particle size as shown by the regression line. MAIAC however does have some issues retrieving accurately during high aerosol loading of coarse or mixed particle sizes (AE between 0.5 and 1.75). Figure 6 also reaffirms the results portrayed in Figure 4, however it shows that the larger biases tend to occur when the aerosol particle size is large, or when the concentration of coarse and fine particles is mixed. Both algorithms appear to perform quite well during cases of smoke or urban pollution.

Figure 6.

AOT errors from the (a) VIIRS, and (b) MAIAC matchups as a function of AERONET Angstrom Exponent, with regression line drawn in black. Data points are color-coded based on the AERONET AOT retrieval associated with those matchups.

While not analyzed directly here for reasons stated in Section 1, the demonstrated performance of the MODIS aerosol product is useful for providing extra context. A study from 2013 by Levy et al. details the performance of the MODIS Collection 6 algorithm and specifically section 4.4 outlines the MODIS Dark Target (DT) algorithm. Results for MAIAC shown here in Figures 4 & 5 compare well with the MODIS algorithm (Figure 11 in Levy et al.) over land with similar levels of accuracy and precision. It is important to note however that the Levy et al. study used global DT data, whereas MAIAC retrieves over both dark and bright surfaces and is constrained to the CONUS region in our analysis.

3.3.2 Dependence of AOT on Viewing Geometry and Surface Reflectance

In an attempt to ascertain which conditions might cause biases in the AOT retrievals, we look at how they are impacted by changing viewing geometry and surface brightness. Only data points where both VIIRS and MAIAC are matched with AERONET observations are used for this purpose, resulting in a dataset of 1034 matchups. Viewing geometry dependence is determined using the following 3 parameters: viewing zenith angle; relative azimuth angle; and scattering angle. The AOT biases are separated into bins using 5 degree increments and plotted as a function of increasing angle. The results are shown in panels a, b, and c of Figure 7.

Figure 7.

Dependence of AOT bias on: (a) viewing zenith angle; (b) relative azimuth angle; (c) scattering angle; and (d) surface reflectance at 555 nm according to MAIAC. VIIRS data is shown in blue and MAIAC in red, while the horizontal zero line (gray) is added for reference.

In terms of viewing angle, both algorithms produce matches that are well distributed across the range of angles with VIIRS having greater range as a result of the increased swath width over MODIS. MAIAC has very little viewing angle dependence, and has a minimal amount of a negative bias. VIIRS has some viewing angle dependence with positive biases at-nadir that approach zero for larger VZA. The number of matchups are not as uniform for RAA, as both MAIAC and VIIRS have a bimodal distribution of angles with limited number of matches near 90°. MAIAC has some small dependence on RAA but biases are generally low except for the 80–110° range and near the extremes of 0° and 180° where matchups are very scarce. VIIRS AOT starts out with positive bias where strong back-scattering is occurring (RAA < 50°) with little dependency, however bias increases dramatically as the relative azimuth angle approaches 180°. It is worth noting that a limited amount of VIIRS matchups are available at RAA > 140°, which is a range with both high bias and variability. Both algorithms have some bias dependence on scattering angle. MAIAC biases are within 0.02 of the zero line for smaller scattering angles, but the negative bias continues to gets larger once SCA surpasses 140°. VIIRS also has a small negative bias which then becomes positive as scattering angle increases.

Figure 7d shows the dependence of the two algorithms in terms of the MAIAC surface reflectance which is binned at intervals of 0.005. Minimal errors are observed for both datasets over dark surfaces up to a reflectance of 0.06, after which the algorithms start slowly trending in different directions. The error becomes larger for VIIRS once the surface reflectance reaches 0.12, while MAIAC dependence on surface reflectance reverses after this point. The brighter surfaces also appear to cause increased fluctuation in bias for both of the algorithms.

As noted previously, there is some dependence on sun-sensor geometry for both of the algorithms analyzed here. Notably, there is a large difference in the level of dependence between retrievals in the back-scattering direction (RAA < 90°) and the forward-scattering direction (RAA > 90°) for VIIRS. The two algorithms also drift away from the zero line in opposite directions for scattering angles greater than 100°. Due to the anisotropy of surface reflectance for many land targets, this change in viewing direction can lead to changes in the perceptible brightness of the surface, a phenomenon known as directional scattering. This effect causes an apparent brightening of the surface when viewed from in the back-scattering direction, and some dimming in the forward-scattering direction [Roujean, 1992]. MAIAC, through its use of the BRDF when retrieving AOT, attempts to account for and mitigate these effects. Based on the results in Figure 7d, it appears as though it is able to remove much of this dependence; VIIRS meanwhile, because of the assumption of a Lambertian surface, produces AOT with higher biases.

To see how each algorithm handles these changes the matchups for surface reflectance have been further stratified based on the scattering direction (using RAA of 90° as a separator). The resulting biases and histograms for both directions are given in Figure 8. VIIRS dependencies are similar regardless of the scattering direction, although errors are markedly higher in the forward-scattering direction for brighter surfaces. On the other hand, the dependency for MAIAC does look quite different depending on the scattering direction. MAIAC errors are near zero over dark surfaces in the back-scattering direction, yet quickly become negative as the surface gets brighter. In the forward-scattering direction, a rather consistent negative bias around −0.05 is found until surface reflectance surpasses 0.12, when it becomes more varied. Comparing these two panels to Figure 6d, we see that the back-scattering retrievals tend to dominate the overall signal due to nearly two-thirds of the retrieval matchups falling within this relative azimuth range; with the only exception being the bright surfaces where MAIAC has few valid retrievals. The histograms also show that MAIAC has some offset in the surface reflectance of its retrievals in both directions when compared to VIIRS. This is likely a result of including the BRDF in its retrieval strategy which accounts for the effects of sun and satellite geometry thereby reducing the brightness in the backward direction and increasing it in the forward direction.

Figure 8.

Dependencies on surface brightness split into observations taken from the forward-scattering (left) and back-scattering (right) direction. Bias is on the left-hand vertical axis and represented by the vertical lines, while the number of matchups in each reflectance bin are given by the vertical bars and occupy the right-hand vertical axis.

3.3.3 Sources of Bias

Matchups of MAIAC and VIIRS with AERONET data in the U.S. and surrounding areas have shown that biases are present that are angular dependent. MAIAC dependencies are less pronounced than VIIRS, but a negative association with geometric surface attributes does exist. Lyapustin et al., (2011) showed that SRC does vary slightly with viewing geometry, and that the use of an average SRC value will cause the algorithm to overestimate surface reflectance in the forward direction and vise-versa for back-scattering geometries. This reduced brightening in the backward direction and increase in the surface reflectance in the forward scattering direction relative to VIIRS is evident in the histogram offsets seen in Figure 8.

The consequence of this would be an underestimated AOT in the forward-scattering direction, and overestimation in the back-scattering direction, however we only find a consistent negative bias in the forward direction. In the back-scattering direction, the surface tends to be brighter due to reduced shadowing and lower aerosol backscattering compared to the forward-scattering direction. This can cause the sensitivity of the TOA reflectance to AOT to decrease, leading to higher uncertainty of AOT in the back-scattering direction. This combined with the limited amount of MAIAC matchups with a high surface reflectance in the back-scattering direction are likely leading to the larger, variable errors over bright surfaces.

Previous global validation studies have focused on VIIRS Aerosol products [Liu et al., 2013 (EDR only); Huang et al., 2016 (EDR and IP)] and have shown that a slight positive bias is observed in AOT over land. As is shown in this analysis, Liu et al., (2013) also found a similar dependence in the EDR data in relationship to viewing zenith angle over land as is shown in this paper, although errors were found to be larger in this case. This is not surprising as more noise is expected in the pixel-level IP AOT data, which does not have the benefit of aggregation and further screening. Even with that in mind, the level of bias seen in this study for VIIRS products is concerning since data at this product level is useful to the air quality community who require highly accurate data for their applications. Therefore, a brief attempt was made to uncover additional sources of bias to those already established by previous studies.

Recall from section 3.2 that urban ‘hotspots’ of AOT were consistently present over medium to large cities across the U.S. in all seasons (more so in warm seasons). A fair amount of AERONET sites that are not surrounded by bright or soil-dominated surfaces in the U.S. are located in or near these urban areas, meaning that some of the bias may be attributed to these sites. In fact, of those matchups which exhibit excessive positive bias (> 0.1), 65% of them are associated with a handful of sites located in Los Angeles or Houston, two large and highly urbanized cities. Over 85% of the highly biased matchups (20% of all matchups) originate from AERONET sites located in a major metro area. When looking at viewing geometry values where large biases are seen, we notice a considerable number of those AERONET sites also being in select urban areas, while sites with lower biases tend to be more random. This suggests that a sizable portion of the large biases and dependencies on viewing geometry in this domain may be due to a lack of accuracy over urban areas and that viewing geometry is an intensifier of those biases.

4. AOT case studies

Up to this point, the geographic inspection of the AOT products from MAIAC and VIIRS have been contained to seasonally gridded AOT. In an attempt to observe and verify some of the findings from the bias analysis, a look at individual cases at the products’ native resolution are presented below. This allows for qualitative comparison of the two products independent of the AERONET matchups which, with respect to VIIRS, were found to be heavily influenced by an urban bias. Two cases; one with a large area of smoke present over the northwestern U.S. and a more typical late-summer AOT case in the eastern half of the country were chosen. Careful attention was paid to make certain that the Aqua and Suomi-NPP overpass times for the selected date were close together (<20 min) so valid spatial comparisons could be made.

4.1 High AOT case

In 2013, a few large historical wildfires took place in North America with one such fire being the Rim fire, which started on August 17th near Yosemite National Park and burned for over two months. Figure 9 shows a VIIRS true-color image over the Western U.S. from August 25^th along with AOT from VIIRS and MAIAC. The two products agree well over regions where both have retrieved AOT, however differences do exist. VIIRS IP AOT is higher over the thickest parts of the smoke plume and is noisier, however this is expected since it is a pixel-level product while the MAIAC AOT has the advantage of using gridded MODIS reflectance, and much of the information used to perform the retrieval is supplied from processing at the block-level.

Figure 9.

Example of high aerosol loading on August 25^th, 2013 over the western U.S. due to regional fires. (a) True-color image from S-NPP VIIRS; (b) VIIRS high quality IP AOT; (c) MAIAC AOT.

Just as the analysis in section 3 showed, VIIRS coverage over brighter surfaces is limited compared to MAIAC, as large sections of Montana, Idaho and Wyoming lack any high quality retrievals. However VIIRS does retrieve more of the smoke in northern Idaho. The missing MAIAC retrievals in the far upper right section of the image are a result of it being outside MAIAC’s North American processing domain. There are also smaller rectangular holes in the MAIAC data near the center of the image which are a product of the block-level SRC retrieval that takes place within the aerosol retrieval. In some cases, SRC may not be retrieved or updated due to cloudiness. This causes AOT to not be retrieved over the brighter surfaces within that block (25 km × 25km).

4.2 Moderate AOT case

Given that strong AOT bias dependencies exist in both the viewing geometry and AOT itself, a second case representing a more moderate aerosol loading scenario was investigated. Figure 10 includes the true-color image and AOT maps from VIIRS and MAIAC on Sept. 5^th, 2013 over the Mid-western and Mid-Atlantic states. In contrast to the previous example, the spatial coverage of VIIRS is much closer to MAIAC in this case due to a majority of the surface being dark. The exception here is over inland water bodies such as the Great Lakes where VIIRS currently does not retrieve AOT. Once again, the two products characterize the spatial variation in AOT in similar ways. Over much of the Ohio River valley, where an area of haze exists, the two algorithms produce results that are very alike, although VIIRS AOT is slightly higher in the vicinity of clouds in northern Illinois. VIIRS is also significantly higher over the Chicago and St. Louis urban areas which are circled in black, lending credibility to the theory that VIIRS is often biased high over cities. No AERONET sites are located in Chicago, but one is located in downtown St. Louis, where data shows that VIIRS is biased high by 0.05 while MAIAC has a bias of −0.11. There are also areas where VIIRS is retrieving slightly higher AOT in a more uniform manner. The clearest example of this is in the Mid-Atlantic where VIIRS is retrieving AOTs which are around 0.05 higher than MAIAC. A similar pattern is also visible over a region stretching from Lake Michigan into Ohio and Pennsylvania.

Figure 10.

Image of a moderate AOT case from September 5, 2013. (a) True-color image from S-NPP VIIRS; (b) VIIRS high quality IP AOT; (c) MAIAC AOT.

5. Conclusions

This study was undertaken to assess the utility of the MAIAC algorithm for retrieving aerosol information from a passive satellite sensor through a comparison with the aerosol products from VIIRS and ground-based sun photometers. With these data sets as benchmarks, we were able to evaluate the spatial coverage and accuracy of the MAIAC AOT product. Using data gridded to 0.25 degrees, we found that MAIAC is capable of providing retrievals over a varied set of surface types, including the bright and soil dominated surfaces which restrict the coverage of the common dark-target only algorithms (VIIRS, MODIS). The number of valid high-quality retrievals MAIAC produces is also greater, leading us to evaluate the cloud mask performance of both algorithms through matchups with CALIOP. Those matchups showed that both MAIAC and VIIRS had similar accuracy, however we found MAIAC to be more conservative in its assignment of clear-sky pixels. When compared directly with VIIRS, MAIAC produces AOT values that on average are 0.017 lower than VIIRS during 2013. There is large seasonality however, with minor differences for winter and fall, and larger separation seen in the summer season.

In order to conduct a more robust accuracy assessment including the dependence of the algorithms on viewing geometry and surface reflectance, both datasets were also evaluated against AERONET Level 2 AOT. MAIAC showed little dependence on viewing zenith, however there was some negative association with the scattering angle and the brightness of the surface. VIIRS showed negative association with viewing angle, but was positive with scattering angle and surface reflectance. Biases as a function of surface reflectance were further stratified based on scattering direction because of the differences in errors seen with both products. Trends in VIIRS bias as a function of surface reflectance were not greatly affected by scattering direction, although overall errors were larger in the forward-scattering direction. Analysis of MAIAC showed that it only has strong dependence on surface reflectance when the surface is viewed in the back-scattering direction.

The results of this bias analysis coincided well with the initial investigations of the MAIAC algorithm. The results after studying the VIIRS biases with respect to scattering direction however were not consistent with previous validation studies; therefore a closer look was taken at those highly biased matchups. It was found that urban backgrounds may be causing, or at least intensifying the positive bias seen in VIIRS AOT. Overall, the MAIAC algorithm has shown the ability to perform well over the North American region with a high level of accuracy given its spatial resolution. Global analysis over a longer time period will be needed to make certain that the product(s) are robust and meet the levels of accuracy needed for aerosol monitoring.

Key Points.

• ■MAIAC algorithm is evaluated for use in future satellite missions to derive information on aerosol properties
• ■MAIAC’s use of time-series observations allow it to derive BRF which in turn improves cloud masking and aerosol-surface retrievals.
• ■Comparison with AOT from VIIRS and AERONET show that MAIAC exhibits low bias over North America with high spatial coverage