MODIS Collection 6 shortwave-derived cloud phase classification algorithm and comparisons with CALIOP

Abstract

Cloud thermodynamic phase (ice, liquid, undetermined) classification is an important first step for cloud retrievals from passive sensors such as MODIS (Moderate-Resolution Imaging Spectroradiometer). Because ice and liquid phase clouds have very different scattering and absorbing properties, an incorrect cloud phase decision can lead to substantial errors in the cloud optical and microphysical property products such as cloud optical thickness or effective particle radius. Furthermore, it is well established that ice and liquid clouds have different impacts on the Earth’s energy budget and hydrological cycle, thus accurately monitoring the spatial and temporal distribution of these clouds is of continued importance. For MODIS Collection 6 (C6), the shortwave-derived cloud thermodynamic phase algorithm used by the optical and microphysical property retrievals has been completely rewritten to improve the phase discrimination skill for a variety of cloudy scenes (e.g., thin/thick clouds, over ocean/land/desert/snow/ice surface, etc). To evaluate the performance of the C6 cloud phase algorithm, extensive granule-level and global comparisons have been conducted against the heritage C5 algorithm and CALIOP. A wholesale improvement is seen for C6 compared to C5.

1. Introduction

In addition to cloud height, thickness, and microphysics (e.g., size distribution), thermodynamic phase (i.e., ice, liquid, mixed) is an important determinant of the role of clouds in the Earth’s radiation budget, weather, and hydrological cycle [Liou, 1986; Ramanathan et al., 1989, 2001; Chahine et al. 1992; Wielicki et al., 1995]. Moreover, correctly determining the phase of a cloudy field of view is a critical initial step for remote sensing retrievals of cloud properties such as optical thickness (COT), effective particle radius (CER), and water path. Because ice and liquid phase clouds have substantially different scattering and absorption properties, an incorrect phase decision can lead to significant errors in remotely retrieved cloud properties. For those reasons several cloud phase classification algorithms have been developed and continue to be improved for several instruments such as AVHRR [Key and Intrieri, 2000], CALIOP [Hu et al., 2009], POLDER [Goloub et al., 2000; Riedi et al., 2010], AIRS [Jin and Nasiri, 2014] and MODIS [Platnick et al., 2003; Baum et al., 2012]. Each of these algorithms is designed to take advantage of the given instrument’s features; here we introduce the new cloud phase algorithm developed for MODIS Collection 6 (C6)

The Moderate-Resolution Imaging Spectroradiometer (MODIS), launched on the Earth Observing System (EOS) Terra and Aqua platforms in 1999 and 2002, respectively, is a key instrument for atmospheric, land, and ocean remote-sensing science [Justice et al., 1998; King et al., 2003; Platnick et al., 2003]. MODIS measures reflected and emitted radiation at 36 spectral channels from the visible to the infrared, with a 1 km spatial resolution at nadir, and provides pixel-level retrievals of numerous geophysical parameters in its Level-2 products. Of particular interest here is the cloud optical and microphysical property product [Platnick et al., 2003], designated MOD06 and MYD06 for Terra and Aqua, respectively (for simplicity, the Terra and Aqua products will be referred to collectively with the identifier “MOD” since the retrieval algorithms are the same for each platform). The MOD06 product includes 1 km pixel-level cloud thermodynamic phase information derived from two approaches, namely an algorithm that exclusively uses infrared (IR) channels [Baum et al., 2000; Baum et al., 2012] whose results are reported for both daytime and nighttime (also available at 5 km resolution), and a daytime-only algorithm that uses a combination of visible (VIS), shortwave IR (SWIR), and IR channels.

The daytime-only algorithm (referred to hereafter as the MOD06 cloud optical property [COP] phase algorithm) that provides the phase decisions for the MOD06 cloud optical and microphysical property retrievals (e.g., COT, CER, cloud water path) has undergone an extensive overhaul in the latest MOD06 C6 reprocessing efforts. The primary motivation for the C6 changes was to overcome some well-known shortcomings in Collection 5 (C5). In particular, the C5 phase decision logic was somewhat opaque to end users, and because the algorithm relied on SWIR channel ratio thresholds specific to MODIS, was inadequate for achieving climate data record continuity from multiple passive sensors such as MODIS, VIIRS, and beyond. In addition, the algorithm underperformed in certain situations, such as broken liquid cloud scenes that were often misidentified as ice and thin ice cloud edges that were often misidentified as liquid. Because the cloud phase decision determines the processing path (i.e., ice or liquid) of the MOD06 retrievals, an incorrect cloud phase classification can introduce substantial errors in the final Level-2 COT, CER and water path products. Furthermore, these errors can impact the global Level-3 product (MOD08) by introducing biases into the grid-level, phase segregated cloud property populations (e.g., ice and liquid phase fractions) and derived statistics.

With these shortcomings in mind, the design goals for the new C6 MOD06 COP phase algorithm were to create a more universal phase algorithm applicable to multiple sensors and to minimize cloud phase decision errors. Algorithm development relied heavily on collocated observations from CALIOP (Cloud-Aerosol Lidar with Orthogonal Polarization) onboard CALIPSO [Winker et al., 2009], and a thorough assessment was performed using CALIOP as the benchmark. Notable changes include a complete restructuring of the phase decision logic, though some C5 tests were retained for C6, in addition to removal of the bulk of the SWIR ratio threshold tests in favor of assessments of ice and liquid phase spectral CER retrievals that inherently account for instrument differences (e.g., spectral channel selection and response functions, etc.). Here, a detailed description of the C6 MOD06 COP phase algorithm is provided, including changes and enhancements with respect to C5. The C6 phase algorithm compares quite well with CALIOP for scenes in which CALIOP observes only one cloud phase. Furthermore, the C6 algorithm is shown to provide a significant performance improvement over C5 for all surface types.

2. Data

The active lidar observations from CALIOP provide an excellent benchmark for developing and evaluating the C6 MOD06 COP phase algorithm. This study uses the CALIOP cloud phase discrimination [Hu et al., 2009] reported in the 1 km and 5 km cloud layer products for two selected months (July 2008 and November 2012). First the CALIOP 1 km layer products are collocated with MODIS by finding the MODIS pixel with the minimum great circle distance with respect to each CALIOP profile. Because some optically thin clouds such as cirrus require lidar horizontal averaging scales longer than 1 km for detection and are only reported in the CALIOP 5 km layer products, the 5 km layer products are also collocated with MODIS by over-sampling the 5 km profiles to 1 km resolution and concatenating with the 1 km layer products. Thus a complete CALIOP phase dataset is created to screen for single-phase ice or liquid profiles only. The importance of this merged dataset is illustrated in Figure 1. Here the CALIOP 1 km (b) and 5 km (d) layer cloud phase, with dark and light blue denoting liquid and ice phases, respectively, is plotted for an example Aqua MODIS granule observed on 3 July 2008 at 0830 UTC (a). Also shown in Fig. 1b,d is a horizontal bar near 20 km altitude indicating the collocated MOD06 C6 cloud phase classification (c). It is evident here that the CALIOP 1 and 5 km cloud layer sampling can be quite different, with more low-altitude broken liquid clouds found in the 1 km layer product and more high-altitude ice clouds found in the 5 km layer product. Note the CALIOP 333m layer products were also evaluated, though only minor differences were found with respect to the 1 km products. Consequently, the 333m layer products are excluded from this investigation.

Figure 1.

Aqua MODIS granule (3 July 2008, 0830 UTC) with the corresponding RGB image (a) and the MODIS C6 cloud phase classification (c), selected to illustrate the collocation between MODIS and CALIOP 1km (b) and 5km (d) cloud layer products.

3. Algorithm Description

The C5 MOD06 COP phase algorithm employed a decision tree logic that was in practice difficult to improve and did not utilize information from all phase tests due to its sequential design [King et al., 2006]. The algorithm was therefore redesigned for C6 to use a simple voting methodology that takes into account all available phase information, with phase test thresholds optimized via evaluation with the collocated CALIOP cloud products. A flowchart describing the C6 MOD06 COP phase algorithm voting logic is presented Figure 2. Note that a complete flowchart describing in detail the C6 MOD06 COP phase algorithm can be found in the MODIS C6 cloud optical properties User Guide [Platnick et al., 2014] and in the supplement attached to the current article.

Figure 2.

MODIS C6 cloud phase classification algorithm general logic flowchart.

For a given 1 km MODIS pixel, the COP cloud phase algorithm is only invoked if the pixel is classified as “cloudy” or “probably cloudy” by the MODIS cloud mask (MOD35), and if it has not also been identified as “not cloudy” by the Clear Sky Restoral (CSR) spatial variability [King et al., 2006; Platnick et al., 2014] and spectral behavior tests [Zhang and Platnick, 2011; Pincus et al., 2012]. The default phase is undetermined, and each phase test then provides a signed integer vote for liquid or ice phase (or no vote if the test is ambiguous), with the cumulative score determining the final cloud phase, i.e., negative for ice, positive for liquid, and zero for undetermined (note that if ice and liquid have the same number of votes the cumulative score is then zero). A final cloud top sanity check, based on cloud top temperature, IR cloud phase, and cloud top property retrieval method, is implemented for pixels that remain undetermined or are low confidence liquid phase (cumulative scores of zero or one, respectively). A description of the four primary phase tests of the C6 algorithm, shown in the flowchart, and their rationale follows. Note the tests now utilize both liquid and ice phase COT and CER retrievals.

3.1. Cloud Top Temperature Tests

An obvious first-order cloud phase test is the application of thresholds on the retrieved cloud top temperature (CTT), here the new 1 km CTT product that is included in MOD06 [Baum et al., 2012]. However, the MOD06 cloud top retrieval is known to lose sensitivity for optically thinner clouds, roughly below COT = 2 [Menzel et al., 2010]. Furthermore, for multilayer scenes, namely ice clouds overlying liquid clouds that are often difficult to identify with passive imager-based techniques, a simple CTT threshold test may yield undesirable phase results. For instance, the cloud top retrieval may give a relatively cold CTT (e.g., less than 240 K) for moderately thick cirrus overlying an optically thick liquid cloud, and thus result in an ice phase vote, even though the underlying liquid cloud may dominate the TOA reflectance in the solar channels; in such a case the more radiatively consistent result may instead be liquid phase. It is therefore important to exercise caution when determining cloud phase from CTT retrievals alone, and the CTT test was designed with these limitations in mind.

For optically thick warm clouds (i.e. liquid COT > 2 and CTT > 270 K), the CTT retrieval is considered to be of high confidence and the cloud phase is forced to liquid via an insurmountably large vote. This is analogous to the “warm sanity check” in the C5 algorithm. Conversely, for cold clouds (i.e., CTT < 240 K) the possibility of multi-layer (or mixed-phase) clouds precludes such confidence, and the test yields only a weak vote for ice phase. Optically thin warm clouds (COT < 2), or those clouds with a more ambiguous warm CTT retrieval (260 K < CTT < 270 K), yield weaker liquid phase votes. Completely ambiguous CTT retrievals (240 K < CTT < 260 K) yield no phase vote (i.e., undetermined).

3.2. Tri-Spectral IR Cloud Phase Test

As part of the MOD06 cloud top property retrieval algorithm, an IR-only cloud phase is also provided at 1 km and 5 km resolution. Previously a two-channel approach, for C6 this product was enhanced with the addition of a third IR channel [Baum et al., 2012], and uses emissivity ratios to infer cloud phase. While the bi-spectral IR cloud phase was used only as an initial guess in the C5 MOD06 COP phase algorithm, the so-called tri-spectral IR phase provides an independent vote in the C6 phase algorithm, albeit with a smaller weight since its results are strongly correlated with the retrieved CTT. Note in addition to ice, liquid, and undetermined designations, the tri-spectral IR phase can also return a mixed-phase designation, though only the ice and liquid designations provide votes here.

3.3. 1.38 μm Channel Test

To help identify optically thin cirrus as ice phase, a test based on the 1.38 μm channel is implemented in C6. An advantage of the 1.38 μm channel is its location within a strong water vapor absorption band; if the atmosphere contains a sufficient amount of water vapor, measured TOA reflectance at 1.38 μm is primarily from high altitude cirrus that lie above most of the water vapor, while low altitude liquid clouds and the surface only negligibly contribute [Gao et al., 1993]. The 1.38 μm test used in the COP cloud phase discrimination algorithm comes directly from the MODIS cloud mask product and is based on simple thresholds to separate thin cirrus from clear and low altitude clouds [Ackerman et al., 2010].

It should be noted that the skill of the 1.38 μm channel to discriminate ice and liquid clouds is strongly tied to the column water vapor amount and the retrieved COT. For example, in more arid atmospheres (such as in subsidence zones), though optically thin low altitude clouds are still expected to negligibly contribute to TOA 1.38 μm reflectance, optically thick low altitude liquid clouds may have a significant contribution. Thus applying the 1.38 μm test in all cases can lead to false ice cloud phase designations. Consequently, the 1.38 μm channel test is coupled with a retrieved ice phase COT threshold, and provides an ice phase vote only when retrieved COT is less than 2. Because the MOD06 COT retrievals use solar window channels, and can thus be considered total column retrievals, applying the 1.38 μm test only when COT is small adds confidence this test only votes ice phase for cirrus cases.

3.4. Spectral Cloud CER Tests

In C5, the primary COP cloud phase tests were a series of thresholds applied to SWIR reflectance ratios. The rationale for these tests is the fact that ice and liquid particles have different imaginary indexes of refraction at 1.6 and 2.1 μm [Kou et al., 1993], i.e., ice particles are more absorptive than liquid droplets at these wavelengths and thus have smaller TOA SWIR reflectances. Figure 3(a) shows a scatter plot of 2.1 μm (y-axis) versus 0.85 μm (x-axis) cloud reflectances over ocean, randomly sampled from the MODIS-CALIOP collocated dataset. The scatter point color indicates the collocated CALIOP cloud phase (ice phase in light blue and liquid phase in burgundy). The corresponding C5 SWIR ratio thresholds are plotted as dashed red lines, such that all points above the upper dashed red line are considered liquid and all points below the lower dashed red line are considered ice; points between the two lines are considered undetermined. It is evident the SWIR ratio approach allows a rough discrimination of ice and liquid phase clouds, though the non-linearity of cloud reflectances, due to their dependence on COT, view geometry, etc., render single linear thresholds inadequate.

Figure 3.

The MODIS C5 bidirectional reflectance thresholds (a) have been replaced by thresholds based on forced ice cloud effective radius (i.e., ice cloud effective radius retrieval is attempted for each cloudy pixel) retrieved at three separate wavelengths: 1.6, 2.1 and 3.7 μm. Example liquid (black) and ice (red) cloud retrieval look-up tables are shown in (b). Figures (c) and (d) show the forced liquid and ice 2.1 μm cloud effective radius histograms, respectively, from the MODIS-CALIOP collocated dataset, color coded by CALIOP-derived phase.

Alternatively, the SWIR ratio tests have been replaced in the C6 COP phase algorithm by thresholds on ice and liquid phase spectral CER retrievals (i.e., at 1.6, 2.1, and 3.7 μm) that inherently account for COT and view geometry (among other) dependencies. The rationale for this change is that it is more appropriate to define single linear thresholds in CER space than in reflectance space. Figure 3(b) shows example ice (red dashed line) and liquid (black dashed line) MOD06 COT-CER look-up tables (LUTs) for a given viewing geometry. Note the C5 ice crystal model that assumed a mixture of crystal shapes has been replaced in C6 by a single-habit severely roughened aggregate column model [Yang et al., 2013] that provides better spectral consistency between MODIS solar- and IR-based COT retrievals as well as those from CALIOP [Holz et al., 2015]. Figures 3(c) and (d) show histograms of forced liquid and ice phase 2.1 μm CER retrievals along the CALIPSO track, respectively, segregated by collocated CALIOP phase (ice phase in light blue and liquid phase in burgundy). It is evident that the distribution of forced ice phase CER retrievals for those pixels identified as ice by CALIOP is quite different from that of the pixels identified as liquid; the forced liquid phase CER histograms are more ambiguous. Note, however, that including information about failed retrievals, i.e., from the new Retrieval Failure Metric (RFM) introduced in C6 MOD06, can reduce the ambiguity in the liquid phase CER histograms in Figure 3(c), though during development of the phase algorithm this information was not yet available. Similar results are found for the 1.6 and 3.7 μm CER retrieval histograms (not shown), though the 3.7 μm distributions are offset towards smaller CER compared to the 1.6 and 2.1μm distributions. Thus it is possible to define simple CER thresholds to discriminate ice and liquid phase clouds; an example is shown by the dashed red lines in Figure 3(d). The C6 spectral CER thresholds were derived via extensive evaluation along the CALIPSO track with the collocated CALIOP cloud layer products, and are summarized in Table 1.

Table 1.

Forced ice cloud effective radius based thresholds (using the severely-roughened compact aggregated columns ice crystal model) derived from the MODIS-CALIOP collocated dataset (Re < Min. liquid; Re > Max. ice; Max. > Re > Min. undetermined).

Forced Ice Re Thresholds	Minimum	Maximum
Re 1.6 micron	20 micron	30 micron
Re 2.1 micron	20 micron	30 micron
Re 3.7 micron	15 micron	25 micron

An important caveat is the fact that not every cloudy pixel will yield successful ice phase CER retrievals. Failed CER retrievals nevertheless retain phase information, specifically in the location of the measured SWIR reflectance with respect to the ice phase LUT. For instance, referring to Figure 3(b), a cloudy pixel lying above the ice phase LUT (point P1) implies liquid phase, and a pixel lying below the LUT (point P2) implies ice phase. For C6, this information for pixels outside the LUT solution space is now available via a new alternate COT-CER retrieval solution logic that provides the COT and CER of the LUT grid point closest to the reflectance observations, as well as a measure of the relative distance to the LUT (note these parameters are reported for the final solution phase in the RFM SDS). Thus for pixels for which any ice phase spectral CER retrieval fails, the C6 COP phase algorithm instead uses the nearest LUT CER information from the alternate solution logic. Note also that, because Aqua MODIS has non-functioning detectors at 1.6 μm, the 2.1 μm CER test is used as a proxy when 1.6 μm is not available, and therefore votes twice in such instances.

Finally, there are two distinct disadvantages to using spectral CER retrievals in the phase logic. First, computational efficiency is greatly reduced since it is necessary to perform two CER retrievals, i.e., both ice and liquid phase, for each of the three COT-CER spectral combinations (VNSWIR-1.6, −2.1, −3.7 μm), thus six independent retrievals for each cloudy pixel. Second, the ice CER thresholds depend on the assumed ice crystal model used in the forward radiative transfer simulations. Therefore changes in the ice model assumption may in turn require changes in the CER thresholds.

5. Algorithm Evaluation

To evaluate the performance of the C6 MOD06 COP phase algorithm, extensive comparisons have been carried out against the heritage C5 MOD06 algorithm, as well as collocated phase retrievals from the CALIOP v3 cloud layer products. In this section, we will first discuss the main differences between C5 and C6 cloud phase results at a granule and global level. We will then discuss the CALIOP and MODIS cloud phase comparison results for a variety of surface types and cloud optical thicknesses, i.e., opaque and non-opaque clouds as determined by CALIOP.

5.1. Evaluation Against C5

A comparison of cloud phase results from the C5 and C6 algorithms is shown in Figure 4 for a selected Aqua MODIS granule observed on 7 August 2007 at 2010 UTC. Panel (a) shows the true color RGB image (0.66, 0.55, 0.47 μm) for this granule. The scene is mainly covered by broken marine boundary layer clouds and what appears to be cirrus on the left. Panel (b) shows the 1 km cloud top temperature retrievals, and panels (c) and (d) show the C5 and C6 cloud phase classification. Note the gray regions within the granule in (b), (c), and (d) correspond to clear sky pixels. Immediately visible here is the increased number of cloud phase pixels in C6 compared to C5. This increase does not represent changes to the MOD35 cloud mask, but is instead a result of the inclusion in C6 MOD06 of pixels identified by the CSR algorithm as either cloud edges or partly cloudy (collectively referred to as PCL pixels) that are presumably inhomogeneous and were previously discarded in C5.

Figure 4.

Example Aqua MODIS granule (7 August 2007, 2010 UTC) with the corresponding RGB image (a), the C6 1 km cloud top temperature (b), and the cloud phase classification for C5 (c) and C6 (d), respectively. Note that for C6 the cloud phase is now reported for partially cloudy pixels leading to an increase of liquid cloud pixels, in particular for the broken cloud area.

A research-level version of the C5 phase algorithm has been run on the PCL pixel population, and results indicate a large amount of the marine boundary layer clouds are misclassified as ice phase (not shown). Broken liquid clouds such as those shown in Figure 5 can be challenging for cloud phase classification for multiple reasons. For example, as can be seen in Figure 5(b), the CTT of broken clouds, particularly at higher latitudes, is often lower than the 270 K liquid phase threshold used in the C5 algorithm. Furthermore, inhomogeneous broken clouds have been shown to be associated with a high CER retrieval failure rate [Zhang and Platnick, 2011; Cho et al., 2015], thus relying heavily on CER tests for phase determination can be problematic. Consequently, an extensive granule-level analysis was used to optimize the vote weights and CTT thresholds in the C6 COP phase algorithm to increase the classification skill for these clouds. These modifications helped to improve the cloud phase classification, as the additional, likely inhomogeneous, PCL pixels in the broken boundary layer cloud field in Figure 5(d) are correctly classified as liquid. Finally, also note that C6 undetermined cloud phase (red color) is mainly reported in the transition between ice and liquid clouds, as we can expect in this ambiguous cloud phase area where multi layer clouds might be found.

Figure 5.

Same as Figure 4, except for an Aqua MODIS granule on 15 January 2008 (1435 UTC). Note here the improvement of ice cloud edge classification over desert surface.

Cloud phase classification improvement can also be observed for C6 compared to C5 at the edge of cirrus clouds, especially over desert surfaces, as is shown by the Aqua MODIS granule (15 January 2008, 1435 UTC) in Figure 5. The RGB in Figure 5(a) indicates a cirrus cloud deck extending from the tropical eastern Atlantic over the western Sahara. The corresponding MOD06 1 km CTT retrievals are shown in Figure 5(b), confirming the clouds are at high altitudes. It is evident in Figure 5(c) that the edges of the cirrus over the desert in this granule were misclassified in C5 as liquid phase clouds; this misclassification is greatly reduced for C6, shown in Figure 5(d).

The granule-level differences between C5 and C6 observed in Figures 4 and 5 can also be observed in global statistical aggregations. As an example, Figure 6 shows MODIS C6 monthly liquid (a) and ice (b) cloud fraction (including both successful and unsuccessful optical property retrievals) gridded at 1×1 degrees for November 2012. Note these fractions correspond only to the population of pixels identified as overcast by the CSR algorithm (i.e., CSR=0). The liquid and ice cloud fractions for the partly cloudy PCL pixel population (i.e., CSR= 1,3) are shown in (c) and (d), respectively. One can see that the PCL pixel population is mostly identified as liquid by the C6 COP phase algorithm, an expected result given that liquid clouds tend to be smaller in scale and have a more broken structure than do ice clouds.

Figure 6.

Monthly gridded cloud phase fractions derived from the MOD06 COP phase product for November 2012. Figures (a) and (b) show the liquid and ice cloud fraction, respectively, for the overcast (CSR=0) pixel population, while (c) and (d) show the partly cloudy PCL (CSR= 1,3) liquid and ice cloud fraction, respectively. The differences between the C5 and C6 overcast liquid (e) and ice (f) cloud phase fractions are also shown.

The difference between the C5 and C6 November 2012 monthly fractions, for the overcast CSR = 0 pixel population only (PCL pixels were previously discarded in C5), is shown in Figure 6(e) and (f) for liquid and ice phase, respectively. Here red shades indicate an increase for C6 over C5, and blue colors indicate a decrease; color bar values denote absolute fraction changes. Several differences are worth noting. The most obvious is that the C6 algorithm identifies more liquid phase clouds in the southern oceans than does C5, along with a corresponding decrease in ice phase. An increase in liquid phase identification over many non-polar vegetated land areas, as well as a decrease over South America, is also evident. Comparisons have also been performed for other months (e.g., summer months), with similar differences observed. As will be shown in subsequent sections, these C6 changes largely represent phase classification improvements over C5.

Although the C6 COP phase classification algorithm is significantly improved over C5, some situations continue to be problematic. For instance, optically thin cirrus over warm surfaces, a particularly acute problem in C5 in which such cases were often incorrectly identified as liquid phase, may continue to be identified as liquid phase though C6 provides better skill in such circumstances, as shown in Figure 5. In addition, at oblique sun angles, especially at high latitudes, the spectral CER tests become less sensitive to phase and may incorrectly vote for liquid phase clouds. False ice phase classification of broken liquid phase clouds also remains problematic despite improvements in low maritime broken cloudy scenes. However, these pixels are often identified as partly cloudy by the CSR algorithm and are therefore excluded from the standard MOD06 retrieval products (though they are reported in separate PCL SDSs).

5.2. Evaluation Against CALIOP

Contingency tables comparing the MOD06 COP phase algorithm to the collocated CALIOP v3 cloud layer product are shown in Figure 7 for C6 (a) and C5 (b). The data used here are from November 2012 for the entire globe (all surface types), and are limited to cases where the MOD06 CSR algorithm identified an overcast scene (CSR= 0) and CALIOP identified only a single phase in the column, regardless of the success/failure status of the various spectral CER retrievals; the CSR= 0 constraint is applied such that the C6 pixel population is consistent with C5. The abscissa denotes CALIOP phase, and the ordinate denotes MODIS phase. The numerical values in each table can be interpreted as the percent of total collocated cloudy scenes for which the given phase condition is observed. For instance, the value corresponding to the second column and second row in the C6 table (b) indicates that MODIS and CALIOP agreed on liquid phase designation in 54.4% of the collocated cloudy pixels; similarly, the value of the first column and second row indicates that in 3.2% of the collocated cloudy pixels CALIOP determined ice phase while MODIS disagreed, determining liquid phase. Note the total CALIOP ice and liquid phase populations, in terms of percent of the total collocated cloudy pixel population, can be found by summing each column; likewise, the MODIS ice, liquid, and undetermined phase populations are found by summing each row.

Figure 7.

Contingency tables corresponding to MODIS C5 (a) and C6 (b) cloud phase calculated from the MODIS and CALIOP collocated dataset during November 2012.

A convenient method of summarizing these contingency tables is to define a simple skill score, referred to as the Phase Agreement Fraction (PAF):

$$PAF=\frac{a_{2,2}+a_{3,1}}{{\sum }_{i,j}{a}_{i,j}}$$

Here, the a values are the number of pixels corresponding to the phase condition of row i and column j. Thus the PAF skill score is simply the ratio between the number of pixels where MODIS and CALIOP phase are in agreement divided by the total number of collocated cloudy pixels. Alternatively, PAF may be found by simply summing the contingency table values corresponding to phase agreement, and dividing by 100 to convert from percent to fraction. Defined in this way, PAF ranges from zero for no phase agreement to one for complete phase agreement. Assuming CALIOP as truth, the C6 COP phase algorithm provides a marked improvement over C5, with the global PAF skill score increasing from 0.83 for C5 to 0.92 for C6. This improvement is primarily due to the increased skill of the C6 algorithm for liquid phase clouds, as the portion of liquid pixels misidentified as ice by MODIS substantially decreased by over a factor of 4 (10.8% to 2.6%), and the liquid phase agreement increased (43.0% to 54.4%). In addition, the portion of pixels identified as undetermined phase decreased by a factor of 2.5 in C6 (5.4% to 2.1%). The overall increase in liquid phase clouds and decrease in ice phase clouds (i.e., the decrease in misidentified ice phase by MODIS) in the C6 algorithm is consistent with what is shown previously from the MODIS C5 and C6 comparisons. On the other hand the fraction of misclassified liquid clouds by MODIS remains roughly constant between C5 and C6 (3.0% to 3.2%). These misclassified pixels are in part due to optically thin ice clouds over warm or bright surfaces but may also be due to insufficiently screening out all multilayer cloud cases from the MODIS-CALIOP collocated dataset. In some cases where ice clouds overlap optically thick liquid clouds, CALIOP might detect only the overlying ice cloud while MODIS may identify the scene as liquid. This “spurious” liquid phase classification might in fact be preferable for the MODIS cloud optical products, as a liquid phase may provide better radiative consistency and reduce retrieval errors.

In addition to the contingency tables that globally summarize the cloud phase classification skill, a more detailed analysis has also been done. Figure 8 shows the global gridded November 2012 PAF score at 10×10 degree resolution for MODIS C5 (a) and C6 (b). The C6 cloud phase improvement is broadly distributed, with a noticeable improvement over ocean. Moreover, the C5 cloud phase skill gradually decreased with increasing latitude, with a pronounced minimum over Antarctica, a shortcoming that has been greatly reduced in C6.

Figure 8.

Gridded PAF (Phase Agreement Fraction) score maps, for C5 (a) and C6 (b), obtained from the MODIS-CALIOP collocated dataset for November 2012.

The PAF score has also been analyzed by surface type (i.e., ocean, permanent snow/ice, desert, and vegetated land) and cloud optical thickness (i.e., opaque clouds vs. non-opaque clouds as determined by CALIOP), as is shown in Figures 9 and 10 for November 2012 and July 2008, respectively. These figures underscore the broad phase identification skill improvement for C6. Only for optically thin (non-opaque) clouds over desert surfaces, specifically in November 2012, does C6 slightly underperform C5; however, it should be noted the pixel count in this category is only 5% of the total November 2012 collocated cloudy pixel population. It is also worth noticing the significant improvement of the cloud phase skill over snow/ice surfaces for optically thick clouds compared to C5, in particular in November 2012. As expected, the cloud phase skill is overall lower for optically thin clouds compared to thick clouds, though C6 performs reasonably well for optically thin clouds over ocean.

Figure 9.

Detailed PAF (Phase Agreement Fraction) scores, derived from the MODIS-CALIOP collocated dataset for November 2012, as a function of surface type (ocean, snow/ice, desert and vegetated land) and cloud opacity (opaque vs. non-opaque clouds) as determined by CALIOP. The percentage of pixels for each classification is also shown (Note that coastal surfaces are not included).

Figure 10.

Same as Figure 9 except the month is July 2008.

Cloud top temperature is a widely used parameter and plays a critical role in the MODIS cloud phase algorithm. Figure 11 shows the probability density functions (PDFs) for CALIOP (a) and MODIS C6 (b) and C5 (c) cloud phase against the MODIS 1 km cloud top temperature calculated for November 2012. Note these distributions again exclude multi-phase scenes as identified by CALIOP (about 20% of cloudy scenes from the MODIS-CALIOP collocated dataset present multi-phase scenes). The main conclusion is that the MODIS C6 ice and liquid PDFs now look quite similar to the CALIOP cloud phase PDFs, in contrast to C5 that yields too much ice in the interval [240 K, 260 K]. This figure also shows that the C6 undetermined cloud phase is roughly in the interval between 240 K and 270 K, as expected since cloud phase discrimination is particularly difficult in these temperature ranges.

Figure 11.

Probability density functions (PDFs) of CALIOP (a) and MODIS C6 (b) and C5 (c) cloud phase against the MODIS 1km cloud top temperature for November 2012.

5. Conclusions

Cloud thermodynamic phase classification is an important component of the MODIS cloud optical products. For MODIS Collection 6 (C6) the cloud retrieval phase classification algorithm has been completely revised and optimized using intensive comparisons between MODIS and CALIOP. The new algorithm is now based on a simple majority vote logic that uses thresholds derived from MODIS and CALIOP comparisons instead of the C5 decision tree logic-based algorithm approach that was difficult to optimize. In addition, the C6 phase algorithm uses four primary tests, based on the 1 km cloud top temperature, the 1 km IR cloud phase, the 1.38 cirrus detection test from the MOD35 cloud mask, and three spectral cloud effective radius tests (derived from 1.6, 2.1, and 3.7 μm channels). The spectral effective radius tests effectively replace the C5 SWIR bidirectional reflectance ratio thresholds; the C5 SWIR ratio thresholds were problematic as they did not account for the reflectance dependence on both the viewing geometry and cloud optical thickness, leading in particular to false ice phase classification for optically thick clouds. The new cloud effective radius tests outperform the C5 reflectance ratio tests, though the radius thresholds now depend on the assumed ice radiative model and are more computationally expensive.

These cloud phase classification algorithm modifications have resulted in noticeable changes between C5 and C6. In particular, global MODIS-CALIOP cloud phase classification agreement has increased by about 10% for C6 compared to C5, leading to a total cloud phase agrement between MODIS C6 and CALIOP of over 90 percent for single-phase cloudy pixels. Moreover, these improvements are observed for several surface types (ocean, land, desert, and snow/ice) and cloud optical thickesses (thin and thick). The most significant improvement is found for opaque clouds (defined by the CALIOP lidar) over snow/ice surfaces. On the other hand, cloud phase discrimination for optically thin clouds over really bright or warm surfaces (such as thin cirrus clouds over desert) continue to be problematic. Another important difference between C5 and C6, though not a result of cloud phase algorithm development, is the cloudy pixel population for which the cloud phase is reported. Previously in C5, only pixels identified as overcast by the clear sky restoral algorithm were optical/microphysical retrieval candidates, and as such cloud phase was only reported for this pixel population (regardless of retrieval success/failure). For C6, optical/microphysical retrievals are also attempted for pixels classified as very inhomogenous (e.g., partly cloudy) and cloud phase is reported for this pixel population as well (again regardless of retrieval success/failure).

Finally, though the CALIOP comparisons show better agreement for C6 compared to C5, numerous challenges remain. Because the collocated MODIS-CALIOP dataset used for development and evaluation only includes pixels for which CALIOP observed a single cloud phase in the column, the extent to which the results presented here hold for multilayer clouds is still an open question. Limiting the analysis to the CALIPSO ground-track also limits the viewing and scattering angle space such that it is unclear whether the C6 improvements are consistent across the entire MODIS swath; the impacts of potential view angle dependencies are at present unknown. Moreover, because spectral channels sets can vary between satellite sensors (e.g., MODIS 2.1μm vs. VIIRS 2.25μm), it is uncertain whether the spectral effective radius tests, as used here, can be applied uniformly across multiple platforms for climate data record continuity, though work to this end is ongoing. Nevertheless, the C6 COP phase algorithm represents a vast improvement over C5, and future work will focus on the remaining challenges such as multilayer clouds and view and scattering angle dependencies.