Retrieval of ice cloud properties using an optimal estimation algorithm and MODIS infrared observations. Part I: Forward model, error analysis, and information content

Abstract

An optimal estimation (OE) retrieval method is developed to infer three ice cloud properties simultaneously: optical thickness (τ), effective radius (r_eff), and cloud-top height (h). This method is based on a fast radiative transfer (RT) model and infrared (IR) observations from the MODerate resolution Imaging Spectroradiometer (MODIS). This study conducts thorough error and information content analyses to understand the error propagation and performance of retrievals from various MODIS band combinations under different cloud/atmosphere states. Specifically, the algorithm takes into account four error sources: measurement uncertainty, fast RT model uncertainty, uncertainties in ancillary datasets (e.g., atmospheric state), and assumed ice crystal habit uncertainties. It is found that the ancillary and ice crystal habit error sources dominate the MODIS IR retrieval uncertainty and cannot be ignored. The information content analysis shows that, for a given ice cloud, the use of four MODIS IR observations is sufficient to retrieve the three cloud properties. However, the selection of MODIS IR bands that provide the most information and their order of importance varies with both the ice cloud properties and the ambient atmospheric and the surface states. As a result, this study suggests the inclusion of all MODIS IR bands in practice since little a priori information is available.

1. Introduction

Ice clouds have received great attention due to their strong impact on regional and global climate [Liou, 1986; Baran, 2012; Yang et al., 2015]. To fully understand ice cloud radiative impacts and associated uncertainties, the study of cloud optical thickness (τ), effective particle radius (r_eff), and cloud-top height (h) on the global scale is necessary. Satellite observations provide the only means to infer global ice cloud properties. Numerous satellite instruments have been used to retrieve radiatively-relevant ice cloud properties over the past several decades [King et al., 1992; Chepfer et al., 1998; Minnis et al., 1993, 1998, 2011; Platnick et al., 2003; Huang et al., 2004; Wei et al., 2004; Meyer et al., 2004; Heidinger and Pavolonis, 2009; Meyer and Platnick, 2010; Watts et al., 2011; Baum et al., 2012; Poulsen et al., 2012; Walther and Heidinger, 2012; Garnier et al., 2012, 2013; Kahn et al., 2014]. For example, passive sensor based methods using imager and sounders include the visible and near-/shortwave-/midwave-infrared (VNIR/SWIR) bi-channel reflectance method [Nakajima and King, 1990], water vapor absorption channel cirrus retrieval method [e.g., Meyer and Platnick, 2010], the thermal infrared split-window (IR-split) method [Inoue, 1985, Parol et al., 1991], and other IR methods [e.g., Minnis et al., 2011; Heidinger et al., 2015, Garnier et al., 2012, 2013] to infer ice cloud optical and microphysical properties. The VNIR/SWIR technique is not applicable to nighttime scenes and can have higher uncertainties in high-latitude regions and optically thin cirrus cloud scenes. On the contrary, the IR-split method has two inherent advantages in the inference of cirrus clouds residing in the upper troposphere: 1) cirrus clouds reduce emission from warm surfaces and emit at a much colder temperature. As a result, radiances at the top of the atmosphere (TOA) in the presence of cirrus can significantly differ from clear-sky radiances. 2) Consistent IR observations in both daytime and nighttime allow us to build a comprehensive cloud climatology and understand the full cloud diurnal cycle. Furthermore, cloud particle absorption strongly depends on particle size. Because of the significant ice absorption difference that occurs between the two IR window channels at 8.5μm and 11.0μm, it is feasible to infer ice cloud r_eff from IR observations. Cloud-top height is another critical variable that determines the outgoing longwave radiation. Passive sensor cloud-top height retrieval algorithms are generally related to above-cloud gas absorption. For example, the widely employed CO₂ slicing technique uses several thermal IR bands near the 15 μm CO₂ absorption region to infer cloud-top height [Smith and Platt, 1978; Menzel et al., 1983; Wylie and Menzel, 1999]. Since the absorptivity of CO₂ across this spectral region changes significantly, observations in these bands are sensitive to different atmospheric levels. Therefore, clouds located in different vertical regions can be retrieved with different combinations of these IR bands.

Despite this remarkable progress, ice cloud properties are still a major source of uncertainty in climate modeling [Waliser et al., 2009; Li et al., 2012]. One major reason is that cloud retrievals strongly rely on assumed cloud microphysical properties such as particle size distribution and particle habits, in particular retrievals using VNIR/SWIR observations [Zhang et al., 2009; Baum et al., 2014]. In situ measurements collected in field campaigns provide direct observational support of ice cloud microphysics and in-cloud physical processes for remote sensing studies. It is found from in situ measurements that a wide and complex variety of ice crystal sizes and habits/structures exist [Heymsfield et al., 2002; de Reus et al., 2009]. Clouds formed through different microphysical processes and under different ambient conditions will consist of ice crystals with very different size and habit distributions [Heymsfield et al., 2002; Baum et al., 2011]. In-cloud physical processes, such as melting, condensation, collision, and coalescence, result in more complicated and irregular habits of ice crystals [Pfalzgraff et al., 2010]. Complicated ice cloud microphysical properties prevent the radiative modeling of real ice clouds for remote sensing applications and climate models for two reasons. First, the single-scattering properties are only available for a relatively small number of geometrically simple ice crystal habits (e.g., pristine hexagonal plates and columns), which cannot fully represent real ice cloud particles. Second, in operational applications, only one particular habit or a certain mixture of several habits is employed for global ice cloud retrievals [e.g., Platnick et al., 2003; Yang et al., 2007; Minnis et al., 2011; Yang et al., 2013], which simplifies the retrieval algorithm but can introduce a significant error source [Zhang et al., 2009].

In addition to errors from an assumed ice crystal habit, errors from ancillary datasets (e.g., atmospheric profiles, surface emission and reflectivity), satellite observations, and forward radiative transfer (RT) models should be considered. However, error estimations in current ice cloud retrieval methods are incomplete. Specifically, in most satellite-based ice cloud retrieval algorithms, uncertainties from ice crystal microphysical properties are ignored. Although several previous sensitivity studies have demonstrated that ice crystal microphysical assumptions can introduce significant uncertainties to cloud retrievals [Cooper et al., 2003, 2006], it is difficult to estimate how much retrieval uncertainty is quantitatively contributed by an assumed habit in operational applications. Furthermore, in order to mitigate the computational burden, IR-based retrieval applications often estimate error statistics (e.g., covariance matrix) for ancillary errors outside of the retrieval process. For example, Kahn et al. [2008] estimated such errors by introducing Gaussian-distributed errors to atmospheric profiles and surface temperature, using Atmospheric Infrared Sounder (AIRS) validation results. A large number of perturbed parameters are generated, with which the perturbed forward model simulations are calculated under different cloudy states. Statistics of simulated error due to the perturbed parameters can be evaluated by comparing the perturbed simulations against the reference. Similar methods are employed in many other retrieval studies [e.g., Iwabuchi et al., 2014; Wang et al., 2014]. The advantage of this method is that the generated simulation error statistics can be directly used in a retrieval algorithm and therefore computational resources are reduced. However, this method is inflexible because the statistics of the ancillary data errors are fixed. In fact, these parameter errors will have spatial and temporal variations, and strongly depend on the atmosphere and cloud conditions. For this reason, a more flexible and computationally efficient ice cloud modeling and retrieval framework is required to estimate simulation/retrieval uncertainties for various atmospheric and surface states.

In this study, we develop an optimal estimation (OE) based algorithm to simultaneously retrieve the three ice cloud properties (τ, r_eff, and h) using MODIS IR observations. A unique feature of this retrieval algorithm is that four different error sources are taken into account: cloud microphysical assumption errors, ancillary data errors, observational errors, and forward RT model errors. A computationally efficient forward model is employed to simulate MODIS IR observations and estimate simulation uncertainty from different error sources within the retrieval process. In this study, we wish to 1) provide a nighttime capability to complement the current MODIS (MOD06) cloud optical and microphysical property daytime-only product; and 2) improve our understanding of IR-based ice cloud retrieval uncertainties.

This paper (hereafter, Part 1) describes the forward model and retrieval algorithm, introduces the four error sources, and presents the information content analysis. A follow-up paper (hereafter Part 2) will report the retrieval results and validation against active sensor-based products. Part 1 is organized as follows: Section 2 describes the forward model, including a clear-sky module and an ice cloud module. Section 3 introduces the error sources. Section 4 discusses the method to estimate the impact from each error source on the observational space. The OE-based retrieval algorithm is introduced in Section 5. Summary and further discussion are given in Section 6.

2. Forward Model

A computationally efficient RT model simulating the MODIS thermal IR observations is used. Only a brief description of the RT model is provided here. The present RT model is a one-dimensional, single-layered cloud model. To mitigate the computational burden, cloud scattering, emission, and absorption are considered using lookup tables (LUTs) calculated using a discrete ordinate method radiative transfer code (DISORT [Stamnes et al., 1988]). Furthermore, the microphysical properties are identical throughout the whole cloud layer (homogeneous cloud assumption) but the cloud layer temperature varies linearly with height. Wang et al. [2011, 2013, hereafter referred to as W11 and W13] discussed the IR RT equation solver of this model in detail. In the LUTs, cloud scattering properties are calculated for 16 incident angles and/or viewing angles, 40 τ, and 18 r_eff values (see Table 1).

Table 1.

Geometries and ice cloud properties selected to calculate the cloud lookup tables (LUTs).

	Ice Cloud LUTs
16 Viewing/Incident angles (cosine)	0.99470, 0.97229, 0.93282, 0.87770, 0.80894, 0.72901, 0.64080, 0.54751, 0.45249, 0.35920, 0.27099, 0.19106, 0.12230, 0.06718, 0.02771, 0.00530.
40 Cloud Optical thickness	0.01, 0.03, 0.05, 0.10, 0.20, 0.30, 0.40, 0.50, 0.60, 0.70, 0.80, 0.90, 1.00, 1.20, 1.40, 1.60, 1.80, 2.00, 2.50, 3.00, 3.50, 4.00, 4.50, 5.00, 5.50, 6.00, 6.50, 7.00, 7.50, 8.00, 8.50, 9.00, 9.50, 10.0, 12.0, 15.0, 20.0, 25.0, 30.0, 50.0.
18 Cloud Effective Radius (μm)	5, 10, 15, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, 80, 85, 90

In W11 and W13, a correlated k-distribution code [CKD, Kratz, 1995] and a large clear-sky transmittance database are used to simulate gas absorption in MODIS IR channels and in the IR window region with a high spectral resolution, respectively. This study employs the clear-sky module of the Community Radiative Transfer Model (CRTM) designed for a wide variety of sensors including the Aqua MODIS [Han et al., 2006; Liu and Weng, 2006]. By comparing the Aqua MODIS IR observations with collocated high-spectral resolution observations, such as from AIRS and IASI, some previous studies evaluated the quality of the spectral response functions (SRFs) of the Aqua MODIS IR channels and demonstrated that the Aqua MODIS SRFs are shifted in some water vapor and CO₂ channels [Tobin et al., 2006; Sohn et al., 2010]. These shifted SRFs lead to simulation biases up to 2–3 K in terms of brightness temperature (BT). The CRTM clear-sky module has corrected the shifted MODIS spectral response functions [Liu and Boukabara, 2014].

In this study, ice crystals are assumed to be aggregates of eight hexagonal columns with severely roughened surfaces (hereafter referred to as aggregate columns) [Yang et al., 2013] and satisfy a gamma distribution with an effective variance of 0.1. This is identical to the ice particle assumption used in the Collection 6 MODIS cloud product (MOD06) [Holz et al., 2015]. It is necessary to emphasize that, even in the IR spectral region, ice cloud property retrievals depend on the assumed cloud microphysical properties. Holz et al. [2015] found that the use of aggregated columns in the IR-split window technique provides consistent cloud optical thickness retrievals in comparison with lidar and VNIR/SWIR retrievals. Other studies also looked into microphysical assumptions that provide consistent retrievals between IR, VNIR/SWIR total reflectance and polarimetric directional observations [e.g., Cole et al., 2013; Baum et al., 2014]. Nevertheless, assuming a single habit and size distribution is not expected to represent the variety of particle shapes and distributions occurring in real ice clouds. Uncertainties for the microphysical assumption will be estimated in Section 4.

The forward RT model (F) can be expressed as a function of cloud properties and other known model input (ancillary parameters):

$$\mathbf{Y}={\left[{\text{BT}}_1,{\text{BT}}_2,\cdots ,{\text{BT}}_{\mathrm{m}}\right]}^{\mathbf{T}}=\mathbf{F}\left[\mathbf{X}\left(\tau ,{\mathit{r}}_{\mathit{eff}},h\right),\mathbf{P}\right]+\mathbf{e},$$ (1)

where Y is a vector consisting of m MODIS IR observations in BT, P is a vector that includes quantities provided by ancillary datasets (e.g., air temperature (T), water vapor concentration (w), and ozone concentration (O₃) profiles, surface emissivity spectrum (ε_{_s}) and temperature (T__s)), and e is a vector including the differences between simulation and measurement. The state vector (X) includes n (n=3) components, namely, ice cloud optical thickness τ, effective particle radius r_eff, and cloud-top height h. Hereafter bold variables are vectors or matrices, unless otherwise stated. W11 and W13 introduced the formula and equations mapping X and P onto the observational space. Several Jacobian matrices are required for the retrieval and error analysis. A m × n matrix K_F/X consists of the partial derivatives of the m MODIS IR observations with respect to the n cloud parameters:

$${\mathbf{K}}_{\mathbf{F}/\mathbf{X}}={\left[\frac{\partial \mathbf{F}}{\partial \tau },\frac{\partial \mathbf{F}}{\partial r_{\mathit{eff}}},\frac{\partial \mathbf{F}}{\partial h}\right]}^{\mathbf{T}}.$$ (2)

Figure 1 shows the output F (in BT) of the present model as a function of cloud optical thickness, and the corresponding Jacobian matrix K_F/X. With given h and r_eff, the TOA BTs in all MODIS IR bands decrease with an increase in τ, but with different slopes. Panels b, c, and d show the sensitivities of satellite observations in each band to the state variables. It is found that the IR measurements have the largest sensitivities to τ and r_eff when 0.3 <τ < 5 (Garnier et al., 2012), while the maximum sensitivities of BTs to h occur when τ > 10. In addition, cloud optical properties in the midwave-IR band (Band 20 at 3.8 μm) are different from those in the thermal IR bands. For example, BT at 3.8 μm decreases slowly with τ when τ < 3, and has a maximum sensitivity to r_eff when τ > 5. With these features, the 3.8 μm channel could provide unique ice cloud property information compared with thermal IR observations (see Section 5). Simulation of daytime observations in this band is complicated due to a non-trivial solar reflectance signal, which is not included in the present forward RTM.

Figure 1.

The forward model simulations of a single-layer cloudy (ice) atmosphere with r_eff = 20 μm, cloud-top height h = 10 km, surface temperature T__s = 294 K, and surface emissivity ε__s = 0.98. Here TOA BT as a function of cloud optical thickness τ (panel a) is shown, as is the first derivative of TOA BTs with respect to t (panel b), r_eff (panel c), and h (panel d).

Other Jacobian matrices include the first derivative of BTs with respect to ancillary data parameters. For example, K_{F/I_s} indicates the derivative of BTs with respect to surface emission spectrum (I_{_s}); K_F/B(T) and ${\mathbf{K}}_{\mathbf{F}/{\tau }_g}$ are Jacobian matrices of air emission and optical thickness (τ_g), respectively. The matrix K_F/B(T) of an l-layer atmosphere can be expressed as:

$${\mathbf{K}}_{\mathbf{F}/\mathrm{B}\left(\mathbf{T}\right)}={\left[\frac{\partial \mathbf{F}}{\partial \mathrm{B}\left(T_1\right)}\frac{\partial \mathbf{F}}{\partial \mathrm{B}\left(T_2\right)}\cdots \frac{\partial \mathbf{F}}{\partial \mathrm{B}\left(T_l\right)}\right]}^{\mathbf{T}},$$ (3)

where B is the Planck function, T_i indicates the temperature of the i^th atmospheric layer. Derivations of Jacobian matrices K_F/X, K_{F/I_s}, ${\mathbf{K}}_{\mathbf{F}/{\tau }_g}$ are given in Appendices A and B.

3. Optimal Estimation based retrieval method

The optimal estimation (OE)-based retrieval method [Rodgers, 2000] is an efficient inversion method widely applied to a wide variety of remote sensing applications [Poulsen et al., 2012; Sourdeval et al., 2013, 2015; Iwabuchi et al., 2014]. The retrieval is essentially a process that reduces the state vector uncertainty from the a priori state, which depends on our knowledge of the state variables, to the posterior state when measurements are made. The OE method retrieves the state variables having the maximum probability of occurrence by minimizing a cost function J:

$$\mathbf{J}={\left[\mathbf{F}\left(\mathbf{X},\mathbf{P}\right)-\mathbf{Y}\right]}^{\mathbf{T}}{\mathbf{S}}_y^{-1}\left[\mathbf{F}\left(\mathbf{X},\mathbf{P}\right)-\mathbf{Y}\right]+{\left[\mathbf{X}-{\mathbf{X}}_a\right]}^{\mathbf{T}}{\mathbf{S}}_a^{-1}\left[\mathbf{X}-{\mathbf{X}}_a\right],$$ (4)

where X_a and X are the a priori and posterior state vectors, S_y and S_a are covariance matrices of the observation to simulation differences and the uncertainty of the a priori state vector, respectively. Generally, we assume large a priori uncertainties so that the cost function J is dominated by the first term of equation 4. Minimizing the cost function J is a non-linear least squares fitting problem. The Levenberg-Marquardt iteration method [Levenberg, 1944; Marquardt, 1963] is an efficient approach to solve this problem and is used in this study. This method is a combination of the gradient descent method and the Gauss-Newton method, expressed as follows:

$$\mathrm{\Delta }{\mathbf{X}}_i={\left({\mathbf{K}}_{\mathbf{F}/\mathbf{X},i}^{\mathbf{T}}{\mathbf{S}}_{y,i}^{-1}{\mathbf{K}}_{\mathbf{F}/\mathbf{X},i}+{\gamma }_i\mathbf{I}\right)}^{-1}{\mathbf{K}}_{\mathbf{F}/\mathbf{X},i}\left[\mathbf{Y}-\mathbf{F}\left({\mathbf{X}}_i\right)\right],$$ (5)

where i indicates the i^th iteration, γ is a positive damping parameter and varies at each iteration according to the variation of the cost function. Generally speaking, in this retrieval algorithm, γ varies from 0.01 and is decreased (or increased) by a factor of 5 if a new cost function is smaller (or larger) than previous iteration step. As γ approaches 0, Eq. (5) essentially reduces to the Gauss-Newton method, while it approaches the gradient descent method for large γ. Marquardt [1963] found that since elements in the state vector X could have different magnitudes, the identity matrix I in Eq. (5) could be replaced by a scaling matrix D = S_a⁻¹. The retrieval stops at the p^th iteration (X_p) when the iteration converges or the simulation fits the measurement [Rodgers, 2000]:

$${\left({\mathbf{X}}_p-{\mathbf{X}}_{p-1}\right)}^{\mathbf{T}}{\mathbf{S}}_p^{-1}\left({\mathbf{X}}_p-{\mathbf{X}}_{p-1}\right)\ll n,$$ (6)

$${\left[\mathbf{F}\left({\mathbf{X}}_p\right)-\mathbf{Y}\right]}^{\mathbf{T}}{\mathbf{S}}_y^{-1}\left[\mathbf{F}\left({\mathbf{X}}_p\right)-\mathbf{Y}\right]\approx m,$$ (7)

where n and m are the number of state variables and measurements, and S_p is the error covariance matrix of the posterior state vector X_p, defined in the form:

$${\mathbf{S}}_p={\left({\mathbf{S}}_a^{-1}+{\mathbf{K}}_{\mathbf{F}/\mathbf{X}}^{\mathbf{T}}{\mathbf{S}}_y^{-1}{\mathbf{K}}_{\mathbf{F}/\mathbf{X}}\right)}^{-1}.$$ (8)

4. Error analysis

Four error sources are taken into account in the retrieval algorithm: measurement uncertainty, fast RT model uncertainty, uncertainties in ancillary datasets, and uncertainties associated with cloud particle property assumptions.

4.1 Measurement Uncertainty

Measurement uncertainty and the corresponding long-term trend of MODIS due to the instrumental noise and degradation are documented for the latest MODIS Collection 6 L1b product. Specifically, the noise-equivalent temperature differences in Aqua MODIS IR bands are generally less than 0.3 K [Xiong et al., 2009]. Scaled uncertainty indices are provided in the product for individual bands and pixels. Shifted SRFs of Aqua MODIS have been found in Bands 27, 28, 34, 35, and 36, resulting in observational biases. The impact of the shifted SRFs has been evaluated quantitatively in previous studies [e.g., Tobin et al., 2006; Sohn et al., 2010], and can be removed by using the latest CRTM clear-sky module designed specifically for Aqua MODIS [Liu and Boukabara, 2014]. It is assumed that observational uncertainties in different bands are independent. Therefore, in this study, a m × m diagonal matrix (S_obs) is used for each pixel to indicate the error covariance of measurement error:

$${\mathbf{S}}_{\mathit{obs}}=\left[\begin{array}{cccc}{{\sigma }_{\mathit{obs},1}}^2& 0& \cdots & 0\\ 0& {{\sigma }_{\mathit{obs},2}}^2& \cdots & 0\\ ⋮& ⋮& \ddots & ⋮\\ 0& 0& \cdots & {{\sigma }_{\mathit{obs},m}}^2\end{array}\right],$$ (9)

where σ_obs,i indicates the standard deviation of the measurement errors in band i.

4.2 Fast RT Model Uncertainty

The fast RT model uncertainty is the difference between the employed fast model, based on a set of LUTs to describe cloud bulk scattering properties, and 1-D RTMs that rigorously solve RT equations in plane-parallel atmospheres (e.g., the DISORT). It is important to emphasize that, even if perfect measurements are made and there is no input error, simulations of the rigorous 1-D RTMs can be different from observations. For example, cloud inhomogeneity effects can introduce significant retrieval errors [e.g., Iwabuchi and Hayasaka, 2002; Marshak et al., 2006; Fauchez et al., 2014, 2015; Cornet et al., 2005]. However, estimation of these errors in practice is difficult and beyond the scope of this paper. In this study, a large number of forward simulations from the present RT model and DISORT under different cloud and atmospheric conditions, and viewing geometries are used to create the statistics of the fast RT model error, which are described using an m × m error covariance matrix S_RT.

4.3 Uncertainties in Ancillary Datasets

Errors associated with non-retrieved variables dominate the IR based retrieval uncertainty and cannot be ignored [Cooper et al., 2006; Iwabuchi et al., 2014]. In this study, an ancillary parameter error covariance matrix (S_anc) is used to estimate these errors quantitatively for each cloudy pixel. Specifically, uncertainties of four ancillary parameters are considered: the surface temperature (T__s), the surface emissivity (ε_{_s}), and the atmospheric profiles of temperature (T) and water vapor concentration (w).

The surface emission (I__s) is the product of B (T__s) and ε_{_s}. The covariance matrix (COV) of surface emission uncertainty (δI__s), under the assumption that the T__s and ε_{_s} are independent, can be expressed as:

$$\mathbf{C}\mathbf{O}\mathbf{V}\left(\delta {\mathbf{I}}_{\_s}\right)={\mathbf{K}}_{{\mathbf{I}}_{\_s}/\mathbf{B}\left(T_{\_s}\right)}\mathbf{C}\mathbf{O}\mathbf{V}\left[\delta \mathbf{B}\left(T_{\_s}\right)\right]{\mathbf{K}}_{{\mathbf{I}}_{\_s}/\mathbf{B}\left(T{\_}_s\right)}^{\mathbf{T}}+{\mathbf{K}}_{\mathbf{I}{\_}_s/{\mathbf{\epsilon }}_{\_s}}\mathbf{C}\mathbf{O}\mathbf{V}\left(\delta {\mathbf{\epsilon }}_{-s}\right){\mathbf{K}}_{{\mathbf{I}}_{\_s}/{\mathbf{\epsilon }}_{\_s}}^{\mathbf{T}},$$ (10)

where δB(T_{_s}) and δε_{_s} are uncertainties of black surface emission and emissivity, respectively, ${\mathbf{K}}_{{\mathbf{I}}_{\_s}/\mathbf{B}\left(T{\_}_s\right)}$ and ${\mathbf{K}}_{{\mathbf{I}}_{\_s}/{\mathbf{\epsilon }}_{\_s}}$ are Jacobian matrices of surface emission with respect to B(T_{_s}) and ε_{_s}, respectively. The ancillary parameter uncertainty covariance matrix due to surface emission (S_{anc_s}) is:

$${\mathbf{S}}_{\mathit{anc}\_s}={\mathbf{K}}_{\mathbf{F}/\mathbf{I}\_s}\mathbf{C}\mathbf{O}\mathbf{V}\left(\delta {\mathbf{I}}_{\_s}\right){\mathbf{K}}_{\mathbf{F}/\mathbf{I}\_s}^{\mathbf{T}},$$ (11)

where K_{F/I_s} is the Jacobian matrix of TOA BT with respect to surface emission.

For simplification, we only consider the uncertainties of air temperature and water vapor concentration in the cloud-free part of the profile, although CO₂, O₃, and other trace gas concentration uncertainties impact simulations in several MODIS bands, in particular MODIS bands 33–36 (13.3–14.2 μm). Meanwhile, the relation between T and w profile uncertainties (δT and δw) is not well quantified. In this study, we simply assume that the two uncertainties are independent. Gas emission is determined by blackbody emission B(T) and gas optical thickness τ_g. Here we separately consider the impacts from δB(T) and δτ_g.

Both δT and δw profiles result in τ_g uncertainties. With independent δT and δw profiles, COV of δτ_g can be expressed as:

$$\mathbf{C}\mathbf{O}\mathbf{V}\left(\delta {\mathit{\tau }}_g\right)={\mathbf{K}}_{{\mathit{\tau }}_g/\mathbf{T}}\mathbf{C}\mathbf{O}\mathbf{V}\left(\delta \mathbf{T}\right){\mathbf{K}}_{{\mathit{\tau }}_g/\mathbf{T}}^{\mathbf{T}}+{\mathbf{K}}_{{\mathit{\tau }}_g/\mathit{w}}\mathbf{C}\mathbf{O}\mathbf{V}\left(\delta \mathit{w}\right){\mathbf{K}}_{{\mathit{\tau }}_g/\mathit{w}}^{\mathbf{T}},$$ (12)

where ${\mathbf{K}}_{{\mathit{\tau }}_g/\mathbf{T}}$ and ${\mathbf{K}}_{{\mathit{\tau }}_g/\mathit{w}}$ are the Jacobian matrices of τ_g with respect to T and w, respectively. It is important to emphasize that COV(δT) and COV(δw) are not diagonal matrices since both δT and δw have vertical correlations. Similarly, COV of clear-sky blackbody emission uncertainty δB(T) is:

$$\mathbf{C}\mathbf{O}\mathbf{V}\left[\delta \mathbf{B}\left(\mathbf{T}\right)\right]={\mathbf{K}}_{\mathbf{B}\left(\mathbf{T}\right)/\mathbf{T}}\mathbf{C}\mathbf{O}\mathbf{V}\left(\delta \mathbf{T}\right){\mathbf{K}}_{\mathbf{B}\left(\mathbf{T}\right)/\mathbf{T}}^{\mathbf{T}},$$ (13)

where K_B(T)/T is the Jacobian matrix of the Planck function with respect to temperature. The ancillary parameter uncertainty covariance matrix due to δB(T) and δτ_g (S_{anc_B} and ${\mathbf{S}}_{\mathit{anc}\_{\tau }_g}$) can be expressed as:

$${\mathbf{S}}_{\mathit{anc}\_{\tau }_g}={\mathbf{K}}_{\mathbf{F}/{\mathit{\tau }}_g}\mathbf{C}\mathbf{O}\mathbf{V}\left(\delta {\mathit{\tau }}_g\right){\mathbf{K}}_{\mathbf{F}/{\mathit{\tau }}_g}^{\mathbf{T}},$$ (14)

$${\mathbf{S}}_{\mathit{anc}\_B}={\mathbf{K}}_{\mathbf{F}/\mathbf{B}\left(\mathbf{T}\right)}\mathbf{C}\mathbf{O}\mathbf{V}\left[\delta \mathbf{B}\left(\mathbf{T}\right)\right]{\mathbf{K}}_{\mathbf{F}/\mathbf{B}\left(\mathbf{T}\right)}^{\mathbf{T}},$$ (15)

where ${\mathbf{K}}_{\mathbf{F}/{\mathit{\tau }}_g}$ and K_F/B(T) are two Jacobian matrices of TOA BT with respect to B(T) and τ_g.

Assuming independence of all error components, the total ancillary parameter uncertainty can be expressed as the summation of the three components, namely the surface uncertainty, the gas optical thickness uncertainty, and air temperature (or blackbody emission) uncertainty:

$${\mathbf{S}}_{\mathit{anc}}={\mathbf{S}}_{\mathit{anc}\_s}+{\mathbf{S}}_{\mathit{anc}\_{\tau }_g}+{\mathbf{S}}_{\mathit{anc}\_B}.$$ (16)

Derivations of Jacobian matrices ${\mathbf{K}}_{{\mathit{\tau }}_g/\mathbf{T}}$, ${\mathbf{K}}_{{\mathit{\tau }}_g/\mathit{w}}$, ${\mathbf{K}}_{\mathbf{F}/{\mathit{\tau }}_g}$, and K_F/B(T) in Eqs. (11)–(15) are given in Appendix B. Figures 2 and 3 show the three components of S_anc for two cases. In the first case, we use a 0.5 K uncertainty of T__s and a 0.01 uncertainty of ε_{_s} for all IR bands. The air T and w uncertainties are 1 K and 15 %, respectively, for all layers. In the second case, the surface temperature and emissivity uncertainties are 2 K and 0.03, respectively. The air T and w uncertainties are 2 K and 25 %, respectively, for all layers. For simplification, we assume that the ε_{_s} uncertainties in all bands are independent. In both cases, the correlation between the T (or w) uncertainties of two vertical layers decreases exponentially with the increase of layer vertical distances and approaches 0 if the distance exceeds 10 km. In the window bands (e.g., Bands 20 (3.75 μm), 29 (8.6 μm), 31 (11 μm), and 32 (12 μm)) uncertainties resulting from surface temperatures and emissivity spectra are important. In absorption bands, uncertainties of air temperature and water vapor profiles lead to larger uncertainties in the gas optical thickness profile, which then result in larger simulation uncertainties. This is because water vapor is a major absorptive species, and both the absorption and emission abilities of water vapor are affected by air temperature. As a result, in these bands, simulation uncertainties are highly correlated. The ice cloud properties used to calculate these covariance matrices are: τ = 1, r_eff = 20 μm, and h = 10 km.

Figure 2.

The three components (S_{anc_s}, ${\mathbf{S}}_{\mathit{anc}\_{\tau }_g}$ and S_{anc_B}) of error covariance matrices S_anc. The surface temperature and emissivity uncertainties (δT_{_s} and δε_{_s}) are 0.5K and 0.01, respectively. The air temperature and water vapor concentration uncertainties (δT and δw) are 1.0K and 15%, respectively, for all atmospheric layers. Note that the four types of uncertainties δT_{_s}, δε_{_s}, δT and δw are independent. Vertical correlations are considered for both δT and δw. Calculations are for a single-layered ice cloud with τ = 1, r_eff = 20 μm, and h = 10km.

Figure 3.

Same as Figure 2, but δT_{_s} and δε_{_s} are 2K and 0.03, respectively. The δT and δw are 2.0K and 25%, respectively, for all atmospheric layers. The other conditions are the same as Figure 2.

4.4 Ice Cloud Particle Habit Uncertainty

Uncertainties in ice crystal habits introduce additional errors to ice cloud retrievals. However, it is difficult to estimate this uncertainty directly since particle habit is not an explicit physical quantity in the radiative transfer model. In our forward RTM, the change of ice particle habit is equivalent to the change of cloud layer scattering properties, such as emissivity, transmissivity, and reflectivity, which have further impact on simulations and retrievals of ice clouds. Therefore, discussions in this section are essentially focused on uncertainties of cloud layer scattering properties considering a large cloud particle habit ensemble. We intend to establish a general estimation of simulation uncertainty using current knowledge of in-situ observed ice crystal habits and size distributions, and theoretically derived single scattering properties of ice particles with simple habits. Obviously, the limited ice particle habits used in theoretical studies cannot fully represent the irregular ice particle habits in real ice clouds. However, theoretical habit models can nevertheless shed light on the range of cloud scattering property uncertainties.

The Yang et al. [2013] ice crystal single-scattering database used here includes 9 non-spherical ice crystal shapes and 3 degrees of particle surface roughness. The database covers wide ranges of ice particle maximum dimension (between 2 and 10,000 μm) and wavelength (between 0.2 and 100 μm). Additionally, the effect of particle surface roughness is simulated by randomly distorted surface slope for each incident ray in the IGOM approach. The slopes of a tilted surface facet along two orthogonal directions are specified in terms of the two-dimensional Gaussian distribution [Yang and Liou, 1998; Liu et al., 2013] following Cox and Munk [1954], with a roughness parameter σ (i.e., the standard deviation of the Gaussian distribution). Specifically, three degrees of surface roughness are considered, namely smooth (σ = 0), moderately roughened (σ = 0.03), and severely roughened (σ = 0.5).

Assuming that particles with the same shapes but different degrees of surface roughness can be considered different habits, there are 27 habits in the database. With these existing habits, we can estimate how cloud scattering property uncertainties impact RT simulations with a cloud scattering property matrix (C). Specifically, C is a mN_p × N_h matrix, where m is the number of measurements, N_p is total the number of cloud scattering properties (e.g., cloud layer emissivity, transmissivity, and reflectivity calculated for 16 angles, 40 τ, and 18 r_eff values, as shown in Table 1) used in the RT simulation, and N_h is the number of possible ice crystal habits. As discussed in previous studies, the impact of an ice cloud layer in the forward RTM is primarily controlled by cloud layer emissivity (ε_{_c}), transmissivity (Γ__c), and reflectivity (R_{_c}). For given cloud τ and r_eff values, these bulk properties are pre-computed at different angles. Therefore, dozens of cloud scattering properties are involved in the cloudy-sky RT simulation. The matrix C can be expressed as follows:

$$\mathbf{C}=\left[\begin{array}{cccc}{\mathbf{C}}_{11}& {\mathbf{C}}_{12}& \cdots & {\mathbf{C}}_{1N_h}\\ {\mathbf{C}}_{21}& {\mathbf{C}}_{22}& \cdots & {\mathbf{C}}_{2N_h}\\ ⋮& ⋮& \ddots & ⋮\\ {\mathbf{C}}_{m1}& {\mathbf{C}}_{m2}& \cdots & {\mathbf{C}}_{m{N}_h}\end{array}\right],$$ (17)

where C_i,j is a vector that includes the cloud scattering properties of the j^th particle habit in the i^th band:

$${\mathbf{C}}_{ij}={\left[c_{ij}^1,c_{ij}^2,\cdots c_{ij}^{N_P}\right]}^{\mathbf{T}},$$ (18)

where c indicates the cloud scattering property, and the superscript denotes the index of a cloud scattering property parameter.

The uncertainty covariance matrix of the assumed ice crystal habits (S_habit) is given by

$${\mathbf{S}}_{\mathit{habit}}={\mathbf{K}}_{\mathbf{F}/\mathbf{C}}\mathbf{C}\mathbf{O}\mathbf{V}\left(\mathbf{C}\right){\mathbf{K}}_{\mathbf{F}/\mathbf{C}}^{\mathbf{T}},$$ (19)

where the m × mN_p matrix K_F/C is the Jacobian matrix of the simulation with respect to cloud scattering properties. Derivation of K_F/C is given in Appendix C. The covariance matrix COV(C) is a mN_p × mN_p matrix.

The covariance matrix (S_y) of the total measurement to simulation difference is the summation of the four components if the four types of error are independent:

$${\mathbf{S}}_y={\mathbf{S}}_{\mathit{obs}}+{\mathbf{S}}_{RT}+{\mathbf{S}}_{\mathit{anc}}+{\mathbf{S}}_{\mathit{habit}}.$$ (20)

Figure 4 shows these components of S_y except S_RT. In comparison with the other three components, S_RT is too small to be visualized. The ice cloud state is the same as the one used in Figures 2 and 3. S_anc is identical to that shown in Figure 2, and S_obs reflects a typical magnitude of MODIS observational uncertainty. Figure 5 shows the fractional contributions of the 4 different errors to the diagonal entries of matrix S_y. It shows that the maximum impact of ancillary parameter errors occurs in absorption bands, in particular Band 27 and 36 (yellow bars in Figure 5). In window bands, such as Bands 20, 29, 31, and 32, although the contribution from S_anc is also important, uncertainty due to ice crystal habit assumption (S_habit) is comparable to S_anc. Contributions from S_obs and S_RT are small in magnitude, in comparison with S_anc and Shabit, suggesting that the satellite measurement error and the fast model error are small. Uncertainties from the four different sources will influence retrieval accuracy, and are shown in Figure 6. As expected, uncertainties from ancillary datasets and ice crystal habits are two important contributors and should not be ignored.

Figure 4.

The three components (S_obs, S_anc, and S_habit) of error covariance matrices S_y. Note that S_RT is not shown in this figure since S_RT is two orders of magnitude smaller than S_anc. S_obs is generated using typical Aqua MODIS observations. Other conditions are the same as Figure 2.

Figure 5.

Fractions of the four components of the diagonal elements in S_y shown in Figure 4.

Figure 6.

Uncertainties of τ, r_eff, and h due to measurement errors (red), uncertainties in ancillary datasets (yellow), ice cloud particle habit (light blue), and all four uncertainties combined (dark blue). Cloud and atmospheric conditions are the same as Figure 2.

5. Information content analysis

The Shannon information content (hereafter information content), defined by Shannon and Weaver [1949], is used to describe the entropy reduction of a variable (or a set of variables) after measurement. Rodgers [2000] showed that the entropy (S) of a multivariate Gaussian distribution with covariance matrix S is

$$S=\frac{1}{2}\ln \left|\mathbf{S}\right|+\text{constant}.$$ (21)

The information content (H) is defined as

$$H=\frac{1}{2}\ln \left|{\mathbf{S}}_a\right|-\frac{1}{2}\ln \left|{\mathbf{S}}_p\right|=\frac{1}{2}\ln \left|{\mathbf{S}}_a{\mathbf{S}}_p^{-1}\right|,$$ (22)

where S_a and S_p are the error covariance matrices of the prior and posterior state vector X as discussed in Section 4. Substituting Eq. (8) in Eq. (22) we have

$$H=\frac{1}{2}\ln \left|{\mathbf{S}}_a\left({\mathbf{S}}_a^{-1}+{\mathbf{K}}_{\mathbf{F}/\mathbf{X}}^{\mathbf{T}}{\mathbf{S}}_y^{-1}{\mathbf{K}}_{\mathbf{F}/\mathbf{X}}\right)\right|.$$ (23)

A binary logarithm is frequently used in information theory when the unit of information is “bit”. The information content H of a measurement quantitatively indicates the factor by which the uncertainty of retrieval variables decreases with the measurement. From Eq. (22), it is obvious that H is 0 if a useless measurement is made, which does not decrease the uncertainty of the a priori estimation. A positive H indicates the knowledge of retrieval variables benefits from a measurement. Eq. (23) shows that the information content not only depends on the sensitivity of the measurement to the retrieval variables (K_F/X), but also on the errors of the forward model and measurements (S_y), and the a priori estimation (S_a). The framework of information content has been frequently used to evaluate the sensitivity of a remote sensing method or algorithm to model parameters [e.g., L’Ecuyer, et al. 2006, Cooper et al. 2006; Coddington et al., 2012].

Figures 7 and 8 show information content analyses associated with different ice cloud states. In each panel of the figures, the H of a single band is calculated first, as indicated by the black curve. Large H indicates better estimation of the three retrieval variables. With the help of one MODIS band, the performance of each band varies with cloud properties. For example, in comparison with other bands, Band 31 has the largest H for optically thin cirrus cloud consisting of relatively small particles because BT in Band 31 is sensitive to both τ and r_eff (Panels a and d of Fig. 7); whereas H for Band 20 improves with the increase of τ because the cloud is more “transparent” in Band 20 (Panels c and f of Fig. 7). For each panel, we first find the best measurement (band) from the black curve. Based on that first band, we continually calculate H values for two measurements for each ice cloud state (see blue curve). The blue curve indicates the information content of the second band and the previously determined first band. Consequently, the second best band can be found from the blue curve. Similar processes can be done to find the third, forth, and fifth best bands (illustrated with green, yellow, and red curves, respectively). Obviously, the overall information content increases with an increase in the number of measurements. However, H increases with the number of measurements slowly when more than four bands are involved, suggesting that the MODIS IR bands are not independent. Furthermore, it is found that under some cloud states, the overall H values are relatively low even if the best five bands are selected. For example, red curves in the first columns of Figures 7 and 8 show the information contents obtained from the best five bands for optically thin cirrus clouds with τ = 0.1. The overall H values are approximately 3, suggesting that the largest IR retrieval uncertainty occurs when the cloud is optically thin.

Figure 7.

Information content H (in bits) for different band combinations and different cloud states. In each panel, the black curve indicates the H of a single band. The first best band with highest H can be found from the black curve. The blue curve indicates H of a combination of the current band and the first best band. Similarly, the second best band can be found from the blue curve. The green, yellow, and red curves indicate the H of three, four, and five bands. Corresponding first 5 best bands are given at the bottom of each panel. Ice cloud r_eff values are 15 μm for all panels. First row: high clouds with h = 15 km; from left to right: τ increases from 0.1 to 5.0. Second row: moderately high clouds with h = 10 km.

Figure 8.

Same as Figure 7, but ice cloud r_eff values are 50 μm for all panels.

Figure 9 shows the change in retrieval uncertainty with an increase in the number of measurements. Generally, retrieval uncertainty decreases and approaches the minimum if more than three channels are used. Here the uncertainty of ln(τ) (first row in Fig. 9) decreases first if only one measurement is made, suggesting that the selected best channel is always sensitive to cloud optical thickness and uncertainty of ln(τ) can be greatly reduced with only one measurement. Uncertainties of r_eff and h begin to reduce if two or more measurements are made. Furthermore, uncertainty of ln(τ) decreases slowly for optically thin cloud (black curve in panel a) and rapidly for high cloud (red curve in panel c). Uncertainty of r_eff does not change significantly for optically thin and thick clouds (black and red curves in panel d), suggesting that IR observations have difficulty in retrieving r_eff if τ is too small or too large. For a transparent cirrus cloud, uncertainty of r_eff decreases rapidly if r_eff is small, as shown with the black curve in panel e. The third row of Fig. 9 shows the uncertainties of h for different clouds. The maximum decrease rate of h uncertainty occurs when the cloud is opaque (τ > 5). On the contrary, it is difficult to retrieve h with existing IR observations when τ is small. In this case, multiple solutions (or a wide range of solution space) may exist and the retrieval algorithm cannot give further details of cloud-top height information. It should be pointed out that another possible situation is when h is above the tropopause, where the ambient air temperature may not change significantly with height, and the retrieval algorithm cannot find an accurate h since in the Jacobian matrix K_F/X approaches 0. As shown in Eq. (8), zero sensitivity does not improve S_p even if S_y is small.

Figure 9.

Uncertainty of each retrieval variable as a function of the number of measurements. First row: uncertainty of ln(τ). Second row: uncertainty of ln(r_eff). Third row: uncertainty of h. First column: fixed r_eff and h (15 μm and 10 km, respectively), τ varies from 0.1 to 10.0. Second column: fixed τ and h (1 and 10 km, respectively), r_eff varies from 5 to 50 μm. Third column: fixed τ and r_eff (1 and 15μm, respectively), h varies from 5 to 12.5 km.

Although using less than 5 measurements can efficiently reduce retrieval uncertainty to the minimum as mentioned, the needed combination of IR measurements varies significantly with cloud properties and the background atmosphere. Therefore, we suggest using all of the bands to maximize the overall information content if computational efficiency is acceptable.

6. Conclusion

We document an OE-based retrieval algorithm we developed to infer three key ice cloud properties (τ, r_eff, and h) from MODIS measurements and analyze the corresponding uncertainties. The retrieval algorithm is based on an accurate and computationally efficient forward model that simulates TOA radiances in the infrared spectral region. This model employs the CRTM clear-sky module, which mitigates simulation biases due to the shifted SRFs of several Aqua MODIS IR bands. Absorption, emission, and scattering of ice cloud particles are considered using pre-computed LUTs in the forward model, as discussed in Section 2 and previous studies. Meanwhile, the present model also simulates critical Jacobian matrices, such as K_F/X, ${\mathbf{K}}_{\mathbf{F}/{\mathit{\tau }}_g}$, K_F/B(T), and K_F/C, facilitating a more comprehensive and computationally efficient IR ice cloud retrieval framework for research and operational applications, which went beyond previous studies by providing quantitative estimation of uncertainties from the atmospheric state, surface, and ice particles.

Four types of uncertainties are considered in the retrieval algorithm: the measurement, fast RT model, the ancillary data fields (e.g., atmospheric state), and the assumed ice crystal habit uncertainties. Except for the measurement uncertainty, the other uncertainties are influenced by cloud properties and clear-sky conditions such as the atmospheric state and surface emission. Inter-comparisons between the four types of uncertainties show that: (1) ancillary parameter uncertainties are dominant; (2) the maximum impact of ancillary parameter uncertainties occur in absorption bands; (3) in window bands, the impact of ice particle habit uncertainty becomes more important and is comparable to the impact of ancillary parameter uncertainties; (4) uncertainties from observations and the fast RT model simulations are relatively small compared with ice particle and ancillary uncertainties.

Information content analysis is conducted based on the aforementioned model and error analysis. Three major conclusions can be gained from the study, which agree well with previous studies [e.g., Cooper et al., 2006]. First, retrieval of the three ice cloud properties using IR observations has the best performance when the cloud is moderately optically thick (τ ~ 1–5), located at a high altitude, and consists of relatively small particles. Under these conditions, the overall information content using the best 5 observations (IR channels) can exceed 10 bits. Overall information content decreases if the cloud is optically thin (τ ~ 0.1) because of the large uncertainty associated with h retrieval, and if the cloud consists of large particles (r_eff > 50 μm) or is optically thick (τ ~ 10) because of the large uncertainty associated with r_eff retrieval. Second, the importance of each band varies with cloud properties. For example, the thermal IR window bands are more important for optically thin cirrus cloud consisting of relatively small particles, whereas the mid-wave IR window band (e.g., 3.8 μm) is more important when the cloud is optically thick. Third, the overall information content increases with the increase of measurements. However, H increases slowly when more than three bands are involved, suggesting that the MODIS IR bands are not independent. Generally, it is difficult to select a fixed combination of several bands for ice cloud retrieval without the knowledge of cloud properties. Therefore, we suggest using all of the bands to maximize the overall information content if computational efficiency is acceptable.

Appendix A

Derivation of Jacobian matrix K_F/X:

A quasi-analytical method is used to derive K_F/X in the forward model. For a single-layer cloudy atmosphere, the downward radiance reaching the surface can be approximately expressed as

$$I_{\mathit{surf}}^{\downarrow }\left(\mu \right)=B\left(T_{\mathit{eff}\_c}^{\downarrow }\right){\epsilon }_{\_c}\left(\mu \right){\mathrm{\Gamma }}_{gl}\left(\mu \right)+I_{gl}^{\downarrow }\left(\mu \right),$$ (A1)

where μ is the cosine of radiance zenith angle (absolute value), ε_{_c} is cloud emissivity, T_eff__c is cloud effective temperature, Γ_gl is the transmittance of the lower atmosphere, and $I_{gl}^{\downarrow }$ is downward clear-sky emission of the lower atmosphere. Here we assume the cloud internal temperature varies linearly with height. The effective temperature of the cloud layer is defined as:

$$T_{\mathit{eff}\_c}^{\uparrow \downarrow }\left(\mu ,T_1,T_2\right)=B^{-1}\left(\frac{I_{\_c}^{\uparrow \downarrow }\left(\mu \right)}{{\epsilon }_{\_c}\left(\mu \right)}\right),$$ (A2)

where B⁻¹ is the inverse Planck function, T₁ and T₂ are the temperatures at the cloud upper and lower boundaries, and $I_{\_c}^{\uparrow \downarrow }\left(\mu \right)$ is the outgoing upward (or downward) cloud emission at the cloud layer boundary. In order to derive $T_{\mathit{eff}\_c}^{\uparrow \downarrow }$, we first calculate the outgoing cloud emission $I_{\_c}^{\uparrow \downarrow }\left(\mu \right)$ with reference boundary temperatures T_ref,1 (upper: 245 K) and T_ref,₂ (lower: 250 K) by using the 32-Stream DISORT, and follow by deriving an effective temperature $T_{\mathit{eff}\_c,\mathit{ref}}^{\uparrow \downarrow }$ of the reference layer with Eq. (A2). Finally, $T_{\mathit{eff}\_c}^{\uparrow \downarrow }\left(\mu ,T_1,T_2\right)$ for a cloud layer with arbitrary boundary temperatures T₁ and T₂ can be derived with a linear interpolation:

$$T_{\mathit{eff}\_c}^{\uparrow \downarrow }=T_1+\left(T_2-T_1\right)\times \frac{T_{\mathit{eff}\_c,\mathit{ref}}^{\uparrow \downarrow }-T_{\mathit{ref},1}}{T_{\mathit{ref},2}-T_{\mathit{ref},1}}.$$ (A3)

Cloud effective temperature is discussed in detail in Wang et al. [2013 2014].

The upward radiance at the surface is

$$I_{\mathit{surf}}^{\uparrow }=B\left(T_{\_s}\right){\epsilon }_{\_s}+\frac{2\left(1-{\epsilon }_{\_s}\right)}{\pi }{\displaystyle {\int }_0^1{I}_{\mathit{surf}}^{\downarrow }}\left(\mu \right)\mu d\mu ,$$ (A4)

where T_{_s} is the surface temperature, ε_{_s} is surface emissivity, and $I_{\mathit{surf}}^{\downarrow }$ is the downward radiance derived with Eq. (A1). The upward radiance at cloud bottom $I_{cb}^{\uparrow }$, omitting multiple reflections between cloud and the surface, is given by

$$I_{cb}^{\uparrow }\left(\mu \right)=I_{\mathit{surf}}^{\uparrow }{\mathrm{\Gamma }}_{gl}\left(\mu \right)+I_{gl}^{\uparrow }\left(\mu \right),$$ (A5)

where $I_{gl}^{\uparrow }$ is the upward clear-sky emission of the lower atmosphere. The upward radiance at cloud-top $I_{ct}^{\uparrow }$ consists of 1) diffusely and directly transmitted upward radiances from cloud bottom, 2) cloud upward emission, and 3) reflected downward radiance from the upper atmosphere:

$$I_{ct}^{\uparrow }\left({\mu }_v\right)=2{\displaystyle {\int }_0^1\left[I_{cb}^{\uparrow }\left(\mu \right){\mathrm{\Gamma }}_{\_c}\left(\mu ,{\mu }_v\right)+I_{gh}^{\downarrow }\left(\mu \right)R_{\_c}\left(\mu ,{\mu }_v\right)\right]}\mu d\mu +I_{cb}^{\uparrow }\left({\mu }_v\right)e^{-\frac{\tau }{{\mu }_v}}+B\left(T_{\mathit{eff}\_c}^{\uparrow }\right){\epsilon }_{\_c}\left({\mu }_v\right),$$ (A6)

where is the cosine of the viewing zenith angle, $I_{gh}^{\downarrow }$ is the upward clear-sky emission of the above cloud atmosphere, Γ_{_c} and R_{_c} are the azimuthal averaged bi-directional transmittance and reflectance of the cloud layer, respectively, and $e^{-\frac{\tau }{{\mu }_v}}$ is the cloud direct transmittance. The satellite observed radiance is

$$I_{\text{TOA}}^{\uparrow }\left({\mu }_v\right)=I_{ct}^{\uparrow }\left({\mu }_v\right){\mathrm{\Gamma }}_{gh}\left({\mu }_v\right)+I_{gh}^{\uparrow }\left({\mu }_v\right),$$ (A7)

where $I_{gh}^{\uparrow }$ is the upward clear-sky emission of the upper atmosphere and Γ_gh is the transmittance of the upper atmosphere. Satellite observations can be simulated using Equations (A4)–(A7) with known cloud and clear-sky optical properties. In these equations, ε_{_c}, Γ_{_c}, R_{_c}, and T_{eff_c} are functions of τ and r_eff. Additionally, $I_{gl}^{\uparrow }$, $I_{gl}^{\downarrow }$, $I_{gh}^{\uparrow }$, $I_{gh}^{\downarrow }$, Γ_gl, and Γ_gh are functions of the cloud-top height (h).

It is straightforward to derive the analytical expression of the Jacobian matrix K_F/X if the first order derivatives of the aforementioned variables with respect to τ, r_eff, and h are given. For example, the derivative of $I_{\mathit{surf}}^{\downarrow }$ with respect to τ is given by

$$\frac{\partial I_{\mathit{surf}}^{\downarrow }}{\partial \tau }={\mathrm{\Gamma }}_{gl}\left(\mu \right)\left[B\left(T_{\mathit{eff}\_c}^{\downarrow }\right)\frac{\partial {\epsilon }_{\_c}\left(\mu \right)}{\partial \tau }+\frac{\partial B}{\partial T}\frac{\partial T_{\mathit{eff}\_c}^{\downarrow }}{\partial \tau }{\epsilon }_{\_c}\left(\mu \right)\right].$$ (A8)

Similarly, derivatives of Equations (A4) ~ (A7) can be expressed as

$$\frac{\partial I_{\mathit{surf}}^{\uparrow }}{\partial \tau }=\frac{2\left(1-{\epsilon }_{\_s}\right)}{\pi }{\displaystyle {\int }_0^1\frac{\partial I_{\mathit{surf}}^{\downarrow }\left(\mu \right)}{\partial \tau }}\mu d\mu ,$$ (A9)

$$\frac{\partial I_{cb}^{\uparrow }\left(\mu \right)}{\partial \tau }=\frac{\partial I_{\mathit{surf}}^{\downarrow }}{\partial \tau }{\mathrm{\Gamma }}_{gl}\left(\mu \right),$$ (A10)

$$\frac{\partial I_{ct}^{\uparrow }\left({\mu }_v\right)}{\partial \tau }=2{\displaystyle {\int }_0^1\left[\frac{\partial I_{cb}^{\uparrow }\left(\mu \right)}{\partial \tau }{\mathrm{\Gamma }}_{\_c}\left(\mu ,{\mu }_v\right)+I_{cb}^{\uparrow }\left(\mu \right)\frac{\partial {\mathrm{\Gamma }}_{\_c}\left(\mu ,{\mu }_v\right)}{\partial \tau }\right]}\mu d\mu +2{\displaystyle {\int }_0^1\left[I_{gh}^{\downarrow }\left(\mu \right)\frac{\partial R_{\_c}\left(\mu ,{\mu }_v\right)}{\partial \tau }\right]}\mu d\mu +\frac{\partial I_{cb}^{\uparrow }\left({\mu }_v\right)}{\partial \tau }{e}^{-\frac{\tau }{{\mu }_v}}+,$$ (A11)

$$\frac{\partial I_{\text{TOA}}^{\uparrow }\left({\mu }_v\right)}{\partial \tau }=\frac{\partial I_{ct}^{\uparrow }\left({\mu }_v\right)}{\partial \tau }{\mathrm{\Gamma }}_{gh}\left({\mu }_v\right).$$ (A12)

Cloud scattering properties are pre-computed and stored in LUTs. The corresponding first order derivatives of the cloud optical properties shown in Equations (A8)–(A12) can be numerically derived using these LUTs.

Appendix B

Derivation of Jacobian matrices ${\mathbf{K}}_{{\mathit{\tau }}_g/\mathbf{T}}$, ${\mathbf{K}}_{{\mathit{\tau }}_g/\mathit{w}}$, ${\mathbf{K}}_{\mathbf{F}/{\mathit{\tau }}_g}$, and ${\mathbf{K}}_{\mathbf{F}/\mathbf{B}\left(\mathbf{T}\right)}$:

We derive the first two matrices numerically. Specifically, in one run of the forward RT model, we conduct three runs of the CRTM clear-sky module with unbiased (T and w) profiles, perturbed temperature profile (T + ΔT and w), and perturbed water vapor concentration profile (T and w + Δw), respectively. The temperature perturbation ΔT is a constant 0.1 K for all atmospheric layers, while Δw is assumed to be 5 % of w for all layers in practice. ${\mathbf{K}}_{{\mathit{\tau }}_g/\mathbf{T}}$ and ${\mathbf{K}}_{{\mathit{\tau }}_g/\mathit{w}}$ are two ml ×l matrices. Here m and l are the number of measurements and atmospheric layers, respectively. For example, ${\mathbf{K}}_{{\mathit{\tau }}_g/\mathbf{T}}$ can be expressed as a vector of m square sub-matrices:

$${\mathbf{K}}_{{\mathit{\tau }}_g/\mathbf{T}}={\left[{\mathbf{K}}_{{\mathit{\tau }}_g/\mathbf{T}}^1,{\mathbf{K}}_{{\mathit{\tau }}_g/\mathbf{T}}^2,\cdots ,{\mathbf{K}}_{{\mathit{\tau }}_g/\mathbf{T}}^i,\cdots ,{\mathbf{K}}_{{\mathit{\tau }}_g/\mathbf{T}}^m\right]}^{\mathbf{T}},$$ (B1)

where the superscript i indicates the i^th measurement, and the square sub-matrix ${\mathbf{K}}_{{\mathit{\tau }}_g/\mathbf{T}}^i\left(l\times l\right)$ is given by

$${\mathbf{K}}_{{\mathit{\tau }}_g/\mathbf{T}}^i=\left[\begin{array}{cccc}\frac{\partial {\tau }_{g1}^i}{\partial T_1}& \frac{\partial {\tau }_{g1}^i}{\partial T_2}& \cdots & \frac{\partial {\tau }_{g1}^i}{\partial T_l}\\ \frac{\partial {\tau }_{g2}^i}{\partial T_1}& \frac{\partial {\tau }_{g2}^i}{\partial T_2}& \cdots & \frac{\partial {\tau }_{g2}^i}{\partial T_l}\\ ⋮& ⋮& \ddots & ⋮\\ \frac{\partial {\tau }_{gl}^i}{\partial T_1}& \frac{\partial {\tau }_{gl}^i}{\partial T_2}& \cdots & \frac{\partial {\tau }_{gl}^i}{\partial T_l}\end{array}\right]=\left[\begin{array}{cccc}\frac{\partial {\tau }_{g1}^i}{\partial T_1}& 0& \cdots & 0\\ 0& \frac{\partial {\tau }_{g2}^i}{\partial T_2}& \cdots & 0\\ ⋮& ⋮& \ddots & ⋮\\ 0& 0& \cdots & \frac{\partial {\tau }_{gl}^i}{\partial T_l}\end{array}\right],$$ (B2)

where the subscript indicates the index of an atmospheric layer. For the j^th layer, $\frac{\partial {\tau }_{gj}^i}{\partial T_j}$ is given by

$$\frac{\partial {\tau }_{gj}^i}{\partial T_j}\approx \frac{{\tau }_{gj}^i\left(T_j+\mathrm{\Delta }T,w_j\right)-{\tau }_{gj}^i\left(T_j,w_j\right)}{\mathrm{\Delta }T}.$$ (B3)

A similar approach can be used to derive ${\mathbf{K}}_{{\mathit{\tau }}_g/\mathit{w}}$.

The last two Jacobian matrices ( ${\mathbf{K}}_{\mathbf{F}/{\mathit{\tau }}_g}$ and K_F/B(T)) have m rows and m × l columns, both of which can be expressed as a block diagonal matrix:

$${\mathbf{K}}_{\mathbf{F}/{\mathit{\tau }}_g}=\left[\begin{array}{cccc}{\mathbf{K}}_{F^1/{\mathit{\tau }}_g^1}& {\mathbf{K}}_{F^1/{\mathit{\tau }}_g^2}& \cdots & {\mathbf{K}}_{F^1/{\mathit{\tau }}_g^m}\\ {\mathbf{K}}_{F^2/{\mathit{\tau }}_g^1}& {\mathbf{K}}_{F^2/{\mathit{\tau }}_g^2}& \cdots & {\mathbf{K}}_{F^2/{\mathit{\tau }}_g^m}\\ ⋮& ⋮& \ddots & ⋮\\ {\mathbf{K}}_{F^m/{\mathit{\tau }}_g^1}& {\mathbf{K}}_{F^m/{\mathit{\tau }}_g^2}& \cdots & {\mathbf{K}}_{F^m/{\mathit{\tau }}_g^m}\end{array}\right]=\left[\begin{array}{cccc}{\mathbf{K}}_{F^1/{\mathit{\tau }}_g^1}& 0& \cdots & 0\\ 0& {\mathbf{K}}_{F^2/{\mathit{\tau }}_g^2}& \cdots & 0\\ ⋮& ⋮& \ddots & ⋮\\ 0& 0& \cdots & {\mathbf{K}}_{F^m/{\mathit{\tau }}_g^m}\end{array}\right].$$ (B4)

Again, superscripts in Eq. (B4) indicate the indices of the measurements. Each sub-matrix in Eq. (B4) is a vector consisting of l elements:

$${\mathbf{K}}_{F^i/{\mathit{\tau }}_g^i}=\left[\frac{\partial F^i}{\partial {\tau }_{g1}^i},\frac{\partial F^i}{\partial {\tau }_{g2}^i},\cdots ,\frac{\partial F^i}{\partial {\tau }_{gj}^i},\cdots ,\frac{\partial F^i}{\partial {\tau }_{gl}^i}\right].$$ (B5)

The j^th element $\frac{\partial F^i}{\partial {\tau }_{gj}^i}$ in vector ${\mathbf{K}}_{F^i/{\mathit{\tau }}_g^i}$ represents the sensitivity of the i^th measurement to the gas optical thickness of the j^th layer. $\frac{\partial F^i}{\partial {\tau }_{gj}^i}$ can be analytically derived by differentiating Equations (A4) ~ (A8) with respect to ${\tau }_{gj}^i$. A similar approach can be used to derive K_F/B(T).

Appendix C

Derivation of Jacobian matrix K_F/C:

Jacobian matrix K_F/C can be analytically derived by differentiating Eqs. (A1), and (A4) ~ (A8) with respect to the corresponding cloud scattering properties. For example, the derivative of $I_{\mathit{surf}}^{\downarrow }$ with respect to ε_{_c} is given by

$$\frac{\partial I_{\mathit{surf}}^{\downarrow }\left(\mu \right)}{\partial {\epsilon }_c\left(\mu \right)}=B\left(T_{\mathit{eff}\_c}^{\downarrow }\right){\mathrm{\Gamma }}_{gl}\left(\mu \right),$$ (C1)

Similarly, derivatives of Equations (A4) ~ (A8) with respect to ε_{_c} (μ_i) can be expressed as

$$\frac{\partial I_{\mathit{surf}}^{\uparrow }}{\partial {\epsilon }_{\_c}\left({\mu }_i\right)}=\frac{2\left(1-{\epsilon }_s\right)}{\pi }{\displaystyle {\int }_0^1\frac{\partial I_{\mathit{surf}}^{\downarrow }\left({\mu }^{\prime }\right)}{\partial {\epsilon }_{\_c}\left({\mu }^{\prime }\right)}\delta \left({\mu }^{\prime }-{\mu }_i\right)}{\mu }^{\prime }d{\mu }^{\prime },$$ (C2)

$$\frac{\partial I_{cb}^{\uparrow }\left(\mu \right)}{\partial {\epsilon }_{\_c}\left({\mu }_i\right)}=\frac{\partial I_{\mathit{surf}}^{\uparrow }}{\partial {\epsilon }_{\_c}\left({\mu }_i\right)}{\mathrm{\Gamma }}_{gl}\left(\mu \right),$$ (C3)

$$\begin{gathered} \frac{\partial I_{ct}^{\uparrow }\left({\mu }_v\right)}{\partial {\epsilon }_{\_c}\left({\mu }_i\right)}=2{\displaystyle {\int }_0^1\left[\frac{\partial I_{cb}^{\uparrow }\left(\mu \right)}{\partial {\epsilon }_{\_c}\left(\mu \right)}{\mathrm{\Gamma }}_{\_c}\left(\mu ,{\mu }_v\right)\delta \left(\mu -{\mu }_i\right)\right]}\mu d\mu + \\ \frac{\partial I_{cb}^{\uparrow }\left({\mu }_v\right)}{\partial {\epsilon }_{\_c}\left({\mu }_i\right)}{e}^{-\frac{\tau }{{\mu }_v}}+B\left(T_{\mathit{eff}\_c}^{\uparrow }\right)\delta \left({\mu }_i-{\mu }_v\right) \end{gathered}$$

$$\begin{gathered} \frac{\partial I_{ct}^{\uparrow }\left({\mu }_v\right)}{\partial {\epsilon }_{\_c}\left({\mu }_i\right)}=2{\displaystyle {\int }_0^1\left[\frac{\partial I_{cb}^{\uparrow }\left(\mu \right)}{\partial {\epsilon }_{\_c}\left(\mu \right)}{\mathrm{\Gamma }}_{\_c}\left(\mu ,{\mu }_v\right)\delta \left(\mu -{\mu }_i\right)\right]}\mu d\mu + \\ \frac{\partial I_{cb}^{\uparrow }\left({\mu }_v\right)}{\partial {\epsilon }_{\_c}\left({\mu }_i\right)}{e}^{-\frac{\tau }{{\mu }_v}}+B\left(T_{\mathit{eff}\_c}^{\uparrow }\right)\delta \left({\mu }_i-{\mu }_v\right), \end{gathered}$$ (C4)

$$\frac{\partial I_{\text{TOA}}^{\uparrow }\left({\mu }_v\right)}{\partial {\epsilon }_{\_c}\left({\mu }_i\right)}=\frac{\partial I_{ct}^{\uparrow }\left({\mu }_v\right)}{\partial {\epsilon }_{\_c}\left({\mu }_i\right)}{\mathrm{\Gamma }}_{gh}\left({\mu }_v\right),$$ (C5)

where δ(μ − μ_i) is the Dirac Delta function, which is 0 if μ ≠ μ_i, and infinity large if μ = μ_i. Similar approaches are used to derive the derivative of TOA radiance with respect to other cloud parameters, such as reflectivity (R__c) and transmissivity (Γ__c).