Skip to main content
NCBI home page
Wiley Open Access Collection logoLink to Wiley Open Access Collection
. 2019 Jul 24;6(7):1043–1056. doi: 10.1029/2019EA000586

Predictability of Weather and Climate

V Krishnamurthy 1,✉
PMCID: PMC6774281  PMID: 31598537

Abstract

The past developments in the predictability of weather and climate are discussed from the point of view of nonlinear dynamical systems. The problems ahead for long‐range predictability extending into the climate time scale are also presented. The sensitive dependence of chaos on initial conditions and the imperfections in the models limit reliable predictability of the instantaneous state of the weather to less than 10 days in present‐day operational forecasts. The existence of slowly varying components such as the sea surface temperature, soil moisture, snow cover, and sea ice may provide basis for predicting certain aspects of climate at long range. The regularly varying nonlinear oscillations, such as the Madden‐Julian Oscillation, monsoon intraseasonal oscillations, and El Niño‐Southern Oscillation, are also possible sources of extended‐range predictability at the climate time scale. A prediction model based on phase space reconstruction has demonstrated that monsoon intraseasonal oscillation can be better predicted at long leads.

Key Points

  • The predictability of weather forecast models is limited to less than 10 days because of the limit imposed by chaos and model imperfections

  • The existence of slowly varying components of the climate system and regularly varying phenomena provide basis for climate predictability

  • A prediction model of monsoon intraseasonal oscillation using phase space reconstruction shows that some aspects of climate can be predicted

1. Introduction

During the decade before the American Geophysical Union marked its 50th anniversary, several momentous scientific developments took place and had far‐reaching consequences in weather prediction. The most important among them were the discovery of chaos by Lorenz (1962, 1963a) and the operational weather forecasting using numerical models and computers in the United States and the United Kingdom (Lynch, 2007). During the 1950s, when the first numerical model forecast was made using the Electronic Numerical Integrator and Computer (ENIAC) by the Princeton group (Charney et al., 1950) and was followed by progress in dynamical prediction, linear statistical forecasting models were still widely used. It was, therefore, natural to ask whether the atmospheric flow, which is predictable by a set of governing equation, is also equally well predictable by linear statistical schemes. This problem was investigated by Lorenz (1962, 1963b) under the Statistical Forecasting Project at the Massachusetts Institute of Technology by deriving a set of 12 ordinary differential equations from a simplified two‐layer baroclinic model of the atmosphere. The solutions of this deterministic system were used as data for a linear statistical model to compare the predictions of the two methods. Since the observed time series of weather does not show any periodic or regular behavior and that the solutions of a linear model would be easily predictable, the deterministic model consisted of nonlinear advection processes, forcing from heating and dissipation through friction. Lorenz recognized that the solutions of the model should not be stationary or periodic. After several integrations of the model using a computer, Lorenz (1962) obtained solutions that varied like weather patterns with no evidence of periodicity. The time series of the solutions were unmistakably nonperiodic and exhibited continuous power spectrum. This discovery of deterministic chaos was reported by Lorenz (1962) at the International Symposium on Numerical Weather Prediction in Tokyo in 1960. It was also shown that the linear regression method using the data from the deterministic model produced mediocre and progressively worsening forecasts.

While repeating numerical experiments with the 12‐variable model, Lorenz (1963b) also discovered that an initial small difference between two solutions would grow in time and become as large as the difference between two randomly selected states. This sensitive dependence on initial condition is now known to be a fundamental property of chaos and was recognized to have tremendous implication on weather prediction. Because of the obvious errors in observations used as initial conditions, the dynamical predictions would be inaccurate in a few days, making long‐range predictions impossible. The seminal paper by Lorenz (1963a) demonstrated chaos in a simple three‐variable model and provided the fundamental instability theory and detailed topological structure of chaos. Any prediction brings up the natural question as to how far it can be trusted. The predictability of a model can be generally defined as the degree of accuracy with which the state of the system can be predicted in the future (Lorenz, 1984). Often, the predictability is measured in terms of the square of the difference (error) between the forecast and the observation. Predictability always refers to the particular model or method that is used for prediction. The specific phenomenon or process that is being forecast may determine the time scale of the predictability. For example, errors due to a convective process may grow faster than those due to baroclinic instability.

The discovery of the sensitive dependence of chaos on initial conditions necessitated quantification of predictability to determine how the errors grow with time and how long the predictions are accurate or reliable. A mathematical theory of the growth of small initial errors was provided by Lorenz (1963b, 1965) using a 28‐variable version of the two‐layer baroclinic model. Typically, small random errors grow linearly at first and then slow down due to nonlinearity before reaching saturation. The Global Atmospheric Research Program (GARP; Charney et al., 1966), an international program to conduct global observations and investigate the possibility of long‐range prediction, recognized the importance of Lorenz's work on chaos and predictability. Earlier theoretical studies (Eady, 1949; Thompson, 1957) had provided some estimates of atmospheric predictability. During the planning of GARP, error growth experiments with three models, two of which were global, were performed to determine the limit of atmospheric predictability. Out of these three models, the Mintz‐Arakawa model (Mintz, 1965) generated nonperiodic variability and revealed error growth somewhat similar to the study by Lorenz (1965). The GARP planning group estimated the doubling time of errors to be about 5 days. Since then, the estimates of doubling time of root mean square (RMS) errors and the time taken to reach saturation have been commonly used quantitative measures of the predictability of models. For example, Lorenz (1982) examined the daily forecasts of the operational model of the European Centre for Medium‐Range Weather Forecasts (ECMWF) and obtained upper and lower bounds of predictability with an estimate of the doubling time of small errors to be 2.4 days. A similar method to estimate of predictability was adopted by the U.S. National Centers for Environmental Prediction (NCEP) and the ECMWF for their weather forecasts for a number of years (e.g., Simmons & Hollingsworth, 2002). The reliability of the weather forecasts issued by present‐day operational models at major operational centers is limited to about 5–7 days (ECMWF, 2018).

The uncertainties in the observations and the imperfections in the prediction models are major sources of errors in weather forecasts. Because of the sensitive dependence on initial conditions, chaos imposes limitation on the accuracy of the prediction of instantaneous state of the weather system, which is less than about 10 days. Beyond the weather time scale, the predictability of climate variability is not well understood. However, in recent years, operational centers have been issuing forecasts of seasonal mean (Saha et al., 2006, 2014; Stockdale et al., 2011) and intraseasonal oscillations (ISOs) such as the Madden‐Julian Oscillation (MJO; Gottschalck et al., 2010). Although the prediction of the instantaneous states in the climate time scale is almost impossible with the current models, it is necessary to explore which aspects of climate variability can be predicted at long range and to establish the predictability of climate.

With the above background description of the history of chaos, this paper will discuss the predictability of weather and climate models. Since this article is part of the special issue of the Nonlinear Geophysics Focus Group for the centennial celebration of the American Geophysical Union, the predictability will be discussed mainly from the point of view of nonlinear dynamical systems, especially relying on the works of Lorenz. An accompanying article (McWilliams, 2019) in this special issue provides a broader account of the major works and legacy of Lorenz. Further, this paper will provide a review of the predictability as well as the methods and tools developed for assessing the predictability and will also present results with current models and newly developed methods of prediction and predictability.

2. Predictability of Weather

The predictability of weather models is generally assessed by determining the rate of error growth. The most commonly used method is to conduct an “identical twin” experiment in which a small perturbation is introduced in the initial condition. The two initial states are allowed to evolve by integrating the model, and the evolution of their difference, usually the RMS of the error, is used to calculate the growth rate of the error. Often, an ensemble of initial perturbations is introduced, and the growth rate of the ensemble mean of the errors is obtained for a better assessment of the predictability. An alternate method to study the predictability is to directly use the equations that govern the atmosphere or the weather system. Such an approach is feasible only if the model is simplified by using suitable approximations. The model should be simple enough to allow the derivation of the equations governing the errors. Although this method is impractical to be used with operational models, a theoretical understanding of the predictability can be achieved by using simple models.

2.1. Theory of Predictability With a Simple Model

The theory of error growth was first provided by Lorenz (1965) who applied it to a 28‐variable two‐layer quasi‐geostrophic model with thermal forcing and mechanical and thermal dissipation. Considering the errors to be small, the theory is limited to the growth of errors that are considered small and are governed by a set of linearized equations. From the theory of Lorenz (1965), for a system of dimension M, if x(t) is a small error at time t, the error growth from time t 0 to t 1 is given by

xt1=At1t0xt0, (1)

where A is a square matrix that depends on the behavior of the model's nonlinear equations between times t 0 and t 1. It was further shown that errors initially on a small sphere of radius ε evolve on to an ellipsoid whose semiaxes are εσ i, where σ 1, … , σ M are the singular values of A or square root of the eigenvalues of AA T, with A T as the transpose of A. If the singular values are arranged in a decreasing order of magnitude, error growth occurs if the singular value σ 1 exceeds unity. The orientation of the errors is represented by the eigenvectors of AA T. During a certain interval of time, depending on the magnitude of σ i, errors will grow in certain directions, while there will be decay in others. The overall or ultimate growth of small errors can be determined by considering the interval (t−t 0) to be large. The limiting values defined by

λi=limt→∞lnσi/t−t0 (2)

are called the Lyapunov exponents. These are generally independent of the initial state for many systems and represent average exponential growth or decay. To understand the local growth of errors, limited‐time Lyapunov exponents for short intervals of time are similarly defined. In the 28‐variable model, Lorenz (1965) showed that the local errors during different lengths of period exhibited both growth and decay depending on the values of σ i of the corresponding periods. A similar definition for the eigenvectors of AA T in the limit of large time interval yields the Lyapunov vectors.

For operational weather forecast models, it is not feasible to compute even a few limited‐time Lyapunov exponents. Alternately, in most predictability experiments with such models, small random errors are introduced at initial states and their evolutions are studied to estimate the error growth rate and the limit of predictability. The relation between the growth rate of random errors and Lyapunov exponents was studied by Krishnamurthy (1993) using a simple quasi‐geostrophic model. This 28‐variable model is similar to that used by Lorenz (1965) except for the inclusion of topography. For certain values of forcing and dissipation, this model's attractors are chaotic. An ensemble of small errors introduced at initial states was shown to grow exponentially for a while and then slow down before reaching the saturation value (Krishnamurthy, 1993).

A similar predictability experiment is presented here using a much larger version of the quasi‐geostrophic model. With a larger spectral expansion that includes 20 spatial scales in both zonal and meridional directions, the present model consists of 1,640 variables (or 1,640 coupled ordinary differential equations). This model also exhibits a chaotic behavior for the same values of forcing and dissipation used by Krishnamurthy (1993). The computed Lyapunov exponent spectrum shows 140 positive values corresponding to growing directions, one zero value along the direction of flow and 1,499 negative values for decaying directions. The largest Lyapunov exponent λ 1 is equal to 0.03. Random errors, each with a magnitude of 10−12, were introduced at 5,000 different points in the attractor, and their evolution was studied. The RMS error for 200 days, plotted in Figure 1, shows exponential growth for about 60 days and then slower growth during the nonlinear phase and saturation at about 100 days. The exponential growth during the first 60 days is governed by linear error dynamics when the errors are considered small. Since the errors do not grow forever, the slowdown occurs because of the nonlinearity of the system. When the errors reach saturation, their magnitude is as large as the difference between two randomly selected states of the attractor. The time taken to reach saturation or limit of predictability depends on the initial size of the error.

Figure 1.

Figure 1

Open in a new tab

Root mean square (RMS) error (blue) growth of initial random errors introduced at 5,000 points in the attractor of the two‐layer quasi‐geostrophic model and the fit (red) of nonlinear empirical formula of equation (3).

This kind of error growth is typical of forced, dissipative nonlinear systems and can be represented by an empirical formula for the RMS error E as

dE/dt=λ1E−SE2, (3)

where E s = λ 1/S is the saturation value of E, similar to the formula introduced by Lorenz (1969). Since the largest Lyapunov exponent λ 1 has been computed by the direct method using the error equations, the error growth according to equation (3) can be compared with the random error growth, as shown in Figure 1. Except for the initial few days, the empirical formula 3 is a good approximation for the random error growth. For most part of the linear regime, the exponential growth is governed by the largest Lyapunov exponent λ 1 = 0.03 (per 3‐hr time unit), which corresponds to a doubling time of about 2.9 days. During a small initial period of adjustment, the error growth is determined by several exponents, and soon after, the largest exponent starts to dominate. The quadratic term of E seems to be a good approximation for the nonlinear growth regime, while the mean saturation value is given by √2 times the standard deviation of the time series of the attractor.

2.2. Predictability of Operational Forecast Models

After the first predictability study with general circulation models (Charney et al., 1966), and as the models improved, the error growth studies were conducted regularly (e.g., Smagorinsky, 1969; Williamson & Kasahara, 1971). When the weather forecasting became fairly well established using more realistic global models, Lorenz (1982) conducted a predictability experiment using the concepts of dynamical systems. For this purpose, the experiment used the daily weather forecasts from 1 to 10 days in advance issued daily by the ECMWF. Exploiting the fact that the forecasts are issued daily, Lorenz (1982) devised a novel method to assess the error growth without conducting any new model predictions with perturbations in the initial states. The analysis based on observation, used as the initial state, for today's weather and the 1‐day forecast for today may differ by a moderately small amount. Therefore, the 1‐day forecast can be considered as today's analysis plus a small error. The growth of this error in 1 day is given by the difference between the 2‐day forecast and the 1‐day forecast for tomorrow. Subsequent evolution of this error can be determined by following the difference between the two forecasts that were initiated 1 day apart. Similarly, the 2‐day error growth can be determined by the difference between two forecasts that were initiated 2 days apart, and so on, although the initial errors may be lager.

Using the ECMWF 10‐day forecasts of 500‐hPa geopotential height field for the period 1 December 1980 to 10 March 1981, Lorenz (1982) computed the global average of the RMS errors. Following the method described earlier, the average RMS errors were computed for 1‐ up to 10‐day errors. Additionally, the errors between the forecasts and the analysis (0‐day forecast) were also presented. The errors computed as the difference between two forecasts implicitly assumes that the model is perfect and, therefore, provide an upper bound for the predictability. The errors between the forecast and the analysis result from both the uncertainty in the observation and the imperfections of the model and represent the lower bound of predictability. The results showed that the smallest initial error occurred for the 1‐day error and increased as the difference in the forecast days increased, and all the errors grew in a similar manner (see Figure 1 in Lorenz, 1982). To compute the growth rate of small errors, the experiment fitted the empirical formula 3 to the errors and estimated the doubling time of small errors to be 2.4 days. For a number of years, the ECMWF adopted the method introduced by Lorenz (1982) to evaluate the predictability of its operational forecasts in a more detailed manner including regional variations (Simmons & Hollingsworth, 2002).

In the present study, the predictability experiment of Lorenz (1982) is repeated with the forecasts from the current operational climate model of the NCEP in the United States to get an understanding of the present status. For this purpose, the retrospective forecasts generated by the NCEP Climate Forecast System version 2 (CFSv2) have been used. CFSv2 is a coupled model with components representing atmosphere (T126 horizontal resolution), ocean, land, and sea ice (Saha et al., 2014) and has been used for operational seasonal forecasts since 2011. The forecasts of the 500‐hPa geopotential heights generated by CFSv2 from initial states during December–February of 1982–2009 were analyzed in exactly the same manner as in the study by Lorenz (1982). However, the present analysis is limited to the growth of 1‐, 5‐, and 10‐day errors, which are differences between forecasts that are initiated 1, 5, and 10 days apart, respectively. These errors are differences between two forecasts of the same model and will be referred to as predictability errors. The error between analysis and the forecast initiated on day 0, referred to as the forecast error, is the difference between solutions of the two different systems, the model, and the true atmospheric system. These errors, which are global averages of the RMS error in 500‐hPa geopotential height, are shown in Figure 2a for the first 35 days. The error growths are similar to those in the study by Lorenz (1982) except that the initial errors in CFSv2 are smaller. The saturation value is reached in about 20–25 days. The three predictability errors grow at the same rate and are displaced from one another because of different initial values. The forecast error (black curve in Figure 2a) also includes the errors due to model imperfection. The forecast error and predictability errors will tend to converge as the models become more realistic.

Figure 2.

Figure 2

Open in a new tab

(a) Global average of RMS errors in CFSv2 forecasts of 500‐hPa geopotential height for the 1‐day error (blue), 5‐day error (green), 10‐day error (red), and forecast error (black). (b) Global average of the 1‐day error (blue) and empirical fit (red) with the nonlinear version of equation (3) and empirical fit (green) with the linear version of equation (3). Units are in m2/s2. RMS = root mean square error; CFSv2 = NCEP Climate Forecast System version 2.

The growth rate of errors is determined by fitting the nonlinear empirical formula given by equation (3) to the 1‐day predictability error, which has the smallest initial error. The 1‐day error and the error curve with the nonlinear formula are shown in Figure 2b, indicating good correspondence. The nonlinear formula was fitted with a value of λ 1 that corresponds to a doubling time of 2.0 days for the errors, which is quite close to the estimate by Lorenz (1982). In Figure 2b, the error curve of the nonlinear empirical formula has been extrapolated backward to start with a very small value so that the exponential growth of small errors (i.e., error growth governed by linear error dynamics) is shown more explicitly. Also shown in Figure 2b is the fit of the empirical formula with just the linear term in equation (3) using the same value of λ 1. The nonlinear growth of error starts when the linear formula and nonlinear formula diverge. From Figure 2b, it is clear that the initial value of the 1‐day error is very close to the point where the nonlinear error growth starts, indicating that even the one forecast has a large initial error. To reach the nonlinear phase in about 20 days, the initial error must be less by about 3 orders of magnitude.

The geopotential height field used in the previous discussion places more weight on the middle and higher latitudes because of its lack of variation in the tropics. The instabilities that are responsible for the weather in the tropics and extratropics are different and may play an important role in determining the error growth in the respective regions. Using the CFSv2 retrospective forecasts for the period 1982–2009, a similar error analysis was performed for the precipitation over the Indian monsoon region and the tropical Pacific with forecasts from initial states during May and November, respectively (Krishnamurthy, 2017a). The forecast errors, shown in Figure 3, are averaged over 70–110°E, 10–30°N, in the Indian region and over 160°E–160°W, 30–10°S, in the Pacific region. The error growth is similar to the height field with growth phase and saturation although there are fluctuations because of using the precipitation field and averaging over a smaller area. The best fits of the nonlinear and linear versions of the empirical formula of equation (3) are also shown in Figure 3 to estimate the growth rates. In both the Indian region (Figure 3a) and the Pacific region (Figure 3b), the initial error is seen to be already in the nonlinear phase of the growth. The doubling time of small errors is estimated to be 7.7 and 13.9 days for the Indian and Pacific regions, respectively. This growth rate is slower compared to the error growth of the height field discussed earlier because of different instabilities. The error growth in the extratropics mainly results from the baroclinic and barotropic instabilities, whereas propagation of convection and ocean and land processes on broad space and time scales are responsible in the tropics. Similar differences in the growth rate between the tropics and the extratropics were also noted in a different earlier version of the NCEP model (Straus & Paolino, 2009).

Figure 3.

Figure 3

Open in a new tab

RMS errors (blue) in CFSv2 forecasts of precipitation averaged over (a) the Indian region of 70–110°E, 10–30°N, and (b) the Pacific region of 160°E–160°W, 30–10°S ,in model forecasts from May and November initial conditions, respectively. Empirical fit (red) with the nonlinear version of equation (3) and empirical fit (green) with the linear version of equation (3). Units are in mm/day. RMS = root mean square error; CFSv2 = NCEP Climate Forecast System version 2.

3. Predictability of Climate

The weather prediction models at major operational forecasting centers undergo periodic improvements by making modifications based on better understanding of the dynamics and various processes of the atmosphere, while the observing systems and data assimilation have also evolved. Because of the sensitive dependence of chaos on initial conditions, these centers have been providing probabilistic forecasts for the last several years using an ensemble of initial conditions that are based on some form of Lyapunov vectors (Buizza, 2006; Molteni et al., 1996; Toth & Kalnay, 1997). The forecasting centers also assess the performance of their models on a regular basis with quantitative estimates of predictability. Even after several decades of progress in weather prediction, reliable forecasts of the instantaneous state of the atmosphere is limited to less than 10 days (e.g., ECMWF, 2018). Therefore, the prediction of climate is even more challenging.

Despite the known limits on predictability beyond the weather time scale, the forecasting centers have been issuing climate forecasts on seasonal time scales for the past several years (Saha et al., 2006, 2014; Stockdale et al., 2011). Even earlier, the prediction of time averages had been investigated based on the assumption that the monthly and seasonally averaged anomalies may be determined by low‐frequency planetary waves rather than by synoptic‐scale instabilities (e.g., Shukla, 1985). The existence of atmospheric regimes, such as blocking, and a similar behavior of almost intransitivity was also considered to offer long‐range predictability of time‐averaged patterns (Lorenz, 1984).

3.1. Basis for Long‐Range Predictability

A more optimistic reason for climate prediction and the scientific basis for long‐range predictability stem from the existence of features in the climate systems that vary slowly (Lorenz, 1984). One such example is the sea surface temperature (SST), which varies slowly because of the large heat capacity of the ocean. Other features, such as sea ice, snow cover, and land processes, also vary slowly. The interaction between these slowly varying components of the climate system and the atmosphere may result in certain phenomena that vary on a longer time scale. Based on a model study, Charney and Shukla (1981) showed that the synoptic‐scale flow instabilities account for the interannual variability in the midlatitudes, whereas the boundary forcing of SST determines the variability in the low latitudes. According to the hypothesis of Charney and Shukla (1981), the interannual variability of the seasonal mean in the tropics is mainly determined by slowly varying components such as SST, albedo, sea ice, and soil moisture. Further model experiments showed that the tropical circulation and precipitation are strongly determined by the underlying SST with very little sensitivity to the changes in the initial conditions of the atmosphere (Shukla, 1998). The El Niño‐Southern Oscillation (ENSO) is the best known example of slowly varying phenomenon that results from ocean‐atmosphere interaction in the tropical Pacific. The prediction of conditions associated with the ENSO has also seen more success in seasonal prediction by the climate forecasting centers.

There has been a growing realization that the predictions at weather and climate time scales should be seamless. So far, with the somewhat better prediction of ENSO (Kim et al., 2012; Krishnamurthy, 2017b), the climate prediction has been focused on seasonal prediction. To bridge the gap between weather and climate predictions, strong suggestions have been made to accord high priority to predictions at the intraseasonal time scale (National Academies of Science, Engineering and Medicine, 2016; National Research Council, 2010). Recently, projects on prediction at subseasonal to seasonal time scales have been examining the scientific basis and sources of predictability (Robertson & Vitart, 2019). More accurate predictions for 2 weeks and beyond will be beneficial to various sectors of the society. Although the observations and models have improved, the prediction of climate at intraseasonal to seasonal time scales is a challenging problem. The complex interactions among atmosphere, ocean, land, and cryosphere must be properly represented in addition to the better specification of the initial state of the climate system. An equally important problem is the understanding of the predictability of climate.

The fundamental question that is not yet satisfactorily answered is whether climate is predictable (Lorenz, 2006). To answer this, it is necessary to identify the sources of climate predictability in terms of processes and phenomena at intraseasonal to seasonal time scales, to start with. The earlier discussion has suggested that the variability associated with slowly varying components, such as SST, soil moisture, and snow cover, is a possible source of long‐range predictability. Another possible source is the existence of more regularly varying phenomena such as the MJO (Madden & Julian, 1971, 1972), monsoon intraseasonal oscillations (MISOs) in the Indian region (e.g., Krishnamurthy & Shukla, 2007, 2008), and extratropical oscillations (Ghil & Robertson, 2002; Ghil et al., 2018). The MJO and MISOs are coherent patterns of convection and circulation varying in a nearly periodic manner and exert considerable influence in the tropical climate and beyond. These are nonlinear oscillations embedded within the climate variability at the intraseasonal time scale. These oscillations can be thought of as unstable periodic orbits embedded in a chaotic attractor from a standpoint of nonlinear dynamical systems. It is well known that the chaotic orbit of a dynamical system exhibits continuous close approaches with the unstable periodic orbits (Lorenz, 1963a). Therefore, oscillations such as the MJO and MISO, offer a pathway to exploit their nearly periodic behavior, although nonlinear, for long‐range prediction. Although it may not be possible to reliably predict the instantaneous states of climate, certain aspects of the climate variability seem to be predictable at long range. Several forecasting centers are engaged in projecting suitably filtered MJO signals from observations on daily model forecasts to predict the amplitude and phase of the MJO at the intraseasonal time scale (Gottschalck et al., 2010). A recent study (Krishnamurthy & Sharma, 2017) demonstrated the long‐range predictability of certain aspects of climate by constructing a prediction model for MISO based on the concepts and methods of nonlinear dynamical systems. Further discussion of the climate predictability will be provided by using the Indian monsoon as an example.

3.2. Climate Variability of the Indian Monsoon

The Indian monsoon is a highly seasonal phenomenon occurring mainly during June–September and covers the Indian subcontinent and the Indian Ocean. The southwesterly winds from the warm Indian Ocean bring copious moisture and rain over India during the monsoon season. However, the seasonal mean rainfall averaged over India exhibits considerable interannual variability, as shown in Figure 4a for the period 1901–2015, with several strong (weak) monsoon years with rainfall in excess (deficit) beyond one standard deviation. There is a strong association between the seasonal rainfall over India and ENSO, in accordance with the Charney‐Shukla hypothesis. As shown in the (strong‐weak) composites of seasonal rainfall (Figure 4b) and SST (Figure 4c), the strong (weak) monsoon years are associated with La Niña (El Niño) conditions. Thus, the monsoon‐ENSO relation offers possible long‐range predictability of the seasonal rainfall.

Figure 4.

Figure 4

Open in a new tab

(a) June–September seasonal anomaly of rainfall averaged over India for the period 1901–2015. (Strong‐weak) monsoon composite of June–September seasonal anomaly of (b) rainfall and (c) sea surface temperature (SST).

Within a season, the daily rainfall averaged over India shows strong intraseasonal variability, as shown in Figure 5a for the monsoon season of 1941. The variability consists of active periods with above‐normal rainfall and break periods with below‐normal or no rainfall, each phase lasting for several days. The active‐break cycle is actually composed of ISOs of different periods (Krishnamurthy & Shukla, 2007, 2008). These ISOs can be extracted by applying multichannel singular spectrum analysis (MSSA; Ghil et al., 2002) on daily rainfall anomalies. For this purpose, the daily rainfall data set over India on a 0.25° × 0.25° horizontal grid, prepared by the India Meteorological Department (IMD; Pai et al., 2014), for the period 1901–2015 was used. The MSSA was applied on daily rainfall anomalies for June–September of 1901–2015, yielding oscillatory and seasonally persisting eigenmodes. The two leading oscillations were found to have spectra centered at 45 and 30 days and showed both zonal and meridional propagation. The leading persisting modes were found to be related to ENSO and the Indian Ocean Dipole. Further discussion will be limited to the leading MISO with a 45‐day period, which has been used to demonstrate long‐range predictability of climate. Similar usage of oscillatory signal in ENSO demonstrated skill in extended prediction (Ghil & Jiang, 1998).

Figure 5.

Figure 5

Open in a new tab

Daily rainfall anomaly (mm/day) averaged over India for June−September of 1941. (b) Power spectrum of the first principal component of the monsoon intraseasonal oscillation (MISO) reconstructed component (RC) in observed rainfall. (c) MISO RC (blue) and total rainfall anomaly (black) averaged over India for the period June–September of 1972.

For analyzing the leading MISO (or MISO for brevity, hereafter), the reconstructed component (RC) of the oscillation was computed, and its phase and amplitude were determined following the method provided by Ghil et al. (2002). The power spectrum of the first principal component of MISO is broadband and centered at 45 days (Figure 5b), consistent with the nonlinear nature of the oscillation. The comparison of the MISO rainfall and total rainfall anomaly, each averaged over India, indicates that MISO is clearly embedded in the total anomaly with proper phases (Figure 5c). The phase composites of the RC of the MISO was constructed for eight phase intervals, each of length π/4, of each cycle for the entire period of analysis. The phase composites (Figure 6) show northward and eastward propagation of the MISO with a peak active period during phases 2 and 3 and a peak break period during phases 6 and 7 over central India. This MISO will be used to construct a new prediction model based on the concepts of nonlinear dynamical systems that will demonstrate predictability of climate at extended range, as described in the next subsection.

Figure 6.

Figure 6

Open in a new tab

Phase composites of the reconstructed component (mm/day) of the monsoon intraseasonal oscillation in observed rainfall for the period 1901–1980. The phase interval is indicated in the top right corner of each panel.

3.3. Prediction and Predictability of MISO

The weather and climate systems are complex dynamical systems governed by a large number of dynamical variables that are nonlinearly coupled. The evolution of such forced‐dissipative systems is confined within an attractor of reduced dimension in phase space. This property has been used as the basis for the reconstruction of phase space from the time series of a limited number of variables (Abarbanel et al., 1993;Packard et al., 1980 ; Takens, 1981). This method captures the dynamics inherent in the data, independent of modeling assumptions (Sharma, 1995). A prediction model, which is called the phase space reconstruction model (PSRM), was developed by first reconstructing the low‐dimensional phase space and then finding an appropriate mapping to predict the future states in the phase space (Krishnamurthy & Sharma, 2017). The time‐delay embedding technique (Packard et al., 1980; Takens, 1981) is at the core of the approach for constructing the PSRM. In this technique, an m‐component phase vector X i is constructed from a time series x(t) as X i = {x 1(t i), x 2(t i), … , x m(t i)}, where x k(t i) = x[t i + (k − 1)τ] and τ is a time delay. The trajectory defined by a sequence of X i captures the dynamics underlying the observed time series in the reconstructed phase space without loss of inherent characteristics.

For constructing the reduced phase space, a method based on an orthonormal basis obtained from MSSA of lagged time series is employed. The lower‐dimensional eigenspace spanned by suitable MSSA modes provides a proper choice for the reconstruction of the reduced phase space. The next step is to find a suitable mapping for prediction. Any trajectory of a dynamical system comes arbitrarily close to any point arbitrarily often while visiting all regions of the attractor (Lorenz, 1963a). If two points in the phase space are very close, their subsequent trajectories remain close for a while before diverging. The local dynamics dictates the length of time the trajectories remain close to each other. The evolution of nearby trajectories from the past data can then be used to predict the evolution of the current state to the future time steps, conceptually similar to the analog method of Lorenz (1969). The closeness of the current state with any other state in the past is quantified by the Euclidean distance between the two in the reconstructed phase space. The states that are within a specified distance of the current state are referred to as the nearest neighbors. The reconstructed phase space thus needs to be an orthonormal set of coordinates so that the Euclidean distances can be computed properly. This is achieved by adopting MSSA to obtain the appropriate coordinate system of the reduced phase space. To predict the evolution of a given current state, the PSRM identifies the nearest neighbors within a sphere of specified small radius from the past data and then models the dynamical evolution using the trajectories of the nearest neighbors. The mean value of such an ensemble is obtained as the predicted evolution (Ukhorskiy et al., 2004).

The MISO over India was extracted by applying MSSA on the IMD daily gridded rainfall for the period 1901−1980. The PSRM was constructed using the reduced space of MISO as described earlier (Krishnamurthy & Sharma, 2017). Using the PSRM, predictions of daily MISO were made for the period 1982−2009. The eigenspace spanned by the principal components of the MISO RC was used as the reduced phase space of the PSRM. For each prediction, the nearest neighbors and their trajectories of the MISO from the past were identified, and the mean value of the ensemble provided the predictions.

An analysis of 700 such predictions of the MISO showed that the correlation between the PSRM prediction and observation was higher than 0.6 for up to a lead of 80 days (Krishnamurthy & Sharma, 2017). For example, the PSRM predictions of the MISO from seven different initial states during May 1991, shown in Figure 7a, indicate that the predictions closely capture the amplitude and phase of the observation for about 20–50 days. The PSRM is able to predict the observed spatial structure and phase of the MISO remarkably well at leads of 15 days (Figure 7b). These results have demonstrated long‐range predictability of MISO and have indicated the potential for predicting certain aspects of climate variability beyond the weather time scale.

Figure 7.

Figure 7

Open in a new tab

(a) Time series of the phase space reconstruction model (PSRM) predictions and observation for the period 1 May 1991 to 30 September 1991. The PSRM predictions are made from seven initial states in May. Monsoon intraseasonal oscillation reconstructed component (mm/day) for (b) PSRM prediction and (c) observation for 10, 15, 20, and 25 July 1991. The PSRM prediction is made from the initial state on 5 July 1991.

4. Summary

An account of some of the past accomplishments and the future challenges of predictability of weather and climate has been provided in this study. A basic question about the predictability of statistical forecasting models led to the discovery of chaos in deterministic nonlinear systems. The important property of sensitive dependence on initial conditions in chaotic systems renders the weather predictions to become progressively worse because of errors in observations used as initial conditions in models. The predictability studies have been carried out since the time when the first few global models were developed, and regular assessment of the predictability of the weather forecasting models is done at operational centers. Studies with idealized low‐order nonlinear models showed that the typical error growth consists of exponential growth followed by slower growth because of nonlinearities before reaching saturation. A linear theory of the dynamics of errors explains the exponential growth when the errors are small with the largest Lyapunov exponent more often controlling the growth rate. Although tremendous progress has been achieved in weather prediction models, observations, and data assimilation, reliable forecasts are limited to less than 10 days. While better representation of some of physical processes is still necessary, the magnitude of the errors in the initial conditions may pose a more challenging problem. The initial errors in the present‐day models are already in the nonlinear phase of the error growth. Any improvement in terms of a few days may require the errors in initial conditions to be reduced by an order of magnitude or more.

While the growing recognition of seamless prediction of weather and climate seems to be a desirable goal, some fundamental questions about the predictability of climate need to be addressed. With the present‐day models being incapable of predicting the instantaneous states beyond the weather time scale, there is a need to find answers for what is predictable at the climate scale and to identify the sources of long‐range predictability of climate. The optimism for long‐range predictability of climate comes from the existence of slowly varying components such as SST and soil moisture and more regularly varying oscillations such as monsoon ISOs, MJO, and ENSO. Instead of instantaneous states, the predictability of time‐averaged variability, the embedded nonlinear oscillations, and signals associated with persisting modes of climate must be investigated. The recent study employing the phase reconstruction method demonstrated that there is reliable long‐range predictability of monsoon ISO. Certain aspects of climate variability at intraseasonal and seasonal time scales can be predicted at extended range, and therefore, the predictability of phenomena such MJO and ENSO must be investigated. Ultimately, the prediction at the climate time scale must be made by global climate models. The demonstration of the predictability of climate by simple models brings optimism for better forecasts by operational models when they are capable of properly representing the sources of long‐range predictability. For example, a major problem with the operational model in predicting the monsoon ISO is the failure of the model to correctly capture the initial phase of the oscillation. Therefore, investigations with simple models and newer methods may shed light on the sources of climate predictability and the problems that need to be addressed by operational centers.

Acknowledgments

This work was supported by the National Science Foundation (grant 1338427), the National Oceanic and Atmospheric Administration (grant NA140OAR4310160), and the National Aeronautics and Space Administration (grant NNX14AM19G) from the United States. Discussions with A. Surjalal Sharma are gratefully acknowledged. The author thanks Michael Ghil and an anonymous reviewer for comments and criticisms. The IMD rainfall data can be obtained from http://www.imdpune.gov.in/library/Data_Sale.html, and NCEP reforecasts from https://www.ncdc.noaa.gov/data‐access/model‐data/model‐datasets/climate‐forecast‐system‐version2‐cfsv2#CFS Reforecasts.

Krishnamurthy, V. (2019). Predictability of weather and climate. Earth and Space Science, 6, 1043–1056. 10.1029/2019EA000586

References

  1. Abarbanel, H. D. I. , Brown, R. , Sidorowich, J. J. , & Tsimring, L. S. (1993). The analysis of observed chaotic data in physical systems. Reviews of Modern Physics, 65, 1331–1392. [Google Scholar]
  2. Buizza, R. (2006). The ECMWF ensemble prediction system In Palmer T., & Hagedorn R. (Eds.), Predictability of weather and climate (pp. 459–488). Cambridge: Cambridge University Press. [Google Scholar]
  3. Charney, J. G. , Fleagle, R. G. , Lally, V. E. , Riehl, H. , & Wark, D. Q. (1966). The feasibility of a global observation and analysis experiment. Bulletin of the American Meteorological Society, 47, 200–220. [Google Scholar]
  4. Charney, J. G. , Fjörtoft, R. , & von Neumann, J. (1950). Numerical integration of the barotropic vorticity equation. Tellus, 2, 237–254. [Google Scholar]
  5. Charney, J. G. , & Shukla, J. (1981). Predictability of monsoons In Lighthill J., & Pearce R. P. (Eds.), Monsoon dynamics (pp. 99–109). New York: Cambridge University Press. [Google Scholar]
  6. Eady, E. T. (1949). Long waves and cyclone waves. Tellus, 1(3), 33–52. [Google Scholar]
  7. European Centre for Medium‐Range Weather Forecast (2018). Annual report 2017. European Centre for Medium‐Range Weather Forecast, pp. 1–40.
  8. Ghil, M. , Allen, M. R. , Dettinger, M. D. , Ide, K. , Kondrashov, D. , Mann, M. E. , Robertson, A. W. , Saunders, A. , Tian, Y. , Varadi, F. , & Yiou, P. (2002). Advanced spectral methods for climatic time series. Reviews of Geophysics, 40(1), 1003 10.1029/2000RG000092 [DOI] [Google Scholar]
  9. Ghil, M. , Groth, A. , Kondrashov, D. , & Robertson, A. W. (2018). Extratropical sub‐seasonal‐to‐seasonal oscillations and multiple regimes: The dynamical systems view In Robertson A. W., & Vitart F. (Eds.), Sub‐seasonal to seasonal prediction: The gap between weather and climate forecasting (pp. 119–142). Amsterdam: Elsevier. [Google Scholar]
  10. Ghil, M. , & Jiang, N. (1998). Recent forecast skill for the El Niño/Southern Oscillation. Geophysical Research Letters, 25(2), 171–174. [Google Scholar]
  11. Ghil, M. , & Robertson, A. W. (2002). “Waves” vs. “particles” in the atmosphere's phase space: A pathway to long‐range forecasting? Proceedings of the National Academy of Sciences of the United States of America, 99, 2493–2500. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Gottschalck, J. , Wheeler, M. , Weickmann, K. , Vitart, F. , Savage, N. , Lin, H. , Hendon, H. , Waliser, D. , Sperber, K. , Nakagawa, M. , Prestrelo, C. , Flatau, M. , & Higgins, W. (2010). A framework for assessing operational Madden‐Julian Oscillation forecasts: A CLIVAR MJO Working Group project. Bulletin of the American Meteorological Society, 91, 1247–1258. [Google Scholar]
  13. Kim, H.‐M. , Webster, P. J. , & Curry, J. A. (2012). Seasonal prediction skill of ECMWF System 4 and NCEP CFSv2 retrospective forecast for the Northern Hemisphere winter. Climate Dynamics, 39, 2957–2973. [Google Scholar]
  14. Krishnamurthy, V. (1993). A predictability study of Lorenz's 28‐variable model as a dynamical system. Journal of the Atmospheric Sciences, 50, 2215–2229. [Google Scholar]
  15. Krishnamurthy, V. (2017a). Predictability of CFSv2 in the tropical Indo‐Pacific region at daily and subseasonal time scales. Climate Dynamics. 10.1007/s00382-017-3855-y [DOI] [Google Scholar]
  16. Krishnamurthy, V. (2017b). Seasonal prediction of South Asian monsoon in CFSv2. Climate Dynamics. 10.1007/s00382-017-3963-8 [DOI] [Google Scholar]
  17. Krishnamurthy, V. , & Sharma, A. (2017). Predictability at intraseasonal time scale. Geophysical Research Letters, 44, 8530–8537. 10.1002/2017GL074984 [DOI] [Google Scholar]
  18. Krishnamurthy, V. , & Shukla, J. (2007). Intraseasonal and seasonally persisting patterns of Indian monsoon rainfall. Journal of Climate, 20, 3–20. [Google Scholar]
  19. Krishnamurthy, V. , & Shukla, J. (2008). Seasonal persistence and propagation of intraseasonal patterns over the Indian monsoon region. Climate Dynamics, 30, 353–369. [Google Scholar]
  20. Lorenz, E. N. (1962). The statistical prediction of solutions of the dynamic equations. Proceedings of the International Symposium on Numerical Weather Prediction, Tokyo, 629–635.
  21. Lorenz, E. N. (1963a). Deterministic nonperiodic flow. Journal of the Atmospheric Sciences, 20, 130–141. [Google Scholar]
  22. Lorenz, E. N. (1963b). The predictability of hydrodynamic flow. Proceedings New York Academy of Sciences, 25, 409–432. [Google Scholar]
  23. Lorenz, E. N. (1965). A study of the predictability of a 28‐variable atmospheric model. Tellus, 17, 321–333. [Google Scholar]
  24. Lorenz, E. N. (1969). Atmospheric predictability as revealed by naturally occurring analogues. Journal of the Atmospheric Sciences, 26, 636–646. [Google Scholar]
  25. Lorenz, E. N. (1982). Atmospheric predictability experiments with a large numerical model. Tellus, 34, 505–513. [Google Scholar]
  26. Lorenz, E. N. (1984). Some aspects of atmospheric predictability In Burridge D. M., & Kallen E. (Eds.), Problems and prospects in long and medium rage weather forecasting (pp. 1–20). Berlin: Springer‐Verlag. [Google Scholar]
  27. Lorenz, E. N. (2006). Predictability—A problem partly solved In Palmer T., & Hagedorn R. (Eds.), Predictability of weather and climate (pp. 40–58). Cambridge: Cambridge University Press. [Google Scholar]
  28. Lynch, P. (2007). The origins of computer weather prediction and climate modeling. Journal of Computational Physics, 227, 3431–3444. [Google Scholar]
  29. Madden, R. A. , & Julian, P. R. (1971). Description of a 40–50 day oscillation in the zonal wind in the tropical Pacific. Journal of the Atmospheric Sciences, 28, 702–708. [Google Scholar]
  30. Madden, R. A. , & Julian, P. R. (1972). Description of global‐scale circulation cells in the tropics with a 40–50 day period. Journal of the Atmospheric Sciences, 29, 1109–1123. [Google Scholar]
  31. McWilliams, J. C. (2019). A perspective on the legacy of Edward Lorenz. Earth and Space Science, 6, 336–350. 10.1029/2018EA000434 [DOI] [Google Scholar]
  32. Mintz, Y (1965). Very long‐term integration of the primitive equations of atmospheric motion. Proc. WMO/IUGG Symposium on Research and Development Aspects of Long‐Range Forecasting, Boulder, Colarado, 1964, WMO Tech. Note, 66, 141–167.
  33. Molteni, F. , Buizza, R. , Palmer, T. N. , & Petroliagis, T. (1996). The ECMWF Ensemble Prediction System: Methodology and validation. Quarterly Journal of the Royal Meteorological Society, 122, 71–119. [Google Scholar]
  34. National Academies of Sciences, Engineering and Medicine (2016). Next generation Earth system prediction: Strategies for subseasonal to seasonal forecasts. Washington, DC: The National Academies Press; http://www.nap.edu/21873, 10.17226/21873 [DOI] [Google Scholar]
  35. National Research Council (2010). Assessment of intraseasonal to interannual climate prediction and predictability. Washington, DC: The National Academies Press; http://www.nap.edu/catalog.php?record_id=12878 [Google Scholar]
  36. Packard, N. , Crutchfield, J. , Farmer, D. , & Shaw, R. (1980). Geometry from time series. Physical Review Letters, 45, 712–715. [Google Scholar]
  37. Pai, D. S. , Sridhar, L. , Rajeevan, M. , Sreejith, O. P. , Satbhai, N. S. , & Mukhopadhyay, B. (2014). Development of a new high spatial resolution (0.25 × 0.25) long period (1901–2010) daily gridded rainfall data set over India and its comparison with existing data sets over the region. Mausam, 65, 1–18. [Google Scholar]
  38. Robertson A. W., & Vitart F. (Eds) (2019). Sub‐seasonal to seasonal forecasting: The gap between weather and climate forecasting (pp. 1–585). Amsterdam: Elsevier. [Google Scholar]
  39. Saha, S. , Moorthi, S. , Wu, X. , Wang, J. , Nadiga, S. , Tripp, P. , Behringer, D. , Hou, Y.‐T. , Chuang, H.‐L. , Iredell, M. , Ek, M. , Meng, J. , Yang, R. , Mendez, M. P. , Van den Dool, H. , Zhang, Q. , Wang, W. , Chen, M. , & Becker, E. (2014). The NCEP climate forecast system version 2. Journal of Climate, 27, 2185–2208. [Google Scholar]
  40. Saha, S. , Nadiga, S. , Thiaw, C. , Wang, J. , Wang, W. , Zhang, Q. , Van den Dool, H. M. , Pan, H.‐L. , Moorthi, S. , Behringer, D. , Stokes, D. , Peña, M. , Lord, S. , White, G. , Ebisuzaki, W. , Peng, P. , & Xie, P. (2006). The NCEP climate forecast system. Journal of Climate, 19, 3483–3517. [Google Scholar]
  41. Sharma, A. S. (1995). Assessing the magnetosphere's nonlinear behavior: Its dimension is low, its predictability, high (US National Report to IUGG, 1991–1994). Reviews of Geophysical, 33, 645–650. [Google Scholar]
  42. Shukla, J. (1985). Predictability. Advances in Geophysics, 28B, 87–122. [Google Scholar]
  43. Shukla, J. (1998). Predictability in the midst of chaos: A scientific basis for climate forecasting. Science, 282, 728–731. [DOI] [PubMed] [Google Scholar]
  44. Simmons, A. J. , & Hollingsworth, A. (2002). Some aspects of the improvement in skill of numerical weather prediction. Quarterly Journal of the Royal Meteorological Society, 128, 647–677. [Google Scholar]
  45. Smagorinsky, J. (1969). Problems and promises of deterministic extended range forecasting. Bulletin of the American Meteorological Society, 50, 286–311. [Google Scholar]
  46. Stockdale, T. N. , Anderson, D. L. T. , Balmaseda, M. A. , Doblas‐Reyes, F. , Ferranti, L. , Mogensen, K. , Palmer, T. N. , Molteni, F. , & Vitart, F. (2011). ECMWF seasonal forecast system 3 and its prediction of sea surface temperature. Climate Dynamics, 37, 455–471. [Google Scholar]
  47. Straus, D. M. , & Paolino, D. (2009). Intermediate error growth and predictability: Tropics versus mid‐latitudes. Tellus A, 61, 579–586. [Google Scholar]
  48. Takens, F. (1981). Detecting strange attractors in turbulence In Rand D. A., & Young L.‐S. (Eds.), Dynamical Systems and Turbulence, Lecture Notes in Mathematics 898 (pp. 366–381). Berlin: Springer. [Google Scholar]
  49. Thompson, P. D. (1957). Uncertainty of initial state as a factor in the predictability of large scale atmospheric flow patterns. Tellus, 9(3), 275–295. [Google Scholar]
  50. Toth, Z. , & Kalnay, E. (1997). Ensemble forecasting at NCEP and the breeding method. Monthly Weather Review, (12), 3297–3319. [Google Scholar]
  51. Ukhorskiy, A. Y. , Sitnov, M. I. , Sharma, A. S. , & Papadopoulos, K. (2004). Global and multi‐scale features of solar wind‐magnetosphere coupling: From modeling to forecasting. Geophysical Research Letters, 31, L08802 10.1029/2003GL018932 [DOI] [Google Scholar]
  52. Williamson, D. L. , & Kasahara, A. (1971). Adaptation of meteorological variables forced by updating. Journal of the Atmospheric Sciences, 28, 1313–1324. [Google Scholar]

Articles from Earth and Space Science (Hoboken, N.j.) are provided here courtesy of Wiley

RESOURCES