Computation of extreme heat waves in climate models using a large deviation algorithm

Significance

We propose an algorithm to sample rare events in climate models with a computational cost from 100 to 1,000 times less than direct sampling of the model. Applied to the study of extreme heat waves, we estimate the probability of events that cannot be studied otherwise because they are too rare, and we get a huge ensemble of realizations of an extreme event. Using these results, we describe the teleconnection pattern for the extreme European heat waves. This method should change the paradigm for the study of extreme events in climate models: It will allow us to study extremes with higher-complexity models, to make intermodel comparison easier, and to study the dynamics of extreme events with unprecedented statistics.

Abstract

Studying extreme events and how they evolve in a changing climate is one of the most important current scientific challenges. Starting from complex climate models, a key difficulty is to be able to run long enough simulations to observe those extremely rare events. In physics, chemistry, and biology, rare event algorithms have recently been developed to compute probabilities of events that cannot be observed in direct numerical simulations. Here we propose such an algorithm, specifically designed for extreme heat or cold waves, based on statistical physics. This approach gives an improvement of more than two orders of magnitude in the sampling efficiency. We describe the dynamics of events that would not be observed otherwise. We show that European extreme heat waves are related to a global teleconnection pattern involving North America and Asia. This tool opens up a wide range of possible studies to quantitatively assess the impact of climate change.

Rare events, for instance extreme droughts, heat waves, rainfall, and storms, can have a severe impact on ecosystems and socioeconomic systems (1–3). The Intergovernmental Panel on Climate Change (IPCC) has concluded that strong evidence exists indicating that hot days and heavy precipitation events have become more frequent since 1950 (4, 5). However, the magnitude of possible future changes is still uncertain for classes of events involving more dynamical aspects, for instance hurricanes or heat waves (4, 6, 7). Estimates of the average time between two events of the same class, called return time (or return period), are key for assessing the expected changes in extreme events and their impact. This is crucial on a national level when considering adaptation measures and on the international level when designing policy to implement the Paris Agreement, in particular its Article 8 (https://en.wikisource.org/wiki/Paris_Agreement). Public or private risk managers need to know amplitudes of events with a return time ranging from a few years to hundreds of thousands of years when the impact might be extremely large.

The 2003 Western European heat wave led to a death toll of more than $\mathrm{70,000}$ people (8). Similarly, the estimated impact of the 2010 Russian heat wave was a death toll of 55,000 people, an annual crop failure of $\sim 25\%$, more than 1 million ha of burned areas, and $\sim$US$15 billion ($\sim 1\%$ of gross domestic product) of total economic loss (9). During that period, the temperature averages over 31 d at some locations were up to 5.5 SDs away from the 1970–1999 climate (9). As no event similar to those heat waves has been observed during the last few centuries, no past observations exist that would allow us to quantify their return times. If return times cannot be estimated from observations, we must rely on models.

Several scientific barriers need to be overcome, however, before we can obtain quantitative estimates of rare event return times from a model. One of them is that extreme events are observed so rarely that collecting sufficient data to study quantitatively their dynamics is prohibitively costly. This has led authors of past studies to either use models which are of a lesser quality than the up-to-date IPCC top class models or focus on a single event or a few events, which does not allow for a quantitative statistical assessment. Making progress for this sampling issue would also allow a better understanding of those rare event dynamics and strengthen future assessment of which class of models is suited for making quantitative predictions.

Rare Event Algorithms

In physics, chemistry, and biology rare events may matter: Even if they occur on timescales much longer than the typical dynamics timescales, they may have a huge impact. During recent decades, new numerical tools, specifically dedicated to the computation of rare events from the dynamics but requiring a considerably smaller computational effort, have been developed. They have been applied for instance to changes of configurations in magnetic systems in situations of first-order transitions (10–12), chemical reactions (13), conformal changes of polymer and biomolecules (14–17), and rare events in turbulent flows (18–22). Since their appearance (23), the analysis of these rare event algorithms also became an active mathematical field (24–28). Several strategies prevail, for instance genetic algorithms where an ensemble of trajectories is evolved and submitted to selections, minimum action methods, or importance sampling approaches.

Here we apply a rare event algorithm for sampling extreme events in a climate model. Given the complexity of the models and phenomena, this has long been thought to be impracticable for climate applications. A key success factor for this approach is to first clearly identify a restricted class of phenomena for which a rare event algorithm may be practicable. Then one has to choose among the dozens of available algorithms which one may be suited for this class of phenomena. Finally one has to develop the tools that will make one specific algorithm effective for climate observables. Matching these concepts coming from the rare event community and climate dynamics requires a genuine interdisciplinary approach, to master both the climate dynamics phenomenology and the probability concepts related to rare event algorithms. We study extreme heat waves as robust phenomena in current climate models, involving the largest scales of the turbulent dynamics, and use an algorithm dedicated to study large deviations of time-averaged quantities: the Giardina–Kurchan–Lecomte–Tailleur (GKLT) algorithm (29–31). As this algorithm was dedicated to compute large deviation rate functions in the infinite time limit, we have to pick the main ideas of the algorithm, but to twist its use to compute finite time observables. Moreover, we have to develop a further adaptation, aimed at computing return times rather than large deviation rate functions or tails of probability distribution functions. With this approach, based on statistical physics concepts, we compute the probability of events that cannot be observed directly in the model, the number of observed rare events for a given amplitude is multiplied by several hundred, and we can predict the return time for events that would require 1,000 times more computational resources.

The Jet Stream Dynamics and Extreme Heat Waves

Midlatitude atmospheric dynamics are dominated by the jet streams (one per hemisphere). The jet streams are strong and narrow eastward air currents, located at about ${45}^{\circ }\text{N}$ or ${45}^{\circ }\text{S}$, with maximum velocity of the order of $40{\text{m}\cdot \text{s}}^{-1}$ close to the tropopause (see Fig. 1A). The climatological position of the Northern Hemisphere jet stream in our model is shown in Fig. 1B, which represents the time average of the kinetic energy due to the horizontal component of the velocity field at 500-hPa pressure surfaces. The jet stream’s meandering dynamics, due to nonlinear Rossby waves, are related to the succession of anticyclonic and cyclonic anomalies which characterize weather at midlatitudes. It is well known that midlatitude heat waves, like the 2003 Western European heat wave or the 2010 Russian heat waves, are due to rare and persistent anticyclonic anomalies (or fluctuations) that arise as either Rossby wave breaking (blockings), or shifts of the jet stream, or more complex dynamical events leading to a stationary pattern of the jet stream.

Fig. 1.

(A) Snapshot of wind speed velocity at the top of the troposphere, showing the jet stream over North America (image courtesy of NASA/Goddard Space Flight Center Scientific Visualization Studio). (B) Average horizontal kinetic energy at 500 hPa (midtroposphere) in the Plasim model, showing the averaged Northern Hemisphere jet stream.

Studying extreme heat waves then amounts to studying the nonlinear and turbulent dynamics of the atmosphere. Two key dynamical variables are the temperature and pressure fields. One could look at pressure maps at some value of the geopotential height (the most convenient vertical coordinate). Equivalently, it is customary to look at the geopotential height value on a surface defined by a fixed pressure.

Heat Waves in the Planet Simulator Model

We use the Planet Simulator (Plasim) model (32). Plasim gives a reasonably realistic representation of atmospheric dynamics and of their interactions with the land surface and with the mixed layer of the ocean; it includes parameterizations of radiative transfers and cloud dynamics. While Plasim features about ${10}^5$ df, it is simpler and less computationally demanding than the top-class IPCC models used for assessing the projection of temperature increase. It is nevertheless in the class of models used to discuss extreme heat waves in the last IPCC report (for instance, ref. 33). Our aim is to demonstrate the huge potential of rare event algorithms for this class of models and to advocate the feasibility of this approach for top-class IPCC models in the near future.

Heat waves can be defined as rare and long-lasting anomalies (fluctuations) of the surface temperature over an extended area (34, 35). We consider

$$a=\frac{1}{T}{\int }_0^T\stackrel{\sim }{A}(t)\text{d}t\hspace{0.17em}\text{where}\hspace{0.17em}\stackrel{\sim }{A}(t)=\frac{1}{\mathcal{A}}{\int }_{\text{Area}}{T}_S(𝐫,t)\text{d}𝐫,$$ [1]

where $𝐫$ is the spatial variable, $t$ is time, $T_S$ is the surface temperature anomaly with respect to its averaged value, $\mathcal{A}$ is the surface area, and $T$ is the heat wave duration. The relevant value for $T$ depends on the application of interest. We vary $T$ from a few weeks to several months and discuss the results for $T=90$ d. The spatial average is over Western Europe, the region over land surface with latitudinal and longitudinal boundaries ${36}^{\circ }$N–${70}^{\circ }$N and ${11}^{\circ }$W–${25}^{\circ }$E (Fig. 2A). We study the upper tail of the probability distribution function (PDF) of $a$, denoted $P(a,T)$.

Fig. 2.

(A) The red color marks the area of Europe over which the temperature is averaged. (B) Time series of European surface temperature anomaly, 6 h (light blue) and 90 d running mean (dark blue), during 360 d and $\mathrm{1,000}$ y (Inset). The red triangles and circles feature one local maximum of the temperature anomalies, as an example of a $2\text{K}$ heat wave lasting $90$ d.

The instantaneous $T_S$ has SD $\sigma \approx 1.6$ K and is slightly skewed toward positive values. Its autocorrelation time is ${\tau }_c\approx 7.5$ d, which is the typical time for synoptic fluctuations (at a scale of about 1,000 km). An example of a time series of the $90$-d averaged Europe temperature anomaly is shown in Fig. 2B.

Importance Sampling and Large Deviations of Time-Averaged Temperature

We first explain importance sampling, a crucial probabilistic concept for the following discussion. We sample $N$ independent and identically distributed random numbers from a PDF $\rho$ and want to estimate ${\gamma }_B$, the probability to be in a small set $B$ (Fig. 3A). We will obtain about $N{\gamma }_B$ occurrences in the set $B$, from which we can estimate ${\gamma }_B$. An easy calculation (24) shows that the relative error of this estimate is of the order of $1/\sqrt{N{\gamma }_B}$. For instance, if ${\gamma }_B$ is of the order of ${10}^{-2}$, estimating ${\gamma }_B$ with a relative error of $1\%$ requires a huge sample size, of the order of ${10}^6$. However, if we rather sample $N$ random numbers ${\stackrel{\sim }{X}}_n$ from the distribution $\stackrel{\sim }{\rho }$ (see Fig. 3(a)), where $\rho (x)=L(x)\stackrel{\sim }{\rho }(x)$, with $L$ conveniently chosen, then the event may become common: this is importance sampling. From the formula $\rho =L\stackrel{\sim }{\rho }$, we have the estimate ${\stackrel{\sim }{\gamma }}_B=\frac{1}{N}{\sum }_{n=1}^{N}L\left({\stackrel{\sim }{X}}_n\right)1_B\left({\stackrel{\sim }{X}}_n\right)$, where $1_B$ is the indicator function of the set $B$. If the rare event is actually common for $\stackrel{\sim }{\rho }$, this estimate gives a relative error of order $1/\sqrt{N}$ (see (24) for a precise formula). Then, in order to estimate ${\gamma }_B$ with a relative error of $1\%$, we need a sample size of order of ${10}^4$; this is a gain of a factor 100. The importance sampling gain grows like the inverse of the probability ${\gamma }_B$. The key question is: How to perform importance sampling, relevant for extreme heat waves, starting from a climate model?

Fig. 3.

(A) We want to estimate the probability to be in the set $B$, for the model PDF $\rho (x)$. We are able to sample instead from the PDF $\stackrel{\sim }{\rho }(x)$ for which the rare event becomes common. We know the relation $L=\rho /\stackrel{\sim }{\rho }$ and can recover the model statistics $\rho$, from the importance sampling $\stackrel{\sim }{\rho }$. (B) PDF of the time-averaged temperature $a$ ($T=90\text{d}$) for the model control run (black) and for the algorithm statistics with $k=50$, illustrating that the algorithm performs importance sampling and that +2 K heat waves become common for the algorithm while they are rare for the model.

Since the climate is a nonequilibrium dynamical system, importance sampling has to be performed at the level of the trajectories. Trajectories generated by the model are distributed according to the unknown PDF ${\mathbb{P}}_0({\left\{X(t)\right\}}_{0\le t\le T}={\left\{x(t)\right\}}_{0\le t\le T})$ [this is a formal notation for the probability of the model variables $X(t)$ to be close to $x(t)$]. We use the GKLT large deviation algorithm, described below, that selects trajectories distributed according to the importance sampling PDF ${\mathbb{P}}_k$,

$${\mathbb{P}}_k({\left\{X(t)\right\}}_{0\le t\le T}={\left\{x(t)\right\}}_{0\le t\le T})=\frac{1}{Z(k,T)}\exp \left(k{\int }_0^{T}A\left(X(t)\right)\text{d}t\right){\mathbb{P}}_0({\left\{X(t)\right\}}_{0\le t\le T}={\left\{x(t)\right\}}_{0\le t\le T}),$$ [2]

where $k$ is a real-valued parameter, and $Z(k,T)$ is a normalization constant such that ${\mathbb{P}}_k$ is a normalized PDF. The surface-averaged temperature is $\stackrel{\sim }{A}(t)=A\left(X(t)\right)$. One observes that for positive values of $k$, the measure ${\mathbb{P}}_k$ is tilted with respect to ${\mathbb{P}}_0$ such that large values of ${\int }_0^{T}A\left(X(t)\right)\text{d}t$ will be favored with an exponential weight. Tuning $k$, we study different ranges of extreme values for $a=\frac{1}{T}{\int }_0^{T}A\left(X(t)\right)\text{d}t$ and thus different classes of extreme heat waves when $a$ is the time-averaged European temperature (Eq. 1).

The large deviation algorithm performs an ensemble simulation with $N$ trajectories (ensemble members), typically $N\sim O({10}^2-{10}^3)$. The trajectories start from independent initial conditions that sample the model’s invariant measure. After time intervals of constant duration $\tau$ we stop the simulation, and for each trajectory we compute a score function based on the dynamics in the previous time interval of length $\tau$ (see SI Data and Methods for the definition of the score function). Trajectories which are going in the direction of the extremes of interest, as measured by the score function, are cloned in one or more copies, while poorly scoring trajectories are killed. We call this step resampling and $\tau$ the resampling time. The different copies of a successful trajectory are slightly perturbed, so that they can evolve differently. Then the ensemble of trajectories is iterated for another resampling time $\tau$. Once the final time $T_a$ has been reached, resampling is performed one last time. With a proper choice of the score function we obtain an ensemble of $N$ trajectories of length $T_a$ distributed according to Eq. 2, where $k$ enters as a chosen parameter of the algorithm. The full details of the algorithm implementation are provided in SI Data and Methods.

In the normalization term of [2], $Z(k,T)={𝔼}_0\left[{\text{e}}^{k{\int }_0^{T}A\left(X(t)\right)\text{d}t}\right]$, the average is taken over the model statistics ${\mathbb{P}}_0$. In large deviation theory (36), $\lambda (k)={lim}_{T\to \mathrm{\infty }}\lambda (k,T)\text{with}\lambda (k,T)=\frac{1}{T}\log Z(k,t)$ is called a scaled cumulant generating function. One can prove that for large times, the PDF $P(a,T)$ of time-averaged temperature $a$ satisfies $P(a,T)\underset{T\to \mathrm{\infty }}{\asymp }{\text{e}}^{-TI\left[a\right]}$. Whenever $I$ is convex, $\lambda$ and $I$ are the Legendre–Fenchel transform of one another: $\lambda (k)={sup}_a\left\{ka-I(a)\right\}$ and $I(a)={sup}_k\left\{ka-\lambda (k)\right\}$. The reader knowledgeable of statistical mechanics or thermodynamics will immediately note the analogies between $Z$ and the partition function, $a$ and the energy, $k$ and the temperature, $\lambda$ and the free energy, and $I$ and the entropy. To summarize, the large deviation algorithm allows us to choose the “temperature” $k$ for which dynamical states of “energy” $a$ (in this case time-averaged European temperature) will become common. Increasing $k$, we can thus study events with more and more extreme heat waves.

Return Times for 90-d Heat Waves

We use the large deviation algorithm and Eq. 2 to compute the return times for heat waves lasting several weeks, following the methodology described in SI Data and Methods. Fig. 4 shows return times vs. amplitude $a=\frac{1}{T}{\int }_0^T\stackrel{\sim }{A}(t)\text{d}t$, for $T=90$ d. The black curve has been plotted from a $\mathrm{1,000}$-y control run. The red curve has been obtained as explained in SI Data and Methods from six experiments with the large deviation algorithm with values of the bias parameter $k$ ranging from 10 to 40 (Eq. 2). Each of these simulations has a computational cost of about $182$ y.

Fig. 4.

Return times for the $90$-d Europe surface temperature, computed from the $\mathrm{1,000}$-y-long control run (black) and from the large deviation algorithm, at the same computational cost as the control run (red). This illustrates both the good overlap on the 10- to 300-y range and the fact that the algorithm can predict probability for events that cannot be observed in the control run.

The first striking result in Fig. 4 is that we can compute return times up to ${10}^6-{10}^7$ y with a total computational cost of the order of ${10}^3$ y. This is thus a gain of more than three orders of magnitude in the sampling efficiency. It is striking that we can compute the return times for events that could not have been observed in a direct numerical simulation with the current or foreseeable computational possibilities.

Another aspect is the improvement of the quality of the statistics. In the control run there is only one heat wave with temperature in excess of 2 K during $90\text{d}$, while in the $k=50$ experiment there are several hundred, at a fraction of the computational cost. We can thus recover the return time of such heat waves either at a much smaller numerical cost compared with the control run or with a much smaller relative error, for a given numerical cost. Such an improvement of the statistics will be crucial to perform a dynamical analysis that involves temperature and pressure fields.

Teleconnection Patterns for Extreme Heat Waves

We use the excellent statistics gathered with the large deviation algorithm to describe the corresponding state of the atmosphere during extreme heat wave events. Fig. 5A shows the temperature and the 500-hPa geopotential height anomalies, conditioned on the occurrence of a $90$-d 2 K heat wave (composite statistics). Those conditional statistics are reminiscent of the teleconnection pattern maps sometimes shown in the climate community. However, while usual teleconnection patterns are computed from empirical orthogonal function (EOF) analysis, and thus describe typical fluctuations, our extreme event conditional statistics describe very rare flows that characterize extreme heat waves. Those global maps are a unique way to consider rare event and atmosphere fluctuation statistics, which is extremely interesting from a dynamical point of view.

Fig. 5.

(A) Northern Hemisphere surface temperature anomaly (colors) and 500-hPa geopotential height anomaly (contours), conditional on the occurrence of heat wave conditions $\frac{1}{T}{\int }_0^{T}A(x_n(t))dt>a$, with $T=90\text{d}$ and a = 2 K, estimated from the large deviation algorithm. (B) Northern Hemisphere anomaly of the averaged kinetic energy for the zonal velocity at 500 hPa conditional on the occurrence of heat wave conditions $𝔼\left[K{E}_{500}\mid \frac{1}{T}{\int }_0^{T}A(x_n(t))dt>a\right]$, with $T=90\text{d}$ and a = 2 K, estimated from the large deviation algorithm, with respect to the long-time average $𝔼\left[K{E}_{500}\right]$ computed from the control run.

By definition, as we plot statistics conditioned on $a=\frac{1}{T}{\int }_0^{T}A(x_n(t))dt>2\text{K}$, with $T=90$ d, Fig. 5A shows a warming pattern over Europe. The geopotential height map also shows a strong anticyclonic anomaly right above the area experiencing the maximum warming, as expected through the known positive correlation between surface temperature and anticyclonic conditions (34). A less expected and striking result is that the strong warming over Europe is correlated with a warming over southeastern Asia and a warming over North America, both with substantial surface temperature anomalies of order of 1 K to 3 K, and anticorrelated with strong cooling over Russia and Greenland, of the order of -1 K to -2 K. This teleconnection pattern is due to a strongly nonlinear stationary pattern for the jet stream, with a wavenumber 3 dominating the pattern, as is clearly seen from the geopotential height anomaly. In Fig. 5B, the anomaly of the kinetic energy gives a complementary view: Over Europe, the succession of a southern blue band (negative anomaly) and a northern red band (positive anomaly) should be interpreted as a northward shift of the jet stream there. Strikingly, over Greenland and North America, the jet stream is at the same position (but it is more intense) for the large deviation algorithm statistics as for the control run, while it is shifted northward over Europe and very slightly southward over Asia. This is related to the strong southwest–northeast tilt of the geopotential height anomalies over the Northern Atlantic. The extended red area (positive anomaly of kinetic energy) over Asia is rather due to a more intense cyclonic activity there, than to a change of jet stream position.

Inspection of the time series of the daily temperature shows that along the long duration of heat waves, the synoptic fluctuations on timescales of weeks are still present (Fig. 2B). The temperature is thus fluctuating with fluctuations of order of 5–10 ^∘C, as usual, but they fluctuate around a larger temperature value than usual. This is also consistent with the northward shift of the jet stream over Europe, but does not seem to be consistent with a blocking phenomenology as hypothesized in many other publications. This calls for using similar large deviation algorithms with other models and other setups to test the robustness of the present observation.

Conclusions

We have demonstrated that rare event algorithms, developed using statistical physics ideas, can improve the computation of the return times and the dynamical aspects of extreme heat waves. One of the future challenges in the use of rare event algorithms for studying climate extremes will be to identify which algorithms and which score functions will be suitable for each type of rare event. We anticipate that this tool will make available a range of studies that have been out of reach to date. First, it will pave the way to the use of state of the art climate models to study rare extreme events, without having to run the model for unaffordable times. The demonstrated gain of several orders of magnitude in the sampling efficiency will also help to make quantitative model comparisons, to assess on a more quantitative basis the skill to predict extreme events, for the existing models. It will also make available a range of dynamical studies. As an example, having a high number of heat waves allowed us to conclude that a Europe heat wave, mainly affecting Scandinavia, is related to a northward jet stream shift rather than a Rossby wave breaking, in the Plasim model. Such a phenomenology may well be model and model resolution dependent. Finally, and maybe more importantly, this tool will be extremely useful in the near future to assess quantitatively anthropogenic carbon dioxide emission impact on heat waves and other classes of extreme events. Assessment of the anthropogenic causes of rare event return time changes requires comparing two different climates (4, 33) and running a rare event algorithm for each case.

Data and Methods

SI Data and Methods contains a complete description of the GKLT algorithm and of the method to compute return times with the rare event algorithm, the description of the implementation of the Plasim model, aspects of the statistical postprocessing, and the description of the dynamical quantities represented in this article.