Learning from single extreme events

Abstract

Extreme climatic events (ECEs) have a disproportionate effect on ecosystems. Yet much of what we know about the ecological impact of ECEs is based on observing the effects of single extreme events. We examined what characteristics affect the strength of inference that can be drawn from single-event studies, which broadly fell into three categories: opportunistic observational studies initiated after an ECE, long-term observational studies with data before and after an ECE and experiments. Because extreme events occur rarely, inference from such single-event studies cannot easily be made under the usual statistical paradigm that relies on replication and control. However, single-event studies can yield important information for theory development and can contribute to meta-analyses. Adaptive management approaches can be used to learn from single, or a few, extreme events. We identify a number of factors that can make observations of single events more informative. These include providing robust estimates of the magnitude of ecological responses and some measure of climatic extremeness, collecting ancillary data that can inform on mechanisms, continuing to observe the biological system after the ECE and combining observational data with experiments and models. Well-designed single-event studies are an important contribution to our understanding of biological effects of ECEs.

This article is part of the themed issue ‘Behavioural, ecological and evolutionary responses to extreme climatic events’.

1. Introduction

After an extreme storm in New England in 1898, Bumpus found 136 injured house sparrows (Passer domesticus), of which 64 later died [1]. When he compared the morphology of survivors with that of deceased individuals, he found that the former were clearly less variable than the latter. This outcome was precisely as predicted by the theory of natural selection, with elimination of those individuals that deviate the most from the norm. A century later, Keller et al. [2] examined inbreeding in a population of song sparrows (Melospiza melodia) on Mandarte Island, British Columbia, before and after a population crash caused by a severe winter storm and found that the survivors were less inbred than the individuals that had died.

Both studies were based on a single extreme event. Bumpus made use of a fortuitous opportunity and was one of the first to observe natural selection in action. Keller et al. analysed an event that occurred during a long-term study during which they had collected detailed pedigree data and provided one of the first demonstrations of selection against inbreeding. Both studies became landmark studies, significantly contributing to our understanding of how natural selection works in the wild.

Extreme climatic events (ECEs) are changing in frequency and magnitude [3], and the concern is that they may have a disproportionate effect on ecosystems [4]. ECE studies therefore need to provide information on the likely biological effects of a climatic event with a particular extremeness, and whether this type of event is likely to push the biological system across thresholds from which it will only recover slowly, if at all [5]. ECEs often induce delayed and cascading responses [5–7], and understanding the underlying mechanisms (e.g. physiological, demographic) of the biological reaction is, therefore, important. Here, we ask what can be learned from studies based on a single ECE and what characteristics can make such studies particularly informative. We suggest ways of making better use of existing studies and to improve the design of future ECE studies.

(a). Characteristics of single-event studies

A substantial proportion of our knowledge about the effects of ECEs on natural biological systems (below, we will use the term ‘system’ for any biological system, which could be an individual, population or community) is based on observing a single event. Are these studies a biased sample of all ECE studies? We examined the characteristics of 242 studies listed in a recent review [8]. More than half (59%) of these studies were based on a single ECE but they covered similar ecological responses, climatic events, habitats and taxonomic groups as the studies with multiple ECEs (figure 1; electronic supplementary material, appendix S1 with figures S1–3).

Figure 1.

Characteristics of a representative sample of observational studies on extreme climatic events (from Bailey & van de Pol [8]). See electronic supplementary material, appendix S1, for details. (a) The types of extreme climatic events investigated (n = 174). (b) The types of ecological responses investigated (n = 170). Bars represent the number of studies in each category and are further subdivided by whether a single extreme event (black) or multiple extreme events (grey) were observed.

The single-event ECE studies in our sample broadly fell into three categories (see electronic supplementary material, appendix S1 for details): (i) Opportunistic observational studies were initiated after an ECE and tended to be short in duration (31% of the studies, median duration 1.5 years). (ii) Long-term observational studies usually followed a system both before the extreme event occurred and after it passed and were generally able to detect delayed responses (38% of the studies, median duration 10 years). (iii) Experimental studies tended to be short, on small spatial scales and were generally restricted to systems that can be manipulated relatively easily (31% of the studies, median duration 0.7 years).

(b). Definitions matter

Most climatic and biological variables of interest are continuous and whether an event is considered extreme—or how many events in a time series are labelled extreme—therefore depends on the definition used [8,9]. Most definitions of an ECE require that either the climatic variable (climatological definition), the biological variable (biological definition) or both (hybrid definition) exceed a certain value or are expected to occur sufficiently rarely [9], e.g. in less than 5% of the years. Single-event studies report a biological response to one event (or treatment in the case of experiments) that is considered extreme according to one of these definitions (an observation falling into areas I, II or III in figure 2a). Beyond that, however, single-event studies vary greatly in their design, which impacts on what can be learned from them.

Figure 2.

Hypothetical datasets illustrating different types of single-event studies of extreme climatic events. (a) Opportunistic single-event studies document the ecological response of a single climatic event. These could be defined as extreme events because the climatic event was unusual (vertical grey line: a climatic event to the right of this line, in areas I and II, is defined to be extreme), the ecological response is unusual (horizontal grey line: a response above this line, in areas I and III, is defined as being extreme) or both (area I). (b) A single extreme event (black triangle) is observed as part of a long-term study (black dots are non-extreme events). In this set-up, it is not possible to distinguish whether the extreme climatic event led to an increased mean response (open grey triangles represent unobserved replications of similarly extreme climatic events) or increased variance (grey circles represent unobserved replications of similarly extreme climatic events), as the observed outcome is likely under either scenario. (c) An extreme climatic event led to a response (black triangle) that does not look unusual based on observations under non-extreme events (black dots). Either the extreme climatic event had no effect on the ecological system (open grey triangles), or it increased the variance in the response but we happened to observe a value that would be typical of less extreme conditions (grey circles). (d) An extreme climatic event led to an extreme response (black triangle) that is nevertheless in line with the relationship between the system's response and the environment under non-extreme climatic conditions (black dots).

(c). Opportunistic single-event studies

Some single-event studies literally just observe a single extreme event and some biological response to it (opportunistic studies, figure 2a). Opportunistic single-event studies using a climatological definition (climate event is to the right of the vertical grey line in figure 2a) typically were initiated after an ECE occurred (e.g. the 2003 heatwave in Europe [10]) and examined ecological responses after the event. For example, a mid-winter rainstorm in the polar desert of continental Antarctica, and subsequent freezing did not damage lichens [11]. Single-event studies using a biological definition (a response above the horizontal grey line in figure 2a) typically observe an unusual ecological response and then examine what may have caused it. For example, an extreme case of heather (Calluna vulgaris) dieback in Scotland was attributed to low humidity combined with low temperatures and an ageing plant population [12]. Opportunistic single-event studies sometimes use a hybrid definition (area I in figure 2a, [5]). For example, Knapp & Soule [13] examined the spatial extent of an extreme frost event after it had led to widespread tree mortality in the Pacific Northwest of the United States.

Opportunistic single-event studies show what can happen to a system under certain climatic conditions. However, observational studies generally cannot attribute the response to the ECE unequivocally because the effect of unobserved confounding variables can never be ruled out.

(d). Single extreme climatic events observed during long-term studies

The other main type of observational single-event study reports ecological responses to an ECE that occurred during a long-term study (figure 2b–d). These studies are able to quantify how much the ecological response to the extreme climate deviates from the response of the system to non-extreme climate. For example, one could model the ecological response, Y_i at occasion i, as a function of the observed climate (clim_i) as

1.1

where g is a suitable transformation of the response and f a suitable function for the relationship between climate and response, e.g. constant, linear, etc. The model above assumes that the errors ɛ_i are normally distributed with variance σ², and related to climate through function h, which may be constant or generalized to account for autocorrelation. δ estimates the difference between a normal (X_i = 0) and an extreme (X_i = 1) event, and therefore how unusual it is, compared with non-extreme events. However, if the response to a single ECE is judged unusual, one cannot distinguish whether the event led to an extreme mean (event is not well described by function f; open triangles in figure 2b) or an increased variance (event is not well described by function h; open circles in figure 2b) in the response, as an extreme response is possible in both cases. Conversely, if a climate extreme does not lead to an observed extreme response (figure 2c), this does not necessarily mean that function f in the above model provides a good description of how the system responds to this type of extreme event. It is possible that the response is more variable under these conditions but the observed event happened to lead to a response that looked typical given the observations from non-extreme events (figure 2c). An increased variance in the ecological response could be expected if climate interacted with other variables. For example, extreme winters lead to high mortality in Dutch oystercatchers (Haematopus ostralegus) only when food availability is low [14].

The model above (equation (1.1)) can be used to test whether an observed response falls outside of the system's response expected under non-extreme climatic variation, which could indicate a threshold-like response. Conversely, if this model predicts an extreme response well, that could indicate that the system does not cross any thresholds over the range of observed climatic events (figure 2d). For example, juvenile survival in barn owls (Tyto alba) was lowest after unusually harsh winters, but in line with what was expected given the extremeness of these years [15].

2. Inference from single-event studies

To understand the effects of ECEs on biological systems, we need to be able to estimate the magnitude of the effect (e.g. as described above) and also to attribute that effect to the ECE [9]. Different types of ECE studies have different strengths and limitation in this regard. Experimental studies adhering to the three statistical principles of study design—replication, randomization and control—allow attributing effects to treatments. Experiments are, therefore, the most powerful tool for examining the effect of particular climatic conditions on a system. However, experiments may not estimate the magnitude of the natural response well [16] and are often not possible to carry out at the desired scale. Well-designed observational studies are important because they estimate the response under real conditions but attribution is more difficult since one can never rule out the possible effects of unobserved covariates [17,18].

(a). Design of observational single-event studies

Observational single-event studies allow for stronger inference if they use random sampling as opposed to convenience sampling. Random sampling ensures representativeness. Bumpus's study [1], for example, was based on sparrows that were injured during the storm and they may not be representative of the sparrow population in general. We, therefore, cannot tell whether this storm reduced morphological variance in the sparrow population as a whole. More generally, single-event studies based on a convenience sample of observational units (individuals, study sites, etc.), rather than a random one, can demonstrate that a particular phenomenon can happen but they cannot make inference about the bigger population of affected biological systems.

Inference from single-event studies can be improved by employing some kind of control. The classical designs for such ‘impact studies’ [19] involve either temporal or spatial controls, or both. Where a single ECE was observed during a long-term study, its impact can be inferred from temporal changes in the response (e.g. equation (1.1) above). Methods for inferring step-changes in time series are also known as intervention analysis [20]. If no data are available from before the impact occurred (as is the case in opportunistic single-event studies), spatial controls could be used instead, e.g. by comparing the system affected by the ECE with one that was not affected. Spatial replication has occasionally been used in single ECE studies (e.g. case study 1, box 1, [21]), allowing researchers to examine interactions between the ECE and factors that vary spatially, like habitat quality [21]. Since the extremeness of an event often varies spatially [10,13], spatial replication can give important information on the shape of the relationship between climate and response, e.g. whether critical thresholds lie within the range of observed values of the climatic driver. Spatial replication could also be used to separate climate extremes from other drivers, if they are not strongly correlated. The strongest designs include spatial and temporal controls, known as Before-After-Control-Impact designs [19], but are difficult to apply to ECEs because we do not know when and where these are going to happen.

Box 1. Examples where observing a single event has led to important insights.

Case study 1: More detailed understanding through spatial replication

Spatial replication—where the response to a single extreme climatic event is observed across multiple sites (e.g. populations of the same species)—can yield additional information on the effects of a single event. Oliver et al. [21] studied the impact of an extreme drought event in 1995 on 79 populations of the wide ranging ringlet butterfly (Aphantopus hyperantus) in the UK. The populations inhabited natural woodlands that varied considerably in size and habitat fragmentation. The drought event was followed by marked population declines in all study sites, but the impact depended on habitat quality. Populations in smaller and more fragmented woodlands declined more during the drought and recovered more slowly. While this study used data collected across large spatial scales, spatial replication across smaller scales can reveal context dependence if there is heterogeneity among sites in the extremeness of the climatic event or relevant variables that interact with climate.

Case study 2: The value of detailed community data

Some studies have been able to gain important understanding from observing a single extreme climatic event because it happened during a long-term study that collected detailed mechanistic data. Rosemary & Peter Grant [22] have studied Darwin's finches on the Galapagos Island of Daphne Major since 1976. Normally arid, this island received an extreme amount of rainfall during the 1982/1983 El Niño event. As a result, small soft seeds on which the medium ground finch (Geospiza fortis) specializes became more abundant and large hard seeds on which the cactus finch (G. scandens) specializes became relatively less common. The change in food availability led to selection for smaller beaks in G. fortis and corresponding evolutionary change over the next years. The population of G. scandens declined. The environmental changes in the wake of this extreme event also increased fitness of hybrid individuals and led to increased gene flow between the two species compared to normal years. Because of the detailed demographic data, knowledge of the relationship between seed size, beak size and fitness, as well as how these variables changed in response to the extreme event, this study was able to document the effects of the event as it propagated through the food web and levels of biological organization.

Case study 3: Combining different approaches

A promising approach is to combine observational data from an extreme climatic event with mechanistic models to produce testable predictions. Godfree et al. [23] studied the demographic responses of the keystone grass, Austrostipa aristiglumis, in eastern Australia during an exceptional 3-year drought. Taking advantage of spatial variability, they used demographic models to estimate the population growth rate under different levels of soil water content. Based on these results, Godfree et al. then made specific predictions about how precipitation and temperature should limit the distribution of this species, and then tested these predictions using species distribution models.

(b). Replication at a lower level

Single-event studies are unreplicated at the level of the ECE and inference is limited to the particular event that was observed. We therefore also cannot estimate the natural variability in the biological response to the observed type of ECE (figure 2b,c), and whether it depends on the state of the system [24] or other variables that may interact with climate [21]. However, the precision with which we can estimate the biological response to a particular event is at least partly under our control as it depends on replication at a lower level. For example, to estimate the effect of a particular heatwave on the mortality of a certain tree population, we could count the number of trees that have died (d) and those that survived (s) in a particular sample and estimate survival (Ф) during this particular event as:

Assuming the trees' fates are independent, the standard error of Ф is

where n is the total number of trees observed. By increasing n, we can estimate tree mortality during this particular event more precisely.

Stronger inference is possible if we have data on tree stands during normal climatic conditions, e.g. by sampling other stands that were not impacted by the heatwave (spatial control), or by observing the same stand during non-extreme years (temporal control). We could then estimate the variability in survival, for example using a generalized linear mixed effects model

where Y_i is the number of surviving trees out of n_i in stand or year i, μ estimates mean survival under non-extreme conditions, δ estimates the difference between a normal (X_i = 0) and an extreme (X_i = 1) event and ɛ_i is a random effect. The variability in survival among stands or years is captured by the estimate of σ². With this model, we can quantify and test the effect of this particular ECE on survival and we can improve precision by increasing n_i or the number of years/stands that we observe.

(c). Mechanistic understanding through ancillary information

Statistically attributing biological responses to an ECE is difficult in observational studies [17,18]. However, a convincing case can usually be made if the mechanistic pathways that led to the response are known and ancillary data are collected that give insights into the mechanisms of the ecological response we are interested in [25]. For example, Grant & Grant ([22], case study 2, box 1) showed how an El Niño event changed selection on beak size of Galapagos finches by favouring plants that produced softer seeds. Observing multiple demographic responses might give insights into the demographic mechanisms, such as life-history trade-offs and constraints [26], and observing multiple species helps understanding community-level effects [27]. Godfree et al. ([23], case study 3, box 1) combined detailed observations with an experiment and models to understand how an extreme drought affected the local occurrence and range boundaries of a grass. Observing how the system recovers from the impact [28] or how a collection of similar systems reacts to the same event (e.g. [21], case study 1, box 1) are additional means for strengthening the inference that can be made from single-event studies. For a mechanistic understanding of the biological effects of ECEs, physiologically important variables such as temperature, or available water, are more useful than measures of climate without a clear mechanistic link to the ecological response, e.g. a climate index [29]. Ideally, the measured climatic conditions should closely reflect the microclimate that a biological system experiences, which may not necessarily be what regular meteorological stations record.

(d). Clear hypotheses lead to stronger tests

There is usually some prior knowledge on the system we are interested in, or on similar systems. If we understand the relevant processes (e.g. physiological limits) well enough to be able to generate specific predictions, observing the response to a single ECE can be a powerful test of our knowledge. Bumpus' [1] study was so influential because it tested a key hypothesis of an important new theory. A single careful observation can also indicate gaps in our understanding. For example, if a known physiological threshold is exceeded during an extreme event, but the expected ecological reaction does not happen, the organisms must have ways to protect themselves [30].

3. Ways to knowledge: learning by data accumulation versus learning through theory development

Progress in our understanding of ecological systems alternates between inductive and deductive inference: we observe a phenomenon, develop hypotheses that might explain the observation, develop a theory, collect more observations, refine or re-develop the theory, etc. [31]. The goal is to progress from a situation of no data and little understanding to a situation of having rich data and thorough understanding. Along this path, observations, experiments and theory (models) are tools that complement each other. We need to observe natural events to make sure that the phenomenon we study is relevant in nature. We need experiments to establish causation. And we need theory to deduce testable predictions and reach a more general understanding.

How a new observation—e.g. of the effect of an ECE—contributes to this process depends on the level of background information that is already available [32]. When we lack understanding of a system, observing one event can tell us what type of responses are possible. A single observation greatly reduces uncertainty compared with the prior state of not having any information at all [33], and can be useful for decision-making [34]. Observations, even patchy or anecdotal ones, are a starting point for the process of gaining knowledge.

Conversely, having a lot of data does not necessarily lead to a good understanding of a system or process. Holling [35] distinguished between situations with lots of data but little understanding (area I in figure 3), lots of understanding but little data (area II in figure 3), little data and little understanding (area IV in figure 3) and lots of data and good understanding (area III in figure 3). The goal is to reach area III but one can do so via different routes, which we call a data-driven pathway if it goes via area I, and a theory-driven pathway if it goes via area II (figure 3).

Figure 3.

Pathways to knowledge. Accumulating knowledge is a continuous interplay between data accumulation (along the x-axis) and development of a mechanistic understanding of the system under study (along y-axis). In some fields—e.g. some areas of molecular biology, market research—data accumulate rapidly and a data-driven pathway to knowledge is most productive. In other fields—e.g. the study of ecological effects of extreme events—observations are hard to come by. These fields need to rely strongly on theory development that may be based on relatively little data at first. This is what we call the theory-driven pathway. Adapted from Holling [35].

(a). Theory-driven pathway

Most importantly, we think, single-event studies can contribute to the theory-driven pathway to knowledge (figure 3) in critical ways. In systems with little prior knowledge (area IV in figure 3), observing a large ecological response to an unusual climatic event can suggest that critical thresholds have been crossed. For example, Salewski et al. [36] observed high mortality among migratory birds arriving at an oasis south of the Sahara desert during the post-breeding southwards migration, following temperatures around 50°C. Because little was known about the migration ecology of these birds crossing the Sahara, it is not clear how unusual such events are or what effect they have on the bird populations. Nevertheless, it clearly showed that these conditions can push many individual birds over the limit, which led to further studies that clarified how migratory birds cross the Sahara desert [37,38].

Where more background information is available, detailed observations of ecological effects of a single ECE can improve our confidence in our understanding of the system. A good understanding of the mechanistic pathways of how extreme events affect ecological systems exists for many situations. For example, we know a lot about the mechanisms by which plants respond to droughts and under what conditions they reach limits [39]. We also have clear hypotheses of the effects of extreme precipitation on terrestrial systems [40], and we understand the pathways of how ECEs affect carbon fluxes [41], riverine systems [42] and arid ecosystems [43]. These reviews provide frameworks against which each new observation can be evaluated.

The slow accumulation of observations of extreme events is similar to the slow accumulation of evidence in some situations where natural resources are harvested. There, adaptive management has been suggested as a tool for making decisions while at the same time learning about a system [44]. Adaptive management relies on a number of alternative models that represent the uncertainty about how the system works, according to the current knowledge at the time. Learning happens by comparing model predictions to observed outcomes and re-evaluating one's confidence in each model.

Ideas of adaptive management could be used to learn about the ecological responses to extreme events more effectively, even if no management decisions are involved. We applied this method to the question how extreme winters affect barn owl survival [15] and found that learning continued long after the last extreme event happened (box 2). Generally, by building alternative models to predict ecological responses to ECEs it becomes clear what kind of information is needed to help distinguish between alternative hypotheses. Each time an extreme event occurs, one can focus on collecting that type of information and thereby make progress along the theory-driven pathway (figure 3). Adaptive management provides a sound framework for learning from consecutive events. This approach is particularly powerful when observations are hard to come by and one needs to make the most out of each observation [46,47] and for detecting ecological surprises [48]. Adopting ideas from adaptive management may only work for effects that we can anticipate and have enough knowledge to model, albeit even with little knowledge, simple models can be very powerful [49].

Box 2. Learning through updating model weights.

Under adaptive management, a number of competing models are confronted with data in an iterative way. Learning happens as the data increasingly favour one model and the associated biological hypothesis.

We illustrate how similar ideas can help learning from single, or a few, extreme events. We used a published time series of survival rates of Swiss barn owls between 1946 and 2001 [15]. Barn owls hunt small rodents but cannot do so if there is a layer of snow greater than about 5 cm. If such a snow layer persists for long enough, barn owls starve [15]. Switzerland experienced extreme snowfall during the winters beginning in 1952 and 1962, leading to high mortality, and crashes in the barn owl populations [15].

Here, we fitted three models to the logit-transformed

survival estimates for adult barn owls (figure 4a). The models were versions of the general model in equation (1.1) in the main text.

3.1

3.2

3.3

Figure 4.

(a) Number of days with more than 5 cm of snow measured at a central location (Bern) in Switzerland between 1946 and 2001 and annual survival estimates of adult barn owls (T. alba) in Switzerland, estimated from ringing data [15]. The arrows mark two extreme climatic events (1952, 1962). (b) Akaike weights of three models representing different hypotheses about how survival in barn owls responds to snow. The Akaike weights measure the relative support each model has from the data up to a particular year, and sum to one in a particular year. A change in Akaike weight, as more data are included over time, represents learning.

These models represent three alternative hypotheses about how we expect the system to react to environmental variation. Model 3.1 assumes that survival is unrelated to the number of days with more than 5 cm snow (snow); model 3.2 assumes a linear relationship between logit survival and the number of days with more than 5 cm snow; and model 3.3 additionally allows for an effect of extreme years (δ), defined as any year with more than 55 days of snow exceeding 5 cm. X_i is an indicator variable taking the value 1 if year i is extreme and 0 otherwise.

First, we pretended that we had only observed survival from the start of the study (1946) until the first extreme event (1952) and fitted the three models to the data from these years. We used Akaike's Information Criterion to compare the models and calculated Akaike weights [45] to estimate the relative support each model had from the data.

The idea of adaptive management is to let each model predict how the system should react during the next time step, observe the response and evaluate how well each model predicted the observation. Then, update the model weights based on how well the data support each model. Diverging model weights represent learning. If one model consistently predicts new observations better than the others, its weight increases over time, lending more credibility to the associated biological hypothesis.

To illustrate this process, we added the next observation (1953) and refitted the three models. Then we recalculated the Akaike weight for each model. We repeated this procedure, adding 1 year at the time, until the last year, 2001.

With the data up to 1952, model 3.2 was the best supported model (figure 4b) but models 3.1 and 3.3 also had considerable support from the data (Akaike weight > 0.2). As time went on, model 3.3 received increasingly more support, especially after the second extreme event in 1962. By 2001, model 3.3 was five times as well supported as model 3.2 (Akaike weights of 0.83 versus 0.17, respectively). The Akaike weight for model 3.1 dropped close to zero during the second half of the study.

Considering all the data, there was therefore clear evidence that survival during the two extreme years was lower than what would have been predicted from variation in survival during non-extreme years, based on a linear relationship. Remarkably, learning about the effects of extreme winters on the survival of barn owls continued after the last extreme event. This is because more data allowed the relationship between survival and snow (function f in equation (1.1), main text) to be estimated ever more precisely. More information on the reaction of a system to non-extreme climatic variation therefore improves our ability to estimate the effects of extreme events when they happen.

(b). Data-driven pathways to learning from single-event studies: meta-analyses

Studies on single extreme events can contribute to the data-driven pathway most effectively if they are reported in a way that makes them comparable with other studies, for example, through formal meta-analysis [50]. This requires reporting effect sizes in a comparable way—a meta-analysis is only possible if the effect sizes from different studies reflect the same thing. The most relevant effect size is often the magnitude of the ecological response to the ECE, i.e. the difference in the ecological variable—e.g. survival, growth—after an extreme event compared to its value under normal climatic conditions. Having some kind of control (§2a) is therefore particularly important. To make single-event studies comparable one also needs to know how extreme the observed climatic event was, since the magnitude of the ecological response likely depends on how climatically extreme the event was. Quantifying the magnitude of climatic extremeness in a biologically relevant way is challenging as long-term climatic data are needed, and the frequency of such events is changing with climate change. Meta-analyses can help attributing extreme responses to particular drivers (different components of climate or non-climatic drivers) if single-event studies report those.

To demonstrate how meta-analyses can be used to draw information from multiple single-event studies, we examined whether the 2003 heatwave that affected much of Europe [10] had different effects on fecundity, growth and survival across different organisms (example 1, box 3). We found some evidence that survival and fecundity declined more than growth but there was a lot of variability among observations. As a second example, we examined whether a change in survival in response to heatwaves depended on the extremeness of the event (example 2, box 3). In the two examples, we had to exclude 24 and 44% of the studies because they lacked critical information on the effect size or extremeness of the climatic event, suggesting that future studies should pay more attention to reporting critical information.

Box 3. Meta-analyses using single-event studies.

We give two examples of questions that can be addressed through meta-analysis using single-event extreme climatic event (ECE) studies. These examples are for illustration only, and to see whether single-event studies typically report the information needed to perform a meta-analysis. We restricted our literature search to the list of 242 studies described in electronic supplementary material, appendix S1 and did not follow the entire protocol of rigorous meta-analyses [50].

European heatwave of 2003

In the first example, we look at the heat wave that affected much of Europe during the summer of 2003 [10] and ask whether the fitness components fecundity, growth and survival were affected in different ways. Our sample contained 17 studies reporting ecological responses to this particular event. Out of these, four studies did not provide enough information for us to extract effect sizes and associated standard errors. Among the remaining studies, nine reported fecundity, growth or survival responses. From these studies, we extracted 25 responses and used per cent change during the heatwave compared to a baseline estimate as effect sizes (electronic supplementary material, table S2).

We adopted a Bayesian approach similar to that of McCarthy & Masters [51] to estimate the mean effect sizes in the three types of response (see electronic supplementary material, appendix S2 for details). Across these nine studies, fecundity and survival were more strongly reduced by the heatwave than growth (figure 5, posterior mean and 95% credible interval for per cent change: fecundity −27 [−60, 10]; growth −10 [−34, 13] and survival −31 [−61, −1]). The uncertainty was relatively large because the responses within each response type were fairly variable. If we had any covariates that could explain part of this variability—e.g. how severe this particular event was in a specific location—the uncertainty could probably be reduced. However, this example demonstrates how single-event studies can be further analysed using meta-analysis techniques. The model we used for this analysis could easily be expanded to include covariates or additional sources of uncertainty.

Figure 5.

Violin plots of the posterior distributions for the mean change in fecundity, growth and survival in response to the heatwave that affected Europe in 2003, obtained through a Bayesian meta-analysis. The dots show the reported values, slightly jittered.

Effects of extreme heat waves on survival

In the next example, we ask how the severity of heatwaves affects survival. The sample of studies in electronic supplementary material, appendix S1 contained nine papers that covered a single ECE and considered survival in response to a heatwave. Four of these papers failed to provide adequate information to estimate either the severity of the extreme event or changes in survival. Of the remaining five, only two provided effect sizes and their standard errors in a straightforward way. Nevertheless, we were able to extract nine unique estimates from the text and graphs of these five studies. We used per cent change as effect size (electronic supplementary material, table S3). However, we were only able to obtain a standard error for the effect size for three of these nine estimates.

In this meta-analysis, we therefore had to ignore the uncertainty in the estimates and decided to treat them like regular observations (electronic supplementary material, appendix S2). The estimated relationship between extremeness of the heat-wave, estimated in number of standard deviations from the mean, and per cent change in survival was negative (figure 6, slope −9.1, 95% CI: −24.5, 6.2) but the confidence interval included zero. The results are inconclusive about the importance of climate extremeness but our difficulty of finding studies that reported extremeness suggests that this is a quantity that should receive more attention in future studies.

Figure 6.

Relationship between the intensity of a heatwave (measured in number of standard deviations from the mean) and change in survival. The dots are the estimates taken from single-event studies, and the black line is the best fitting linear regression line.

One issue that needs particular attention when conducting formal meta-analyses of ecological responses to ECEs is that there could be bias due to particular definitions of ECEs. For example, studies observing no large ecological response after a climatological ECE (area I in figure 2a) would not be called extreme events under the biological or hybrid definitions. Meta-analyses might consequently overestimate the effects of ECEs, particularly where sample sizes are low [52].

4. Conclusion: how to make single-event studies most useful

As ECEs happen rarely we get few opportunities to study them. Single-event studies will therefore remain an important source of knowledge about the biological effects of ECEs. We found that single-event studies broadly fall into three categories that each have their strengths and limitations but can contribute to our knowledge of the biological effects of ECE in complementary ways [53]. (i) Long-term studies can collect information on a system before and after an extreme event. However, they need a lot of investment to be maintained over long enough time spans. (ii) Opportunistic studies are easier to set up, but lack information on the system's dynamics before the extreme event. (iii) Experiments can uncover causal relationships, but tend to be limited to certain types of systems, and relatively small spatial and temporal scales.

We examined factors that make single-event studies more useful contributions to both a theory-driven (§3a) and a data-driven pathway (§3b) of learning, and summarise these factors in box 4. The location and timing of the next extreme event is uncertain and it can therefore be difficult to measure the right thing in the right place (§2c). However, having clear hypotheses (§2d) and some prior understanding of the mechanisms (§2c) help when deciding what responses should be measured and for how long. To make the most of each opportunity, we argue that attention to rigorous study design is particularly important. This involves using appropriate controls and random sampling (§2a), and enough power to estimate the effect size reliably (§2b).

Box 4. Checklist of factors that improve the strength of single-event studies.

• — Control: temporal and/or spatial controls help quantify and attribute the effects of an ECE (§2a).

• — Robust measure of biological response: ensure that sample sizes are sufficient to estimate the biological response with sufficient precision (§2b); ensure that relevant responses are measured (§2d).

• — Measure of climatic extremeness: quantifying climatic extremeness helps in making studies comparable (§3b); measuring biologically relevant climate helps in understanding mechanisms (§2c) and testing predictions (§2d).

• — Clear hypotheses: observing a single extreme event can be a strong test of a hypothesis if clear (preferably quantitative) a priori predictions exist of what is expected to happen under these conditions (§2d).

• — Ancillary data: collecting data that illuminate the mechanisms that led to the biological response can make a study more informative (§2c).

• — Observe multiple responses: observing several demographic quantities (e.g. survival, reproduction and growth; or multiple species in a community) can lead to a better understanding of the biological impact of the extreme climate (§2c).

• — Keep observing the system for long enough to be able to detect delayed responses, cascading effects, and (perhaps) the return of the system to its normal state (§2c). Better knowledge of the system's response to non-extreme climatic variation helps in estimating the effects of extreme events (§3a).

• — Reporting possible alternative drivers helps in attributing a response to a particular component of climate, or non-climatic driver through meta-analyses (§3b).

• — Augment observations with targeted experiments to establish causal relationships (§2c).

• — Select your sample at random to make it representative of the population of interest (§2a).

Studies reporting on ecological responses to a single ECE may have their limitations [8]. However, due to the difficulty of studying multiple extreme events, these studies play an important role in our understanding of such effects. With this paper we hope to improve the value of single-event studies by taking a critical look at the value and limitations of such studies, by suggesting ways of making better use of existing studies and by suggesting ways to improve on the design of future studies.

Authors' contributions

R.A. led the writing and concept development, all authors contributed to reviewing the literature, discussing ideas and writing.

Competing interests

We have no competing interests.

Funding

National Research Foundation of South Africa (grant no. 85802), who accepts no liability for opinions, findings and conclusions or recommendations expressed in this publication.