Incorporating uncertainty in predictive species distribution modelling

Abstract

Motivated by the need to solve ecological problems (climate change, habitat fragmentation and biological invasions), there has been increasing interest in species distribution models (SDMs). Predictions from these models inform conservation policy, invasive species management and disease-control measures. However, predictions are subject to uncertainty, the degree and source of which is often unrecognized. Here, we review the SDM literature in the context of uncertainty, focusing on three main classes of SDM: niche-based models, demographic models and process-based models. We identify sources of uncertainty for each class and discuss how uncertainty can be minimized or included in the modelling process to give realistic measures of confidence around predictions. Because this has typically not been performed, we conclude that uncertainty in SDMs has often been underestimated and a false precision assigned to predictions of geographical distribution. We identify areas where development of new statistical tools will improve predictions from distribution models, notably the development of hierarchical models that link different types of distribution model and their attendant uncertainties across spatial scales. Finally, we discuss the need to develop more defensible methods for assessing predictive performance, quantifying model goodness-of-fit and for assessing the significance of model covariates.

1. Introduction

The spatial (and temporal) distribution of a species is one of the most fundamental pieces of ecological information. Knowing what drives or constrains geographical distribution is pivotal for many purposes: for example, for many macro-ecological questions searching for pattern and process in aggregate distribution statistics [1]; knowledge of the distribution of disease vectors for targeting of health programmes [2]; predicting spread of invasive species [3] and development of successful conservation plans [4,5]. From origins in conceptual models based on expert opinion and reasoned extrapolation, the development of formal species distribution modelling (SDM) using a variety of statistical and machine-learning techniques has formalized the process. Distribution modelling is now a major field of ecological research: between 2005 and 2010, there were over 850 publications in the Thomson Reuters web of knowledge database (found by a search for ‘species distribution mode*’ or ‘niche mode*’) compared with only 79 between 1999 and 2004.

Many recent uses of SDMs are to make two types of predictions of practical use to conservation: predictions of where species may be present but unrecorded (or indeed, where they might be found if human activities had not eliminated them [6]), and predictions of where species may be in the future as environmental change alters distributions [7]. The first predictions are essentially spatial interpolation and extrapolation, whereas the second uses a space for time substitution to make extrapolations into the future, based on projections of climate (and/or land-use) change. Both predictions have important uses, particularly for conservation where predictions have been used to identify spatial priorities for conservation [8,9] and in attempts to assess risks from climate change to particular species [10,11]. Given the interest in predicting future distributions, it is unsurprising that many methods have been developed. As with all ecological models, different methods have advantages and disadvantages, and are appropriate for different questions. All methods, however, are subject to uncertainty; an issue which, while not overlooked [12–15], is often only partially considered.

Identifying uncertainty is important for two main reasons. Firstly, without acknowledging sources of uncertainty, it is hard to see where future improvements can be made. For example, if there is uncertainty in a particular parameter estimate, or if one covariate is poorly measured, then future work on this will improve prediction. Equally important is the confidence or scepticism of the general public: many activities that rely on SDMs are of high profile [16] and if scientists are perceived as overstating problems or downplaying uncertainty, then the public is likely to further lose confidence in scientists [17]. Indeed, although minimizing uncertainty (increasing precision) is worthwhile, a whole field of theory has been developed to help make decisions in the face of uncertainty, but this is possible only when uncertainties are acknowledged and quantified [18]. Instead of trying to eliminate uncertainty completely, we suggest a pragmatic approach that embraces uncertainty and seeks to make accurate predictions with correctly identified precision.

Here, we review methods used to predict species distributions and focus particularly on their associated uncertainty, identifying where methodological development can reduce prediction uncertainty, and other areas where uncertainty is inherent.

2. Distribution modelling methods

There are many distribution models in current use [19–22]. Here, we seek not to review individual methods, but to identify classes of distribution models, discuss the assumptions they make and explore predictions that are possible with each class. In classifying distribution models, we recognize an axis from the purely statistical that seek through correlation to identify process (ecological niche) from pattern (geographical distribution), to methods that try to determine directly the processes and ecological mechanisms that underlie pattern. Although individual models fall anywhere on this continuum, for the purposes of this review we split models into three classes: statistical (and machine-learning) methods, called niche models; process-based models at the opposite end of the continuum, and an intermediate class such as metapopulation models that typically combine statistical associations between climate and habitat variables with demographic components to identify distribution and range dynamics. Along this axis, there is a rapid increase in the need for data and a concomitant decrease in the practicality of application across many species.

Uncertainty in predictions originates from many different sources, and we follow others in identifying three main areas where uncertainty occurs: data, model and prediction uncertainty (table 1). Although uncertainty from each area occurs in all models, not all sources are relevant to all model types. In the sections that follow, we start with a brief description of the model class, describe relevant sources of uncertainty and conclude with our view of where future development in each model class should focus to adequately model uncertainty.

Table 1.

Sources of uncertainty relevant to classes of species distribution model.

uncertainty class	uncertainty source	niche-based models	demographic models	process-based models
data	observed distribution data (including recorder effort)	relevant both to model building and model fit assessment	relevant to model fit assessment	relevant to model fit assessment
	current covariate (habitat and/or climate) data	relevant to model building and model fit assessment	may be relevant to model building (if spatial information on demography is required), always to model fit assessment	relevant to model fit assessment
	other data sources (e.g. noise in demographic data)	not relevant	relevant to demographic data for model building	relevant to model building, e.g. through observations on phenology
model	model fitting methods (generalized-linear model, generalized-additive model, etc.)	highly relevant for niche identification	relevant for, e.g. identification of demographic links to weather	not (or minimally) relevant
	structural model misspecification	highly relevant	highly relevant	highly relevant
	parameter estimation	relevant, but often fewer in number than more complex models	highly relevant	highly relevant and often involving large numbers of parameters
	uncertain model fit to true niche/distribution	highly relevant	highly relevant	highly relevant
prediction	uncertainties in covariate data	highly relevant	highly relevant	highly relevant
	no-analogue conditions	highly relevant	highly relevant	not important

3. Niche-based distribution models

The majority of papers describing predicted species distributions fall into this class. A huge range of methods have been developed to fit these models from statistical regression methods, such as generalized-linear models and generalized-additive models to machine-learning approaches, including artificial neural networks and implementations of genetic or other learning algorithms [20]. Reviews of these methods that describe their strengths and weakness are available and we do not seek to repeat this information here. Instead, we focus on the common properties of this class of method and describe the uncertainties involved in predictions from their application.

In general, niche-based models are straightforward and efficient to fit. They rely on simple (primarily presence/absence records) and relatively little data (as few as five observed data points have been suggested as a minimum for some methods [23,24]); some methods address cases where no information is available on species absence. With such little information required to generate output and the widespread availability of software to fit even sophisticated models, it is unsurprising that many studies use these methods, including numerous high-profile publications [8,16]. However, the same reasons should provide warning that there is considerable uncertainty in predictions from these methods: all models are only as good as the data upon which they are built.

As the name suggests, niche-based distribution models implicitly focus on estimation of a species' niche from the geographical distribution of species [22,25]. Once the niche is estimated, then predictions (in space and time) can be made by re-projection from niche- to geographical space, provided data are available on the important niche dimension for the locations or time for which predictions are required.

Data quality is an obvious source of uncertainty and much can be undertaken to improve matters. Unfortunately, knowledge of species distributions is incomplete: many of the most biodiverse regions remain poorly known, with discoveries of new species regularly reported [26,27]. Species misidentifications can also be common and individual observers vary in their error rates (C. Beale 2011, personal observation). Even when species distributions are perfectly observed, the raw data may still be a source of uncertainty, e.g. some presence records refer to vagrants that have dispersed away from their usual range, adding to the error in modelling. Similarly, presence records may refer to sink populations [28,29]: while including such presence records is inevitable, it raises the unwelcome possibility that the populations predicted to be present at some time in the future may consist only of potential sink populations that would not be viable on their own.

Unknown or incompletely known recording effort is a pervasive and an important source of problems for species data, as it is almost always spatially non-random [30–32]. This can have unfortunate consequences, because spatially systematic variation in effort (or indeed habitat-specific differences in detectability) may match some explanatory covariates, resulting in incorrect estimation of niche [31]. Covariate quality is another source of uncertainty. Typically, in large-scale analyses, covariates are themselves predictions from models (e.g. based on reflectance values of satellite-derived data [33]), or interpolation of data measured from fewer sample locations. These covariates are, therefore, uncertain and incorporation of this within the modelling process is desirable—for example, gridded climate data are generated by interpolation from weather stations using modelling methods that give estimates of precision [34]; yet these uncertainty measures are not generally used in SDM.

Because niche-based models focus on identification of that part of a species' ecological niche central to its geographical distribution, modelling this as accurately as possible is important [25]. Before starting to fit a niche model, there is uncertainty over which niche to model: the fundamental niche or the realized niche? With niche-based modelling, data are most directly available on the realized niche (i.e. the environment where the species currently occurs), yet the fundamental niche of a species is often much wider than the realized niche [35]. Using the realized niche to predict distribution is probably appropriate when filling gaps in known distribution owing to lack of observations, but introduces considerable (and unmeasurable) uncertainty when predicting future distributions. Realized niches differ from fundamental niches owing to dispersal limitation and through interactions with other species [25,36,37]. Competitive exclusion of one species by another is common and there are numerous examples of range limits (or holes within distributions) determined by known competitors [38,39]. Competitors, however, may respond to environmental change in different ways; this could result in currently competing species sharing no future locations but instead interacting with other species in different (and currently unobservable) ways [40,41]. When competition (or facilitation) limits distribution, predicting from the realized niche is subject to inestimable errors, making predictions dependent on the unrealistic assumption that newly interacting species will have the same net interactions as current interactions. Furthermore, if dispersal barriers (e.g. sea crossings, mountains) exist and prevent a species from filling its fundamental niche, then future distributions are likely to be equally constrained in geography. Any niche model excluding these constraints fails to identify the limiting process and may incorrectly predict range expansion. Similar arguments apply to problems associated with source–sink dynamics and historical events that have constrained current distributions (e.g. glaciation cycles and volcanism).

These issues notwithstanding, there is information in current distributions that is useful for estimating the fundamental niche—though there is no certainty that sufficient information on niche limits exist in any given dataset. Indeed, it seems unlikely that many species have distributions suitable for full identification of any niche, realized or fundamental: even for taxa with distributions completely within a study region, there is no guarantee that all niche axes have limits within the area. Equally, there is no certainty that species with only partial ranges within a study area do not reach tolerance limits within the area (figure 1, cf. [42,43]): geographical distribution and distribution within niche space are different phenomena. Modelling the fundamental niche from geographical data is harder than modelling the realized niche, but doing so avoids some problems highlighted above. In practice, no matter how sophisticated might the modelling method be, full recovery of the fundamental niche is unlikely. Instead, an approximation somewhere between the realized and fundamental niche is likely, meaning that some issues associated with the realized niche remain in predictions from models of the fundamental niche. For prediction, however, further problems are generated: consider a species where most of its geographical range is limited by competition. For the region of niche space occupied by this competitor, no accurate prediction is possible (though simultaneously modelling both species makes this feasible). Acknowledging this prediction uncertainty is preferable to confidently asserting that areas with such environmental characteristics will always be unsuitable, even if the competitor's distribution changes. Thus, methods that identify the fundamental niche in preference to the realized niche are preferable, despite the greater uncertainty associated with their predictions, because the narrower precision of the realized niche model probably underestimates uncertainty.

Figure 1.

A graphical demonstration that there is no necessary correlation between endemicity within a study area and ability to fit niche-based models. (a) Hypothetical temperature gradient across a square continent, with the eastern three-quarters (unmarked) being within the modelling area. (b) Hypothetical rainfall gradient across the same square continent, hashes again indicate portion of the continent outside the study area. (c) Niche-space within the continent—black rimmed circles fall within the surveyed area; plus symbols (‘+’) within the unsurveyed space. Fill indicates presence of two hypothetical species: dark or medium grey. (d) Geographical projections of the niches indicated in (c) for the two hypothetical species. Note that while panel (d) shows that the medium grey species' range falls completely within the surveyed area, it is impossible to identify upper niche limits for temperature for this species, while the dark grey species is not endemic to the study area, but all four niche limits can be identified from the data availability in panel (c).

Another source of modelling error in niche models involves identifying niche dimensions. Trivially, excluding an important covariate leads to a poor model and poor prediction. Equally, including redundant variables reduces the accuracy of parameter estimation, particularly, if unnecessary covariates correlate with useful variables [44]. Furthermore, addition of numerous redundant covariates increases the probability of over-fitting. This is of particular concern because many niche-based models use automated fitting methods with minimal or no selection of variables: it is not unusual for models to be built with ca 13 highly correlated covariates [45,46]. This problem is only one form of statistical model misspecification, all forms of which reduce prediction accuracy, while exaggerating precision. A further example is misspecification of the error term: many modelling methods either have no such term, or implicitly assume spatially independent errors when a model of spatially structured residual errors is appropriate (though leads to apparently less-precise predictions, as they do not overestimate precision [44]). Other often overlooked sources of uncertainty relate to the functional form of the niche model: to fit a plausible model requires considering the likely shape of the niche and, if necessary, constraining the statistical model to a biologically plausible form. It is, for example, unlikely that any real species has a fundamental niche that includes temperatures of 10–15°C, excludes 16–20°C but includes again 21–25°C, so fitting a unimodal model is better than one that is more flexible [19,47]—although additional flexibility may be preferable if modelling the realized niche [12]. Structural model misspecification also results in misidentification of covariates when a multi-model or covariate selection process is undertaken because the model likelihoods (and/or Akaike's information criterion (AIC)) will be wrong. When predicting, it is important to remember that fitted distributions are smoother than real distributions: they are more strongly autocorrelated than real distributions and generally have fewer isolated patches of presence. While inevitable (correct classification of presence and absence is weighted towards the main presence and absence areas, rather than correct classification of peripheral populations), this is important where small and isolated populations may be valuable. For example, calculating extinction risk requires knowing when the last population goes extinct: smooth modelled distributions with no outliers are more likely to predict total extinction than a real distribution with populations existing beyond the main climate envelope.

An important additional source of uncertainty in niche-based distribution models is in the assessment of model performance and goodness-of-fit [12,48,49]. Ultimately, all models must be assessed for performance and predictive ability, but this is not as straightforward as implied by many papers in this field. Typically, assessment is undertaken by comparison of observed with predicted distribution for current conditions. Apart from the problem of making genuinely independent predictions of distribution when these data (or at best a highly autocorrelated subset of test data) are used in model building [19], measures assessing pattern match are not without problems. Measures such as the area under the receiver-operating characteristics curve (AUC), or the true skill statistic are typical, but do not offer unbiased measures of pattern-matching ability [12,50,51] and are applied without penalization for model complexity, resulting in risks of over-fitting, which greatly reduces predictive ability [48,52]. A problem not generally noted is that increasing the study area by including additional unsuitable areas always results in an increased AUC or similar score, though the model is neither better nor worse than before: these scores are not a true goodness-of-fit statistic and do not explain how well a model predicts distribution, rather they estimate how well a prediction fits a particular dataset; a subtle, but important distinction. Furthermore, measures that assess overall significance of a model are not necessarily suitable for determining how much variation is explained by the model: statistical significance means neither biological significance nor strong predictive power. Moreover, models estimating the fundamental, rather than realized niche, are far harder to assess using simple pattern-matching: a geographical projection of the fundamental niche may be substantially larger than that of the realized niche, so a perfect model may have a relatively poor match to the current observed distribution, which reflects the realized niche. This source of uncertainty in ability to assess model fit applies to all classes of SDMs.

Prediction uncertainty of niche-based distribution models also occurs from the covariates used to predict future distribution. Such projections rely on the output of climate models and can be extremely uncertain, particularly for precipitation-related variables [53]. Moreover, not all variables are available as output of climate models, and projections of future values of covariates may be lacking. The availability of these future data can restrict initial covariate choice, despite the knowledge that factors for which future predictions are unavailable may be important [54]; i.e. models may be deliberately misspecified to reflect data availability, rather than for ecological reasons. Such sources of error are compounded when the uncertainty of climate predictions is ignored. Another source of prediction uncertainty comes from ‘novel climates’: future climates may have no analogues within the study area and accordingly the species response to this new environment cannot be known [55].

Many causes of uncertainty can be addressed by statistical methods that, while not eliminating uncertainty, allow correct assessment of precision. Usually, correctly modelling precision reduces apparent certainty in the output of SDMs because many exiting niche-based models give overconfident predictions with false precision. Suitable tools to improve uncertainty estimation are available for all stages of the modelling process. Many errors associated with distribution data can be incorporated within occupancy models to provide estimates of niche unbiased by observer ability [56,57]. Similarly, hierarchical models can explicitly incorporate uncertainty in covariates (both in input variables and in predicted future values where available), through resampling of plausible covariate values and model refitting [58]. The novelty of future climates may be a source of irreducible uncertainty for model extrapolations.

Additionally, appropriate spatial error models can be incorporated during model building [12,19,41]. It is more challenging to ascertain how to model interactions between species given the complexity seen in even simple food webs and the analytical challenge presented by multi-species spatial data. Yet, methods such as Bayesian network tools are developing that allow complex interaction networks to be inferred from distribution data while simultaneously estimating the influence of covariates [59,60]. Ultimately, new methods will develop that incorporate this full range of statistical advances, allowing credible estimation of uncertainty.

4. Demographic models

In this category, we identify two main types of model. First, there are classic metapopulation models explaining local patch occupancy as a dynamic process of birth, death, immigration and emigration models linked by dispersal [61]. Such models can be parametrized by correlating demographic components with weather and then used for future prediction using climate projections [62]. Second, models have been developed that again correlate demographic components with climate or weather, often identifying intermediate linkage via food availability but are not patch-based [63–66]. In reality, models of both types fall on a continuum from the purely statistical to those that include climate links to physiology and/or intermediate processes such as food availability that tend towards process-based models.

Metapopulation models of the stochastic patch occupancy type would be excellent tools for modelling fragmented species distributions (species interactions aside). They can accommodate patch structure, fine-scale habitat fragmentation and landscape architecture, population density and Allee effects, local adaptation and evolution, local environmental conditions, and even the behaviour of individuals [67]. Metapopulation models can predict future distributions using two approaches. Firstly, future environments may modify the availability of suitable habitat patches, with some patches becoming unsuitable, and others becoming suitable as the environment changes [62,68,69]. Alternatively, demographic parameters within patches may be adjusted according to climate-related processes measured in the field or laboratory. For example, temperature-related hatching success is widespread among many insect species to which metapopulation models have been applied [70]. Clearly, the patch loss and demographic shifts are not mutually exclusive and both can be incorporated into the same predictive model. Uncertainty in these models is explicitly recognized in their stochastic nature, provided variation in demographic components is accurately measured.

While useful when applied to particular species at the landscape scale (but see [71,72]), in reality these models require intensive fieldwork to parametrize and are typically practical only for small proportions of a species range [71], thereby limiting their usefulness to meet the challenge of global environmental impacts on biodiversity. Furthermore, metapopulation models are criticized for their difficulty in dealing with landscapes other than simply suitable patches within a matrix of unsuitable (though maybe permeable) habitat, and also have little to offer when biotic interactions (beyond direct host–plant relationships) are important in determining distribution limits [67,71].

The second type of model within the class which we call demographic models is less well-defined. At base, these models use information on breeding and survival, correlate these with weather variables and from this generate a simple demographic model [64]. When this statistical model is embedded in a geographical grid with cells with different climate conditions, estimates of the intrinsic rate of population change become spatially explicit, generating an SDM. Complexity (and further realism) can be added by identifying the mechanism linking weather and demographic components. Typically, this involves the linkage between weather and food availability, which, in turn, is linked to breeding success. For example, Both et al. [66] describe how climate change is likely to alter the distribution of pied flycatchers (Ficedula hypoleuca) through generating a temporal mismatch between timing of the bird's breeding and the peak spring availability of their caterpillar prey, resulting in reduced breeding success in areas where early spring has rapidly advanced. Similar studies of other birds also identify range-margin populations under pressure from climate change [65]. So far, there are few examples of this model type and particularly their projection into geographical distributions. To date, those that have use only mean estimates from estimated relationships [65] and therefore underestimate the uncertainties in predictions, yet there is no reason why future predictions may not be generated stochastically to quantify uncertainty appropriately.

In general, demographic models are appealing as SDMs, as they are explicit about the demographic hypotheses involved and offer transparent estimation of uncertainty. Unfortunately, geographical predictions from these methods are accurate only if the demographic model correctly identifies the important factors determining population limitation. As these correlative methods implicitly (but often incorrectly) assume that the demographic impacts of weather and climate are identical, model misspecification errors may be large. For example, consider that many upland birds are restricted in geographical range to places with cool, wet climates [73]. While these species may struggle in a warm, dry climate, many of them show higher breeding success and better survival in warm, dry years than in the typical conditions they usually experience [74–76]. Clearly, demographic models would predict a more widespread population than their preferred upland habitats, suggesting that while warm, dry weather may be good, a warm, dry climate would not. Instead, the distribution of these species relates to climate via potentially complex interactions with other species which, if this class of model is to be correctly specified, must be simultaneously modelled.

Predictions of future distribution from metapopulation models are clouded by uncertainties in determining patch suitability for currently unoccupied patches. This has been attempted using statistical models of habitat quality (with some success [77]), but the uncertainties in determining unoccupied patch suitability are rarely quantified. As with niche-based models, there is also a problem extrapolating to future conditions when expected conditions exceed the variability sampled in current data: wherever relationships between climate and demography are based on observation, extrapolations are subject to large but unmeasurable uncertainties.

Looking forward, it seems likely that the fundamental components of demographic models will not change greatly, but there will continue to be refinement of individual models as data on additional mechanisms accrue. We expect that these models will be fitted to more species: usable data are routinely collected in existing monitoring schemes for many more species than so far modelled. As these models are developed further, the same uncertainties about measures of model fit that affect niche-based model predictions will become important.

5. Process-based models

This third and final class of SDM takes an approach opposite to that of niche-based models. Instead of building a statistical model of niche from geographical distribution, process-based models instead determine niche physiology and then reconstruct geographical distribution from this. This is a relatively new field, with comparatively few working models. Examples of the genre include models for trees that identify niche using aspects of phenology (e.g. PHENOFIT [78]), which assumes that a species can survive only if all components of its life cycle are completed within a calendar year: producing leaves while avoiding frost, growing flowers, setting seed, etc. Similar approaches for animals may be possible, but the best examples focus on determining whether animals meet energetic demands [79]. Kearney et al. [80] describe a model of the greater glider (Petauroides volans) using assessments of energetic and evaporative costs converted to units of milk. They identify the physiological niche where sufficient milk can be produced to successfully raise young and project this to geographical space. Similarly, Kearney et al.'s [81] model of the cane toad (Bufo marinus) measures the impacts of temperature on movement, reproduction and survival, then projects this to geographical distribution. At an even more fundamental level, there are models of plants that use the chemistry of photosynthesis and physiology of water-stress to model tree responses to global change (indeed, some argue that only models based directly on physiology should truly be described as process-based models, all other approaches being descriptions of the relationship between climate and demography). Functioning at this level means such models can incorporate processes such as responses to changing CO₂ levels [82].

Process-based models often require numerous parameters (e.g. timing of phenological events, or sensitivity of activities to climate), each of which carries uncertainty. Correctly assessing uncertainties in process-based models involves identification of sources of variation for each parameter, including individual variation and population differences caused by local adaptation. It is simple to incorporate this variation into output through resampling of observed variation and re-running the model, though to date most models discard information on variation and instead use averages [78] or at best undertake sensitivity analyses [81]. More complex models have more parameters and suffer more from parameter estimation uncertainty, as error propagation results in increasingly imprecise estimates as complexity increases. Indeed, creating geographical predictions from the output of the most complex process-based models is problematic as, for example, tree growth rates (raw output from some physiological models) must be translated into population dynamics [82].

As with niche-based models, a further source of error in process-based models is the challenge of identifying the important niche axes. Although niche-based models use pattern-matching to identify likely niche components, process-based models rely on an understanding of the species' ecology to identify important niche axes. Where direct impacts of temperature or water-stress are important, experiments can measure tolerance and identify limiting factors [78,81]. Often, however, the interaction between environment and organism is complicated and experiments are impossible or unethical: it is hard to justify using temperature chambers to identify stressful environmental conditions on many animal species, especially rare ones. In such cases, tolerance must be measured differently [80] with increasing errors. Moreover, identification of climate impacts on life-history components is challenging, requiring detailed knowledge of the ecology of the target species: once options are identified, the only appropriate test is to run the model and see how well predicted matches observed distribution. This comparison is even more challenging than for niche-based models because the prediction is based on the fundamental niche (though if dispersal is incorporated within the model, the prediction can be expected to be closer to the realized niche), while the geographical distribution is formed by the realized niche: over-prediction is virtually guaranteed.

An important advantage of process-based models is that true prediction into conditions beyond observed conditions (e.g. into areas with novel climates) is possible [82], and can incorporate appropriate uncertainty. Moreover, genetic evolution can be included within process-based models, enabling this potentially important mechanism to be incorporated [83]. Indeed, models containing competitive interactions with other species are possible, though have not yet been attempted. Without such multi-species modelling, however, the same problems with lack of interactions remain for process-based models as other distribution modelling approaches.

Distribution models based on physiology can be further abstracted to generic trait types and not for individual species. Such models predict not species, but collections of plant functional types defining vegetation classes, such as biomes. They are termed dynamic global vegetation models (DGVMs; [84,85]) and not usually considered as SDMs, but their conceptual and mathematical similarity to single-species process-based models makes it appealing to consider their use alongside more standard distribution models. In contrast to single-species models, DGVMs explicitly model vegetation assemblages and the boundaries between major community types from basic plant physiology, using plant functional types to define communities [47]. Most DGVMs are run only for single or small sets of parameter values, providing sensitivity and not uncertainty estimates, but increased computer power will soon enable uncertainty assessments [86]. Other refinements, such as the incorporation of additional traits, will follow [47]. Assessment of DGVM accuracy, like that of typical SDMs, lies in the match between predicted current distribution and that of observed distribution [84,85]. As with all models, this comparison is subject to uncertainty in the observed distributions (usually assessed from satellite imagery), but also in the definition of plant communities: many biomes do not have clear edges, but rather grade from one to another through intermediate states, and boundary choice is arbitrary and introduces uncertainty. Unlike single-SDMs, however, DGVMs are not subject to uncertainty from interactions, as the competition between biomes and associations between species are explicitly included in the models. There are, however, uncertainties in this for those systems where factors other than climate drive biome limits, such as grassland–forest interfaces mediated through herbivory and fire [87], only the latter of which is sometimes incorporated into DGVMs. Furthermore, current DGVMs are subject to uncertainties from the lack of explicit modelling of dispersal [88]. Despite these uncertainties, however, DGVMs are remarkably accurate in their overall prediction of global vegetation communities [85]. It seems that sacrificing information on species identity makes it possible to improve predictions about collections of species as vegetation types. This may be because community or local assemblage membership rules operating via species interactions combine with environmental filters acting on species traits to make the average properties more predictable than specific presence or absence of individual species.

Uncertainty in process-based models can easily be incorporated into predictions. However, uncertainties from the modelling process are not the only sources of uncertainty, as predictions are generated from an assumed model of the niche. Using a process-based model, one cannot quantify the uncertainty that occurs owing to missing niche components. For example, if Kearney et al. [81] had not measured mobility and assumed only reproduction and survival were temperature-sensitive they could have built a process-based model including full uncertainty from the variability in temperature and demography—but their model would have been less accurate than the uncertainty thus derived supposes. Unfortunately, this uncertainty is irreducible and the only test of the model is to compare observed with modelled distribution, with all the problems this involves. So far, few process-based models have been published and it remains unknown how difficult it is to identify niche parameters—studies published to date have been successful, predicting current distributions similar to observed patterns [79]. However, it is unlikely that process-based models which do not predict current distribution well will be publishable, when one cannot determine if it is the model that is misspecified, or if realized and fundamental niche are simply too different [89].

Thanks to their bottom-up approach and the ability to predict distributions independent of the data used to generate them, process-based models have an important contribution to make in this field. Models of individual species will become increasingly well-defined, as information accrues about niche dimensions and physiological responses to temperature and water-stress are better understood. There is little doubt that models will be developed for more species and new software is becoming available to assist the process. We suggest, however, that the requirement for detailed, species-specific information (often involving laboratory experiments) means that process-based models will never be available for the number and diversity of species that are required, for example to make global assessments of species sensitivity to climate change. Instead, we suggest that an abstraction is likely, similar to that of DGVMs, where species identity is ignored and trait-based analyses develop that allow sensitivity to be assessed based on readily available trait information.

6. Discussion

From this overview, it is clear that multiple sources of uncertainty apply to all model types. Data uncertainties in environmental covariates affect output from all models, and should always be incorporated in model predictions but this is rarely done [90]. Uncertainties in species distribution data present particular challenges to niche-based models, but also need considering when validating predictions from all model types using observed data—another issue often overlooked. Prediction uncertainty of future climate from climate models needs to be considered in predictions from all model types; e.g. rather than using median outputs for each climate model variable, individual realizations should be used. Parameter uncertainty should be estimated and recorded, whether using a niche-based model, or a model of fundamental niche based on laboratory-measured physiological tolerance. And all examples are subject to uncertain influences of biotic interactions. Other sources of uncertainty are specific to model class: niche-based models are subject to uncertainty from having no mechanism to include dispersal and source–sink processes, whereas mechanistic niches modelled in process-based models never reflect true complexity of biological systems. Although most sources of uncertainty have been previously identified and occasionally measured [15], we know of no example where all known sources of uncertainty have been measured. As many sources of uncertainty have multiplicative effects, it is inevitable that uncertainty in SDMs is generally underestimated. Indeed, once appropriate measures of prediction uncertainty are available, we expect that for many species our best models are completely uninformative.

We have further identified gaps where research can reduce the uncertainties associated with current SDMs. Primary among these is an appropriate measure of model fit. Analysts have relied heavily on internal cross-validation and AUC measures of predictive performance, but this approach can misidentify over-fitted models as well-fitting and strongly predictive. Until we know with confidence how well a model performs on independent datasets, we remain fundamentally uncertain about predictions of all SDMs. Such a measure must identify how well predictions fit observed data (dealing appropriately with the uncertainty in the observations themselves) while penalizing for model complexity, and should remain unaffected by both autocorrelation in the observed pattern [51] and prevalence [50]. Ideally, models should be tested against independent data, such as in an introduced range [91,92], through use of historic [93] or palaeontological [47] datasets to retrodict distribution, and through the use of simulated data where the ability of the model to recover known processes is a measure of performance.

Measures appropriate to many uncertainty sources are already available, both through mechanisms to incorporate uncertainty into the modelling framework (e.g. hierarchical models allow estimates and errors to propagate through various submodels [57]) and through those designed to increase accuracy. In the latter class fall ensemble forecasts which average across many alternative models [94]. However, while averaging is useful for increasing precision, it does not affect the accuracy and current implementations of ensemble modelling do not preserve measures of uncertainty through the averaging process. Nevertheless, comparisons of multiple models are useful for reducing uncertainty and examples exist where comparisons have been made between different classes of distribution model, allowing differences and similarities to be identified. Notably, Kearney et al. [80] compared niche-based models and a process-based model of the distribution of the invasive cane toad in Australia. They showed that while some niche-based models give different outputs, one method in particular (maximum entropy [95]) gave a similar predicted distribution to the process-based model. By contrast, Keenan et al. [82] compared niche-based models and process-based models for three tree species and found that if impacts of increasing CO₂ are ignored, then both niche- and process-based models give qualitatively similar predictions, but if CO₂ is incorporated into the process-based model, the predictions are reversed: the effect of CO₂ is overwhelmingly important, but cannot be identified by niche-based modelling methods. Other comparisons report similarly disparate results, and in cases of disagreement, we suggest the more fundamental insights coming from a good process-based model should be preferred to those of a pattern-based approach. These comparisons are important because they allow uncertainties from different sources to be identified, and the potential magnitude of mistakes to be estimated.

While comparisons are useful, they are not the only way multiple model types can be combined: different model types can be joined statistically, in similar ways different types of climate models have been linked to generate localized predictions [96]. For example, niche-based models and demographic- or process-based models could be integrated across spatial scales in a hierarchical framework [97], or more simply, DGVM output could feed into SDMs to better predict species reliant on particular biomes. These methods could further be linked to network recovery tools to estimate and incorporate the impacts of biotic interactions. Implementing such ‘second-generation’ SDMs will require further statistical research developing methods to identify biotic interactions and to develop specific hierarchical models relevant to this problem. The bases for such models are already available [59,60], and progress in these areas will likely be rapid: any attempt to move models from a simple pattern-based approach to a more fundamental understanding of ecological processes is to be welcomed.

Returning to the use of SDMs to inform management decisions, the final area for further development is in the incorporation of uncertainty into decision-making frameworks. Decision theory is a well-developed field, but has only recently appeared in the ecological management literature [98–100]. Perhaps ironically, for many of the species of most interest to conservation (often those with small ranges), modelling uncertainty will always be large. Indeed, not only may uncertainty be great, but also some sources are unquantifiable, such that we are uncertain about the degree of uncertainty in many distribution models. We argue that it is just as important to quantify uncertainty in model predictions as to make the predictions themselves, yet the importance of prediction uncertainty is rarely emphasized. By correctly identifying the existence and sources of uncertainty, even if in the worst case predictions are impossible, advice and management actions will be better informed than if based on false certainty.