A unifying theory for two-dimensional spatial redistribution kernels with applications in population spread modelling

Abstract

When building models to explain the dispersal patterns of organisms, ecologists often use an isotropic redistribution kernel to represent the distribution of movement distances based on phenomenological observations or biological considerations of the underlying physical movement mechanism. The Gaussian, two-dimensional (2D) Laplace and Bessel kernels are common choices for 2D space. All three are special (or limiting) cases of a kernel family, the Whittle–Matérn–Yasuda (WMY), first derived by Yasuda from an assumption of 2D Fickian diffusion with gamma-distributed settling times. We provide a novel derivation of this kernel family, using the simpler assumption of constant settling hazard, by means of a non-Fickian 2D diffusion equation representing movements through heterogeneous 2D media having a fractal structure. Our derivation reveals connections among a number of established redistribution kernels, unifying them under a single, flexible modelling framework. We demonstrate improvements in predictive performance in an established model for the spread of the mountain pine beetle upon replacing the Gaussian kernel by the Whittle–Matérn–Yasuda, and report similar results for a novel approximation, the product-Whittle–Matérn–Yasuda, that substantially speeds computations in applications to large datasets.

1. Introduction

Ecologists are concerned with questions of an inherently spatial nature, since movement and environmental heterogeneity are ubiquitous features of the natural world. They are therefore often rewarded by new insights when the mechanism underlying a spatial effect can be worked into a process model [1]. Redistribution kernels are a popular means to this end, with applications as diverse as predator–prey interactions [2]; range expansion and invasion biology [3]; grouping/swarming behaviour [4]; chemical communication [5]; and cellular transport [6].

Statistical ecologists study many of the same questions, but with a focus on characterizing the randomness in measurements. This, too, is a spatial problem. Ecological datasets typically exhibit spatial autocorrelation (SAC) [7], which, if ignored, reduces the precision of estimators [8]. A common solution is to use phenomenological models, known as covariograms (or covariance kernels), that map separation distances to correlations [9].

Covariograms are qualitatively similar to redistribution kernels—both describe a tapering of relatedness with spatial scale, so it is perhaps unsurprising that the same function families often take on both roles. An example that we find particularly interesting is the function known variously as the Whittle–Matérn [10] or K-Bessel kernel [11]. Its versatility and mathematical elegance make it one of the most important covariograms in spatial statistics [12]. Yet, in spite of many advantages over more familiar alternatives (e.g. the Gaussian), this function receives little attention in the context of movement modelling.

Up to a normalization constant (and a restriction on the shape parameter), the Whittle–Matérn is identical to the K-Bessel redistribution kernel first derived by Yasuda [11], and later by Yamamura [13] and Hapca et al. [14]. We focus in this paper on its potential as a movement model, offering a new mechanistic derivation. However, as the literature on covariograms contains decades worth of analysis into its mathematical properties, we refer to this function as the Whittle–Matérn–Yasuda (WMY) kernel.

1.1. Redistribution kernels

A redistribution kernel D(x, x′) maps pairs of coordinates (source x′ and destination x) to the probability density for the redistribution event from x′ to x. When modelling movement events having a random character, kernels provide a simple means of parametrizing the probability density function (PDF) for position following the movement event. For example, in process-based models, redistribution kernels are often derived from partial differential equations (PDEs) representing the (time) evolution of the position PDF during a random walk [15].

The simplest such models are stationary, meaning their kernels are functions of the separation vector r = x − x′ only, independent of location. When isotropy (radial symmetry) is also assumed, kernels can be defined more simply as a function of r = |r|. Of these simpler kernels, the isotropic Gaussian (table 1) is the most widely used. It solves a PDE for Fickian diffusion proceeding until a particular (fixed) time.

Table 1.

Notable examples of stationary isotropic kernels from the WMY family, $\mathcal{D}(r;\kappa ,\rho )$. All arise from 2D Fickian diffusion with gamma-distributed settling times [11]. The special cases listed here have been derived independently under various movement models. Alternatively, the full WMY family can be derived by repeated iterations of the kernel in the top row (§2.2). ^*for brevity the normalization $1/2\pi {\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}rD\hspace{0.17em}\mathrm{d}r$ is omitted.

shape (κ)	kernel name	density* D(r)	mechanistic derivation
(−1, 0)	—	(r/ρ)κKκ(r/ρ)	2D fractal diffusion with constant settling hazard (§2.1)
0	Bessel	K0(r/ρ)	2D Fickian diffusion with constant settling hazard [16]
1/2	2D Laplace	exp (−r/ρ)	2D turbulent diffusion with instantaneous settling [17]
∞	Gaussian	exp (−(r/ρ)2)	2D Fickian diffusion with instantaneous settling [18]

With two-dimensional (2D) isotropic models, one must take care to distinguish between the full density D(r) = D(r, θ) of the random point (R, Θ) and the marginal density D_r(r) of the random radius R (i.e. distance). The two are related by

$$D_r(r):={\int }_0^{2\pi }D(r,\theta )r\hspace{0.17em}\mathrm{d}\theta =2\pi rD(r,\theta )=2\pi rD(r),$$ 1.1

where factor r in the integrand is the Jacobian of the polar transformation, so that ${\int }_{{\mathbb{R}}^2}D={\int }_0^{\mathrm{\infty }}{D}_r(r)\hspace{0.17em}\mathrm{d}r=1$. This distinction between marginal and full densities is important, but sometimes unclear in the ecological literature. Our notation will always indicate the marginal by a subscript, as we do in (1.1).

The WMY is an important example of a stationary and isotropic kernel:

$$\mathcal{D}(r;\kappa ,\rho )=A(\kappa ,\rho ){\left(\frac{r}{\rho }\right)}^{\kappa }{K}_{\kappa }\left(\frac{r}{\rho }\right),\text{with}\hspace{0.17em}\frac{1}{A(\kappa ,\rho )}=2^{\kappa +1}\pi {\rho }^2\mathit{\Gamma }(\kappa +1),$$ 1.2

where $K_{\kappa }$ denotes the κth order modified Bessel function of the second kind (electronic supplementary material, appendix 1.1), ρ > 0 is a distance-scaling (range) parameter and κ is a kurtosis (shape) parameter. The domain of κ depends on the application: for covariance, κ > 0 (to guarantee positive definiteness); and for redistribution, κ > −1 (to guarantee integrability). By studying κ, we will see that the WMY is closely related to a number of other kernels in common use, generalizing some the most prominent ones (table 1), while providing a more flexible range of tail behaviours (see §2.3).

This makes the WMY an unusually flexible model for the spread of populations. An important example from our research area is the mountain pine beetle (MPB). Dispersal flights of this forest pest allow populations to spill outward into neighbouring areas, and are therefore a key component of models for the progression of outbreaks moving across the landscape. We discuss the use of redistribution kernels in such applied situations in §3, showing how the WMY (and a computationally efficient approximation, the pWMY) can improve model fit when used in place of the less flexible Gaussian kernel in a popular model for MPB forest damage patterns.

The bulk of this paper, however, is devoted to developing a new process-based mechanism to explain how this kernel arises. To motivate the use of equation (1.2) more generally, we derive the WMY kernel as the solution to a PDE for diffusion through inhomogeneous habitat, with a constant hazard of settling (§2). In the course of this derivation, we highlight the many appealing mathematical properties of the WMY, and draw connections between the WMY and number of other, better-established, isotropic models for redistribution.

2. The Whittle–Matérn–Yasuda as a model for diffusion with settling

In Yasuda’s 1975 paper [11], the WMY was derived as the settled density solution to a Fickian diffusion process with gamma-distributed settling times. Equation (1.2) can therefore be understood as an extension of the Bessel kernel (table 1), with a more general characterization of stopping times. Both models, however, assume the random walk takes place in unrestricted 2D space. We will relax this assumption, by assuming instead that obstacles in the environment may inhibit movements to some degree.

In statistics, the WMY is derived from the stationary random field solution of a fractional stochastic PDE resembling a generalized Helmholtz equation [19]. This suggests that as a redistribution kernel, the WMY might also solve a similar deterministic PDE involving fractional derivatives. These exotic dynamical systems often appear in connection with the statistical mechanics of (non-Fickian) diffusion through complex media that hinder movement. They are studied in physics, for example, to explain the physical and chemical properties of porous substrates [20]. Here, we adapt those results in the ecological context to find a versatile macro-scale description of random walks.

2.1. Diffusion over fractal media in two dimensions

To constrain the movements of our random walker, we view the domain of movement as a porous 2D medium with a fractal structure, rather than a full 2D space. This more faithfully reflects the type of movement behaviours that actively track patchy arrangements of habitat, or navigate around complex arrangements of physical obstacles.

Fractal environments are not an unfamiliar idea in ecology. Fractal aspects of tree crown cover, hydrological networks and topography have been recognized for decades [21]. These structures often exhibit power laws under scaling that allow their pertinent features to be summarized by simple parameters, such as the Hausdorff dimension d_f [22]. The flight of a MPB, for example, is constrained within a complex network of gaps in the forest vegetation. Like many aspects of forest structure, the porosity of this abstract medium can be conveniently described using d_f [23]. For example Jonckheere et al. improved predictions of light penetration in pine forests by estimating the d_f value in hemispherical images of scots pine canopy gaps, finding them to be highly fractal [24]. We will use d_f to describe the porosity of the patchy space navigated by random walker.

d_f summarizes space-filling properties. In full 2D space, the area enclosed in a disc of radius r scales as πr², whereas in an embedded fractal space it scales as $\pi r^{d_f}$ [25]. Unlike the topological dimension (in our case, d = 2), Hausdorff dimension can assume non-integer values. One can define spaces with 0 < d_f < 2 that nearly fill the plane, yet leave a complex arrangement of patches inaccessible, with the availability of habitat decreasing with d_f. [26], for example, used Hausdorff dimension to characterize the quality of marten habitat, estimating a d_f in the range of 1.7–1.9 for pine and spruce forests in their Utah study area.

In the context of random walks, a fractal medium offers less space for movement. This prompts some adjustments of the balance law behind Fickian diffusion: suppose u(r, t) is a PDF for occupancy within the available space at radius r, at time t. If all of ${\mathbb{R}}^2$ were available, the density within the annulus Ω would be measured by $2\pi {\int }_{\mathrm{\Omega }}u(r,t)r\hspace{0.17em}\mathrm{d}r$. On a d_f-dimensional fractal it is:

$$2\pi {\int }_{\mathrm{\Omega }}u(r,t)p(r)r\hspace{0.17em}\mathrm{d}r\text{where}\hspace{0.17em}p(r)=r^{d_f-2},\hspace{0.17em}\text{and }0<d_f\le 2.$$ 2.1

The scaling function $p(r)=\pi r^{d_f}/\pi r^2$ is the proportion of the area inside radius r that is available for movement. For notational convenience, we will suppress this dependence on r (and d_f) and simply write p. Function u(r, t) is therefore an occupancy PDF with the (unusual) distance-scaled probability measure $pr\hspace{0.17em}\mathrm{d}r$. Under the more familiar Lebesgue measure, the PDF is D(r, t) = u(r, t)p with measure $r\hspace{0.17em}\mathrm{d}r$.

In their 1985 paper [27], O'Shaughnessy and Procaccia described how the usual equation for Fickian diffusion may be modified in order to remain consistent with the scaling property (2.1). Their generalized 2D heat equation describes the time-evolution of D(r, t)/p = u(r, t)

$$\frac{\mathrm{\partial }u}{\mathrm{\partial }t}=\frac{1}{pr}\frac{\mathrm{\partial }}{\mathrm{\partial }r}\left(\alpha pr\frac{\mathrm{\partial }u}{\mathrm{\partial }r}\right)\text{with}\hspace{0.17em}\alpha >0,\hspace{0.17em}u(r,0)=\frac{\delta (r)}{2\pi pr},$$ 2.2

where δ(r) denotes the one-dimensional (1D) Dirac delta function (and δ(r)/2πpr its 2D analogue), representing the initial departure of the disperser from the origin. Note the diffusivity coefficient $\alpha p=\alpha r^{d_f-2}$ scales with distance, approximating correlations in movements due to geometrical constraints [20]. Under the condition u → 0 as r → ∞ (required for a valid PDF), the general solution to (2.2) is known. Switching to Lebesgue measure, it can be written

$$D(r,t)=u(r,t)p={\left(\pi \mathrm{\Gamma }\left(\frac{d_f}{2}\right){(4\alpha t)}^{d_f/2}\right)}^{-1}{r}^{d_f-2}\exp (-r^2/4\alpha t).$$ 2.3

D(r, t) is the PDF for the position (r, θ) at time t of a random walker that departs the origin at t = 0 and diffuses through a medium with Hausdorff dimension d_f. The effect of decreasing d_f is to make this density function fatter-tailed, with density shifted from the shoulders of the distribution towards its extremities.

Note that u(r, t) is simply a Gaussian kernel renormalized for measure $pr\hspace{0.17em}\mathrm{d}r$. Indeed when d_f = 2 we have D(r, t) = u(r, t) (since p = 1), and equation (2.2) is the classical 2D heat equation in radial coordinates, with the 2D Gaussian kernel its well-known solution [4]. When d_f = 1 equation (2.2) simplifies to become a 1D diffusion equation (since pr = 1), so in the resulting marginal density function D_r = 2πu(r, t)pr = 2πu(r, t) we find the 1D Gaussian kernel.

2.1.1. Settling via constant hazard

To build a redistribution kernel directly from (2.2) to (2.3), one could simply assume movement proceeds until a particular fixed time t. However, in reality, the duration of dispersal is often stochastic. Unpredictable environmental factors such as temperature may compel the random walker to wait out unfavourable conditions [28]. Moreover, settling events can be prompted by chance encounters, such as the detection of a prey item [29] or mate [11].

Preferring a model that accounts for randomly cued settling events, we suggest a simple extension of (2.2) that introduces a constant settling hazard λ > 0. We then define our kernel as the total settled density over all time. Thus we have $D={\int }_0^{\mathrm{\infty }}\lambda up\hspace{0.17em}\mathrm{d}t$, with p defined as in (2.1), and:

$$\frac{\mathrm{\partial }u}{\mathrm{\partial }t}=\frac{\alpha }{pr}\frac{\mathrm{\partial }}{\mathrm{\partial }r}(\hspace{0.17em}pr\frac{\mathrm{\partial }u}{\mathrm{\partial }r})-\lambda u\text{with}\hspace{0.17em}u(r,0)=\frac{\delta (r)}{2\pi pr}\hspace{0.17em}\text{and}\hspace{0.17em}\underset{r\to \mathrm{\infty }}{lim}u(r,t)=0.$$ 2.4

Here, the disperser moves about the domain as in (2.2) but settles at a randomly determined time, drawn from an exponential distribution with mean 1/λ. D(r) now expresses the PDF for position at the time of settling. To find D, one can integrate the PDE (2.4) over all time and consider weak solutions u (electronic supplementary material, appendix 1.3.2). The resulting kernels are the singular members of the WMY family

$$D(r)={\int }_0^{\mathrm{\infty }}\lambda up\hspace{0.17em}\mathrm{d}t=\mathcal{D}(r;\kappa ,\rho )\text{where }\kappa =\frac{d_f}{2}-1\hspace{0.17em}\text{and}\hspace{0.17em}{\rho }^2=\frac{\alpha }{\lambda }.$$ 2.5

These WMY kernels (−1 < κ ≤ 0) also emerge as the long-time limit of u(r, t)p when the point source is stationary in time rather than instantaneous. To see this, we modify (2.4) by viewing λ as a mortality hazard and adding a source term F = λδ(r)/2πpr to the right-hand side of the PDE. Dispersers are therefore continuously introduced from the origin, and continuously removed throughout ${\mathbb{R}}^2$ in a density-dependent manner. The steady state in this smokestack-like system is $\underset{t\to \mathrm{\infty }}{lim}u(r,t)p=\mathcal{D}(r;\kappa ,\rho )$, with κ, ρ defined as in (2.5) (electronic supplementary material, appendix 1.3.1).

The (non-fractal) case of d_f = 2 produces a Bessel kernel, or $\mathcal{D}(r;0,\rho )$. This model has a long history in ecology, as a description of the movements of worms [16] and moths [30]; and later as a model for the biological activity of a chemical diffusing outward from the centre of a petri dish [31]. More recently, approximations of the Bessel kernel have been used to model bark beetle dispersal flights [29,32].

2.2. Multi-stage extensions

Redistribution events may naturally split into multiple stages. For example, diurnal periods of flight activity occur in many insect orders [33]. Moreover, forest-dwelling insects like the MPB may initially fly in the unrestricted space above the canopy before switching to subcanopy dispersal [28]. In this section, we look at two simple ways of extending (2.4) to model multi-stage processes.

2.2.1. Switching via constant hazard

The first idea is to connect each stage by a switching hazard with rate constant λ > 0. Writing u_m(r, t)p for the density in the mth stage, we assume an instantaneous point release of unit density initializes dispersers in the first stage, so that the dynamics of u₁ are the same as u in (2.4). As time progresses, this initial impulse trickles through subsequent stages, eventually exiting the nth stage at rate λ (as settled density). We will be interested in the long-term settled density

$$D(r)=U_n(r)p=\lambda {\int }_0^{\mathrm{\infty }}{u}_n(r,t)p\hspace{0.17em}\mathrm{d}t,\text{where }$$ 2.6a

$$\begin{array}{rl}& \frac{\mathrm{\partial }{u}_1}{\mathrm{\partial }t}=\frac{\alpha }{pr}\frac{\mathrm{\partial }}{\mathrm{\partial }r}(\hspace{0.17em}pr\frac{\mathrm{\partial }{u}_1}{\mathrm{\partial }r})-\lambda u_1,\text{with }{u}_1(r,0)=\frac{\delta (r)}{2\pi pr},\\ & \text{and, for }1<m\le n,\end{array}$$ 2.6b

$$\begin{array}{r}\frac{\mathrm{\partial }{u}_m}{\mathrm{\partial }t}=\frac{\alpha }{pr}\frac{\mathrm{\partial }}{\mathrm{\partial }r}(\hspace{0.17em}pr\frac{\mathrm{\partial }{u}_m}{\mathrm{\partial }r})+\lambda u_{m-1}-\lambda u_m,\text{with }{u}_m(r,0)=0.\end{array}$$ 2.6c

Assuming lim _r→∞u_m(r, t) = 0, an analytic solution is available (electronic supplementary material, appendix 1.3.3):

$$D(r)=U(r)p={(2^{n-1+d_f/2}\pi {\rho }^2\mathrm{\Gamma }(n)\mathrm{\Gamma }\left(\frac{d_f}{2}\right))}^{-1}{\left(\frac{r}{\rho }\right)}^{n-2+d_f/2}{K}_{n-d_f/2}\left(\frac{r}{\rho }\right),$$ 2.7

where ρ² = α/λ. The PDF in (2.7) is simply the (renormalized) product of the WMY kernel $\mathcal{D}(r;n-d_f/2,\rho )$ with the scaling function p. Thus in the non-fractal case of d_f = 2 it produces the kernel family $D=U=\mathcal{D}(r;n-1,\rho )$. This extends the Bessel kernel (n = 1) to yield a sequence of PDFs that are concave and bounded in their approach to the origin. These are special cases of Yasuda’s gamma-distributed settling time model [11], if we interpret settling time as a sum of exponentials (representing time spent in each of the stages).

For d_f < 2, however, the distribution (2.7) remains unbounded at the origin for all n. In general, by increasing n we shift density away from the tails, effectively stalling dispersers near the origin. Decreasing d_f has the opposite effect, producing fatter tails and a highly peaked shape.

This approach of linking PDEs for n-stage processes was suggested by Neubert, Kot & Lewis [2] to describe (non-fractal) diffusion with settling on the real line. In their 1D model, the solution is a product of a Laplace kernel and a polynomial of order n − 1. These same kernels were later proposed as extensions of the Laplace that are robust to changes in sampling frequency [34]. We show in the electronic supplementary material, appendix 1.3.3, that they emerge also from (2.6) to (2.7), as the marginal density functions (D_r) for d_f = 1.

2.2.2. Convolutions of Whittle–Matérn–Yasuda kernels

A simpler approach to the n-stage modelling problem is to suppose a long time delay separates stages. Thus within each stage, we assume the diffusion process (2.4) operates until (nearly) all density has settled. The t → ∞ limit in (2.5) then becomes initial data for the next stage. Under this assumption, the n^th stage settled density can be written as a convolution of n WMY kernels.

This is because when a population independently undergoes the redistribution process represented by D twice in succession, the resulting composite kernel is the autoconvolution D * D. More generally if a population undergoes a sequence of n independent redistribution stages described by the kernels D⁽¹⁾, D⁽²⁾, …, D⁽ⁿ⁾ then their combined effect is D = D⁽¹⁾*D⁽²⁾*· · ·*D⁽ⁿ⁾.

Certain kernels have the property of closure under n-part convolutions. Chesson and Lee explained how this property aids interpretability, using it to develop redistribution kernels for lattice data [35]. The Gaussian kernel is an example. The WMY is another, provided the range parameter ρ > 0 is fixed in all stages (electronic supplementary material, appendix 1.2). In particular, if ${\mathcal{D}}^{(m)}=\mathcal{D}(r;{\kappa }_m,\rho )$ is a 2D WMY,

$$\begin{array}{rl}{\mathcal{D}}^{(1)}\hspace{0.17em}\ast \hspace{0.17em}{\mathcal{D}}^{(2)}\hspace{0.17em}\ast \hspace{0.17em}\cdots \hspace{0.17em}\ast \hspace{0.17em}{\mathcal{D}}^{(n)}& =\mathcal{D}(r;\kappa ,\rho ),\\ \text{where }\kappa & =n-1+\sum _{m=1}^n{\kappa }_m.\end{array}$$ 2.8

Note that we may assume the shape parameters κ_m belong to (−1, 0], since 0 < d_f ≤ 2. So by allowing arbitrary n > 0, the model (2.8) generates the complete set of thinner-tailed and bounded WMY kernels (0 < κ < ∞) not captured by the single-stage model (2.4) in the previous section.

By adding stages, we increase κ, shifting density away from the tails and origin, and towards the shoulders of the distribution. Thus as κ increases through zero, $\mathcal{D}$ becomes bounded at the origin; and as κ increases through 1/2, its approach to the origin switches from convex to concave (electronic supplementary material, appendix 1.2). With further increases in κ, the WMY increasingly resembles a 2D Gaussian kernel; In fact, if we parametrize ρ² ∝ 1/κ, then $\mathcal{D}$ limits to a Gaussian kernel as κ → ∞ [36].

When κ = 1/2 the WMY simplifies to $\mathcal{D}(r;1/2,\rho )=$ ${(2\pi {\rho }^2)}^{-1}\exp (-r/\rho )$, which has the same functional form as a (1d) Laplace kernel. The Laplace is the 1D analogue of the Bessel; it arises from diffusion with constant-hazard settling in ${\mathbb{R}}^1$ [4]. In the 2D setting, this exponential form arises from turbulent limnological diffusion [17]. However, in ecological applications, it is frequently invoked simply as a phenomenological model [e.g. as in 37,38]. Equations (2.4)–(2.5) and (2.8) provide a new mechanistic origin for this 2D Laplace kernel. κ = 1/2 can arise, for example, in a two-stage process where the first stage of diffusion takes place in unrestricted 2D space (with d_f = 2) and the second in a fractal medium with d_f = 1 (so that κ₁ = 0 and κ₂ = −1/2).

Note that if all the κ_m are identical, (2.8) expresses that the WMY family is robust to changes in sampling frequency. In fact, Schlägel and Lewis described the WMY family implicitly (in Fourier space) as their first example of a 2D kernel with this property [34]. Equation (1.2) provides its explicit (back-transformed) density function.

2.3. Flexibility in kurtosis

The foregoing derivations are meant to illustrate how WMY patterns of redistribution might arise in a very wide range of ecological systems. In ecology, inhomogeneous environments are the rule rather than the exception. It, therefore, makes sense to relax the assumption of unrestricted movement while retaining it as a special case (the Bessel), as we did in the model (2.4). The natural extension in (2.8) generates a wide spectrum of kernel shapes, ranging from highly singular and heavy-tailed examples like the Bessel, to the highly smooth and thin-tailed Gaussian kernel. Let us now consider how this flexibility in shape is also attractive from a phenomenological standpoint.

One of the more important factors to consider when modelling redistribution is the balance of peakedness and tailedness in a kernel, or its kurtosis. In ${\mathbb{R}}^1$, kurtosis is the standardized fourth moment. In ${\mathbb{R}}^2$, we use the bivariate kurtosis measure k(D) suggested in [39],

$$k(D)=\frac{{\int }_0^{\mathrm{\infty }}{r}^4{D}_r\hspace{0.17em}\mathrm{d}r}{{({\int }_0^{\mathrm{\infty }}{r}^2{D}_r\hspace{0.17em}\mathrm{d}r)}^2}.$$ 2.9

This definition is similar to the univariate case, but differs in using moments of D_r (rather than D), and centring them on zero (rather than the nonzero marginal mean of r). This concisely summarizes the relevant behaviour of isotropic kernels; Larger values of k(D) indicate fatter tails (a higher density assigned to long-distance events) and a sharper peak at the origin. Note that while we use the term ‘fat-tailed’ to describe kernels with higher kurtosis than the Gaussian (here, and throughout the paper), tail fatness is sometimes defined in terms more subtle than equation (2.9). A detailed discussion of this topic can be found in [40].

For the WMY, kurtosis is determined entirely by κ (electronic supplementary material, appendix 1.2)

$$k(\mathcal{D})=2\left(\frac{\kappa +2}{\kappa +1}\right),\text{where }\kappa \in (-1,\mathrm{\infty }).$$ 2.10

The 2D Gaussian kernel sits at the low end of this range with a kurtosis of 2 (recall it is the κ → ∞ limit of $\mathcal{D}$). The WMY family extends it to capture a wide range of leptokurtic (higher than Gaussian) alternatives. The 2D Laplace, for example, has kurtosis 10/3; For the Bessel, it is 4. The 1-stage diffusion model (2.5) spans (4, ∞), with extremely fat-tailed examples emerging as d_f becomes small. This kind of flexibility in kurtosis is of particular importance in dynamical systems for population spread, where the tail behaviour can determine both the success and speed of invasions during a range expansion [3]. For example, a model that uses a (thin-tailed) Gaussian kernel out of mathematical convenience might fail to account for tail-like long-distance range expansion events.

3. The Whittle–Matérn–Yasuda in applications

Redistribution kernels are the central feature of the integrodifference equation (IDE), a model that combines growth (G) with spatially explicit dispersal (D) for a population density variable N_t(x) indexed continuously in space and discretely in time [41]. D is stationary in the most well-studied case, and the IDE reads N_t(x) = (D * G (N_t−1))(x), where * denotes convolution. A rich theory has emerged in connecting the mathematical properties of the function D with the spread dynamics of N_t as t becomes large [e.g. 3,42]. Redistribution kernels are also often applied in analyses of data from individual and aggregated years [e.g. 39,43]. In this more general setting, $\tilde{N}(\mathit{x})=(D\ast N)(\mathit{x})$ simply connects the pre (N) and post-dispersal ($\tilde{N}$) levels in a population.

In applications, populations are sampled at a finite set of locations, with centroids $\{{\mathit{x}}_k\}$ (k ∈ 1, …, m). The (continuous) kernel convolution D * N must therefore be replaced with a discretized version. In place of D one specifies an m × m redistribution matrix D, where the entry [D]_ij (a discretization of D, see electronic supplementary material, appendix 2.1) assigns density to the redistribution event from location x_j to x_i. Thus $\tilde{\mathit{n}}=\mathit{D}\mathit{n}$ connects the length-m pre (n) and post-dispersal ($\tilde{\mathit{n}}$) population vectors.

3.1. Separable approximations for gridded data

In many applied situations (e.g. numerical likelihood maximization), the matrix-vector product Dn must be evaluated many times. The operation has $\mathcal{O}(m^2)$ complexity, leading to issues of computability with large sample sizes (m). However, large spatial datasets are often gridded, and for this configuration there are remedies: fast Fourier transforms (FFTs) reduce the complexity to $\mathcal{O}(m\log m)$ [44]; and if D is spatially separable, Kronecker products can reduce complexity to $\mathcal{O}(m_x^2+m_y^2)$ (where the grid has dimensions m_y × m_x = m). On square domains that is $\mathcal{O}(m)$, a considerable improvement (figure 1). The implementation of this trick is straightforward, but notationally awkward, so we relegate the details to the electronic supplementary material, appendix 3.3.

Figure 1.

Time to compute the length-n post-dispersal vector Dn, for redistribution matrices from the WMY (dashed line) and pWMY (solid line) kernels. The pWMY is computed using Kronecker products (electronic supplementary material, appendix 3.3), while the non-separable WMY uses FFTs.

Separability is the property that D(r) factors into a product of two 1D kernels, D_x(r_x) and D_y(r_y), where r = (r_x, r_y)^T, each depending on only one of the spatial dimension. The (2D) Gaussian kernel is an example: it has the form D_x(r_x)D_y(r_y), where $D_x(r)=D_y(r)={\left(\sqrt{\pi }\rho \right)}^{-1}\exp (-{(r/\rho )}^2)$ are 1D Gaussian kernels. The 2D WMY kernel, on the other hand, is not separable. In seeking a computationally simple alternative, we propose the separable product-WMY (pWMY) kernel, ${\mathcal{D}}_{\otimes }$, in which the x and y components (D_x and D_y) are 1D WMY kernels. These have the same functional form as the 2D WMY (1.2), differing only in normalization constants. Thus,

$${\mathcal{D}}_{\otimes }(\mathit{r};{\kappa }_x,{\kappa }_y,{\rho }_x,{\rho }_y)=\frac{\mathcal{D}(|r_x|;{\kappa }_x,{\rho }_x)\mathcal{D}(|r_y|;{\kappa }_y,{\rho }_y)}{{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}\mathcal{D}(|r_x|;{\kappa }_x,{\rho }_x)\mathcal{D}(|r_y|;{\kappa }_y,{\rho }_y)\hspace{0.17em}\mathrm{d}{r}_x\hspace{0.17em}\mathrm{d}{r}_y}.$$ 3.1

In the electronic supplementary material, appendix 2.3, we report on a simulation study indicating that the pWMY closely approximates the WMY over much of its parameter space. Note that the 2D Gaussian is a limiting case of ${\mathcal{D}}_{\otimes }$, similar to $\mathcal{D}$. These (and other) computational properties of separable models are discussed in more detail in [45], where the pWMY is employed as a covariogram.

3.2. Extensions for geometric anisotropy

While mathematically pleasant, the assumption of isotropic redistribution is often unsatisfactory in ecological applications. Wind, for example, is directional, and wind-assisted migratory flights occur regularly in insects [28]. Indeed these were the cause of a recent range expansion of the MPB [46].

One of the simplest ways to incorporate anisotropy into an isotropic kernel is to compose it with an affine transformation of coordinates (electronic supplementary material, appendix 2.2). This effectively incorporates two stages of movement: drift (by a translation of coordinates) and directed diffusion (by a linear transformation of coordinates). In spatial statistics, such transformations are common extensions of the isotropic covariance kernel [36]. We see them far less often with redistribution kernels. However, they are easy to implement, and their effect on D is easy to visualize and understand (figure 2).

Figure 2.

A geometric model for anisotropic redistribution. Source populations (a) drift via translation D_τ then diffuse according to a redistribution kernel D, which is rotated and stretched by a linear transformation of coordinates. Two examples of D are shown: the WMY (D^a, c), and an approximation, the pWMY (with components D_x and D_y, b).

3.3. Case study: predicting mountain pine beetle infestations with the red-top model

To demonstrate the efficacy of the WMY and pWMY in applied movement modelling, we use the example of the MPB. Populations of this tree-killing beetle exhibit periods of elevated activity (outbreaks) in western North America that can persist for decades [47]. Pine mortality becomes visible on the landscape during outbreaks, with clusters of crown fade (red-tops) spreading through space from year to year, reflecting macro-scale movement patterns [48].

Annual aerial overview surveys (AOS) of MPB damage, carried out by the government of British Columbia (BC), provide a spatio-temporal record of outbreak spread. We analysed subsets of the AOS data using a slight modification of the red-top model of Heavilin & Powell [37]. Their IDE relates the pine mortality, M_t(x), in a given year (t) to its value in the following year. Pine deaths are a proxy for the beetle population in this model, and dispersal across the landscape is represented by the kernel D. Our modified version adds a (spatially uniform) constant 1 to all populations (adjusting for the unobserved, but ubiquitous endemic beetle community), and accounts for model error by an additive Gaussian term on the logit (i.e. log-odds) scale instead of the percent scale. The modified red-top model is thus written:

$$\text{logit}(M_{t+1})=2\log (1+D\hspace{0.17em}\ast \hspace{0.17em}(M_t{S}_t))-2\log (\alpha )+Z_t,$$ 3.2

where S_t(x) is the observed density of susceptible (healthy) pine, and Z_t(x) is a noise term representing process and measurement errors (electronic supplementary material, appendix 3.2). The growth parameter α > 0 parametrizes a nonlinear growth function representing the dynamics of beetle-pine interactions. The reader will find a more detailed explanation of (3.2) and its discretization in the electronic supplementary material, appendix 3.

3.3.1. Kernel selection: comparing the Gaussian, Whittle–Matérn–Yasuda and product-Whittle–Matérn–Yasuda

In their model selection for D [37], Heavilin and Powell compared the Gaussian and 2D Laplace kernels (table 1). We conducted a more exhaustive model selection (figure 3), in order to emphasize three questions: Which WMY kernel (i.e. which κ) best characterizes MPB spread? Is the product-WMY a useful surrogate for the WMY? And do the anisotropy extensions increase explanatory power?

Figure 3.

Flow chart for estimating model error. Data are split into blocks partitioned into test and training sets. Within-block likelihood is maximized on training data (for a given kernel), and predictions compared with test data. Blockwise errors were pooled and compared for six redistribution kernels.

Using numerical maximum likelihood, we fit the (discretized) modified red-top model independently to M = 81 non-overlapping blocks of AOS data, each of size 10 × 10 km, with a grid resolution of 1 hectare (ha). Data were split into test and training sets, facilitating an evaluation of within-block model errors (figure 3). To reduce the effect of SAC on these error estimates, we first fitted a Gaussian covariogram to the full dataset (containing all blocks), using it as a common (global) estimate of covariance in all of the within-block likelihood calculations (electronic supplementary material, appendix 3.4).

By pooling results over all M blocks, and repeating this process for different redistribution kernels D, we compared their overall explanatory power. We compared kernels from three families: the Gaussian; the WMY with κ > 0 (to exclude singularities at r = 0); and the pWMY (with κ_x, κ_y > 0). In each case, we tested the kernel with and without the anisotropy extensions of §3.2, and measured model performance on test data (unseen by the model in training) using root-mean-square prediction error (RMSPE), log-likelihood (LL) and Akaike’s information criterion (AIC) [49].

Figure 4 summarizes the results from all blocks and kernels, by centring each test statistic on that of the best performing kernel for that block. For all three kernel families, the anisotropy extensions improved performance dramatically. Without these extensions, there was little difference among kernels, with the WMY favoured slightly overall. However, with the extensions, the WMY produced substantial improvements over the Gaussian, and the pWMY performed best overall.

Figure 4.

Model performance over 81 model-fitting trials for six redistribution kernels: the Gaussian, WMY and pWMY; and their anisotropic extensions (Gaussian-A, WMY-A and pWMY-A); which have 1, 2, 4, 5, 6 and 7 parameters, respectively. RMSPE, negative LL and AIC were computed on test data in each block, and their differences (Δ) with the best model within that block are summarized as boxplots. Extreme outliers (points to the right of the boxplot whiskers) were associated with blocks having very low levels of MPB activity, where it appears there was insufficient information to reliably parametrize movement patterns.

For the isotropic WMY kernels, the fitted value of κ varied substantially among blocks. We observed a Laplace/Bessel-like spread pattern ($\hat{\kappa }\le 1/2$) in around one quarter of the 81 tested blocks. For around a third of the blocks, the parameter estimate stalled at its upper bound (κ = 25), indicating a tendency towards Gaussian spread patterns ($\hat{\kappa }\to \mathrm{\infty }$). On the remaining blocks it was intermediate. Thus we observed no particular value (or range) of κ to be favoured overall in the point estimates.

4. Discussion

Radar evidence suggests that wind-assisted MPB flights, carrying the insects unusually long distances, are not uncommon [50]. One such long-distance event recently led the MPB to cross the Rocky Mountains, from BC into central Alberta—a range expansion with severe ecological and economic consequences [46]. Equation (2.10) shows how by using WMY kernels, the probability of such tail-like events in models can be more finely tuned using κ.

Previous kernel selections in MPB spread models have led to somewhat contradictory conclusions on this tail behaviour [32,37]: One favoured the Gaussian kernel over a 2D Laplace, suggesting fixed-time settling events leading to thin-tailed movement patterns; the other favoured the (fat-tailed) Bessel over the Gaussian kernel, suggesting a movement mechanism with constant settling hazard (table 1).

This inconsistency can be resolved by simply dropping the assumptions of single-stage dispersal through homogeneous habitat. Our results on n-stage redistribution processes in fractal media (§2) give a mechanistic explanation of how disparities in dispersal patterns can arise simply as a result of geographical variation in habitat type (as represented by d_f), or in the number of times the dispersal event is interrupted and restarted, as might occur with local temperature swings [50].

This explanation is consistent with the wide range of fitted values for κ observed in our case study (§3.3.1), which indicated that no single (stationary) kernel adequately described the scope of MPB spread patterns exhibited in the AOS data. Note that since these observations were based on the maximum-likelihood point estimates alone, their statistical significance cannot be verified here. However, it is very likely that movement patterns will vary locally due to habitat structure [5]. For this reason (among others), we also expect a certain directionality in movement patterns, so it is perhaps unsurprising that the anisotropy extensions of §3.2 improved model predictive performance across the board. This also indicated that adding directionality to the red-top model did not lead to issues of overfitting, despite the introduction of 3–4 additional parameters (see electronic supplementary material, appendix 3.5). In a companion paper, we discuss applications of these directed models, using them to estimate the size of cryptic endemic subpopulations of MPB [51].

Yasuda’s model for ordinary Fickian diffusion with gamma-distributed settling times [11] provides an alternative mechanistic view on geographical variations in kernel shape. However, it requires dropping the parsimonious hypothesis of a constant settling hazard, which, for the MPB, is supported by evidence from laboratory flight-mill experiments [52,53].

More generally, the fractal diffusion kernels derived in §2 provide a plausible process-based description of random walks through complex media, and so have wide applicability beyond MPB dispersal. Recall that §2.1.1 extended the Bessel kernel (originally a model for nematodes) by relaxing the classical assumption of unrestricted movement space. Section 2.2.2 then showed how the full WMY family is recovered through n-stage iterations, where the Gaussian kernel (originally a model for muskrats) is approached in the limit n → ∞. The WMY kernel unifies both extremes of qualitative behaviour under a common process model, defining a wide spectrum of possible redistribution patterns, suitable for a wide range of study organisms.

The WMY has for a long time been embraced in spatial statistics community as the covariogram of choice, for its mathematical elegance and flexibility. In light of the mathematical connections to movement processes outlined in this paper, we would encourage the ecological modelling community to embrace the WMY as the redistribution kernel of choice.

Flexibility in tail behaviour is also desirable in phenomenological applications for redistribution kernels. In §2.3, we explained how this notion of flexibility can be made more precise, using 2D kurtosis. The WMY admits an exceptionally large range of kurtosis values. To our knowledge, the only other comparable redistribution kernel is the 2Dt [39], a phenomenological model of seed shadow distributions. In fact, the 2Dt and WMY are closely related: they are Fourier duals.

Unsurprisingly, the red-top model performed better on replacing the Gaussian with a more flexible WMY. More interesting was the superior performance of the pWMY. This phenomenological kernel takes the best of both worlds; it closely approximates the (mechanistic) WMY kernel, matching its flexibility while retaining the computational advantages of separability.

Data accessibility

All data and R code required to reproduce our analysis have been supplied as electronic supplementary material.

Authors' contributions

D.C.K. wrote the first draft of the manuscript and participated in all stages of the project. M.A.L. and S.R.L. contributed to the model development and critically revised the manuscript. All authors gave final approval for publication and agree to be held accountable for the work performed therein.

Competing interests

We declare we have no competing interest.

Funding

This work was supported by a grant to M.A.L. from the Natural Science and Engineering Research Council of Canada (grant no. NET GP 434810-12) to the TRIA Network, with contributions from Alberta Agriculture and Forestry, Foothills Research Institute, Manitoba Conservation and Water Stewardship, Natural Resources Canada-Canadian Forest Service, Northwest Territories Environment and Natural Resources, Ontario Ministry of Natural Resources and Forestry, Saskatchewan Ministry of Environment, West Fraser and Weyerhaeuser. M.A.L. is also grateful for support through NSERC and the Canada Research Chair Program.