Survival and Aging in the Wild via Residual Demography

Summary

Information about the age distribution and survival of wild populations is of much interest in ecology and biodemography, but is hard to obtain. Established schemes such as capture-recapture often are not feasible. In the proposed residual demography paradigm, individuals are randomly sampled from the wild population at unknown ages and the resulting captive cohort is reared out in the laboratory until death. Under some basic assumptions one obtains a demographic convolution equation that involves the unknown age distribution of the wild population, the observed survival function of the captive cohort, and the observed survival function of a reference cohort that is independently raised in the laboratory from birth. We adopt a statistical penalized least squares method for the deconvolution of this equation, aiming at extracting the age distribution of the wild population under suitable constraints. Under stationarity of the population, the age density is proportional to the survival function of the wild population and can thus be inferred. Several extensions are discussed. Residual demography is demonstrated for data on fruit flies Bactrocera oleae.

1. Introduction

Understanding aging in the wild is one of the most important problems in biodemography, yet available methodology is limited. With appropriate demographic tools, empirical data derived from field studies are expected to be used to frame and test theories of aging, inform research concerned with aging mechanisms, and establish baselines for the natural history of aging (Finch 1990, 2001; Promislow 1991; Gaillard et al. 1994; Reznick et al. 2001; Tatar and Yin 2001). We introduce the new concept of residual demography with the aim to deduce the age distribution of individuals in the wild. Residual demography utilizes data on laboratory survival from both wild-caught individuals of unknown age and wild-type individuals of known age. We demonstrate that the age structure of the wild population can be deduced from the survival of these individuals in the laboratory, in the general situation where survival in the wild and in the laboratory might differ. This study is motivated by the need to develop a method to estimate age structure in field populations of insects.

Residual demography includes two experimental steps, namely (1) the capture and transfer of wild individuals from the field to the laboratory where they are reared out under defined conditions and their date of death is recorded (captive cohort); (2) Creation of a reference cohort by obtaining individuals at birth from the wild and rearing them under the same defined laboratory conditions and recording death rates. The analysis of these data is then based on statistical deconvolution methodology. The aim is to construct an age distribution of the wild population that, when combined with the survival schedule of the reference cohort, yields the distribution of deaths observed in the captive cohort. If the population is stationary and a stable equilibrium of the age distribution has been reached, the density of the age distribution in the wild corresponds to the survival function of the population, a fact that is known from renewal theory (Feller, 1950).

A demographic identity to infer aging and survival in the wild has been discussed previously under the assumption that survival in the wild and after marking individuals is subject to the same force of mortality (Müller et al. 2004). This major restriction is dropped in the proposed residual demography approach, where the force of mortality is allowed to change upon transferring an individual of unknown age from the wild to the laboratory. This reflects underlying biology more closely and makes it possible to obtain inference for the age distribution in the wild in the face of differing mortality rates between field and laboratory. We also briefly discuss extensions of this new approach to the case of non-stationary birth rates and unequal sampling probabilities of capture, and to the closely related problems of estimating age-at-capture of individuals, and of estimating the force of mortality (hazard function) in the wild.

We show that the assumption that the force of mortality acting on an individual depends solely on the individual’s age and on whether the individual is in the wild or in the laboratory implies a demographic convolution equation. Consequently, the recovery of the unknown age distribution requires to solve a statistical deconvolution problem (see Madden et al. 1996 for an overview). An implementation by penalized least squares is shown to provide a feasible solution to this inverse problem. Crucial for this specific deconvolution problem are functional constraints such as smoothness, non-negativity and monotonicity which can be conveniently incorporated via suitable penalty terms.

Assumptions made for the application of the proposed approach in biodemographic studies are that the survival of sampled individuals in captivity is determined by the corresponding age-specific mortality in captivity, irrespective of when the capture occurs, and that each individual present in the wild population has the same chance of being sampled, irrespective of age. If additional information is available, some of these assumptions can be relaxed. For inference about survival in the wild, going beyond inferring merely the age distribution in the wild, another needed assumption is stationarity of the population. This assumption also may be relaxed if additional information is available. We demonstrate the proposed deconvolution methodology with cohorts sampled from the fruit flies Bactrocera oleae.

In section 2, we derive the novel demographic convolution equation that underlies the concept of residual demography and discuss various extensions. In section 3, we discuss the deconvolution and its actual implementation. A biodemographic data set is used in section 4 to demonstrate the methodology in action. Further discussion follows in section 5.

2. A demographic convolution equation

In residual demography, subjects from a wild population are randomly caught and placed in a cohort of wild-caught flies of unknown age, the captive cohort. It is assumed that the unknown age A of a captured individual is distributed according to the age distribution of the population, i.e., all individuals in the wild are equally likely to be sampled. The captive cohort is reared under well-defined laboratory conditions until all subjects in this cohort are dead. In addition to the captive cohort, a reference cohort is assembled which consists of newborn subjects of the same type as the wild population. The reference cohort is raised under identical conditions as the captive cohort until age-at-death is recorded for all subjects. If this cohort is sufficiently large, reasonably accurate estimates of the survival schedule of subjects under laboratory conditions can be constructed. For an individual that is captured at unknown age A, then enters the captive cohort and dies after an observed residual life time in captivity C, age-at-death T is

$$T=A+C.$$ (1)

Since A is unknown, so is T and therefore common methods of survival or life table analysis do not apply.

The following notation will be used throughout where the existence of the underlying quantities is assumed: For a non-negative random variable X, let F_X(t) = P (X ≤ t) denote the distribution function, F̄_X(t) = 1 − F_X(t) the survival function, $f_X(t)=\frac{\text{d}}{\text{dt}}{F}_X(t)$ the probability density function, λ_X(t) = f_X(x)/F̄_X(t) the hazard rate, and ${\mathrm{\Lambda }}_X(t)={\int }_0^t{\lambda }_X(s)ds$ the cumulative hazard rate. Here the choice X = W will label these quantities for subjects in the wild, X = C for subjects in the captive cohort, and X = R for subjects in the reference cohort. Furthermore, the density of the distribution of age-at-capture is denoted by f_A.

In the remainder of this section, we derive the convolution equation which is basic for residual demography, list the needed assumptions in section 2.2 and then proceed to discuss several specific features and extensions in sections 2.3 to 2.7.

2.1 The basic equation

The hazard rate of a subject at age t is then

$$\lambda (t)=\{\begin{array}{ll}{\lambda }_W(t)\hfill & \text{if }t\le A\hfill \\ {\lambda }_R(t)\hfill & \text{if }t>A.\hfill \end{array}$$ (2)

Observing F̄_C(t) = E{P (C > t|A)}, we obtain from F̄_C|A(t|A) = e^{−(Λ_Rt+A) − Λ_R(A))} that

$${\overline{F}}_C(t)={\int }_0^{\infty }{e}^{-({\mathrm{\Lambda }}_R(t+a)-{\mathrm{\Lambda }}_R(a))}{f}_A(a)da={\int }_0^{\infty }{\overline{F}}_R(t+a)\frac{f_A(a)}{{\overline{F}}_R(a)}da.$$ (3)

From the reference cohort, one easily obtains an estimate of F̄_R and from the captive cohort an estimate of F̄_C, so that the only unknown function in this convolution equation is f_A.

By differentiating both sides of (3) w.r. to t, one obtains

$$f_C(y)={\int }_0^{\infty }{f}_R(x+y)\frac{f_A(x)}{{\overline{F}}_R(x)}dx.$$ (4)

Upon integrating both sides of (3), R denoting the lifetime random variable in the reference cohort,

$$EC={\int }_0^{\infty }{\overline{F}}_C(t)dt={\int }_0^{\infty }E(R-a|R>a)f_A(a)da,$$ (5)

so that residual life expectancy in the captive cohort is seen to be a mixture of expected residual lifetimes in the reference cohort.

If we make the additional assumption of a stationary and stable wild population, the relationship between f_A and survival function F̄_W of the wild population is given by

$${\overline{F}}_W(t)=\frac{f_A(t)}{f_A(0)}.$$ (6)

Deconvolving equation (4) then leads to estimates of the wild survival schedule F̄_W. We note that by integrating (6) on both sides, we find the relationship EW = 1/f_A(0), i.e., the age density at age 0 is equal to the reciprocal life expectancy in the wild.

The convolution equations (3), (4) can be extended to more general situations. These include the case of sampling with unequal probabilities, for example if older individuals of the wild population are more likely to be sampled than younger individuals, and the case where the population is not stationary and has time-varying birth rates. This is of interest when seasonal variations in birth rates in the wild would be expected. Further details on these extensions are given in the following section 2.3, where we also discuss the situation of a nonstationary survival schedule. The special cases F_R = F_W, predicted age at capture and the estimation of the force of mortality for the wild population are discussed in sections 2.4, 2.5 and 2.6, respectively, while issues of identifiability are the topic of section 2.7.

2.2 Assumptions

A basic assumption underlying all elements of our approach is that the force of mortality depends only on age of an individual and the current environment, and does not depend on past environmental exposure. This assumption implies that an individual in captivity has the same mortality at a given age whether it has spent its prior life in captivity or in the wild.

In the biodemographic context, this assumption is supported by findings of Carey et al. (1998) and Sgo and Partridge (1999), who report the fast adaptation of both Mediterranean fruit flies and Drosophila to new survival schedules corresponding to changed dietary environments, irrespective of past exposures.

A second assumption is that age-bias of captures in the wild is minimal. If age-bias cannot be neglected, a correction factor can be introduced as described in section or, alternatively, the wild age density f_A can be taken to refer to the age structure of wild-captured flies rather than wild flies. This alternative interpretation continues to lead to useful information, for example regarding the presence of old flies in the wild.

A third assumption for the basic convolution equation in section 2.2 is that the population is stable, and birth rates are stationary throughout the observation period. This assumption may be violated in wild populations and it can be discarded under a slighly modified scheme that is discussed in section 2.3.

These three assumptions suffice to derive the convolution equation, which determines the age distribution in the population. In order to solve the equation, we assume that the age density is smooth, monotone decreasing and that the wild survival schedule is such that the solution of the equation is identifiable. None of these assumptions imposes a serious restriction, for identifiability this is discussed in section 2.7.

In order to use the solution of the basic convolution equation to derive the wild survival schedule, we need to make the additional assumption that population survival is stationary. This means that population (cohort) hazard rates depend only on age of an individual but do not depend on calendar time. As survival may depend on available resources or other changing environmental conditions, hazard rates in some cases may vary with calendar time. Extensions of the basic model to cover this case are discussed in section 2.3.

2.3 Extensions of the convolution equation

We first consider the extension to the case with unequal sampling probabilities. In this situation availability of a sampling function β (a) = P(Individual is sampled|Individual is of age a) is useful; this function may be known or may be determined from additional experiments. We then define a selected age density function f̆_A(a) = β(a)f_A(a)/∫β(a)f_A(a) da and replace (3) with a generalized version,

$${\overline{F}}_C(t)={\int }_0^{\infty }{\overline{F}}_R(t+a)\frac{{\check{f}}_A(a)}{{\overline{F}}_R(a)}da.$$

A second extension addresses the non-stationary case where the survival schedule of the wild population is stationary, but the age distribution of a population is changing over time, due to a birth rate that is itself changing over time, so that the wild age distribution is not stationary. A modified sampling plan, tailored to this situation, is to sample individuals from the wild not just at one but at each of various calendar times z, and then to rear several captive cohorts captured at these times z. Using the same arguments as before, the ensemble of the corresponding captive cohorts then gives rise to a family of convolution equations

$${\overline{F}}_C(t,z)={\int }_0^{\infty }{\overline{F}}_R(t+a)\frac{f_A(a,z)}{{\overline{F}}_R(a)}da,$$ (7)

where f_A(a, z) is the probability density of the age distribution for the population at calendar time z, which can be recovered by deconvolution.

With birth rate at calendar time y quantified by a function γ(y), the assumption of stationary survival probabilities (i.e., the probability for an individual to survive from one age to the next does not depend on calendar time) leads to f_A(a, z) = γ(z − a)F̄_W(a)/∫γ(z − t)F̄_W(t) dt. Setting a = 0, one finds ∫γ(z − t)F̄_W(t) dt = γ(z)/f_A(0, z) and therefore

$${\overline{F}}_W(t)=\frac{f_A(t,z)}{f_A(0,z)}\frac{\gamma (z)}{\gamma (z-t)},$$ (8)

Hence the target survival schedule F̄_W can be recovered in the same way by deconvolution as in the stationary case, given the birth rates γ(z), which need to be known or determined from field studies. In the stationary case, γ(z) = γ(z − a) and the original sampling schedule with a captive cohort sampled at just one calendar time z will suffice, as (8) is then equivalent to (6). For stationary survival in the wild it is advantageous to pool estimates obtained for the r.h.s of equation (8) for various values of z, as the l.h.s. does not depend on z. If estimates f̂_A(a, z) of age densities f_A(a, z) are obtained from samples collected at K ≥ 1 calendar times z₁, …, z_K, such a pooled estimate is given by

$${\widehat{\overline{F}}}_W(t)=\frac{1}{K}\sum _{j=1}^K\frac{{\widehat{f}}_A(t,z_j)}{{\widehat{f}}_A(0,z_j)}\frac{\gamma (z_j)}{\gamma (z_j-t)}.$$ (9)

A third extension of the basic convolution equation to the case of non-stationary survival in the wild is a consequence of the above considerations. The nonstationary case arises for example if hazard rates are season-dependent. In this case “cross-sectional” survival functions ${\overline{F}}_W(t,z)=P(W>t|z)=exp(-{\int }_0^t{\lambda }_W(s|z-t+s)ds))$ at calendar time z are still within reach. Adapting (8), we obtain

$${\overline{F}}_W(t,z)=\frac{f_A(t,z)}{f_A(0,z)}\frac{\gamma (z)}{\gamma (z-t)}.$$ (10)

This suggests to sample subjects from the wild on a sufficiently dense grid of calendar times z over a domain D and then to construct a survival surface F̄_W(t, z) for all relevant t and z ∈ D. In some cases an additional local smoothing step across the values of z in local neighborhoods may improve efficiency of such surface estimates, analogously to (9). Pooling the deconvolved wild survival estimates according to (9) under the nonstationary model (10) has the effect to reduce variability, thus counteracting the smaller cohort sizes which are a consequence of the more frequent sampling. Such a pooling method will target F̄_Wave(t) = ∫_DF̄_W (t, z) dz, an “overall average” survival function which may serve as a useful summary measure. A similar pooling effect is achieved if various captive samples obtained at various calendar dates are pooled together into one larger sample before deconvolution.

2.4 The case of identical survival schedules

In some situations it is of interest to consider the case where survival in the wild (characterized by the survival function F_W) and survival in the laboratory reference cohort (characterized by the survival function F_R) are identical, i.e., where one has F_R = F_W. In this case,

$${\overline{F}}_W(a)=f_A(a)/f_A(0)$$

and this implies that (4) simplifies to

$$f_C(y)=f_A(0){\int }_0^{\infty }{f}_W(y+x)dx=f_A(0){\overline{F}}_W(y),$$

so that

$${\overline{F}}_W(y)=f_C(y)/f_C(0),$$

obtained by plugging in and observing that F̄_W (0) = 1:

This corresponds to an identity from renewal theory (Feller 1950) that has been revisited in a demographic context in Müller et al. (2004). An example from anthropological research was given there. Since an explicit solution can be found in this special case, a more complex deconvolution method is not needed.

2.5 Predicted age at capture

It may be sometimes of interest to predict age-at-capture A for an individual for which a remaining lifetime after capture C = c has been observed, but nothing else is known. In this case, the best prediction is the conditional expectation E(A|C = c).

Straightforward calculations lead to

$$E(A|C=c)=\int a{f}_{A|C}(a,c)da=\frac{1}{f_C(c)}\int a\frac{f_R(a+c)f_A(a)}{{\overline{F}}_R(a)}da.$$ (11)

Estimates for age-at-capture can then be obtained by substituting smooth estimates such as kernel density estimators (e.g. Müller 1997) for f_C; f_R and F̄_R, and smooth deconvolution estimates (18) for f_A, followed by numerical integration.

2.6 Force of mortality in the wild

Under stationarity assumptions, (6) implies for the density f_W of the lifetime distribution in the wild that

$$f_W(t)=-\frac{f_A^{\prime }(t)}{f_A(0)}.$$

The force of mortality λ_W is thus found to be

$${\lambda }_W(t)=\frac{f_W(t)}{{\overline{F}}_W(t)}=-\frac{f_A^{\prime }(t)}{f_A(t)}.$$ (12)

Given estimates f̂_A(t) from (18), one possibility to obtain the needed derivative $f_A^{\prime }(t)$ by forming difference quotients,

$${\widehat{f}}_A^{\prime }(t)=\frac{1}{\mathrm{\Delta }}[{\widehat{f}}_A(t+\mathrm{\Delta }/2)-{\widehat{f}}_A(t-\mathrm{\Delta }/2)]$$

and then plugging ${\widehat{f}}_A^{\prime }(t)$, f̂_A(t) into (12).

2.7 Identifiability issues

We provide here a discussion of the problem under which assumptions a (unique) solution of the convolution equation (4) exists. The mathematical arguments provided below imply that deconvolution will not work if wild survival is a mixture of exponential distributions. As these distributions are associated with constant (in case the mixture has just one component) or decreasing hazard rates as age increases, they are not likely to be encountered as survival distributions in wild aging applications.

More specifically, define the deconvolution problem as finding the solution of

$${argmin}_{f_A\in \mathcal{F}}\parallel f_C(y)-{\int }_0^{\infty }K(x,y)g(x)dx\parallel ,$$

where we redefine the target as g(x) = f_A(x)= F̄_R(x) and view K(x, y) = f_R(x + y) as the (symmetric) kernel of a linear operator Ω_K in Hilbert space, so that f_C = Ω_K(g). Here $\mathcal{F}$ is the space of all smooth density functions with non-negative and bounded support, and || · || is a suitable norm, usually chosen as the L² norm for the case of square integrable functions. In general, Ω_K is a compact operator and as such is not globally invertible.

If the kernel is degenerate, i.e., there exist finitely many linearly independent functions α_j, j = 1, …, p, such that

$$K(x,y)=\sum _{j=1}^p{\alpha }_j(x){\alpha }_j(y),$$

then the convolution equation at best determines the integrals ∫α_j(x)g(x) dx, j = 1, … p; and therefore in general the function g is not determined. An extreme case is p = 1 where only R ∫α₁(x)g(x) dx is determined by the convolution equation for some function α₁, which clearly shows that deconvolution is not identifiable in this case.

The property K(x, y) = α₁(x)α₁(y) is characteristic for the exponential density for which f_R(x + y) = λ exp(− λx) exp(− λy) for a parameter λ > 0. This density therefore provides an example where deconvolution is not feasible. The case where f_R is a mixture of exponential distributions also leads to a degenerate kernel with p > 1. If on the other hand the eigenfunctions of the operator Ω_K form a basis of the underlying function space, a solution can be found in the image space ℐ = Ω_Kℱ under certain assumptions (e.g., He et al. 2002).

3. Deconvolution for recovering age distribution and survival in the wild

The basic demography concept requires is to solve the convolution equation (4). Deconvolution generally is a difficult task and falls into the class of inverse problems (Nowak 1998; Carroll and Hall 2004). The proposed approach to solve equation (4) for f_A is based on the idea to approximate pertinent density functions, including f_A, by step functions. The step functions approximating densities f_R and f_C can then be estimated from the data. Plugging these estimates into convolution equation (4) leads to a linear system for the coefficients of the step function approximations whose solution provides an estimate for f_A. This linear system is generally ill-conditioned and must be regularized. For the necessary regularization we propose to use penalized least squares, as detailed in the following.

3.1 Deconvolution by regularization

Regularization is partially achieved by the step function approximation which can be coarsened to any desired degree in order to reduce the dimension of the system, and to a larger extent by introducing a penalized least squares algorithm. The solution must satisfy certain properties such as being a density and being smooth. When estimating a survival schedule via (6) it also must be monotone decreasing. Directly solving the linear system under hard constraints was found to be inferior to the proposed deconvolution via penalized least squares, with penalties ρ₁ for violating smoothness, ρ₂ for violating non-negativity, and ρ₃ for violating that the integral under the curve is 1 (as required for a density). When estimating the survival function in the wild, we also introduce a penalty ρ₄ for violating monotonicity. Details about the implementation are given in subsection 3.2, with the target criterion (18).

From Eq. (17) below, the linearized and discretized version of the convolution equation (4) can be written as

$$\sum _{j\ge 0}[\frac{f_{Rk+j}}{{\overline{F}}_{Rj}}]f_{Aj}=f_{Ck},$$ (13)

where f_Xj denotes the fraction of a cohort that is at cohort age j, and F ¯_Xj denotes the probability to survive to age j, for subjects in the reference cohort (X = R), the captive cohort (X = C) or in the wild cohort (X = A), where A stands for chronological individual age. We thus see that the feasibility of deconvolution hinges on the properties of the design matrix B defined by (4) with elements $b_{jk}=\frac{f_{Rk+j}}{{\overline{F}}_{Rj}}$.

A simple discrete example to illustrate Eq. (13) is as follows. Assume there are only three age groups, such that aging in the wild corresponds to f_A0 = 1/3, f_A1 = 1/3, f_A2 = 1/3, while aging in captivity is accelerated and one observes the values f_R1 =1/2, f_R2 = 1/3, f_R1 = 1/6, with the associated values F̄_R0 = 1, F̄_R1 = 1/2, F̄_R2 = 1/6, F̄_R3 = 0 for the reference cohort. Plugging into the above equation, a straightforward calculation shows that one will observe for the captive cohort f_C1 = 13/18, f_C2 = 4/18, f_C3 = 1/18. In this simple toy example the deconvolution problem can be directly solved as the 3 × 3 matrix corresponding to (13) has determinant -1 and is thus directly invertible. In general, when the number of bins is large, this matrix is ill-conditioned. Direct inversion is then not possible and the estimate one would obtain by non-penalized least squares would be neither smooth nor a density; regularization is crucial.

3.2 Regularized deconvolution via penalized least squares

Assume the captive cohort consists of n_C subjects with observed residual lifetimes C₁, …, C_nC and the reference cohort consists of n_R subjects with observed residual lifetimes R₁, …, R_nR. To approximate the densities appearing in (4) by step functions, we define a suitable equidistant grid of M points x₁, …, x_M, such that x_j = (j − 1)Δ for a constant Δ > 0. By choosing a small value for Δ, this approximation can be made as precise as desired. Choosing an integer M such that x_M = max₁ ≤ i ≤ n_R{R_i} − Δ, densities f_A and f_C will be approximated on intervals [0, x_M] and density f_R on interval [0, x_2M]. Defining right-open intervals I_j = [x_j, x_j+1), j ≥ 1, the approximated densities are

$${\stackrel{\sim }{f}}_A(x)=\sum _{j=1}^{M-1}{f}_{Aj}{1}_{I_j}(x),{\stackrel{\sim }{f}}_C(x)=\sum _{j=1}^{M-1}{f}_{Cj}{1}_{I_j}(x),{\stackrel{\sim }{f}}_R(x)=\sum _{j=1}^{2M-1}{f}_{Rj}{1}_{I_j}(x),$$ (14)

where 1_S(x) = 1 if x ∈ S, and 1_S(x) = 0 otherwise, for any set S.

In a first step, we estimate densities f_R of the survival time distribution of the reference cohort and f_C of the residual lifetime of the captive cohort by estimating the respective coefficients in (14),

$$\begin{array}{l}{\widehat{f}}_{Rj}=\frac{1}{\mathrm{\Delta }{\sum }_{i=1}^{n_R}{1}_{\{R_i\in [0,x_{2M}]\}}}\sum _{i=1}^{n_R}{1}_{\{R_i\in I_j\}},\\ {\widehat{f}}_{Cj}=\frac{1}{\mathrm{\Delta }{\sum }_{i=1}^{n_C}{1}_{\{C_i\in [0,x_M]\}}}\sum _{i=1}^{n_C}{1}_{\{C_i\in I_j\}},\end{array}$$

which leads to the density estimates

$${\widehat{f}}_R(x)=\sum _{j=1}^{2M-1}{\widehat{f}}_{Rj}{1}_{I_j}(x),{\widehat{f}}_C(x)=\sum _{j=1}^{M-1}{\widehat{f}}_{Cj}{1}_{I_j}(x).$$ (15)

Defining ${\widehat{\overline{F}}}_R(x)=1-{\int }_0^x{\widehat{f}}_R(z)dz$, substituting these estimates then transforms convolution equation (4) into

$$\begin{array}{l}{\widehat{f}}_C(x_k)={\int }_0^{\infty }{\widehat{f}}_R(x_k+x)\frac{{\stackrel{\sim }{f}}_A(x)}{{\widehat{\overline{F}}}_R(x)}dx\\ =\sum _{j=1}^{M-1}{\stackrel{\sim }{f}}_A(x_j){\int }_{I_j}{\widehat{f}}_R(x_k+x)/{\widehat{\overline{F}}}_R(x)dx\\ =\sum _{j=1}^{M-1}{\stackrel{\sim }{f}}_A(x_j){\widehat{f}}_R(x_k+x_j){\int }_{I_j}\frac{1}{{\widehat{\overline{F}}}_R(x)}dx,\end{array}$$ (16)

for 1 ≤ k ≤M − 1. This corresponds to a linear system

$${\widehat{f}}_C=B{\stackrel{\sim }{f}}_A,$$ (17)

for the unknown coefficients f_Aj of the target function f̃_A, where the (M − 1) × (M − 1) matrix B has elements

$$b_{kj}={\widehat{f}}_R(x_k+x_j){\int }_{I_j}\frac{1}{{\widehat{\overline{F}}}_R(x)}dx,1\le k,j\le M-1,$$

and

$${\widehat{f}}_C=({\widehat{f}}_C(x_1),\cdots ,{\widehat{f}}_C(x_{M-1}){)}^{\prime },{\stackrel{\sim }{f}}_A=({\stackrel{\sim }{f}}_A(x_1),\cdots ,{\stackrel{\sim }{f}}_A(x_{M-1}){)}^{\prime }.$$

We aim at solving this system for the coefficients f_Aj. Constructing the estimate f̂_A as minimizer of a penalized least squares criterion leads to a solution that is a density and also a smooth function.

Denote by || · || the Euclidean norm, and by H a (M − 1) × (M − 1) matrix with diagonal elements -2 and both side diagonal elements 1, suitably modified near the end points of the diagonal, so that for a (M − 1)-vector υ we have (Hυ)_k = υ_k+1 − 2υ_k + υ_k−1, k = 2, …, M − 2 and ${\widehat{f}}_A^{\prime }{H}^{\prime }H{\widehat{f}}_A$ corresponds to the vector of squared second order difference quotients of f̂_A, apart from a normalization factor. Furthermore, denote by ρ₁, ρ₂, ρ₃, ρ₄ ≥ 0 four nonegative penalty parameters, to be chosen carefully. The target function is a density, therefore it is nonnegative and integrates to 1. It is also assumed to be a smooth and monotone falling function. Accordingly, we include penalty terms that penalize against violations of these four properties.

The penalized least squares criterion is then

$$\begin{array}{l}\mathcal{P}({\stackrel{\sim }{f}}_A)=\parallel B{\stackrel{\sim }{f}}_A-{\widehat{f}}_C{\parallel }^2+{\rho }_1\sum _{j=1}^{M-1}[{\stackrel{\sim }{f}}_A{(x_j)}^2]1_{\{{\stackrel{\sim }{f}}_A(x_j)<0\}}+{\rho }_2{[\mathrm{\Delta }\sum _{j=1}^{M-1}{\stackrel{\sim }{f}}_A(x_j)-1]}^2\\ +{\rho }_3{\stackrel{\sim }{f}}_A^{\prime }{H}^{\prime }H{\stackrel{\sim }{f}}_A+{\rho }_4\sum _{j=1}^{M-1}{({\stackrel{\sim }{f}}_A(x_{j+1})-{\stackrel{\sim }{f}}_A(x_j))}^2{1}_{\{{\stackrel{\sim }{f}}_A(x_{j+1})-{\stackrel{\sim }{f}}_A(x_j)>0\}}.\end{array}$$ (18)

Here the first penalty term penalizes against negative estimates and the second against the estimated function not integrating to 1. The third penalty term promotes smoothness of estimates, by penalizing against the sum of squared second order difference quotients, while the fourth penalty term penalizes against the solution not being monotone falling. Then the minimizer of $\mathcal{P}({\stackrel{\sim }{f}}_A)$ with regard to f̃_A is the desired estimate f̂_A of the density of the age distribution. We note that the least squares approach can be easily extended to weighted least squares by adding weights for each histogram bin of f̃_C, using for example the Poisson approximation to the histogram bin counts(which are binomially distributed), or alternatively to a penalized maximum likelihood. Once f̂_A has been obtained, the estimate of the wild survival function ${\widehat{\overline{F}}}_W$ is an immediate consequence according to (6).

The choice of the grid x_k is often tied to the implicit scaling of the lifetables of captive and reference cohorts. In studies of flies or nematodes, it is customary to assess age-at-death in days, so that the natural choice is Δ = 1 day (see Müller et al. 1997 and Wang et al. 1998, regarding issues of discretization in biodemographic analysis). In human studies, the natural unit might rather be a year. The penalty parameters can be chosen by simulations that mimic the observed data. Optimization routine fminimax (Matlab) was used in the implementation of the penalized least squares solutions f̂_A.

4. Residual demography for fruit flies

To assess wild aging and survival schedule of the tephritid fruit fly (Bactrocera oleae) in the wild, flies were sampled from a wild population in olive groves near Davis/California. A captive cohort consisting of 457 olive flies and a reference cohort of 82 flies were assembled and reared under controlled conditions until all flies were dead and the proposed residual demographic method was applied to these data. The fitted survival schedule was then used as basis for a simulation study to determine the variability of the deconvolution algorithm.

4.1 Wild survival for Bactrocera oleae

Flies were collected from infested olives in Davis, California from June through August, 2004. Newly-eclosed individuals (82 flies) were collected for the reference cohort and placed individually in 1-oz clear plastic containers (condiment cups), provided with adult food (mixture of 3 parts sugar and 1 part yeast hydrolysate) and water, maintained at 25C (±3), 65% (±10) relative humidity; 12:12 light:dark cycle, and monitored each day to record age of death. Captive cohort information was collected on 457 adult olive flies captured in McPhail traps during this same period and in the same orchard. These flies were pooled into one large captive cohort (see the discussion in section 2.3). Individuals were removed from the traps in the laboratory using an aspirator, placed in individual 1-oz cages with food and water, maintained under the same conditions as in the baseline life tables studies, and monitored daily for survival.

The proposed deconvolution algorithm was applied and yielded reasonable results for the estimate of the density of the age distribution in the wild, as shown in Fig. 1 (with penalty parameters defined in (18) chosen as (ρ₁, ρ₂, ρ₃, ρ₄) = (30, 20, 400, 0.5)), along with 95% bootstrap confidence bands. The construction of these bands is described in the following subsection.

Figure 1.

Upper panel: Monotone deconvolution estimate of the age density f_A in the wild for Bactrocera oleae, with 95% bootstrap confidence intervals. Middle panel: Estimated density f_A (thick solid) and 100 simulated deconvolution density estimates (grey lines), created as in Fig. 1; the dotted curve denotes the mean of the simulated age densities. Lower panel: Corresponding estimate of the survival function in the wild, with 95% bootstrap confidence intervals.

Simulated age densities, assuming that the age density shown in Fig. 1 corresponds to the true underlying distribution, are shown in Fig. 2. Further details on these simulations can be found in the following section. Fig. 3 displays the estimate of the wild survival function, obtained via (6), and the corresponding 95% bootstrap confidence bands. Some characteristics of this estimate of the wild survival function are first quartile Q1, at 15d (d=days), second quartile Q2 (median), at 31d, third quartile Q3, at 53d, and mean age at death, at 35d.

Figure 2.

Estimated density f_A (thick solid) and 100 simulated deconvolution density estimates (grey lines), each one obtained by sampling age-at-capture from the original wild age density estimate shown in Fig. 1 and then creating an artificial captive cohort which serves as input for the deconvolution algorithm. The cross-sectional average of the deconvolution estimates (dotted line) is seen to be close to the target function.

Figure 3.

Estimate of the survival function in the wild Bactrocera oleae, with 95% bootstrap confidence intervals.

4.2 Confidence bands and simulations

In Fig. 1, pointwise 95% bootstrap confidence bands are shown along with the estimate of the density f_A. These confidence bands were obtained by creating bootstrap samples of captive cohorts of the same size as the actual observed captive cohort, by sampling 457 ages-at-death with replacement from the captive cohort data. Then for each bootstrap sample the deconvolution procedure was carried out, using the observed lifetime data for the reference cohort. In this way, 1000 bootstrap samples were created. The empirical 2.5% and 97.5% quantiles of the resulting sample of bootstrap density estimates were calculated at each age t and are shown as lower and upper confidence bands in the figure. Bootstrap confidence bands can be analogously constructed for the estimated wild survival function as shown in Fig. 3.

To simulate the behavior of the proposed deconvolution method, we proceeded as follows: For each simulation, we fixed the reference cohort data at the actually observed values, taking the estimate of the reference age-at-death distribution as the true distribution. We assume that the true underlying age density is the estimate f_A, as shown in Fig. 1. For each simulation run, we construct a cohort of 457 captive flies by generating their lifetimes. This step is described next.

We first obtain simulated ages-at-capture A by sampling from the assumed density f_A, using the graph of the estimated density in Fig. 1 and rejection sampling. Then we generate a simulated residual lifetime after capture C for each age-at-capture as follows: Independently from simulated age-at-capture A, we generate a random survival time R, distributed according to f_R, again by rejection sampling. If R < A, another pair (A, R) is generated. If R ≥ A, we record C = R − A. Then C is distributed as the residual lifetime after capture, with density f_C. For each simulation run, we generate in this way a captive cohort consisting of 457 residual lifetimes. These residual lifetimes are then entered into the deconvolution algorithm, along with the estimated reference density and distribution function. The result is a simulated estimate of the wild age density f_A.

These simulated captive lifetimes are then entered into the deconvolution algorithm. The resulting wild age estimate is plotted as one single estimated wild age density (grey line) in Fig. 2. The described procedure is carried out 100 times, yielding 100 wild estimated age densities, all of which are shown in Fig. 2. As the deviation between the simulated densities and the target density is relatively small, given the complexity of the deconvolution task, we conclude from this simulation that the method performs relatively well in a real-life setting.

5. Discussion and concluding remarks

The convolution equation (4) forms the basis to determine wild age and survival schedules from data obtained from both a captive and a reference cohort. A methodological difficulty is in devising a workable deconvolution scheme. We found that penalized least squares provides a viable method for biodemographic deconvolution. If one aims at recovering the wild survival schedule, the age distribution in the wild is obtained first and the assumption of a stationary population is needed to determine the survival schedule from the age distribution. The stationarity assumption can be relaxed by invoking a slightly more complicated sampling design, combined with knowledge about relative number of births, to be obtained from additional field studies.

We note that this methodology is of interest beyond demography: It provides a novel instance of a deconvolution problem under constraints, motivating further statistical research. In survival analysis, information about the onset of a condition may be missing for a group of subjects who enter a study (this would correspond to the unknown age of a subject in the biodemographic framework), while the distribution of the onset times is of great interest; an example is infection with HIV (Bacchetti and Jewell 1991). If the cohort sampled from the “wild” has an unknown onset time (captive cohort) and a second cohort of subjects is observed for which onset occurs during the time subjects are being studied (reference sample), survival information from the subjects in both groups can be combined to obtain inference about the timing of the onset. This can be done by setting up an analogue of equation (3) and then solving the deconvolution problem. Similarly, in reliability applications the convolution model will be useful in situations where one wishes to infer the longevity of manufactured items under realistic everyday use conditions. The reference cohort in this case corresponds to a sample of items tested under laboratory conditions, while the captive cohort consists of items that are randomly sampled from the population of items in use and subsequently tested under laboratory conditions.

Studies concerned with aging in the wild have traditionally focused on either survival estimates using mark-recapture techniques (Caughley 1977; Austad 1993; Krebs 1999) or relatively rough life table differences between cohorts that were subjected to different selection pressures in the field (Reznick et al. 2004; Tatar et al. 1997). The applicability of these tools for demographic analysis is limited, as age-distribution information is not provided, in contrast to residual demography.

On one hand, residual demography complements mark-recapture methods which require the capture, marking, and re-capture of large numbers of individuals of known age (Buckland, 1982, Lebreton et al. 1992, Pradel 1996, Williams 2002, Moorhouse and MacDonald 2005), by relaxing the requirement of known age at capture. On the other hand, it provides an alternative methodology that requires capture only once and is amenable to parameter-free models. This is not the case for mark-recapture, where the sparseness of available information usually necessitates parametric model specifications (with associated maximum likelihood or Bayesian statistical methodology). However, one persistent finding in biodemographic studies has been the enormous plasticity of hazard rates found under various experimental conditions (Carey et al. 2002); this puts a premium on flexible nonparametric approaches. Whereas capture-mark-recapture methods can be used for large insects such as butterflies (Boggs et al. 2004), these methods are of limited usefulness for very invertebrates because of both low recapture rates and the likelihood of injury. Thus residual demography might be the only available methodology to study age structure and longevity in wild populations of certain organisms.

Further analytical and algorithmic developments will be needed to more directly address the various non-stationarities that likely exist in real populations, as discussed in sections 2.2 and 2.3. It will also be of interest to combine the proposed method in a suitable way with mark-recapture methodology, enhancing both residual demography and mark-recapture methods in the process. Further discussion of such extensions can be found in Carey et al. (2007).

While the deconvolution step requires careful implementation, as it involves an ill-posed inverse problem, we have demonstrated the feasibility to obtain and use information contained in captured individuals of unknown age. Residual demography provides a concept to assess age distribution in the wild, which is of interest in its own right, and under additional assumptions allows to draw inference about wild survival. Setting up the prerequisite captive and reference cohorts is particularly feasible for species such as flies or nematodes and other easy-to-raise and easy-to-sample short-lived species. The combination of both laboratory and field studies, extracting information from both the captive and the reference cohort, is poised to shed further light on aging and survival in the wild.