DESCARTES’ RULE OF SIGNS AND THE IDENTIFIABILITY OF POPULATION DEMOGRAPHIC MODELS FROM GENOMIC VARIATION DATA

Abstract

The sample frequency spectrum (SFS) is a widely-used summary statistic of genomic variation in a sample of homologous DNA sequences. It provides a highly efficient dimensional reduction of large-scale population genomic data and its mathematical dependence on the underlying population demography is well understood, thus enabling the development of efficient inference algorithms. However, it has been recently shown that very different population demographies can actually generate the same SFS for arbitrarily large sample sizes. Although in principle this nonidentifiability issue poses a thorny challenge to statistical inference, the population size functions involved in the counterexamples are arguably not so biologically realistic. Here, we revisit this problem and examine the identifiability of demographic models under the restriction that the population sizes are piecewise-defined where each piece belongs to some family of biologically-motivated functions. Under this assumption, we prove that the expected SFS of a sample uniquely determines the underlying demographic model, provided that the sample is sufficiently large. We obtain a general bound on the sample size sufficient for identifiability; the bound depends on the number of pieces in the demographic model and also on the type of population size function in each piece. In the cases of piecewise-constant, piecewise-exponential and piecewise-generalized-exponential models, which are often assumed in population genomic inferences, we provide explicit formulas for the bounds as simple functions of the number of pieces. Lastly, we obtain analogous results for the “folded” SFS, which is often used when there is ambiguity as to which allelic type is ancestral. Our results are proved using a generalization of Descartes’ rule of signs for polynomials to the Laplace transform of piecewise continuous functions.

1. Introduction

Given a sample of homologous genomic sequences from a large population, an important inference problem with a wide variety of important applications is to determine the underlying demography of the population. The population demography can be used to calibrate null models of neutral genome evolution in order to find regions under selection [2, 25, 45]; to stratify samples in genome-wide association studies [3, 28, 33, 37]; to date historical population splits, migrations, admixture and introgression events [10, 18, 24, 26, 40, 43]; and so on. Recently, several large-sample genome- and exome-sequencing datasets have become available [1, 4, 6, 31, 44], shedding new light on patterns of genetic variation that were not previously observable in smaller datasets. Such large-sample studies offer an exciting opportunity to infer demography in unprecedented detail.

One widely-used measure of genetic variation in a set of homologous genome sequences is the sample frequency spectrum (SFS). For a sample of size n, the SFS counts the proportion of dimorphic (i.e., with exactly two distinct observed alleles) sites as a function of the frequency ( $\frac{b}{n}$, where 1 ≤ b ≤ n − 1) of the mutant allele in the sample. The SFS is useful for several reasons. First, the SFS is a succinct summary of a large sample of genomic sequences, where the information in n sequences of arbitrary length can be summarized by just n−1 numbers. This makes the SFS both mathematically and algorithmically tractable. In particular, since the SFS ignores linkage information between sites, one can avoid challenging mathematical and computational issues associated with rigorously modeling genetic recombination. Furthermore, the statistical properties of the SFS and their dependence on the population demographic history are well understood under the coalescent and the diffusion models of neutral evolution [7, 11, 12, 19, 35, 46]. This dependence of the SFS on demography, along with the assumption of free recombination between sites, has been exploited in several efficient methods for inferring historical population demography [5, 13, 27, 29]. Second, the SFS can effectively capture the impact of recent demography on genetic variation. Recent large-sample studies [4, 6, 31, 44] have consistently shown that there is an excess of rare polymorphisms compared to the predictions of previously inferred demographic models, which might be explained by recent rapid population expansion [16]. Because the leading entries of the SFS count the rare variants in the sample, one might be able to use this information to infer demographic events in the recent past at a much finer resolution than possible using smaller samples. Third, the SFS also provides a simple way of visualizing the goodness of fit of a demographic model to data, since one can easily compare the SFS observed in the data with the SFS predicted by the fitted demographic model.

While the SFS has algorithmic advantages for demographic inference, it is believed to suffer from a statistical shortcoming. Specifically, Myers, Fefferman and Patterson [30] recently showed that even with perfect knowledge of the population frequency spectrum [i.e., the proportion of polymorphic sites with population-wide allele frequency in (x, x + dx) for all x ∈ (0, 1)], the historical population size function η(t) as a function of time is not identifiable. Using Müntz–Szász theory, they showed that for any population size function η(t), one can construct arbitrarily many smooth functions F(t) such that both η(t) and η(t) + αF(t) generate the same population frequency spectrum for suitably chosen values of α. They also constructed explicit examples of such functions η(t) and F(t). While this nonidentifiability could pose serious challenges to demographic inference from frequency spectrum data, the population size functions involved in their example are arguably unrealistic for biological populations. In particular, their explicit example involves a population size function which oscillates at an increasingly higher frequency as the time parameter approaches the present. Real biological population sizes can be expected to vary over time in a mathematically more well-behaved fashion. In particular, populations can be expected to evolve in discrete units of time, which, when approximated by a continuous-time model, restricts the frequency of oscillations in the population size function to be less than the number of generations of reproduction per unit time. Furthermore, since a population size model being inferred must have a finite representation for obvious algorithmic reasons, most previous demographic inference analyses have focused on inferring population size models that are piecewise-defined over a restricted class of functions, such as piecewise-constant and piecewise-exponential models [10, 17, 24, 26, 31, 41, 44]. Motivated by the large number of rare variants observed in several large-sample sequencing studies, recent works [38, 39] have also focused on more general population growth models which allow for the population to grow at a faster than exponential rate. Each piece in such piecewise models has two parameters that control the rate and acceleration of population growth. Since these models contain the family of piecewise-constant and piecewise-exponential population size functions, we refer to them as piecewise-generalized-exponential models in the remainder of this paper.

In this paper, we revisit the question of demographic model identifiability under the assumption that the population size is a piecewise-defined function of time where each piece comes from a family of biologically-motivated functions, such as the family of constant or exponential functions. We also re-examine the assumption that one has access to the population-wide patterns of polymorphism. In real applications, we do not expect to know the allele frequency spectrum for an entire population but rather only the SFS for a randomly drawn finite sample of individuals. Here, we investigate whether one can learn piecewise-defined population size functions given perfect knowledge of the expected SFS for a sufficiently large sample of size n. Unlike in the case of arbitrary continuous population size functions considered by Myers, Fefferman and Patterson, the answer to this question is affirmative. More precisely, we obtain bounds on the sample size n that are sufficient to distinguish population size functions among piecewise demographic models with K pieces, where each piece comes from some family of functions (see Theorems 6 and 11). Our bound on the sample size can be expressed as an affine function of the number K of pieces, where the slope of the function is a measure of the complexity of the family to which each piece belongs. In the cases of piecewise-constant, piecewise-exponential and piecewise-generalized-exponential models, which are often assumed in population genetic analyses, the slope of this affine function can be calculated explicitly, as shown in Corollaries 7–9. We also obtain analogous results for the “folded” SFS (see Theorem 12), a variant of the SFS which circumvents the ambiguity in the identity of the ancestral allele type by grouping the polymorphic sites in a sample according to the sample minor allele frequency.

There are two main technical elements underlying our proofs of the identifiability results mentioned above. The first step is to show that the expected SFS of a sample of size n is in bijection with the Laplace transform of a time-rescaled version of the population size function evaluated at a particular sequence of n − 1 points. This reduces the problem of identifiability from the SFS to that of identifiability from the values of the Laplace transform at a fixed set of points. The second step relies on a generalization of Descartes’ rule of signs for polynomials to the Laplace transform of general piecewise-continuous functions. This technique yields an upper bound on the number of roots of the Laplace transform of a function by the number of sign changes of the function. We think that this proof technique based on sign changes might be of independent interest for proving statistical identifiability results in other settings. We also provide an alternate proof of identifiability for piecewise-constant population models, where the aforementioned second step is replaced by a linear algebraic argument that has a constructive flavor. We include this alternate proof in the hope that it could be used to develop an algebraic inference algorithm for piecewise-constant models.

The remainder of this paper is organized as follows. In Section 2, we introduce the model and notation, and describe our main results. We also discuss the counterexample of Myers, Fefferman and Patterson in light of our findings. The proofs of our results are provided in Section 3, and we conclude with a discussion in Section 4.

2. Main results

Here, we summarize our identifiability results. All proofs are deferred to Section 3.

2.1. Model and notation

We consider a population evolving according to Kingman’s coalescent [21–23] with the infinite-sites model of mutation [20] and selective neutrality. Under this model, the genome is assumed to be infinite and every mutation occurs at a different site in the genome that has never experienced a mutation before. This model is applicable in the regime where the mutation rate is very low, and hence the probability of multiple mutations at a given site is vanishingly small. Any polymorphic site in a sample of sequences is dimorphic under this model. The population size is assumed to change deterministically with time and is described by a function η : ℝ_≥0 → ℝ₊, such that the instantaneous coalescence rate between any pair of lineages at time t is 1/η(t).

Let $T_{n,k}^{(\eta )}$ denote the time (in coalescent units) while there are k ancestral lineages for a sample of size n obtained at time 0. Defining R_η(t) as

$$R_{\eta }(t):={\int }_0^t\frac{1}{\eta (x)}dx,$$

the expected time $\mathbb{E}[T_{m,m}^{(\eta )}]$ to the first coalescence event for a sample of size m is given by

$$\mathbb{E}[T_{m,m}^{(\eta )}]={\int }_0^{\infty }t\frac{\left(\begin{array}{c}m\\ 2\end{array}\right)}{\eta (t)}exp\left[-\left(\begin{array}{c}m\\ 2\end{array}\right)R_{\eta }(t)\right]dt.$$ (1)

Following the notation of Myers, Fefferman and Patterson, define a time-rescaled version η̃ of the population size function η as

$$\stackrel{\sim }{\eta }(\tau )=\eta (R_{\eta }^{-1}(\tau )),$$ (2)

where τ ∈ ℝ_≥0. The function η̃(τ) reparameterizes the population size as a function of the cumulative rate of coalescence τ = R_η(t). For a given population size function η̃ parameterized by the total coalescence rate τ, there corresponds a unique population size function η parameterized by time t. Specifically, $\eta (t)=\stackrel{\sim }{\eta }(S_{\stackrel{\sim }{\eta }}^{-1}(t))$, for all t ∈ ℝ_≥0, where S_η̃(t) is an invertible function given by

$$S_{\stackrel{\sim }{\eta }}(t)={\int }_0^t\stackrel{\sim }{\eta }(x)dx.$$

Applying integration by parts to (1) and using the condition that $\mathbb{E}[T_{m,m}^{(\eta )}]<\infty$, we have

$$\mathbb{E}[T_{m,m}^{(\eta )}]={\int }_0^{\infty }exp\left[-\left(\begin{array}{c}m\\ 2\end{array}\right)R_{\eta }(t)\right]dt.$$ (3)

Furthermore, since R_η is monotonically increasing and continuous from ℝ_≥0 to ℝ_≥0, it is a bijection over ℝ_≥0. For notational convenience, for any interval I ⊆ ℝ_≥0, we define R_η(I) to be the interval

$$R_{\eta }(I)=\{R_{\eta }(x)\mid x\in I\}.$$

By making the substitution τ = R_η(t) in (3) and using (2), we have the following expression for $\mathbb{E}[T_{m,m}^{(\eta )}]$:

$$\mathbb{E}[T_{m,m}^{(\eta )}]={\int }_0^{\infty }\stackrel{\sim }{\eta }(\tau )exp\left[-\left(\begin{array}{c}m\\ 2\end{array}\right)\tau \right]d\tau .$$ (4)

Equation (4) states that the time to the first coalescence event for a sample of size m is given by the Laplace transform of the time-rescaled population size function η̃ evaluated at the point $\left(\begin{array}{c}m\\ 2\end{array}\right)$. For a sample of size n, let ξ_n,b denote the probability that a dimorphic site has b mutant alleles and n − b ancestral alleles. We refer to (ξ_n,1, … , ξ_n,n−1) as the expected sample frequency spectrum (SFS).

2.2. Determining the expected times to the first coalescence from the SFS

The following lemma shows that the expected SFS for a sample of size n tightly constrains the expected time to the first coalescence event for all sample sizes 2, … , n:

Lemma 1

Under an arbitrary variable population size model {η(t), t ≥ 0}, suppose ξ_n,1, … , ξ_n,n−1 are known and define $c_m:=\mathbb{E}[T_{m,m}^{(\eta )}]$ for 2 ≤ m ≤ n. Then, up to a common positive multiplicative constant, the quantities c₂, … , c_n can be determined uniquely from ξ_n,1, … , ξ_n,n−1.

This implies that the problem of identifying the population size function η(t) from ξ_n,1, … , ξ_n,n−1 can be reduced, up to a multiplicative constant, to the problem of identifying η(t) from c₂, … , c_n.

2.3. Piecewise population size models and sign change complexities

To state our main result in full generality, we first need a few definitions.

Definition 1

(ℱ, family of continuous population size functions). A family ℱ of continuous population size functions is a set of positive continuous functions f : ℝ_≥0 → ℝ₊ of a particular type parameterized by a collection of variables.

We use ℱ_c to denote the family of constant population size functions; that is, functions of the form f(t) = ν for all t, where ν ∈ ℝ₊ is the only parameter of the family. Further, we use ℱ_e to denote the family of exponential population size functions of the form f(t) = ν exp(βt), where ν ∈ ℝ₊ and β ∈ ℝ are the parameters of the family. In human genetics, there has been recent interest [38, 39] in modeling superexponential growth in the effective population size via models that generalize exponential growth by incorporating an additional acceleration parameter γ. Such population size functions f satisfy the differential equation df/dt = βf (t)^γ with initial condition f(0) = ν, where β ∈ ℝ, γ ∈ ℝ_≥0, and ν ∈ ℝ₊. When 0 ≤ γ <1 (resp., γ >1), this represents superexponential (resp., subexponential) population growth/decline, while γ = 1 corresponds to exponential population growth/decline. We let ℱ_g denote the family of such generalized-exponential population size functions.

Definition 2

[ℳ_K(ℱ), piecewise models over ℱ with at most K pieces]. Given a family ℱ of continuous population size functions, a population size function η(t) defined over ℝ_≥0 is said to be piecewise over ℱ with at most K pieces if there exists an integer p, where 1 ≤ p ≤ K − 1, and a sequence of p time points 0 < t₁ < ··· < t_p < ∞ such that for each 1 ≤ i ≤ p + 1, there exists a positive continuous function f_i ∈ ℱ such that η(t) = f_i (t − t_i−1) for all t ∈ [t_i−1, t_i). For convention, we define t₀ = 0 and t_p+1 =∞. Note that η may not be continuous at the change points t₁, … , t_p. We use ℳ_K(ℱ) to denote the space of such piecewise population size models with at most K pieces, each of which belongs to function family ℱ. Illustrated in Figure 1 is an example of piecewise-exponential population size function η ∈ ℳ_K(ℱ) where K ≥ 5 and ℱ = ℱ_e.

Fig. 1.

A piecewise-exponential population size function η ∈ ℳ_K(ℱ_e), where K ≥ 5. Note that the y-axis is in a log scale. This piecewise-exponential function depicts the historical population size changes of a European population that was estimated from the SFS of a sample of 1351 (diploid) individuals of European ancestry [44].

Definition 3

[σ(f), number of sign changes of a function]. For a function g (not necessarily continuous) defined over some interval (a, b), we say that t ∈ (a, b) is a sign change point of g if there exist some ε > 0, t′ ≥ t, and an interval (t′, t′ + ε) ⊆ (a, b) such that:

1. (t − ε, t) ⊆ (a, b),

2. g(z) = 0 for z ∈ (t, t′),

3. g(x)g(y) < 0 for all x ∈ (t − ε, t) and y ∈ (t′, t′ + ε).

We define the number σ(g) of sign changes of g as the number of such sign change points in its domain (a, b). See Figure 2 for an illustration.

Fig. 2.

Illustration of the sign changes of a function. For the domain shown, σ(g) = 3 and the sign change points of g are denoted t₁, t₂, and t₃.

Note that the above definition of the number of sign changes counts the number of times the function g changes value from positive to negative (and vice versa) while ignoring intervals where it is identically zero. While the above definition is not restricted to piecewise continuous functions, we will restrict our attention to such functions for the remainder of this paper.

Definition 4

[𝒮(ℱ) and 𝒮(ℳ_K(ℱ)), sign change complexities]. For a family ℱ of continuous population size functions, we define the sign change complexity 𝒮(ℱ) as

$$\begin{array}{l}\mathcal{S}(\mathcal{F})=\underset{\begin{array}{c}{f}_1,f_2\in \mathcal{F},\\ a_1,a_2\in {ℝ}_{\ge 0}\end{array}}{sup}\left\{\sigma (g)|\begin{array}{l}g(\tau ):={\stackrel{\sim }{f}}_1(\tau -a_1)-{\stackrel{\sim }{f}}_2(\tau -a_2)\text{with}\text{domain}\hfill \\ \text{Dom}(g)=\left\{\tau \in {ℝ}_{\ge 0}|\begin{array}{l}\tau -a_1\in \text{Dom}({\stackrel{\sim }{f}}_1),\hfill \\ \tau -a_2\in \text{Dom}({\stackrel{\sim }{f}}_2)\hfill \end{array}\right\}\hfill \end{array}\right\}\\ =\underset{\begin{array}{c}{f}_1,f_2\in \mathcal{F},\\ a\in {ℝ}_{\ge 0}\end{array}}{sup}\left\{\sigma (g)|\begin{array}{l}g(\tau ):={\stackrel{\sim }{f}}_1(\tau )-{\stackrel{\sim }{f}}_2(\tau -a)\text{with}\text{domain}\hfill \\ \text{Dom}(g)=\left\{\tau \in {ℝ}_{\ge 0}|\begin{array}{l}\tau \in \text{Dom}({\stackrel{\sim }{f}}_1),\hfill \\ \tau -a\in \text{Dom}({\stackrel{\sim }{f}}_2)\hfill \end{array}\right\}\hfill \end{array}\right\},\end{array}$$ (5)

where f̃_j are the time-rescaled versions of f_j as defined in (2), and Dom(f̃_j) = R_fj (ℝ_≥0) is the domain of f̃_j. Similarly, for the space ℳ_K(ℱ) of piecewise population size models with at most K pieces over some function family ℱ, we define the sign change complexity 𝒮(ℳ_K(ℱ)) as

$$\mathcal{S}\left({\mathcal{M}}_K(\mathcal{F})\right)=\underset{{\eta }_1,{\eta }_2\in {\mathcal{M}}_K(\mathcal{F})}{sup}\{\sigma ({\stackrel{\sim }{\eta }}_1-{\stackrel{\sim }{\eta }}_2)\},$$

where, again, η̃_j are related to η_j as given in (2).

The following lemma gives a bound on the sign change complexity of a model with at most K pieces in terms of the underlying family of population size functions for each piece.

Lemma 2

The sign change complexity of the space ℳ_K(ℱ) of piecewise models with at most K pieces in a function family ℱ is bounded by the sign change complexity of ℱ as

$$\mathcal{S}({\mathcal{M}}_K(\mathcal{F}))\le (2K-2)+(2K-1)\mathcal{S}(\mathcal{F}).$$

Note that the bound in Lemma 2 is tight for the family ℱ_c of constant population sizes, for which 𝒮(ℱ_c) = 0 and 𝒮(ℳ_K(ℱ_c)) = 2K −2.

2.4. Identifiability results

Our main results on identifiability will be proved using a generalization of Descartes’ rule of signs for polynomials.

Theorem 3 (Descartes’ rule of signs for polynomials)

Consider a degree-n polynomial p(x) = a₀ + a₁x + ··· + a_nxⁿ with real-valued coefficients a_i. The number of positive real roots (counted with multiplicity) of p is at most the number of sign changes between consecutive nonzero terms in the sequence a₀, a₁, …, a_n.

The following theorem generalizes the above classic result to relate the number of sign changes of a piecewise-continuous function f to the number of roots of its Laplace transform.

Theorem 4 (Generalized Descartes’ rule of signs)

Let f :ℝ_≥0 → ℝ be a piecewise-continuous function which is not identically zero and with a finite number σ(f) of sign changes. Then the function G(x) defined by

$$G(x)={\int }_0^{\infty }f(t)e^{-tx}dt$$ (6)

has at most σ(f) roots in ℝ (counted with multiplicity).

The statement of Theorem 4 and the proof provided in Section 3 are adapted from Jameson [15], Lemma 4.5, for our setting. Using Theorem 4, we prove in Section 3 the following identifiability theorem for population size function families with finite sign change complexity.

Theorem 5

For a sample of size n, let c = (c₂, … , c_n), where $c_m=\mathbb{E}[T_{m,m}^{(\eta )}]$, for 2 ≤ m ≤ n, defined in (3). If 𝒮(ℱ) <∞ and n ≥ 𝒮(ℱ) + 2, then no two distinct models η₁,η₂ ∈ ℱ can produce the same (c₂, … , c_n). In other words, for n ≥ 𝒮(ℱ) + 2, the map c : $\mathcal{F}\to {ℝ}_{+}^{n-1}$ is injective.

Note that the sample size bound in Theorem 5 applies to an arbitrary function family ℱ which need not have any special structure. Using Lemma 2 for bounding the sign change complexity of piecewise-defined function families ℳ_K(ℱ) in terms of the sign change complexity of the underlying function family ℱ, we immediately obtain the following theorem.

Theorem 6

For a sample of size n, let c = (c₂, … , c_n), where $c_m=\mathbb{E}[T_{m,m}^{(\eta )}]$, for 2 ≤ m ≤ n, defined in (3). If 𝒮(ℱ) < ∞ and n ≥ 2K + (2K − 1)𝒮(ℱ), then the map c : ${\mathcal{M}}_K(\mathcal{F})\to {ℝ}_{+}^{n-1}$ is injective.

Using Theorem 6, it is simple to derive identifiability results for piecewise-defined population size models over several function families ℱ that are of biological interest. In particular, we have the following result for the case of piecewise-constant models.

Corollary 7 [Identifiability of piecewise-constant population size models in ℳ_K(ℱ_c)]

The map c : ${\mathcal{M}}_K({\mathcal{F}}_c)\to {ℝ}_{+}^{n-1}$ is injective if the sample size n ≥ 2K.

The bound in Corollary 7 on the sample size sufficient for identifying piecewise-constant population models is actually tight, since ℳ_K(ℱ_c) has 2K − 1 parameters in ℝ₊ and there is no continuous injective function from ${ℝ}_{+}^{2K-1}$ to ℝⁿ⁻¹ if n < 2K. (This fact can be proved in multiple ways, such as by the Borsuk–Ulam theorem or the Constant Rank theorem.) An alternate proof of Corollary 7 that does not rely on Theorem 6 is also provided in Section 3. This alternate proof is based on an argument from linear algebra, and it might be possible to adapt this approach to develop an algebraic algorithm for inferring the parameters of a piecewise-constant population function from the set of expected first coalescence times c_m.

Another class of models often assumed in population genetic analyses are piecewise-exponential functions, for which we have the following result.

Corollary 8 [Identifiability of piecewise-exponential population size models in ℳ_K(ℱ_e)]

The map c : ${\mathcal{M}}_K({\mathcal{F}}_e)\to {ℝ}_{+}^{n-1}$ is injective if the sample size n ≥ 4K −1.

For the generalized-exponential growth models considered by Reppell, Boehnke and Zöllner [39], we have the following result.

Corollary 9 [Identifiability of piecewise-generalized-exponential population size models in ℳ_K(ℱ_g)]

The map c : ${\mathcal{M}}_K({\mathcal{F}}_g)\to {ℝ}_{+}^{n-1}$ is injective if the sample size n ≥ 6K −2.

For the identifiability of piecewise population size models from the SFS data, we first note the following lemma.

Lemma 10

Consider a piecewise population size function η ∈ ℳ_K(ℱ). Consider a sample of size n ≥ 2K + (2K − 1)𝒮(ℱ) and suppose the function η produces $\mathbb{E}[T_{m,m}^{(\eta )}]=c_m$ for 2 ≤ m ≤ n. Then, for every fixed κ ∈ ℝ₊, there exists a unique piecewise population size function ζ ∈ ℳ_K(ℱ) with $\mathbb{E}[T_{m,m}^{(\zeta )}]=\kappa c_m$ for 2 ≤ m ≤ n. Furthermore, this population size function ζ is given by ζ(t) = κη(t/κ).

Given two models η, ζ ∈ ℳ_K, we say that η and ζ are equivalent, and write η ~ ζ, if they are related by a rescaling of change points and population sizes as described in Lemma 10. Let [η] denote the equivalence class of population size functions that contain η, and let ℳ_K(ℱ)/~ = {[η]]η ∈ ℳ_K(ℱ)} be the set of equivalence classes for the equivalence relation ~. Then, combining Lemma 1, Theorem 6 and Lemma 10, we obtain the following theorem.

Theorem 11

If 𝒮(ℱ) <∞ and n ≥ 2K + (2K − 1)𝒮(ℱ), then, for each expected SFS (ξ_n,1, … , ξ_n,n−1), there exists a unique equivalence class [η] of models in ℳ_K(ℱ)/~ consistent with (ξ_n,1, … , ξ_n,n−1).

2.5. Extension to the folded frequency spectrum

To generate the SFS from genomic sequence data, one needs to know the identities of the ancestral and mutant alleles at each site. To avoid this problem, a commonly employed strategy in population genetic inference involves “folding” the SFS. More precisely, for a sample of size n, the ith entry of the folded SFS χ = (χ_n,1, … , χ _⌊n/2⌋) is defined by

$${\chi }_{n,i}=\frac{{\xi }_{n,i}+{\xi }_{n,n-i}}{1+{\delta }_{i,n-i}},$$

where 1 ≤ i ≤ ⌊n/2⌋. In particular, χ_n,i is the proportion of polymorphic sites that have i copies of the minor allele. For any sample size n, since χ is a vector of approximately half the dimension as ξ, we might expect to require roughly twice as many samples to recover the demographic model from χ compared to ξ. This is indeed the case. Given the folded SFS χ, the following theorem establishes a sufficiency condition on the sample size for identifying demographic models in ℳ_K(ℱ).

Theorem 12

If 𝒮(ℱ) <∞ and n ≥ 2(2K −1)(1 + 𝒮(ℱ)), then, for each expected folded SFS χ = (χ_n,1, … , χ_⌊n,n/2⌋), there exists a unique equivalence class [η] of models in ℳ_K(ℱ)/~ consistent with χ.

2.6. The counterexample of Myers, Fefferman and Patterson

Myers, Fefferman and Patterson [30] provided an explicit counterexample to the identifiability of population size models from the allelic frequency spectrum. In our notation, they provided two time-rescaled population size functions η̃₁ and η̃₂ given by

$$\begin{array}{l}{\stackrel{\sim }{\eta }}_1(\tau )=N,\\ {\stackrel{\sim }{\eta }}_2(\tau )=N(1-9F(\tau )),\end{array}$$

where N is an arbitrary positive constant, and the function F is given by the convolution

$$F(\tau )={\int }_0^{\tau }{f}_0(\tau -u)f_1(u)du,$$

where f₀ and f₁ are given by

$$\begin{array}{l}{f}_0(\tau )=exp(-1/{\tau }^2),\\ f_1(\tau )=\frac{cos({\pi }^2/\tau )exp(-\tau /8)}{\sqrt{\tau }}.\end{array}$$

Both functions f₁ and F have increasingly frequent oscillations as τ ↓ 0 so that σ(η̃₁ − η̃₂) = σ(F)=∞. This is why Theorem 5 does not apply to this example. Indeed, by an argument using the Laplace transforms of f₁ and F, Myers, Fefferman and Patterson showed that the function G(x) defined in (6) in terms of F has roots at $-\left(\begin{array}{c}m\\ 2\end{array}\right)$ for each m ≥ 2.

3. Proofs

We now provide proofs of the results presented earlier.

Proof of Lemma 1

In the coalescent for a sample of size n, let γ_n,b denote the total expected branch length subtending b leaves, for 1 ≤ b ≤ n−1. Then ${\xi }_{n,b}={\gamma }_{n,b}/{\sum }_{k=1}^{n-1}{\gamma }_{n,k}$, which implies that there exists a positive constant κ such that γ_n,b = κξ_n,b for all 1 ≤ b ≤ n−1. We now prove that c₂, … , c_n can be determined uniquely from γ_n,1, … , γ_n,n−1.

Let ${\varphi }_{n,k}=\mathbb{E}[T_{n,k}^{(\eta )}]$. Then, by a result of Griffiths and Tavaré [12],

$${\gamma }_{n,b}=\sum _{k=2}^{n-b+1}k\frac{\left(\begin{array}{c}n-b-1\\ k-2\end{array}\right)}{\left(\begin{array}{c}n-1\\ k-1\end{array}\right)}{\varphi }_{n,k},$$ (7)

for 1 ≤ b ≤ n − 1. The system of equations (7) can be rewritten succinctly as a linear system

$$\mathit{\gamma }=\mathbf{M}\mathit{\varphi },$$

where γ = (γ_n,1, … , γ_n,n−1), ϕ = (ϕ_n,2, … , ϕ_n,n), and M = (m_bk) with $m_{bk}=k\left(\begin{array}{c}n-b-1\\ k-2\end{array}\right)/\left(\begin{array}{c}n-1\\ k-1\end{array}\right)$, for 1 ≤ b ≤ n − 1 and 2 ≤ k ≤ n. The matrix M is upper-left triangular since $\left(\begin{array}{c}n-b-1\\ k-2\end{array}\right)=0$ if k > n − b + 1, and the anti-diagonal entries are $\frac{k}{\left(\begin{array}{c}n-1\\ k-1\end{array}\right)}>0$. Hence, det(M) ≠ 0 and M is therefore invertible. Thus, given γ, we can determine ϕ uniquely as M⁻¹γ.

Let ${\psi }_{n,k}={\sum }_{j=k}^n\mathbb{E}[T_{n,j}^{(\eta )}]$. Then, defining ψ_n,n+1 := 0, observe that ψ_n,k = ϕ_n,k + ψ_n,k+1 for 2 ≤ k ≤ n. This implies that ψ_n,2, … , ψ_n,n can be determined uniquely from ϕ_n,2, … , ϕ_n,n. Polanski, Bobrowski and Kimmel [35] showed that ψ_n,k can be written as

$${\psi }_{n,k}=\sum _{m=k}^n{a}_{km}{c}_m,$$ (8)

where a_km, for k ≤ m ≤ n, are given by

$$a_{km}=\frac{{\prod }_{l=k,l\ne m}^n\left(\begin{array}{c}l\\ 2\end{array}\right)}{{\prod }_{l=k,l\ne m}^n[\left(\begin{array}{c}l\\ 2\end{array}\right)-\left(\begin{array}{c}m\\ 2\end{array}\right)]},$$

and $c_m=\mathbb{E}[T_{m,m}^{(\eta )}]$, shown in (3). Again, the system of equations (8) can be written as a triangular linear system

$$\mathit{\psi }=\mathbf{Ac},$$

where ψ = (ψ_n,2, … , ψ_n,n), c = (c₂, … , c_n), and A = (a_km), for 2 ≤ k, m ≤ n. Note that A is an upper triangular matrix since a_km := 0 if m < k. Since A has nonzero entries on its diagonal, A⁻¹ exists, and c can be determined uniquely as A⁻¹ψ.

Proof of Lemma 2

Given a pair of piecewise population size functions η₁, η₂ ∈ ℳ_K (ℱ), let η̃₁ and η̃₂ be their respective time-rescaled versions, defined by (2). Let $0<t_1^{(1)}<\cdots <t_{p_1}^{(1)}<\infty$, where 0 ≤ p₁ ≤ K − 1 (resp., $0<t_1^{(2)}<\cdots <t_{p_2}^{(2)}<\infty$, where 0 ≤ p₂ ≤ K − 1) be the change points of the pieces of η₁ (resp., η₂). We define $t_0^{(1)}=t_0^{(2)}=0$ and $t_{p_1+1}^{(1)}=t_{p_2+1}^{(2)}=\infty$. The change points of η̃₁ are given by $R_{{\eta }_1}(t_i^{(1)})$, where 1 ≤ i ≤ p₁, while the change points of η̃₂ are given by $R_{{\eta }_2}(t_i^{(2)})$, where 1 ≤ i ≤ p₂. Let 0 < τ₁ < ··· < τ_p < ∞ be the union of the change points of η̃₁ and η̃₂, where 0 ≤ p ≤ p₁ + p₂. For convention, let τ₀ = 0 and τ_p+1 = ∞.

Consider the piece (τ_i, τ_i+1) for 0 ≤ i ≤ p. Let $I_1=(t_k^{(1)},t_{k+1}^{(1)})$, where 0 ≤ k ≤ p₁, and $I_2=(t_l^{(2)},t_{l+1}^{(2)})$, where 0 ≤ l ≤ p₂, be the pieces of the original population size functions η₁ and η₂, respectively, such that (τ_i, τ_i+1) ⊆ R_η1(I₁) and (τ_i, τ_i+1) ⊆ R_η2(I₂). Since η₁ ∈ ℳ_K(ℱ), there exists a function f₁ ∈ ℱ such that ${\eta }_1(t)=f_1(t-t_k^{(1)})$for all t ∈ I₁. Then, for all τ ∈ R_η1(I₁),

$$\begin{array}{l}{\stackrel{\sim }{\eta }}_1(\tau )={\eta }_1\left(R_{{\eta }_1}^{-1}(\tau )\right)=f_1\left(R_{{\eta }_1}^{-1}(\tau )-t_k^{(1)}\right)\\ ={\stackrel{\sim }{f}}_1\left(R_{f_1}\left(R_{{\eta }_1}^{-1}(\tau )-t_k^{(1)}\right)\right)\\ ={\stackrel{\sim }{f}}_1\left(\tau -R_{{\eta }_1}\left(t_k^{(1)}\right)\right).\end{array}$$ (9)

Similarly, there exists some function f₂ ∈ ℱ such that, for all τ ∈ R_η2(I₂),

$${\stackrel{\sim }{\eta }}_2(\tau )={\stackrel{\sim }{f}}_2\left(\tau -R_{{\eta }_2}\left(t_l^{(2)}\right)\right).$$ (10)

Using (9) and (10), we see that the number of sign change points of η̃₁ − η̃₂ in the piece (τ_i, τ_i+1) is at most the number of sign change points of ${\stackrel{\sim }{f}}_1(\tau -R_{{\eta }_1}(t_k^{(1)}))-{\stackrel{\sim }{f}}_2(\tau -R_{{\eta }_2}(t_l^{(2)}))$ for τ ∈ (τ_i, τ_i+1). Hence, by (5), it follows that within each piece (τ_i, τ_i+1) for 0 ≤ i ≤ p, η̃₁ − η̃₂ has at most 𝒮 (ℱ) sign change points. Also, the point τ_i+1 itself could be a sign change point in the interval between the last sign change point in piece (τ_i, τ_i+1) and the first sign change point in piece (τ_i+1, τ_i+2) where 0 ≤ i ≤ p − 1. These are all the possible sign change points of η̃₁ − η̃₂. Hence,

$$\begin{array}{l}\sigma ({\stackrel{\sim }{\eta }}_1-{\stackrel{\sim }{\eta }}_2)\le p+(p+1)\mathcal{S}(\mathcal{F})\\ \le (p_1+p_2)+(p_1+p_2+1)\mathcal{S}(\mathcal{F})\\ \le (2K-2)+(2K-1)\mathcal{S}(\mathcal{F}).\end{array}$$ (11)

Since (11) holds for all η₁, η₂ ∈ ℳ_K (ℱ), the lemma follows.

Proof of Theorem 4

The proof is by induction on the number of sign changes of f. If f has zero sign changes, then without loss of generality, f(t) ≥ 0 for t ∈ (0, ∞) and f(t) > 0 for some interval (a, b) ⊆ (0, ∞). Hence, G(x) > 0 for all x, and the base case holds. Suppose f has m + 1 sign change points t₀,..., t_m, where m ≥ 0. Note that G(x) and F(x) = e^t₀x G(x) have the same real-valued roots (with multiplicity) since e^t₀x > 0 for all x ∈ ℝ. F′ (x) is given by

$$F^{\prime }(x)=\frac{d}{dx}\left({\int }_0^{\infty }f(t)e^{-(t-t_0)x}dt\right)={\int }_0^{\infty }(t_0-t)f(t)e^{-(t-t_0)x}dt,$$

where the interchange of the differential and integral operators in the second equality is justified by the Leibniz integral rule because f is piecewise continuous over ℝ_≥0, and both f(t)e^{−(t−t₀)x} and $\frac{d}{dx}(f(t)e^{-(t-t_0)x})$ are jointly continuous over (p_i, p_i+1) × (−∞, ∞) for each piece (p_i, p_i+1) over which f is continuous. Note that the set of sign change points of (t₀ − t)f(t) is {t₁,..., t_m}. Hence, (t₀ − t)f(t) has only m sign changes. By the induction hypothesis, F′ has at most m real-valued roots. By Rolle’s theorem, the number of real-valued roots of F is at most one more than the number of real-valued roots of F′. Hence, F has at most m + 1 real-valued roots, implying that G has at most m + 1 real-valued roots.

Proof of Theorem 5

Suppose there exist two distinct population size functions η₁, η₂ ∈ ℱ that produce exactly the same c_m for all 2 ≤ m ≤ n. From (4), we have that

$${\int }_0^{\infty }({\stackrel{\sim }{\eta }}_1(\tau )-{\stackrel{\sim }{\eta }}_2(\tau ))e^{-\left(\begin{array}{c}m\\ 2\end{array}\right)\tau }d\tau =0$$ (12)

for 2 ≤ m ≤ n. If we define the function G(x) as

$$G(x)={\int }_0^{\infty }\left({\stackrel{\sim }{\eta }}_1(\tau )-{\stackrel{\sim }{\eta }}_2(\tau )\right)e^{-x\tau }d\tau ,$$

then from (12), we see that $\left(\begin{array}{c}m\\ 2\end{array}\right)$ is a root of G(x) for 2 ≤ m ≤ n, and hence, G has at least n − 1 roots. Applying Theorem 4 to the piecewise continuous function η̃₁ − η̃₂, we see that G can have at most σ (η̃₁ − η̃₂) roots. Taking the supremum over all population size functions η₁ and η₂ in ℱ, we see that G can have at most 𝒮(ℱ) roots. Hence, if n − 1 > 𝒮 (ℱ), we get a contradiction. This implies that if n ≥ 𝒮(ℱ) + 2, no two distinct population size functions in ℱ can produce the same (c₂,..., c_n).

Proof of Corollary 7

As remarked after Lemma 2, for the constant population size function family ℱ_c, 𝒮 (ℱ_c) = 0. Hence, by Theorem 6, if n ≥ 2K, the map c: ${\mathcal{M}}_K({\mathcal{F}}_c)\to {ℝ}_{+}^{n-1}$ is injective.

An alternate proof of Corollary 7 based on linear algebra

Let n ≥ 2K, and suppose there exist two distinct models η⁽¹⁾, η⁽²⁾ ∈ ℳ_K(ℱ_c) that produce exactly the same c_m for all 2 ≤ m ≤ n. Let η̃⁽¹⁾ and η̃⁽²⁾ denote the time-rescaled versions of η⁽¹⁾ and η⁽²⁾, respectively, as in (2). Since η^(j) is piecewise constant with at most K pieces, η̃^(j) is also piecewise constant with the same number of pieces as η^(j), and η⁽¹⁾ ≠ η⁽²⁾ implies η̃⁽¹⁾ ≠ η̃⁽²⁾. Therefore, Δ̃ := η̃⁽¹⁾ − η̃⁽²⁾ is a piecewise-constant function over [0, ∞) with p pieces, where 1 ≤ p ≤ 2K − 1, and Δ̃ is not identically zero. Let τ₁ < ··· < τ_p−1 denote the change points of Δ̃, and define τ₀ = 0 and τ_p = ∞. Suppose Δ̃(τ) = δ_i ∈ ℝ for all τ ∈ [τ_i−1, τ_i), where 1 ≤ i ≤ p. Since η̃⁽¹⁾ and η̃⁽²⁾ produce the same for c_m all 2 ≤ m ≤ n, we know that Δ̃ satisfies

$${\int }_0^{\infty }\stackrel{\sim }{\mathrm{\Delta }}(\tau )e^{-\left(\begin{array}{c}m\\ 2\end{array}\right)\tau }d\tau =0,$$ (13)

for all 2 ≤ m ≤ n. Substituting the definition of Δ̃ into (13) and multiplying by $\left(\begin{array}{c}m\\ 2\end{array}\right)$, we obtain

$$\sum _{i=1}^p{\delta }_i[e^{-\left(\begin{array}{c}m\\ 2\end{array}\right){\tau }_{i-1}}-e^{-\left(\begin{array}{c}m\\ 2\end{array}\right){\tau }_i}]=0,$$ (14)

for 2 ≤ m ≤ n. This defines a linear system Aδ = 0, where δ = (δ₁,..., δ_p) and A = (a_mi) is an (n − 1) × p matrix with $a_{mi}:=e^{-\left(\begin{array}{c}m\\ 2\end{array}\right){\tau }_{i-1}}-e^{-\left(\begin{array}{c}m\\ 2\end{array}\right){\tau }_i}$ for 2 ≤ m ≤ n and 1 ≤ i ≤ p.

Let B = (b_mi) be the (n − 1) × p matrix formed from A such that the ith column of B is the sum of columns i, i + 1,..., p of A. Defining α_i = e^−t_i−1, note that $b_{mi}={\alpha }_i^{\left(\begin{array}{c}m\\ 2\end{array}\right)}$ for 2 ≤ m ≤ n and 1 ≤ i ≤ p. Now, consider the p × p submatrix C of B consisting of the first p rows of B. Since α₁ > α₂ > ···> α_p > 0, note that C is a generalized Vandermonde matrix, which implies det(C) ≠ 0 [8], Chapter XIII, Section 8. Hence, rank(B) = p. The rank of A is invariant under elementary column operations and, therefore, rank(A) = rank(B) = p. Therefore, the kernel of A is trivial, and the only solution to (14) is δ₁ = δ₂ = ··· = δ_p = 0, which contradicts our assumption that Δ̃ = η̃⁽¹⁾ − η̃⁽²⁾ ≡ 0.

Proof of Corollary 8

Let f₁, f₂ ∈ ℱ_e be given by

$$\begin{array}{l}{f}_1(t)={\nu }_{1}exp({\beta }_{1}t),\\ f_2(t)={\nu }_{2}exp({\beta }_{2}t),\end{array}$$

where t ∈ ℝ_≥0, ν₁, ν₂ ∈ ℝ₊ and β₁, β₂ ∈ ℝ. Then, for i = 1, 2, the time-rescaled function f̃_i is given by

$${\stackrel{\sim }{f}}_i(\tau )=\frac{{\nu }_i}{1-{\nu }_i{\beta }_i\tau },$$ (15)

for $\tau \in \text{Dom}({\stackrel{\sim }{f}}_i)=R_{fi}({ℝ}_{\ge 0})=[0,\frac{1}{{\nu }_i{\beta }_i})$. From (15), it can be seen that f̃₁ and f̃₂ are continuous in their domains. Furthermore, for any given a ∈ ℝ_≥0, there is at most one τ, where τ ∈ Dom(f̃₁) and τ − a ∈ Dom(f̃₂), such that g(τ) := f̃₁(τ) − f̃₂(τ − a) = 0, implying σ (g) ≤ 1. By the definition of sign change complexity in (5), it then follows that 𝒮 (ℱ_e) ≤ 1 for the exponential population family ℱ_e. Hence, applying Theorem 6, we conclude that n ≥ 4K − 1 suffices for the map c: ${\mathcal{M}}_K({\mathcal{F}}_e)\to {ℝ}_{+}^{n-1}$ to be injective.

Proof of Corollary 9

Let f₁, f₂ ∈ ℱ_g be generalized-exponential functions which satisfy the following differential equations and initial conditions:

$$\begin{array}{ll}\hfill \frac{d{f}_i}{dt}& ={\beta }_i{f}_i{(t)}^{{\gamma }_i},\\ \hfill f_i(0)& ={\nu }_i,i\in \{1,2\},\end{array}$$

where ν_i ∈ ℝ₊, β_i ∈ ℝ and γ_i ∈ ℝ_≥0. The solutions for f_i are given by

$$f_i(t)=\{\begin{array}{ll}{\nu }_{i}exp({\beta }_{i}t),\hfill & {\gamma }_i=1,\hfill \\ {[{\nu }_i^{1-{\gamma }_i}+{\beta }_{i}t(1-{\gamma }_i)]}^{1/(1-{\gamma }_i)},\hfill & {\gamma }_i\ne 1.\hfill \end{array}$$

It can be shown that the time-rescaled population size functions f̃_i are given by

$${\stackrel{\sim }{f}}_i(\tau )=\{\begin{array}{ll}{\nu }_{i}exp({\beta }_i\tau ),\hfill & {\beta }_i=0\text{or}{\gamma }_i=0,\hfill \\ {({\nu }_i^{-{\gamma }_i}-{\beta }_i{\gamma }_i\tau )}^{-1/{\gamma }_i},\hfill & {\beta }_i\ne 0\text{and}{\gamma }_i>0.\hfill \end{array}$$ (16)

In order to obtain an upper bound on 𝒮 (ℱ_g), we consider the following three cases depending on the functional form of f̃₁ and f̃₂ in (16):

• Case 1: f̃₁(τ) = ν₁ exp(β₁τ) and f̃₂(τ) = ν₂ exp(β₂τ). Since f̃₁ and f̃₂ are continuous functions of τ, the number of sign changes of g(τ) := f̃₁(τ) − f̃₂(τ − a) is at most the number of roots of g(τ). Taking the logarithm of f̃₁(τ) and f̃₂(τ − a), it is easy to see that g(τ) has at most one root for any a ∈ ℝ_≥0. Hence, σ(g) ≤ 1.

• Case 2: f̃₁ and f̃₂ have different functional forms. Suppose f̃₁(τ) = ν₁ exp(β₁τ) and ${\stackrel{\sim }{f}}_2(\tau )={({\nu }_2^{-{\gamma }_2}-{\beta }_2{\gamma }_2\tau )}^{-1/{\gamma }_2}$. For any a₁, a₂ ∈ ℝ_≥0 such that τ − a_i ∈ Dom(f̃_i), the number of sign changes of g(τ) := f̃₁(τ − a₁) − f̃₂(t − a₂) is at most the number of roots of g(τ). By raising f̃₁(t − a₁) and f̃₂(t − a₂) to the power of −γ₂, we see that the number of roots of g(τ) is the number of solutions to

  $${\mu }_{1}exp(-{\gamma }_2{\beta }_1\tau )={\mu }_2^{-{\gamma }_2}-{\beta }_2{\gamma }_2\tau ,$$ (17)

  where μ₁ = ν₁ exp(γ₂β₁a₁) and ${\mu }_2={({\nu }_2^{-{\gamma }_2}+{\beta }_2{\gamma }_2{a}_2)}^{-1/{\gamma }_2}$. Equation (17) represents the intersection of an exponential function with a line and has at most 2 solutions for τ. Hence, σ(g) ≤ 2.

• Case 3: ${\stackrel{\sim }{f}}_i(\tau )={({\nu }_i^{-{\gamma }_i}-{\beta }_i{\gamma }_i\tau )}^{-1/{\gamma }_i}$ for i = 1, 2. Let g(τ) := f̃₁ (τ ) − f̃₂(τ − a) where a ∈ ℝ_≥0 such that τ − a ∈ Dom(f̃₂). Since g is a continuous function, the number of sign changes of g(τ) in ℝ_≥0 is bounded by the number of distinct positive roots of g(τ). The number of distinct positive roots of g is the number of distinct positive solutions τ to

  $${\left({\nu }_1^{-{\gamma }_1}-{\beta }_1{\gamma }_1\tau \right)}^{-1/{\gamma }_1}={\left({\nu }_2^{-{\gamma }_2}-{\beta }_2{\gamma }_2(\tau -a)\right)}^{-1/{\gamma }_2},$$
  which is also the number of distinct positive solutions to

  $${\left({\nu }_1^{-{\gamma }_1}-{\beta }_1{\gamma }_1\tau \right)}^{{\gamma }_2/{\gamma }_1}={\nu }_2^{-{\gamma }_2}-{\beta }_2{\gamma }_2(\tau -a).$$ (18)
  Let $x:={\stackrel{\sim }{f}}_1{(\tau )}^{-{\gamma }_1}={\nu }_1^{-{\gamma }_1}-{\beta }_1{\gamma }_1\tau$. Since f̃₁ is a time-rescaled population size function, x > 0 when τ ∈ ℝ_≥0. Since β_i ≠ 0 and γ_i > 0, (18) can be rewritten as

  $$x^{{\gamma }_2/{\gamma }_1}+Ax+B=0,$$

  where $A=-\frac{{\beta }_2{\gamma }_2}{{\beta }_1{\gamma }_1}$ and $B=\frac{{\beta }_2{\gamma }_2}{{\beta }_1{\gamma }_1}{\nu }_1^{-{\gamma }_1}-{\nu }_2^{-{\gamma }_2}-{\beta }_2{\gamma }_{2}a$. Letting h(x) := x^γ₂/γ₁ + Ax + B, the number of distinct positive solutions for τ in (18) is at most the number of distinct positive roots for the generalized polynomial h. For any real-valued function g(x) possessing infinitely many derivatives and any interval I ⊆ ℝ, let Z(g, I) the number of zeroes of g contained in I, counted with multiplicity. By a consequence of Rolle’s theorem [15], Proposition 2.1, Z(g, I) ≤ Z(g′, I) + 1. Observing that $h^{\prime }(x)=\frac{{\gamma }_2}{{\gamma }_1}{x}^{{\gamma }_2/{\gamma }_1-1}+A$ has at most one root in ℝ₊, Z(h, ℝ₊) ≤ Z(h′, ℝ₊) + 1 ≤ 2. Hence, the number of distinct positive solutions τ to (18) is at most 2, and σ(g) ≤ 2.

From the definition of sign change complexity in (5) and the bound on σ(g) in the three cases above, it follows that 𝒮 (ℱ_g) ≤ 2 for the generalized-exponential population family ℱ_g. Hence, applying Theorem 6, we conclude that n ≥ 6K − 2 suffices for the map c: ${\mathcal{M}}_K({\mathcal{F}}_g)\to {ℝ}_{+}^{n-1}$ to be injective.

Proof of Lemma 10

For the population size function ζ(t) defined by ζ(t) = κη(t/κ), note that R_ζ(t) is given by

$$R_{\zeta }(t)={\int }_0^t\frac{1}{\zeta (x)}dx={\int }_0^t\frac{1}{\kappa \eta (x/\kappa )}dx={\int }_0^{t/\kappa }\frac{1}{\eta (x)}dx=R_{\eta }(t/\kappa ).$$

$\mathbb{E}[T_{m,m}^{(\xi )}]$ is then given by

$$\begin{array}{l}\mathbb{E}[T_{m,m}^{(\xi )}]={\int }_0^{\infty }exp\left[-\left(\begin{array}{c}m\\ 2\end{array}\right)R_{\eta }\left(\frac{t}{\kappa }\right)\right]dt\\ =\kappa {\int }_0^{\infty }exp\left[-\left(\begin{array}{c}m\\ 2\end{array}\right)R_{\eta }(t)\right]dt\\ =\kappa \mathbb{E}[T_{m,m}^{(\eta )}].\end{array}$$

Since n ≥ 2K + (2K − 1)𝒮(ℱ), by Theorem 6, ζ is the unique population size function in ℳ_K(ℱ) with $\mathbb{E}[T_{m,m}^{(\zeta )}]=\kappa c_m$ for 2 ≤ m ≤ n.

To prove Theorem 12, we first need a lemma that characterizes a certain symmetry property of the invertible matrix that relates the genealogical quantities γ and c introduced in the proof of Lemma 1.

Lemma 13

For a sample of size n, let W be the (n − 1) × (n − 1) invertible matrix such that ${\gamma }_{n,b}={\sum }_{m=2}^n{W}_{b,m}{c}_m$, where γ_n,b is the total expected branch length subtending b leaves and $c_m=\mathbb{E}[T_{m,m}^{(\eta )}]$ . Then, for every b and m, where 1 ≤ b ≤ n − 1 and 2 ≤ m ≤ n, we have the following identities:

$$\begin{array}{l}{W}_{b,m}+W_{n-b,m}=0ifmis\mathit{odd},\\ W_{b,m}-W_{n-b,m}=0ifmis\mathit{even}.\end{array}$$

Proof

From the proof of Lemma 1, it can be seen that the matrix W is the product of 3 matrices whose entries are explicitly given combinatorial expressions. However, using Zeilberger’s algorithm [34], Polanski and Kimmel [36], equations (13)–(15), also derived the following recurrence relation for the entries of W:

$$\begin{array}{ll}\hfill W_{b,2}& =\frac{6}{(n+1)},\hfill \\ \hfill W_{b,3}& =30\frac{(n-2b)}{(n+1)(n+2)},\hfill \\ W_{b,m+2}\hfill & =f(n,m)W_{b,m}+g(n,m)(n-2b)W_{b,m+1},\hfill \end{array}$$ (19)

where f(n, m) and g(n, m) are rational functions of n and m given by

$$\begin{array}{l}f(n,m)=-\frac{(1+m)(3+2m)(n-m)}{m(2m-1)(n+m+1)},\\ g(n,m)=\frac{(3+2m)}{m(n+m+1)}.\end{array}$$

It will be easy to prove our lemma by induction on m using (19). The base cases are easy to check:

$$\begin{array}{l}{W}_{b,2}-W_{n-b,2}=0,\\ W_{b,3}+W_{n-b,3}=30\frac{(n-2b)+(n-2(n-b))}{(n+1)(n+2)}=0.\end{array}$$

Using (19), we see that if m is odd,

$$\begin{array}{l}{W}_{b,m+2}+W_{n-b,m+2}=f(n,m)(W_{b,m}+W_{n-b,m})+g(n,m)\{(n-2b)W_{b,m+1}+[n-2(n-b)]W_{n-b,m+1}\}\\ =f(n,m)(W_{b,m}+W_{n-b,m})+g(n,m)(n-2b)(W_{b,m+1}-W_{n-b,m+1})\\ =0,\end{array}$$

where the last equality follows from the induction hypothesis which implies W_b,m + W_n−b,m = 0 and W_b,m+1 − W_n−b,m+1 = 0. Similarly, if m is even,

$$\begin{array}{l}{W}_{b,m+2}-W_{n-b,m+2}=f(n,m)(W_{b,m}-W_{n-b,m})+g(n,m)\{(n-2b)W_{b,m+1}-[n-2(n-b)]W_{n-b,m+1}\}\\ =f(n,m)(W_{b,m}-W_{n-b,m})+g(n,m)(n-2b)(W_{b,m+1}-W_{n-b,m+1})\\ =0,\end{array}$$

where again the last equality follows from the induction hypothesis.

Proof of Theorem 12

For a sample of size n in the coalescent, let γ_n,b be the total expected branch length subtending b leaves, for 1 ≤ b ≤ n − 1. Then there exists a positive constant κ such that

$$\frac{{\gamma }_{n,d}+{\gamma }_{n,n-d}}{1+{\delta }_{d,n-d}}=\kappa {\chi }_{n,d},$$ (20)

for all 1 ≤ d ≤ ⌊n/2⌋. Let $f_{n,d}=\frac{{\gamma }_{n,d}+{\gamma }_{n,n-d}}{1+{\delta }_{d,n-d}}$. The relationship between f = (f_n,1,..., f_{n, ⌊n/2⌋}) and γ = (γ_n,1,..., γ_n,n−1) can be described by the linear equation

$$\mathbf{f}=\mathbf{Z}\mathit{\gamma },$$

where Z is an ⌊n/2⌋ × (n − 1) matrix with entries given by

$$Z_{dj}=\{\begin{array}{ll}1,\hfill & \text{if}j=d\text{or}j=n-d,\hfill \\ 0,\hfill & \text{otherwise},\hfill \end{array}$$

for 1 ≤ d ≤ ⌊n/2⌋ and 1 ≤ j ≤ n − 1. Hence, dim(ker(Z)) = ⌊ (n − 1)/2⌋.

From Lemma 1, we know that γ; and c = (c₂,..., c_n) are related as γ = Wc, where W = (W_b,m) is an (n − 1) × (n − 1) invertible matrix, where 1 ≤ b ≤ n − 1 and 2 ≤ m ≤ n. Hence,

$$\mathbf{f}=\mathbf{Yc},$$ (21)

where Y := ZW. Since Y_b,m = W_b,m + W_n−b,m, we know from Lemma 13 that Y_b,m = 0 for all odd values of m. Therefore, every other column of the matrix Y is zero. This implies that span({e₃, e₅,..., e_{n−𝟙{n even}}}) ⊆ ker(Y), where e_i is an (n − 1)-dimensional unit vector defined as e_i = (e_i,2,..., e_i,n), with e_i,i = 1 and e_i,j = 0 for i ≠ j. Note that n − 𝟙{n even} = 2⌊ (n − 1)/2⌋ + 1 and dim(span({e₃, e₅,..., e_{2⌊ (n−1)/2⌋+1}})) = ⌊ (n − 1)/2⌋. Now, since W is invertible, dim(ker(Y)) = dim(ker(ZW)) = dim(ker(Z)) = ⌊ (n − 1)/2⌋. Therefore,

$$ker(\mathbf{Y})=\text{span}(\{{\mathbf{e}}_3,{\mathbf{e}}_5,\dots ,{\mathbf{e}}_{2\lfloor (n-1)/2\rfloor +1}\}).$$ (22)

Suppose there exist two distinct models η₁, η₂ ∈ ℳ_K (ℱ) that produce the same folded SFS f. Let c⁽¹⁾ and c⁽²⁾ be the vector of genealogical quantities for models η₁ and η₂, respectively, where $c_m^{(1)}=\mathbb{E}[T_{m,m}^{({\eta }_1)}]$ and $c_m^{(2)}=\mathbb{E}[T_{m,m}^{({\eta }_2)}]$, 2 ≤ m ≤ n. From (21), we know that c⁽¹⁾ −c⁽²⁾ ∈ ker(Y). Using (22), $c_m^{(1)}-c_m^{(2)}$ can be written as

$$c_m^{(1)}-c_m^{(2)}=\sum _{l=1}^{\lfloor (n-1)/2\rfloor }{\alpha }_l{e}_{2l+1,m},$$ (23)

for some α_l ∈ ℝ. Since e_ij = 0 for i ≠ j, (23) implies that $c_m^{(1)}-c_m^{(2)}=0$ for all even values of m, where 2 ≤ m ≤ n. Now applying a similar argument as in the proof of Theorem 6 to $c_m^{(1)}-c_m^{(2)}$ for even values of m, we conclude that if ⌈(n − 1)/2⌉ > (2K − 2) + (2K − 1)𝒮(ℱ), then no two distinct models η₁, η₂ ∈ ℳ_K (ℱ) can produce the same f. This implies that a sample size n ≥ 2(2K − 1)(1 + 𝒮 (ℱ)) suffices for identifying the population size function in ℳ_K (ℱ) from the folded SFS f, and the conclusion of the theorem follows from (20) and Lemma 10.

4. Discussion

In human genetics, several large-sample datasets have recently become available, with sample sizes on the order of several thousands to tens of thousands of individuals [1, 4, 6, 31, 44]. The patterns of polymorphism observed in these datasets deviate significantly from that expected under a constant population size, and there has been much interest in inferring recent and ancient human demographic changes that might explain these deviations [10, 24, 26]. Clearly, model identifiability is an important prerequisite for such statistical inference problems. In this paper, we have obtained mathematically rigorous identifiability results for demographic inference by showing that piecewise-defined population size functions over a wide class of function families are completely determined by the SFS, provided that the sample is sufficiently large. Furthermore, we have provided explicit bounds on the sample sizes that are sufficient for identifying such piecewise population size functions. These bounds depend on the number of pieces and the functional type of each piece. For piecewise-constant population size models, which have been extensively applied in demographic inference studies, our bounds are tight. We have also given analogous results for identifiability from the folded SFS, a variant of the SFS that is oblivious to the identities of the ancestral and mutant alleles.

Recent large-sample sequencing studies have consistently found a substantially higher fraction of rare variants compared to the predictions of the coalescent with a constant population size, even in regions of the genome that are believed to have evolved neutrally [9]. Keinan and Clark [16] suggested that recent rapid expansion of the population has given rise to variants which are private to single individuals in the population, and that this signature of population expansion is particularly apparent now due to the larger sample sizes involved in sequencing studies. We illustrate this point with a specific example. The blue plot in Figure 3 shows the expected SFS for a sample of size n = 19 under the piecewise-exponential population size history with 5 epochs recently inferred by Tennessen et al. [44] and illustrated in Figure 1. (Note that n = 19 is the sample size bound given by Corollary 8 for identifying piecewise-exponential models with up to 5 pieces.) The red plot in Figure 3 shows the expected SFS for the same sample size under a constant population size model. For this small sample size, the two expected frequency spectra are very similar despite the large difference in demographic models, indicating the difficulty of accurately recovering the details of recent exponential population growth using small-sample data. In contrast, for a much larger sample of size n = 2702, which corresponds to the actual sample size for Tennessen et al.’s data, the expected frequency spectra under the two demographic models mentioned above are considerably more different; see the green and purple plots in Figure 3.

Fig. 3.

The leading entries of the expected SFS ξ_n for a piecewise-exponential population size model inferred b Tennessen et al. [44]. This demographic model, shown (up to scaling) in Figure 1, was fitted using the observed SFS from a sample of 1351 (diploid) individuals of European ancestry [44]. The blue plot is the expected SFS for n = 19, which matches the sample size bound in Corollary 8 for identifying piecewise-exponential models with up to 5 pieces, while the green plot is the first 18 entries of the expected SFS for n = 2702 (1351 diploids). The red and purple plots are the expected SFS for n = 19 and n = 2702, respectively, for a constant population size function.

On the other hand, our identifiability results show that perfect data (i.e., the exact expected SFS) from even a small sample size of n = 4K − 1 are sufficient to uniquely identify a piecewise-exponential model with K pieces. This gap between theoretical identifiability and practical inference needs to be better addressed through robustness results that can account for the finite genome length, which limits the resolution to which the expected SFS of a random sample can be estimated. Our identifiability results apply in the limit that the genome length is infinite, which allows one to estimate the entries of the expected SFS exactly. On the other hand, a finite length genome does not permit exact estimation of the expected SFS, which can make it difficult in practice to resolve the details of ancient demographic events even if the sample size is large. This is because population size changes sufficiently far back in the past are likely to have only a marginal effect on the SFS since the individuals in the sample are highly likely to have found a common ancestor by such ancient times.

Our work suggests several interesting avenues for future research. An important problem is to understand the sensitivity of the SFS to perturbations in the demographic parameters. A related problem is quantifying the extent to which errors in estimating the expected SFS from a finite amount of data affect the parameter estimates in inferred demographic models.

It would also be interesting to consider the possibility of developing an algebraic algorithm for demographic inference that closely mimics the linear algebraic proof of Corollary 7 provided in Section 3. For example, using a sample of size K + 1, one could consider inferring a piecewise-constant model with K pieces, with one piece for each of the most recent K − 1 generations and another piece for the population size further back in time. (Here, we are considering a restricted class of piecewise-constant population size functions with fixed change points, so the minimum sample size needed for distinguishing such models using the SFS is K + 1 rather than 2K.) Such an algebraic algorithm could provide a more principled way of inferring demographic parameters, compared to existing inference methods that rely on optimization procedures which lack theoretical guarantees for functions with multiple local optima.

In our work, we focused on the identifiability of demography from the expected SFS data. However, if one were to use the complete sequence data or other summary statistics such as the length distribution of shared haplotype tracts, it might be possible to uniquely identify the demography using even smaller sample sizes than that needed when using only the SFS. Indeed, several demographic inference methods have been developed to infer historical population size changes from such data using anywhere from a pair of genomic sequences [14, 24, 32] to tens of such sequences [42], and it is important to theoretically characterize the power and limitations of both the data and the inference methods.