Geometry of the Sample Frequency Spectrum and the Perils of Demographic Inference

Numerous studies in population genetics have been based on analyzing the sample frequency spectrum (SFS) summary statistic. Most SFS-based inference methods can display pathological behavior in optimization: some demographic model parameters can degenerate to 0...

Abstract

The sample frequency spectrum (SFS), which describes the distribution of mutant alleles in a sample of DNA sequences, is a widely used summary statistic in population genetics. The expected SFS has a strong dependence on the historical population demography and this property is exploited by popular statistical methods to infer complex demographic histories from DNA sequence data. Most, if not all, of these inference methods exhibit pathological behavior, however. Specifically, they often display runaway behavior in optimization, where the inferred population sizes and epoch durations can degenerate to zero or diverge to infinity, and show undesirable sensitivity to perturbations in the data. The goal of this article is to provide theoretical insights into why such problems arise. To this end, we characterize the geometry of the expected SFS for piecewise-constant demographies and use our results to show that the aforementioned pathological behavior of popular inference methods is intrinsic to the geometry of the expected SFS. We provide explicit descriptions and visualizations for a toy model, and generalize our intuition to arbitrary sample sizes using tools from convex and algebraic geometry. We also develop a universal characterization result which shows that the expected SFS of a sample of size n under an arbitrary population history can be recapitulated by a piecewise-constant demography with only ${\kappa }_n$ epochs, where ${\kappa }_n$ is between $n/2$ and $2n-1.$ The set of expected SFS for piecewise-constant demographies with fewer than ${\kappa }_n$ epochs is open and nonconvex, which causes the above phenomena for inference from data.

THE sample frequency spectrum (SFS), also known as the site or allele frequency spectrum, is a fundamental statistic in population genomics for summarizing the genetic variation in a sample of DNA sequences. Given a sample of n sequences from a panmictic (i.e., randomly mating) population, the SFS is a vector of length $n-1$ of which the kth entry corresponds to the number of segregating sites, each with k mutant (or derived) alleles and $n-k$ ancestral alleles. The SFS provides a concise way to summarize n sequences of arbitrary length into just $n-1$ numbers, and is frequently used in empirical population genetic studies to test for deviations from equilibrium models of evolution. For instance, the SFS has been widely used to infer demographic history where the effective population size has changed over time (Nielsen 2000; Gutenkunst et al. 2009; Gravel et al. 2011; Keinan and Clark; 2012; Excoffier et al. 2013; Bhaskar et al. 2015) and to test for selective neutrality (Kaplan et al. 1989; Achaz 2009). In fact, many commonly used population genetic statistics for testing neutrality, such as Watterson’s ${\theta }_W$ (Watterson 1975), Tajima’s ${\theta }_{\pi }$ (Tajima 1983), and Fu and Li’s ${\theta }_{FL}$ (Fu and Li 1993) can be expressed as linear functions of the SFS (Durrett 2008).

In the coalescent framework (Kingman 1982a,b,c), the unnormalized expected SFS ${\mathit{\xi }}_n$ for a random sample of n genomes drawn from a population is obtained by taking the expectation of the SFS over the distribution of sample genealogical histories under a specified population demography. In this work, we will be concerned with well-mixed, panmictic populations with time-varying historical population sizes, evolving according to the neutral coalescent process with the infinite-sites model of mutation. The coalescent arises as the continuum limit of a large class of discrete models of random mating—such as the Wright–Fisher, Moran, and Cannings exchangeable family of models (Möhle and Sagitov 2001)—by a suitable rescaling of time and taking the population size to infinity. The infinite-sites model postulates that every mutation in the genealogy of a sample occurs at a distinct site and is commonly employed in population genetic studies for organisms with low population-scaled mutation rates, such as humans. The SFS also appears in the context of statistical modeling as a vector of probabilities. In particular, the normalized expected SFS ${\widehat{\mathit{\xi }}}_n,$ defined by normalizing the entries of ${\mathit{\xi }}_n$ so that they sum to one, gives the probability that a mutation chosen at random is present in k out of n sequences in the sample. Unless stated otherwise, we use the term expected SFS to refer to the unnormalized quantity ${\mathit{\xi }}_n.$

The expected SFS is strongly influenced by the demographic history of the population, and extensive theoretical and empirical work has been done to characterize this dependence (Fu 1995; Wakeley and Hey 1997; Polanski et al. 2003; Marth et al. 2004; Chen 2012; Jouganous et al. 2017; Kamm et al. 2017). Fu (1995) showed that under the infinite-sites model for a panmictic population with constant size and no selection, the expected SFS is given by ${\mathit{\xi }}_n=\theta \cdot \left(1,1/2,\dots ,1/\left(n-1\right)\right),$ where $\theta /2$ denotes the population-scaled mutation rate. When the population size is variable, however, the formula for the expected SFS depends on the entire population size history. In particular, Polanski and Kimmel (2003) (in equations 13–15) showed that the expected SFS under a time-varying population size is given by ${\mathit{\xi }}_n=A_{n}c,$ with $A_n$ being an $\left(n-1\right)$-by-$\left(n-1\right)$ invertible matrix that only depends on n (formula presented in Appendix) and $c=\left(c_2,\dots ,c_n\right),$ where $c_m$ denotes the expected time to the first coalescence event in a random sample of size m drawn from the population at present. For any time-varying population size function $\eta \left(t\right),$ the quantity $c_m$ is given by the following expression:

$$c_m={\displaystyle {\int }_0^{\mathrm{\infty }}\left(\begin{array}{c}m\\ 2\end{array}\right)\frac{t}{\eta \left(t\right)}\exp \left[-\left(\begin{array}{c}m\\ 2\end{array}\right){\displaystyle {\int }_0^t\frac{1}{\eta \left(x\right)}\text{d}x}\right]\text{d}t.}$$ (1)

Pathologies of SFS-based inference algorithms

Let us consider a hypothetical scenario. Suppose we would like to learn about the population history of a group of finches on a remote island. Fossil evidence indicates that the island experienced many generations with ample resources leading to a large, roughly constant population size. Then, some catastrophe occurred, rendering the island’s resources scarce, leading to a small constant population size until the present. We are given four haplotypes from the population and we hope to infer the following parameters for a demographic model based on the history described above:

1. How big was the population during the epoch of plenty?

2. How big was the population during the epoch of scarcity?

3. When did the catastrophe occur, marking the break point?

First, we compute the SFS for the four haplotypes we collected. (Our choice of a sample size of four is for simplicity in this example, but the principles apply for larger samples.) We count singleton (appearing in only one of the haplotypes), doubleton, and tripleton mutations. We do not attempt to track nonsegregating sites. Now we have the SFS, a vector of three real numbers.

Next, we ask ourselves: would we expect to obtain this SFS for some particular set of parameters, based on our model? If the answer is yes, then that set of parameters is our best guess. In Figure 1, the green region describes the set of SFS we would expect for various parameters under this model. Blue dots indicate measured SFS. When the blue dots land in the green region, we simply infer the parameters corresponding to that point. The red crosses are the expected SFS computed for those parameters, so they coincide with the blue dots.

Figure 1.

The green region, denoted ${\widehat{\Xi }}_{4,2},$ represents the set of expected SFS for two-epoch, piecewise-constant demographies for sample size $n=4.$ Each blue hollow circle is the observed SFS simulated using msprime (Kelleher et al. 2016) under a constant population size coalescent with recombination using realistic mutation and recombination rates of ${10}^{-8}$ mutations and $2.2\times {10}^{-8}$ crossovers per base pair per generation per haploid. Each sequence has 1000 unlinked loci of length 10 kb each, resulting in an average of 7300 segregating sites. The red crosses are the expected SFS inferred for these simulated SFS using fastNeutrino (Bhaskar et al. 2015); the dotted blue lines show the correspondence between the observed SFS and their projections onto ${\widehat{\Xi }}_{4,2}.$ For observed SFS lying in the interior of ${\widehat{\Xi }}_{4,2},$ the observed SFS and their projections coincide, while the observed SFS lying outside ${\widehat{\Xi }}_{4,2}$ project onto the boundaries of one of the two convex regions that form ${\widehat{\Xi }}_{4,2}.$

What if the answer is no? That is, what if the SFS we measured would not be expected for any choice of parameters in our population history model? We have two options to interpret this situation: (1) statistical noise is making the SFS appear inconsistent with the model, or (2) our model is mis-specified. Let us suppose that noise is the culprit. Our strategy is then to look for the closest SFS that would be expected in our model, and infer the parameters associated with that one.

This runs into two problems: First off, the parameters inferred in this way are often nonsensical. In Figure 1, the blue dots outside of the green region are connected by dotted lines to the closest SFS vectors in the green region. Naturally, these mainly lie on the boundary of the green region. The problem is that the boundary points (with one exception that we will discuss later) do not actually correspond to achievable expected SFS vectors! Those points correspond to population size histories where one of the epochs is infinity or zero.

The second problem: Even though there is, in general, a unique closest SFS to a given point outside of the green region, the process of finding the closest point is highly sensitive to noise. Specifically, if you change the quantities in the vector by a small amount, the resulting closest point may change by a large amount. The reason for this is that the set is nonconvex, meaning that not all of the straight lines between points in the green region lie inside the green region. As a consequence, some of the blue dots point to the left-hand green region, while others nearby point to the right-hand green region. Sensitivity to noise is a big problem for inference. Any demographic inference method would manifest these pathologies; indeed, the commonly used $\partial a\partial i$ (Gutenkunst et al. 2009), fastsimcoal2 (Excoffier et al. 2013), and fastNeutrino (Bhaskar et al. 2015) all encounter these issues.

If we hypothesize that the model may be mis-specified, we need to support this assertion. The following question will arise: How far away is our measured SFS from the type of SFS that we would expect under the rejected population model? Furthermore, we may be asked to offer an alternative hypothesis, i.e., is there another model that actually does allow for an SFS equal to or near the one that we measured? Both of these questions require an understanding of the set of all possible SFS.

Minimal demographic complexity for SFS reconstruction

Let us slightly change our finch example. Suppose we have no a priori assumptions regarding the demographic history. Instead, we are only interested in determining whether the SFS is consistent with a null hypothesis of a single panmictic population under neutrality. If the measured SFS is equal to the expected SFS for some demography, we may be asked to produce the simplest demography with the expected SFS we want. Work by Myers et al. (2008) implies that there are infinitely many population size histories with a given expected SFS, as long as we allow the demographies to be arbitrarily complicated. Bhaskar and Song (2014) (two of the authors of this article) demonstrated that when we constrain ourselves to a simpler family of population size histories, we may have a unique function achieving the desired expected SFS.

Now suppose that the SFS does not equal the expected SFS for any demography. Again, we would need to quantify how far away it is from being achieved by some demography. This is an intimidating task. How can we be certain to find the SFS corresponding to every demography without leaving any SFS vectors out? After all, the space of possible population size histories is infinite-dimensional! Our hope is to understand the shape of the set of all possible SFS vectors so we know that we have covered everything when we reject the null hypothesis.

For the small example of a sample size of four, we have demonstrated a sequence of constraints placed on SFS vectors in Figure 2. The vectors of interest have three coordinates corresponding to singleton, doubleton, and tripleton mutations. Note that any vector of probabilities must be nonnegative and must sum to one. This means we are constrained to the triangle with vertices $\left(1,0,0\right),\left(0,1,0\right),$ and $\left(0,0,1\right).$ We can ignore the third coordinate since it will always be one minus the others. This triangle is depicted in yellow in Figure 2. One might naively hope that every one of these probability vectors is achievable as the expected SFS of some demography.

Figure 2.

Eliminating candidate normalized SFS vectors for ${\widehat{\Xi }}_{4,2}.$This image considers candidate vectors and eliminates them for different reasons. A priori, any vector adding up to one is a possible SFS. This is represented by the yellow triangle whose third coordinate (not shown) is simply one minus the sum of the other two. Sargsyan and Wakeley (2008) showed that the SFS is nondecreasing, ruling out any vectors outside the blue triangle. Furthermore, they showed that the SFS is convex, therefore ${\xi }_{4,2}\le \frac{1}{2}\left({\xi }_{4,1}+{\xi }_{4,3}\right),$ ruling out anything outside the red triangle. Finally, our algebraic analysis of the expected SFS for a piecewise-constant demography with two epochs rules out vectors outside the green region at the right.

A result proved by Sargsyan and Wakeley (2008) is that SFS vectors must be nonincreasing—this means that we are left with the triangle with vertices $\left(1/3,1/3,1/3\right),$ $\left(1/2,1/2,0\right),$ and $\left(1,0,0\right).$ This is depicted in blue in Figure 2. They further proved that the SFS is convex. This implies that the second coordinate is less than the average of the other two. This further cuts down our possibilities to the triangle with vertices $\left(1/3,1/3,1/3\right),$ $\left(2/3,1/3,0\right),$ and $\left(1,0,0\right),$ depicted in red in Figure 2. If we want SFS vectors for population size histories with two constant pieces, we are further constrained to the green region, which we will describe algebraically later.

We will be able to completely describe the shape of all SFS for a sample size of four using algebraic formulas for the boundary. In fact, we will show that to find all possible SFS for a sample size of four it is sufficient to consider piecewise-constant functions with at most three constant pieces. Furthermore, we will use tools from convex and algebraic geometry to extend our intuition from this small case study to the SFS for all sample sizes.

Summary of main results

Studying the geometry of the set of expected SFS will address both of the areas discussed above:

1. Explaining the pathologies in SFS-based inference.

2. Describing the full set of SFS for a fixed sample size.

In this way, we can help researchers understand why fitting parameters to certain demographic models will lead to runaway behavior. We also enable researchers to reject a null hypothesis of a single panmictic population under neutrality.

Our main result is Theorem 8 which focuses on piecewise-constant demographies. It shows that for every sample size n, there is a crucial threshold in demographic complexity, which we denote ${\kappa }_n$. If we are fitting to a demographic model with fewer than ${\kappa }_n$ constant pieces, then the set of all SFS will be nonconvex and we must expect pathological behavior as described above. However, once we allow for ${\kappa }_n$ constant pieces , we get the full set of SFS for all demographies. Proving that this set is convex is left for later work.

Piecewise-Constant Demographies

In this section, we will define two sets: one of them will be the set of expected SFS for piecewise-constant population size histories. As described in the Introduction, this is an important set for inference. The other set is the set of expected coalescence vectors; this is not as commonly used as the SFS, but it helps us build a strong understanding of the SFS. This is because it is related to the SFS by a simple transformation and yet it is much easier to formulate.

Let ${\text{Π}}_k$ be the set of piecewise-constant population size functions with k pieces. Any population size function in ${\text{Π}}_k$ is described by $2k-1$ positive numbers, representing the k population sizes $\left(y_1,\dots ,y_k\right)$ and the $k-1$ time points $\left(t_1,\dots ,t_{k-1}\right)$ when the population size changes. Let ${\Xi }_{n,k},$ which we call the $\left(n,k\right)$ SFS “manifold,” denote the set of all expected SFS vectors for a sample of size n that can be generated by population size functions in ${\text{Π}}_k.$ (Note that the sets ${\Xi }_{n,k}$ and ${\mathcal{C}}_{n,k}$ are not technically manifolds; they would be more accurately described as semialgebraic sets. However, for expository purposes, we use the widely known term manifold.) Similarly, let ${\mathcal{C}}_{n,k},$ called the $\left(n,k\right)$-coalescence manifold, denote the set of all vectors $c=\left(c_2,\dots ,c_n\right)$ giving the expected first coalescence times of samples of size $2,\dots ,n$ for population size functions in ${\text{Π}}_k.$ Let ${\widehat{\Xi }}_{n,k}$ and ${\widehat{\mathcal{C}}}_{n,k},$ respectively, be equal to the normalization of all points in ${\Xi }_{n,k}$ and ${\mathcal{C}}_{n,k}$ by their ${\ell }_1$ norms (i.e., the sums of their coordinates). Note that both manifolds live in ${\mathrm{ℝ}}^{n-1}$ and their normalized versions live in the $\left(n-2\right)$-dimensional simplex ${\Delta }^{n-2};$ this is the set of nonnegative vectors in ${\mathrm{ℝ}}^{n-1}$ whose coordinates sum to 1.

Now that we have defined our basic objects of study, we can describe the remainder of the article: First, we provide a complete geometric picture of the ${\Xi }_{4,k}$ SFS manifold describing the expected SFS for samples of size $n=4$ under piecewise-constant population size functions with an arbitrary number k of pieces. We make explicit the map between regions of the demographic model space and the corresponding probability vectors, and this will foreshadow some of the difficulties with population size inference in practice. Next, we develop a characterization of the space of expected SFS for arbitrary population size histories. In particular, we show that for any sample size n, there is a finite integer ${\kappa }_n$ such that the expected SFS for a sample of n under any population size history can be generated by a piecewise-constant population size function with at most ${\kappa }_n$ epochs. Stated another way, we show that the ${\Xi }_{n,{\kappa }_n}$ SFS manifold contains the expected SFS for all possible population size histories, no matter how complicated their functional forms. We establish bounds on ${\kappa }_n$ that are linear in n and along the way prove some interesting results regarding the geometry of the general ${\Xi }_{n,k}$ SFS manifold.

Before proceeding further, we state a proposition regarding the structure of the map from ${\text{Π}}_k$ to ${\mathcal{C}}_{n,k},$ which we will call $\chi \left(\overrightarrow{x},\overrightarrow{y}\right);$ the vector of $k-1$ transformed break points is denoted by $\overrightarrow{x}=\left(x_1,\dots ,x_{k-1}\right)$ and defined below, while the vector of population sizes in the k epochs is denoted by $\overrightarrow{y}=\left(y_1,\dots ,y_k\right).$ It turns out that we can formulate the expected coalescence times as polynomial functions of the x and y variables. Two different ways of writing those functions down will give us two perspectives on their shape. All proofs of the results presented in this article are deferred to the Appendix.

Proposition 1. Fix a piecewise-constant population size function in ${\text{Π}}_k$ with epochs $[t_0,t_1),$$[t_1,t_2),$$[t_{k-1},t_k),$ where $0=t_0<t_1<\cdots <t_{k-1}<t_k=\mathrm{\infty },$ and which has constant population size value $y_j$ in the epoch $[t_{j-1},t_j)$ for $j=1,\dots ,k$. Let $x_j=\exp \left[-\left(t_j-t_{j-1}\right)/y_j\right]$ for $j=1,\dots ,k,$ where $x_k=0$ (corresponding to time $T=\mathrm{\infty }$), and define $x_0=1$ (corresponding to time $T=0$) for convenience. The vectors $\left(x_1,\dots ,x_{k-1},y_1,\dots ,y_k\right),$ where $0<x_j<1$ and $y_j>0$ for all j, (uniquely) identify the population size functions in ${\text{Π}}_k$ and they satisfy both of the following equations:

$$\left[\begin{array}{ccc}{x}_0\left(1-x_1\right)& \dots & \left({\displaystyle \prod }_{i=0}^{k-1}{x}_i\right)\left(1-x_k\right)\\ \frac{1}{3}{x}_0^3\left(1-x_1^3\right)& \dots & \frac{1}{3}\left({\displaystyle \prod }_{i=0}^{k-1}{x}_i^3\right)\left(1-x_k^3\right)\\ ⋮& \ddots & ⋮\\ \frac{1}{\left(\begin{array}{c}n\\ 2\end{array}\right)}{x}_0^{\left(\begin{array}{c}n\\ 2\end{array}\right)}\left(1-x_1^{\left(\begin{array}{c}n\\ 2\end{array}\right)}\right)& \dots & \frac{1}{\left(\begin{array}{c}n\\ 2\end{array}\right)}\left({\displaystyle \prod }_{i=0}^{k-1}{x}_i^{\left(\begin{array}{c}n\\ 2\end{array}\right)}\right)\left(1-x_k^{\left(\begin{array}{c}n\\ 2\end{array}\right)}\right)\end{array}\right]\left[\begin{array}{c}{y}_1\\ y_2\\ ⋮\\ y_k\end{array}\right]=\left[\begin{array}{c}{c}_2\\ c_3\\ ⋮\\ c_n\end{array}\right],$$ (2)

$$\left[\begin{array}{cccc}1& x_1& \dots & {\displaystyle \prod }_{i=0}^{k-1}{x}_i\\ \frac{1}{3}& \frac{1}{3}{x}_1^3& \cdots & \frac{1}{3}{\displaystyle \prod }_{i=0}^{k-1}{x}_i^3\\ ⋮& ⋮& \ddots & ⋮\\ \frac{1}{\left(\begin{array}{c}n\\ 2\end{array}\right)}& \frac{1}{\left(\begin{array}{c}n\\ 2\end{array}\right)}{x}_1^{\left(\begin{array}{c}n\\ 2\end{array}\right)}& \cdots & \frac{1}{\left(\begin{array}{c}n\\ 2\end{array}\right)}{\displaystyle \prod }_{i=0}^{k-1}{x}_i^{\left(\begin{array}{c}n\\ 2\end{array}\right)}\end{array}\right]\left[\begin{array}{c}{y}_1\\ y_2-y_1\\ ⋮\\ y_k-y_{k-1}\end{array}\right]=\left[\begin{array}{c}{c}_2\\ c_3\\ ⋮\\ c_n\end{array}\right],$$ (3)

where $c_m$ is the expected first coalescence time for a sample of size m, as defined in (1).

These two formulations provide two different ways of looking at the coalescence manifold ${\mathcal{C}}_{n,k}:$

1. In (2), the left-hand matrix called $M_1\left(n,k\right)$ has each column of the same form with two parameters; this indicates they all live in a two-dimensional surface. Imagine, for example, the surface of the earth. There are two degrees of freedom: north–south and east–west. Here, too, specifying the value of each column, regardless of the value of n, is dependent on two numbers. Explicitly, each column is given by $f_n\left(a,b\right)=\left(a\left(1-b\right),\dots ,a^{\left(\begin{array}{c}n\\ 2\end{array}\right)}\left(1-b^{\left(\begin{array}{c}n\\ 2\end{array}\right)}\right)/\left(\begin{array}{c}n\\ 2\end{array}\right)\right)$ for some inputs a and b.

   • Additionally, the vector $\left(y_1,\dots ,y_k\right)$ has all positive entries. That means that, when we combine columns from our surface, they will not cancel in unexpected ways due to negative coefficients. The set of positive combinations of a set of points is called a cone, and it is very nicely behaved geometrically. This means that the vector $c=\left(c_2,\dots ,c_n\right)$ is contained in the cone over the surface described by the columns of $M_1.$

2. In (3), the left-hand matrix called $M_2\left(n,k\right)$ has each column of the same form with one parameter; this indicates they all live on a curve. Like a train on a track, this has one degree of freedom, only forward–backward. Explicitly, each column is given by $g_n\left(a\right)=\left(a,\dots ,a^{\left(\begin{array}{c}n\\ 2\end{array}\right)}/\left(\begin{array}{c}n\\ 2\end{array}\right)\right)$ for some input a.

   • The vector $\left(y_1,y_2-y_1,\dots ,y_k-y_{k-1}\right)$ on the left-hand side has entries with possibly negative coordinates. So the vector $c=\left(c_2,\dots ,c_n\right)$ is contained in the linear span of the curve described by the columns of $M_2.$ Unfortunately, a linear span is not quite as nicely behaved as a cone. Still, this formulation gains the simplicity of having one degree of freedom instead of two.

Proposition 1 gives us the algebraic mappings that will serve as our objects of interest. Since the SFS manifold is simply a linear transformation of the coalescence manifold, we will use these maps as our entry into understanding the SFS manifold.

The ${\Xi }_{4,k}$ SFS Manifold

A toy model

The first in-depth study will involve the set of all possible expected SFS for a sample size of four. We choose $n=4$ for a number of reasons: First, the cases of sample sizes of two and three are not interesting. When we only have two haplotypes, there is only one entry in the SFS vector, i.e., singletons. The resulting set of possible expected SFS is just the set of all positive numbers. When we have three haplotypes, it is only slightly better. Because there must be fewer doubletons than singletons, the possible expected SFS is somewhere in the wedge between $0$ and $45°$ from the origin; this turns out to be the only constraint.

Second, when $n=4,$ the SFS manifold lives in ${\mathrm{ℝ}}^3,$ which can be nicely visualized, and the normalized SFS manifold lives in the triangle with vertices $\left(1,0,0\right),\left(0,1,0\right),$ and $\left(0,0,1\right).$ Finally, as observed in Proposition 1, the most interesting phenomena in SFS manifolds of any dimension are fundamentally phenomena of curves and surfaces. These are already captured in the $n=4$ case.

For the sake of completeness, we begin by formally describing the coalescence manifolds ${\mathcal{C}}_{n,k}$ for the trivial cases of $n=2$ and $n=3.$

Proposition 2. We list some basic results on the coalescence manifolds ${\mathcal{C}}_{n,k},$ with sample size n and k population epochs, for small values of $\left(n,k\right):$

1. The manifold ${\mathcal{C}}_{n,1}=\left\{\lambda \cdot \left(1,\frac{1}{3},\dots ,\frac{1}{\left(\begin{array}{c}n\\ 2\end{array}\right)}\right):\lambda >0\right\},$ for all n.

2. The manifold ${\mathcal{C}}_{2,k}={\mathcal{C}}_{2,1}=\left\{a:a>0\right\},$ for all $k\ge 1.$

3. The manifold ${\mathcal{C}}_{3,k}={\mathcal{C}}_{3,2}=\left\{\left(a,b\right):a>0 \text{and} 0<b<a\right\},$ for all $k\ge 2.$

Note that from (2) and (3) for $\chi \left(\overrightarrow{x},\overrightarrow{y}\right),$ it follows that $\chi \left(\overrightarrow{x},a\overrightarrow{y}\right)=a\chi \left(\overrightarrow{x},\overrightarrow{y}\right)$ for $a>0.$ In words, rescaling the population sizes in each epoch by a constant a also rescales the first coalescence times by a. This implies that every point in the coalescence manifold ${\mathcal{C}}_{n,k}$ generates a full ray contained in the ${\mathcal{C}}_{n,k}$ coalescence manifold. Another consequence is that the normalized coalescence manifold ${\widehat{\mathcal{C}}}_{n,k}$ is precisely the intersection of the coalescence manifold ${\mathrm{C}}_{n,k}$ with the simplex ${\Delta }^{n-2}.$

With that justification, we begin to consider the normalized coalescence manifold ${\widehat{\mathcal{C}}}_{4,k}$ living in the simplex. As stated in Proposition 2, ${\mathcal{C}}_{4,1}$ is a ray, which implies that ${\widehat{\mathcal{C}}}_{4,1}$ is a single point. We now characterize the set ${\widehat{\mathcal{C}}}_{4,2}.$ Again, this is the set of possible SFS for two-epoch, piecewise-constant population size histories considered as a subset of all vectors summing to one.

Proposition 3. The manifold ${\widehat{\mathcal{C}}}_{4,2},$ describing normalized expected times to first coalescence for sample size 4 and two population epochs, is a two-dimensional subset of the 2-simplex which can be described as the union of the point ${\widehat{\mathcal{C}}}_{4,1}$ with the interiors of the convex hulls of two curves ${\gamma }_1$ and ${\gamma }_2.$ The curves are parametrized as follows:

$${\gamma }_1=\{\left(\frac{6}{6+2t^2+t^5},\frac{2t^2}{6+2t^2+t^5},\frac{t^5}{6+2t^2+t^5}\right):0<t<1\},$$

$$\text{and }{\gamma }_2=\left\{\left(\frac{6}{6+2{\left[2\right]}_t+{\left[5\right]}_t},\frac{2{\left[2\right]}_t}{6+2{\left[2\right]}_t+{\left[5\right]}_t},\frac{{\left[5\right]}_t}{6+2{\left[2\right]}_t+{\left[5\right]}_t}\right):0<t<1\right\},$$

where ${\left[n\right]}_t$ denotes $1+\cdots +t^n.$

This set has some highly unpleasant geometry. First of all, the set is nonconvex; topologically, it is also neither closed nor open because most of the boundary is excluded with the exception of the point $\left(2/3,2/9,1/9\right).$ The set is visualized in Figure 3A.

Figure 3.

Coalescence and SFS manifolds for sample of size four and two population epochs. (A) The coalescence manifold ${\mathcal{C}}_{4,2}$ is the union of red and green cones. The 2-simplex, shaded in blue, intersects ${\mathcal{C}}_{4,2}$ in the normalized coalescence manifold ${\widehat{\mathcal{C}}}_{4,2}.$ The green region corresponds to recent-small, ancient-large demographies; the red region to recent-large, ancient-small demographies. (B) The SFS manifold ${\text{Ξ}}_{4,2}$ is the union of red and green cones. The 2-simplex intersects ${\text{Ξ}}_{4,2}$ in the normalized SFS manifold ${\widehat{\text{Ξ}}}_{4,2}.$ Here, too, the green region corresponds to small-then-large demographies; the red region to large-then-small demographies. As mentioned earlier, ${\text{Ξ}}_{4,2}$ is obtained from ${\mathcal{C}}_{4,2}$ by a linear transformation.

To precisely illustrate the geometry of $\chi \left(\overrightarrow{x},\overrightarrow{y}\right),$ we will consider how contours in the domain map to contours in the image. Specifically, we plot the images of lines with fixed values of $x_1,$ respectively fixed values of $\left(y_1,y_2\right),$ to ${\mathcal{C}}_{4,2}$ in the 2-simplex. The resulting contours are pictured in Figure 4.

Figure 4.

Fixed-time and fixed-size contours in ${\widehat{\text{C}}}_{4,2}$. (A) The blue line segments correspond to the image of ${\chi }_{4,2}\left(x*,\overrightarrow{y}\right)$ where $x*$ is a constant fixing the break point between the two demographies. The other input $\overrightarrow{y}=\left(y_1,y_2\right)$ varies over all positive vectors, though scaled $\overrightarrow{y}$ vectors point to the same normalized value. As $y_1/y_2\to 0,$ the image approaches ${\gamma }_1$ and as $y_2/y_1\to 0,$ the image approaches ${\gamma }_2.$ (B) The blue curves correspond to the image of ${\chi }_{4,2}\left(x,\overrightarrow{y}*\right)$ where $\overrightarrow{y}*$ is a fixed vector indicating the population values and x takes all values in $\left(0,1\right).$ The end points 0 and 1 correspond to break points at $\mathrm{\infty }$ and 0, respectively. For $y_1^{*}<y_2^{*},$ x traces a loop in the green region; for $y_1^{*}>y_2^{*},$ x traces a loop in the red region.

Finally, we consider how the map χ acts on the boundaries of the domain. To aid visualization, we limit the inputs to $x_1$ and $y_1/y_2,$ since all rescalings of $y_1$ and $y_2$ by the same positive constant while keeping $x_1$ fixed map to the same normalized coalescence vector. The resulting map is illustrated in Figure 5.

Figure 5.

Pairing the boundaries of demography space and ${\widehat{\mathrm{C}}}_{4,2}.$ (A) The domain of ${\chi }_{4,2}.$ Note that for fixed $y_1/y_2,$ the normalized coalescence vector is the same. (B) The normalized SFS manifold ${\widehat{\Xi }}_{4,2}$ projected onto its first two coordinates. (C) The normalized coalescence manifold ${\widehat{\mathcal{C}}}_{4,2}$ projected onto its first two coordinates. The red square on the left, corresponding to $y_1>y_2,$ maps to the red regions on the right; the green square on the left, corresponding to $y_2<y_1,$ maps to the green regions on the right. The black line segments on the left [corresponding to $y_1/y_2=1;$ $y_2<y_1$ and $x_1=0$ (equivalently $t_1=\mathrm{\infty }$); $y_2>y_1$ and $x_1=1$ (equivalently $t_1=0$)] all map to the central black points on the right, since they each mimic a constant demography. The green line corresponding to $y_1=0$ maps to the curve ${\beta }_1$ in ${\widehat{\Xi }}_{4,2}$ and the curve ${\gamma }_1$ in ${\widehat{\mathcal{C}}}_{4,2};$ the red line corresponding to $y_2=0$ maps to the curve ${\beta }_2$ in ${\widehat{\Xi }}_{4,2}$ and the curve ${\gamma }_2$ in ${\widehat{\mathcal{C}}}_{4,2}.$ The orange point $\left(x_1=1,y_2=0\right)$ maps to $\left(1/3,1/3,1/3\right)$ in ${\widehat{\mathcal{C}}}_{4,2}$ and maps to $\left(1,0,0\right)$ in ${\widehat{\Xi }}_{4,2}.$ The blue point $\left(x_1=0,y_1=0\right)$ maps to $\left(1,0,0\right)$ in ${\widehat{\mathcal{C}}}_{4,2}$ and $\left(1/3,1/3,1/3\right)$ in ${\widehat{\Xi }}_{4,2}.$ The remaining aqua and violet segments map to the segments of the same color.

We note that the map fails to be one-to-one within the domain only when $y_1/y_2=1;$ this is also in the preimage of the point $\left(2/3,2/9,1/9\right)\in {\widehat{\mathcal{C}}}_{4,2.}.$ The inverse function theorem implies that on the complement of $y_1/y_2=1,$ the map is a homeomorphism (a map that preserves topological features like number of components). This is consistent with our observation that the two rectangles in Figure 5A correspond to the two envelopes in Figure 5C. Now, we consider demographies with more than two epochs. This proposition implies that any expected SFS for a sample size of four coming from a single panmictic population under neutrality, regardless of the true population size history, is equal to the expected SFS for some piecewise-constant history with only three pieces. It also shows that all of these SFS vectors live inside of the convex hull of one curve.

Proposition 4. For all values $k\ge 3,$ the manifold ${\widehat{\mathcal{C}}}_{4,k}={\widehat{\mathcal{C}}}_{4,3},$ and ${\widehat{\mathcal{C}}}_{4,3}$ is the interior of the convex hull of the following curve:

$${\gamma }_3=\left\{\left(\frac{1}{1+t^2+t^5},\frac{t^2}{1+t^2+t^5},\frac{t^5}{1+t^2+t^5}\right):0<t<1\right\}.$$

As we can see from Proposition 4, ${\widehat{\mathcal{C}}}_{4,3}$ is open and convex; however, we lose one useful property of the normalized map $\widehat{\chi }:{\mathrm{ℝ}}^3\to {\widehat{\mathcal{C}}}_{4,2}.$ Specifically, let $\widehat{{\chi }^{\prime }}:{\mathrm{ℝ}}^2\to {\widehat{\mathcal{C}}}_{4,2}$ be given by $\widehat{{\chi }^{\prime }}\left(x_1,y_1\right)=\widehat{\chi }\left(x_1,y_1,1\right),$ noting that $\widehat{\chi }\left(x_1,\lambda y_1,\lambda y_2\right)=\widehat{\chi }\left(x_1,y_1,y_2\right)$ for $\lambda >0.$ Under this definition $\widehat{{\chi }^{\prime }}$ is generically one-to-one (i.e., one-to-one away from a set of measure zero). Meanwhile, the analogous construction $\widehat{{\chi }^{\prime }}:{\mathrm{ℝ}}^4\to {\widehat{\mathcal{C}}}_{4,3},$ mapping the three-epoch demography with break points $\left(x_1,x_2\right)$ and population sizes $\left(y_1,y_2,1\right)$ to the corresponding normalized coalescence vector has two-dimensional preimages, generically. For this reason, contour images do not lend themselves to easy description.

However, as a heuristic, we can choose a distinguished member of this preimage with nice properties. In the orange region adjacent to ${\beta }_3$ depicted in Figure 6, every preimage contains a limit demography with first and third epochs set to zero, and second epoch set to one. This can be thought of as a demography with a population boom in the second epoch. In the blue region adjacent to the line segment from $\left(1/3,1/3,1/3\right)$ to $\left(1,0,0\right),$ every preimage contains a limit demography with second epoch set to zero. This corresponds to a demography with a population bottleneck in the second epoch. Because the set of demographies mapping to each point is two dimensional, this does not describe all demographies characterized by a chosen SFS, but it does give us intuition for the types of demographies to expect.

Figure 6.

Regions of ${\widehat{\Xi }}_{4,3}$ and sample demographies. The image depicts ${\text{Ξ}}_{4,3}$ partitioned into different colored regions. The purple point in the center is the SFS corresponding to the constant demography. The green region contains SFS corresponding to recent-small, ancient-large demographies. The red region corresponds to recent-large, ancient-small demographies. The orange region contains SFS corresponding to three-epoch demographies with a boom in the second epoch. The blue region contains SFS corresponding to three-epoch demographies with a bottleneck in the second epoch. These are not the unique demographies mapping to each region of ${\text{Ξ}}_{4,3},$ but they depict, in some sense, the simplest demographies yielding those SFS.

We can also describe the image of the map $\widehat{{\chi }^{\prime }}:{\mathrm{ℝ}}^4\to {\widehat{\mathcal{C}}}_{4,3}$ on the boundaries of our domain. The easiest way to visualize the map is first to understand how the time variables affect the value of the columns of $M_1\left(4,3\right)$ and to view the y variables as specifying points in the convex hull of those three columns. The boundaries of the square $\left(x_1,x_2\right)\in \left[0,1\right]\times \left[0,1\right]$ map the columns (after rescaling to the simplex) as follows:

$$\begin{array}{ccc}{x}_1=0& \mapsto & \left[\begin{array}{c}6/9\\ 2/9\\ 1/9\end{array}\left|\begin{array}{c}1\\ 0\\ 0\end{array}\right|\begin{array}{c}1\\ 0\\ 0\end{array}\right],\\ x_1=1& \mapsto & \left[\begin{array}{c}1/3\\ 1/3\\ 1/3\end{array}\left|\begin{array}{c}|\\ {\gamma }_2\left(x_2\right)\\ |\end{array}\right|\begin{array}{c}|\\ {\gamma }_1\left(x_2\right)\\ |\end{array}\right],\\ x_2=0& \mapsto & \left[\begin{array}{c}|\\ {\gamma }_2\left(x_1\right)\\ |\end{array}\left|\begin{array}{c}|\\ {\gamma }_1\left(x_1\right)\\ |\end{array}\right|\begin{array}{c}1\\ 0\\ 0\end{array}\right],\\ x_2=1& \mapsto & \left[\begin{array}{c}|\\ {\gamma }_2\left(x_1\right)\\ |\end{array}\left|\begin{array}{c}|\\ {\gamma }_3\left(x_1\right)\\ |\end{array}\right|\begin{array}{c}|\\ {\gamma }_1\left(x_1\right)\\ |\end{array}\right].\end{array}$$

The case of $x_2=1$ is the most interesting: when we fix $y_1=y_3=0$ and $y_2=1,$ we obtain the boundary curve ${\gamma }_3\left(t\right).$ Note that $x_2=1$ corresponds to a second epoch of length zero. The intuition is that very short population booms at the second epoch lead to coalescence vectors close to ${\gamma }_3.$ The maps encoded by a general column of $M_1\left(4,k\right)$ correspond to the interior of the orange region in Figure 7A. Adding in convex combinations of points gives the lined region, which is the remainder of ${\mathcal{C}}_{4,3};$ this is discussed more rigorously in the Appendix. When the number of epochs k steps higher, all columns of $M_1\left(4,k\right)$ still map to the same region of the simplex, so ${\mathcal{C}}_{4,k}$ will still be contained in this convex hull. The region ${\mathcal{C}}_{4,3}$ is depicted in Figure 7A.

Figure 7.

Coalescence and SFS manifolds for a sample of size four and three population epochs. (A) The coalescence manifold ${\mathcal{C}}_{4,3}$ is the entire yellow and orange region. The 2-simplex, shaded in blue, intersects ${\mathcal{C}}_{4,3}$ in the normalized coalescence manifold ${\widehat{\mathcal{C}}}_{4,3}.$ The orange region of ${\widehat{\mathcal{C}}}_{4,3},$ bounded by ${\gamma }_1,{\gamma }_2,$ and ${\gamma }_3,$ is the image of the surface described by the columns of $M_1\left(4,3\right),$ while the yellow region adds in vectors gained by using convex combinations. (B) The SFS manifold ${\text{Ξ}}_{4,3}$ is the entire yellow and orange region. The 2-simplex intersects ${\text{Ξ}}_{4,3}$ in the normalized SFS manifold ${\widehat{\Xi }}_{4,3}.$ The SFS manifold ${\text{Ξ}}_{4,3}$ is obtained from ${\mathcal{C}}_{4,3}$ by a linear transformation. The orange region of ${\widehat{\text{Ξ}}}_{4,3},$ bounded by ${\beta }_1,{\beta }_2,$ and ${\beta }_3,$ is the image of the surface described by the columns of $M_1\left(4,3\right),$ while the yellow region adds in vectors gained by using linear combinations.

As mentioned earlier, the SFS manifold ${\Xi }_{n,k}$ is merely a linear transformation of ${\mathcal{C}}_{n,k};$ however, since it is of interest in its own right, we include the formulas for ${\Xi }_{4,k}$ analogous to those derived in this section.

Proposition 5. The following hold for the normalized $\left(4,k\right)$ SFS manifold:

$${\widehat{\Xi }}_{4,1}=\left(\frac{6}{11},\frac{3}{11},\frac{2}{11}\right).$$

${\widehat{\Xi }}_{4,2}$ is the union of ${\widehat{\Xi }}_{4,1}$ with the convex hulls of two curves:

$$\begin{array}{l}{\beta }_1=\left\{\left(\frac{18+10t^2+2t^5}{54+t^5},\frac{18-3t^5}{54+t^5},\frac{18-10t^2+2t^5}{54+t^5}\right):0<t<1\right\},\\ {\beta }_2=\left\{\left(\frac{18+10{\left[2\right]}_t+2{\left[5\right]}_t}{54+{\left[5\right]}_t},\frac{18-3{\left[5\right]}_t}{54+{\left[5\right]}_t},\frac{18-10{\left[2\right]}_t+2{\left[5\right]}_t}{54+{\left[5\right]}_t}\right):0<t<1\right\}.\end{array}$$

Here, also, ${\left[n\right]}_t$ denotes $1+t+\cdots +t^n.$ Finally, ${\widehat{\Xi }}_{4,k}={\widehat{\Xi }}_{4,3}$ for all k, and ${\widehat{\Xi }}_{4,3}$ is the convex hull of ${\beta }_3,$ where

$${\beta }_3=\left\{\left(\frac{3+5t^2+2t^5}{9+t^5},\frac{3-3t^5}{9+t^5},\frac{3-5t^2+2t^5}{9+t^5}\right):0<t<1\right\}.$$

Visualizations of ${\Xi }_{4,2}$ and ${\Xi }_{4,3}$ may be found in Figure 3B and Figure 7B.

General properties

In this section, we examine the constant ${\kappa }_n,$ defined earlier as the smallest index for which ${\mathcal{C}}_{n,k}\subseteq {\mathcal{C}}_{n,{\kappa }_n}$ for all k. The tools for the proofs in this section come from algebraic geometry (for the derivation of the lower bound) and convex geometry (for the upper bound).

The gist of the algebraic geometry argument is that, under the $M_2\left(n,k\right)$ formulation, the manifold ${\mathcal{C}}_{n,k}$ can be seen to be part of another manifold built by a sequence of well-understood algebraic constructions. Details of this perspective are reserved for the proofs section in the Appendix.

Two concrete consequences follow from this observation:

1. The ability to compute all equations satisfied by ${\mathcal{C}}_{n,k}$ using computer algebra.

2. A formula for the dimension of the coalescence and SFS manifolds.

While the former is harder to explain without more setup, the latter can be simply stated: the dimension of the normalized coalescence manifold ${\widehat{\mathcal{C}}}_{n,k}$ is 0 when we have the constant demography $\left(k=1\right).$ If we allow k constant pieces, the manifold has dimension $2k-2$ unless $2k-2$ is greater than $n-2,$ the dimension of the simplex ${\Delta }^{n-2}.$ In that case, it has dimension $n-2.$

Proposition 6. The dimension of ${\widehat{\mathcal{C}}}_{n,k}$ is given by:

$$\dim {\widehat{\mathcal{C}}}_{n,k}=\{\begin{array}{ll}0,\hfill & k=1,\hfill \\ \min \left(2k-2,n-2\right),\hfill & \text{else}.\hfill \end{array}$$

In particular, ${\mathcal{C}}_{n,k}$ is a proper subset of ${\mathcal{C}}_{n,k+1}$ for $k<\lceil n/2\rceil .$

While Proposition 6 is useful for analyzing individual coalescence manifolds, it also leads to the observation that ${\kappa }_n\ge \lceil n/2\rceil ,$ since the inclusions are proper until that index. It is worth remarking that a slightly weaker lower bound of ${\kappa }_n\ge \lceil n/2\rceil$ follows immediately from the identifiability result of corollary 7 in Bhaskar and Song (2014), which states that for a piecewise-constant population size function with k pieces, the expected SFS of a sample of size $n\ge 2k$ suffices to uniquely identify the function.

We will illustrate how these algebraic ideas can be applied in the next case we have not seen, namely sample size $n=5.$

Example 7. Note that ${\widehat{\mathcal{C}}}_{5,1}=\left(30/48,10/48,5/48,3/48\right),$ by Proposition 2. We will use the new ideas above to describe ${\widehat{\mathcal{C}}}_{5,k}$ for higher values of k.

Since the normalized coalescence manifold has dimension $\min \left(2k-2,n-2\right),$ we know that ${\widehat{\mathcal{C}}}_{5,2}$ has dimension 2 inside of the 3-simplex; therefore, we anticipate that it will satisfy one equation, matching its codimension. The degree of the algebraic variety implies that this polynomial should have degree 8. Indeed, when we compute this equation using computer algebra software Macaulay2 (Grayson and Stillman 2002), we obtain a huge degree-8 polynomial with 105 terms, whose largest integer coefficient is $5,598,720.$ Finally, ${\widehat{\mathcal{C}}}_{5,3}$ is full-dimensional in the 3-simplex, so it will satisfy no algebraic equations relative to the simplex. It would be defined instead by the inequalities determining its boundary.

The convex geometry argument is more elementary. As we noted, the $M_1$ formulation is contained in the convex hull over the surface described by a general column of $M_1.$ Because the columns are related, our selection of points in the surface is not unrestricted. For this reason, it is not obviously equal to the convex hull. However, once we fix some collection of values $x_1,\dots ,x_k$ to be input in the formula for ${\mathcal{C}}_{n,k},$ we can use convex geometry for the resulting polytope. In particular, we use Carathéodory’s theorem [Carathéodory (1907) or Barvinok (2002), theorem 2.3], which states that for X a subset of ${\mathrm{ℝ}}^n,$ every $x\in \text{cone}\left(X\right)$ can be represented as a positive combination of vectors $x_1,\dots ,x_m\in X$ for some $m\le n.$

The argument, roughly, allows us to construct any point in that convex hull, with as few as $n+1$ points. This allows us to place the point in ${\mathcal{C}}_{n,j}$ for $j\le 2n-1.$ Since no new SFS are generated by using more than $2n-1$ epochs, we learn that ${\kappa }_n$ is bounded above by $2n-1.$

Combining the two bounds obtained in this section, we have the main theorem described in the Introduction.

Theorem 8. For any integer $n\ge 2,$ there exists a positive integer ${\kappa }_n$ such that ${\Xi }_{n,k}\subseteq {\Xi }_{n,{\kappa }_n}$ for all $k\ge 1.$ Furthermore, ${\kappa }_n$ satisfies

$$\lceil n/2\rceil \le {\kappa }_n\le 2n-1.$$

Additionally, ${\Xi }_{n,k}$ is nonconvex for all values of $2\le k<{\kappa }_n.$

This allows us to express the SFS from any piecewise-constant demography as coming from a demography with relatively few epochs. Because the SFS is an integral over the demography, the SFS from a general measurable demography can be uniformly approximated by a piecewise-constant demography with sufficiently many epochs. Our results imply that it can be precisely obtained by a demography with at most $2n-1$ epochs.

Data availability

The authors affirm that all data necessary for confirming the conclusions of the article are present within the article, figures, and tables.

Discussion

In this work, we characterized the manifold of expected SFS ${\Xi }_{n,k}$ generated by piecewise-constant population histories with k epochs, while giving a complete geometric description of this manifold for the sample size $n=4$ and $k=2$ epochs. This special case is already rich enough to shed light on the issues that practitioners can face when inferring population demographies from SFS data using popular software programs. While we demonstrated these issues in Figure 1 using the fastNeutrino program (Bhaskar et al. 2015), the issues we point out are inherent to the geometry of the SFS manifold and not specific to any particular demographic inference software. Our simulations showed that the demographic inference problem from SFS data can be fraught with interpretability issues, due to the sensitivity of the inferred demographies to small changes in the observed SFS data. These results can also be viewed as complementary to recent pessimistic minimax bounds on the number of segregating sites required to reliably infer ancient population size histories (Terhorst and Song 2015; Baharian and Gravel 2018).

Our investigation of piecewise-constant population histories also lets us show a general result that the expected SFS for a sample of size n under any population history can also be generated by a piecewise-constant population history with at most $2n-1$ epochs. This result could have potential applications for developing nonparametric statistical tests of neutrality. Most existing tests of neutrality using classical population genetic statistics such as Tajima’s D (Tajima 1989) implicitly test the null hypothesis of selective neutrality and a constant effective population size (Stajich and Hahn 2004). We have characterized the expected SFS of samples of size n under arbitrary population histories in terms of the expected SFS under piecewise-constant population histories with at most ${\kappa }_n$ epochs. As a result, the Kullback–Leibler (KL) divergence of an observed SFS ${\mathit{\xi }}_n^{\text{ obs}}$ to the expected SFS ${\mathit{\xi }}_n\left(\eta *\right)$ under the best-fitting, piecewise-constant population history $\eta *\in {\text{Π}}_{{\kappa }_n}$ with at most ${\kappa }_n\le 2n-1$ epochs is also equal (up to a constant shift) to the negative log-likelihood of the observed SFS ${\mathit{\xi }}_n^{\text{ obs}}$ under the best-fitting population size history without any constraints on its form. (This assumes the commonly used Poisson random field model where sites being analyzed are unlinked.) One can then use the KL divergence inferred by existing parametric demographic inference programs to create rejection regions for the null hypothesis of selective neutrality without having to make any parametric assumption on the underlying demography. Such an approach would also obviate the need for interpreting the inferred demography itself, since the space of piecewise-constant population histories is only being used to compute the best possible log-likelihood under any single population demographic model. This approach could serve as an alternative to recent works which first estimate a parametric demography using genome-wide sites, and then perform a hypothesis test in each genomic region using simulated distributions of SFS statistics like Tajima’s D under the inferred demography (Rafajlović et al. 2014). We leave the exploration of such tests for future work.

Appendix

Formula for A_n

Recall that the SFS can be related to times to first coalescence by the formula ${\mathit{\xi }}_n=A_{n}c.$ The formula for $A_n$ is given recursively in Polanski and Kimmel (2003) (equations 13–15) by the following formulas (with variable names changed for clarity):

$$\begin{array}{c}{\left(A_n\right)}_{b,2}=\frac{6}{n+1}\\ {\left(A_n\right)}_{b,3}=\frac{30\left(n-2b\right)}{\left(n+1\right)\left(n+2\right)}\\ {\left(A_n\right)}_{b,j+2}=-\frac{\left(1+j\right)\left(3+2j\right)\left(n-j\right)}{j\left(2j-1\right)\left(n+j+1\right)}{\left(A_n\right)}_{b,j}+\frac{\left(3+2j\right)\left(n-2b\right)}{j\left(n+j+1\right)}{\left(A_n\right)}_{b,j+1}.\end{array}$$

Proof of Proposition 1

First, we reduce the integral expression for $c_m$ to a finite sum; then we make appropriate manipulations until we arrive at the desired expressions.

Coalescence in the Wright–Fisher model is an inhomogeneous Poisson process with parameter $\left(\begin{array}{c}m\\ 2\end{array}\right)/\eta \left(t\right).$ Therefore, the probability density of first coalescence at time T is:

$$\mathrm{\mathbb{P}}\left(\text{no coalescence in }\left[0,T\right)\right)\mathrm{\mathbb{P}}\left(\text{coalescence at time }T\right)=\exp \left[-{\displaystyle {\int }_0^T\frac{\left(\begin{array}{c}m\\ 2\end{array}\right)}{\eta \left(t\right)}\text{d}t}\right]\frac{\left(\begin{array}{c}m\\ 2\end{array}\right)}{\eta \left(T\right)}\text{dt}.$$

Let $R_{\eta }\left(t\right)={\displaystyle {\int }_0^{T}1/\eta \left(t\right)\text{dt}}.$ To compute the expected time to first coalescence, we have the integral:

$$\begin{array}{c}{c}_m={\displaystyle {\int }_0^{\mathrm{\infty }}t\cdot \frac{\left(\begin{array}{c}m\\ 2\end{array}\right)}{\eta \left(t\right)}\exp \left[-\left(\begin{array}{c}m\\ 2\end{array}\right)R_{\eta }\left(t\right)\right]\text{dt}}\\ ={\displaystyle {\int }_0^{\mathrm{\infty }}\exp \left[-\left(\begin{array}{c}m\\ 2\end{array}\right)R_{\eta }\left(t\right)\right]\text{dt}\hspace{0.17em}\left(\text{integration by parts}\right)}.\end{array}$$

Substituting variables, $\tau =R_{\eta }\left(t\right),$ note that $\text{dt}=\eta \left(R^{-1}\left(\tau \right)\right)\text{d}\tau .$ Therefore, the integral becomes:

$$c_m={\displaystyle {\int }_0^{\mathrm{\infty }}\stackrel{\sim }{\eta }\left(\tau \right)\exp \left[-\left(\begin{array}{c}m\\ 2\end{array}\right)\tau \right]\text{d}\tau },$$

where $\stackrel{\sim }{\eta }\left(\tau \right)=\eta \left[R^{-1}\left(\tau \right)\right].$

The population size $\eta \left(t\right)$ is a piecewise-constant function, whose value is $\eta \left(t\right)={\eta }_j$ if $t_{j-1}\le t<t_j.$ As specified in Proposition 1, $t_0=0,$ $t_k=\mathrm{\infty },$ and $\left(y_1,\dots ,y_k\right)$ is the vector of population sizes. Observe that $\stackrel{\sim }{\eta }\left(\tau \right)$ is also piecewise constant. In particular,

$$\stackrel{\sim }{\eta }\left(\tau \right)=\{\begin{array}{ll}{y}_1,\hfill & 0\le \tau <\frac{t_1}{y_1},\hfill \\ y_2,\hfill & \frac{t_1}{y_1}\le \tau <\frac{t_1}{y_1}+\frac{t_2-t_1}{y_2},\hfill \\ ⋮\hfill & ⋮\hfill \end{array}$$

Let $s_j=t_j-t_{j-1}$ for brevity. The resulting formula is:

$$\stackrel{\sim }{\eta }\left(\tau \right)=y_j,\text{ for }{\displaystyle \sum _{k=1}^{j-1}\frac{s_k}{y_k}}\le \tau <{\displaystyle \sum _{k=1}^j\frac{s_k}{y_k}}.$$

We turn the integral into a sum of integrals on the constant epochs:

$$\begin{array}{c}{c}_m={\displaystyle {\int }_0^{\mathrm{\infty }}\stackrel{\sim }{y}\left(\tau \right)\exp \left[-\left(\begin{array}{c}m\\ 2\end{array}\right)\tau \right]\text{d}\tau }\\ ={\displaystyle \sum _{j=1}^k{\displaystyle {\int }_{{\displaystyle {\sum }_{}^{j-1}{s}_l/y_l}}^{{\displaystyle {\sum }_{}^j{s}_l/y_l}}{y}_j\exp \left[-\left(\begin{array}{c}m\\ 2\end{array}\right)\tau \right]\text{d}\tau }}\\ ={\displaystyle \sum _{j=1}^k{y}_j{\left[\frac{-1}{\left(\begin{array}{c}m\\ 2\end{array}\right)}\exp \left[-\left(\begin{array}{c}m\\ 2\end{array}\right)\tau \right]\right]}_{\tau ={\displaystyle {\sum }_{}^{j-1}{s}_l/y_l}}^{\tau ={\displaystyle {\sum }_{}^j{s}_l/y_l}}}\\ =\left\{\frac{1}{\left(\begin{array}{c}m\\ 2\end{array}\right)}{\displaystyle \sum _{j=1}^k{y}_j\left({\displaystyle \prod }_{l=1}^{j-1}\exp \left[-\left(\begin{array}{c}m\\ 2\end{array}\right)s_l/y_l\right]\right)}\left(1-\exp \left[-\left(\begin{array}{c}m\\ 2\end{array}\right)s_j/y_j\right]\right)\right\}.\end{array}$$

We now make the substitution $x_j=\exp \left(-s_j/y_j\right).$ Note that the old restriction $t_{j+1}>t_j>0$ becomes the new constraint $0<x_j<1.$ Our formula for the $c_m$ is now:

$$c_m=\frac{1}{\left(\begin{array}{c}m\\ 2\end{array}\right)}\left[{\displaystyle \sum _{j=1}^k{y}_j\left({\displaystyle \prod }_{l=1}^{j-1}{x}_l^{\left(\begin{array}{c}m\\ 2\end{array}\right)}\right)\left(1-x_j^{\left(\begin{array}{c}m\\ 2\end{array}\right)}\right)}\right].$$

Noting the linear form of this expression, we factor as a matrix multiplication:

$$\left[\begin{array}{cccc}1& & & \\ & \frac{1}{3}& & \\ & & \ddots & \\ & & & \frac{1}{\left(\begin{array}{c}n\\ 2\end{array}\right)}\end{array}\right]\times \left[\begin{array}{cccc}1& x_1& \dots & {\displaystyle \prod }_{i=1}^{k-1}{x}_i\\ 1& x_1^3& \cdots & {\displaystyle \prod }_{i=1}^{k-1}{x}_i^3\\ ⋮& ⋮& \ddots & ⋮\\ 1& x_1^{\left(\begin{array}{c}n\\ 2\end{array}\right)}& \cdots & {\displaystyle \prod }_{i=1}^{k-1}{x}_i^{\left(\begin{array}{c}n\\ 2\end{array}\right)}\end{array}\right]\times \left[\begin{array}{ccccc}1& 0& 0& \cdots & 0\\ -1& 1& 0& \ddots & 0\\ 0& -1& 1& \ddots & ⋮\\ ⋮& \ddots & \ddots & \ddots & 0\\ 0& \cdots & 0& -1& 1\end{array}\right]\times \left[\begin{array}{c}{y}_1\\ ⋮\\ y_k\end{array}\right]=\left[\begin{array}{c}{c}_2\\ ⋮\\ c_n\end{array}\right].$$

Combining the first three matrices yields (2); combining the first two and last two separately yields (3). $\square$

Proof of Proposition 2

We justify each equation in turn:

1. As mentioned in the Introduction, this is a classical result in population genetics, and can be derived directly from (3).

2. The inclusion ${\mathcal{C}}_{2,1}\subset {\mathcal{C}}_{2,k}$ is immediate, so we need only show that any $a\in {\mathcal{C}}_{2,k}$ satisfies $a>0.$ Using (2), a is written as a sum of products of strictly positive numbers; so ${\mathcal{C}}_{2,k}\subset {\mathcal{C}}_{2,1}.$

3. First, we show that ${\mathcal{C}}_{3,2}$ is the interior of the open cone spanned by $\left(1,0\right)$ and $\left(1,1\right).$ Fix $y_1=a/\left(1-x_1\right)$ (for a positive) and consider $\chi \left[x_1,a/\left(1-x_1\right),y_2\right]:$

   $$\chi \left(x_1,\frac{a}{1-x_1},y_2\right)=\left[\begin{array}{c}a+x_1{y}_2\\ \frac{1}{3}a\left(1+x_1+x_1^2\right)+\frac{1}{3}{x}_1^3{y}_2\end{array}\right]=a\left[\begin{array}{c}1\\ \frac{1}{3}\left(1+x_1+x_1^2\right)\end{array}\right]+x_1{y}_2\left[\begin{array}{c}1\\ \frac{1}{3}{x}_1^2\end{array}\right].$$

When $x_1\to 0,$ the second vector approaches $\left(1,0\right);$ when $x_1\to 1,$ the first vector approaches $\left(1,1\right).$ The vectors are in the interior of that cone for all other permissible values of $x_1$ and $y_2.$ To show that ${\mathcal{C}}_{3,k}={\mathcal{C}}_{3,2},$ note that for larger values of k, the same cone of vectors are produced. In particular, $\chi \left(x_1,\dots ,x_{k-1},y_1,\dots ,y_k\right)$ yields

$${\displaystyle \sum _{j=1}^{k-1}\left\{y_j\left({\displaystyle \prod _{i=1}^{j-1}{x}_i}\right)\left(1-x_j\right)\left[\begin{array}{c}1\\ \frac{1}{3}\left({\displaystyle \prod _{i=1}^{j-1}{x}_i^2}\right)\left(1+x_j+x_j^2\right)\end{array}\right]\right\}+y_k\left({\displaystyle \prod _{i=1}^{k-1}{x}_i}\right)\left[\begin{array}{c}1\\ \frac{1}{3}\left({\displaystyle \prod }_{i=1}^{k-1}{x}_i^2\right)\end{array}\right]}.$$

Clearly, the second coordinate of all vectors is bounded between zero and one.

Proof of Proposition 3

First we observe that ${\gamma }_1$ and ${\gamma }_2$ are normalizations of the curves defined by parameterizations $(t,\frac{1}{3}{t}^3,\frac{1}{6}{t}^6)$ and $\left(1-t,\frac{1}{3}(1-t^3),\frac{1}{6}(1-t^6)\right)$ where t is constrained to the open interval $\left(0,1\right).$

Now we claim that the definition in terms of the map $\chi \left(x,y\right)$ is equivalent to the definition in terms of these two curves. We can use the first formulation of χ to prove this:

$$\begin{array}{l}\chi \left(x_1,y_1,y_2\right)=y_1\left[\begin{array}{c}1-x_1\\ \left(1-x_1^3\right)/3\\ \left(1-x_1^6\right)/6\end{array}\right]+y_2\left[\begin{array}{c}{x}_1\\ x_1^3/3\\ x_1^6/6\end{array}\right]\\ =y_1\left[\begin{array}{c}1\\ \frac{1}{3}\\ \frac{1}{6}\end{array}\right]+\left(y_2-y_1\right)\left[\begin{array}{c}{x}_1\\ \frac{1}{3}{x}_1^3\\ \frac{1}{6}{x}_1^6\end{array}\right]\\ =\left(y_1-y_2\right)\left[\begin{array}{c}1-x_1\\ \frac{1}{3}\left(1-x_1^3\right)\\ \frac{1}{6}\left(1-x_1^6\right)\end{array}\right]+y_2\left[\begin{array}{c}1\\ \frac{1}{3}\\ \frac{1}{6}\end{array}\right].\\ \\ \end{array}$$

When $y_2=y_1,$ the image is the point $\left(2/3,2/9,1/9\right)=X$ as stated. When $y_2>y_1,$ we can use the left-hand expression to view the image as a point on the line segment between ${\mathcal{C}}_{4,1}$ and the curve $\left(t,t^3/3,t^6/6\right).$ When $y_2<y_1,$ the right-hand expression can be used to write the image as a point on the line segment between X and $\left[1-t,\left(1-t^3\right)/3,\left(1-t^6\right)/6\right].$ This means that the image of χ is contained in the regions and point specified.

To show that the reverse inclusion holds, we fix a point P in the interior of the convex hull of ${\gamma }_1$. By convexity, the line segment from X to P is contained in the region; continue in the direction $P-X$ until the line intersects the curve. This must occur because all points in the region are further from the bounding line than X. The point of intersection q is specified as $q={\gamma }_1\left(\tau \right)$ for some $\tau \in \left(0,1\right).$ By convexity, there exists some ρ such that $\rho {\mathcal{C}}_{4,1}+\left(1-\rho \right)q=P.$ Fixing $x_1=\tau ,$ $y_1=\rho ,$ and $y_2=1$ shows that P is in the image of χ. The same argument holds with slight variation for ${\gamma }_2.$

Proof of Proposition 4

The strategy to prove the equality of ${\mathcal{C}}_{4,3}$ and the cone over $\left\{t,t^3,t^6\right\}$ comes in two steps:

1. Show that the columns of $M_1\left(4,k\right)$ are always contained in the region R whose boundary is ${\gamma }_1{{\displaystyle \cup }}^{\text{}}{\gamma }_2{{\displaystyle \cup }}^{\text{}}{\gamma }_3.$

2. Divide the convex hull of R into two regions and show that each of these regions are included in ${\widehat{\mathcal{C}}}_{4,3}.$

First we demonstrate that the regions map precisely into R. We have already shown in the main text that the boundaries of $\left(0,1\right)\times \left(0,1\right)$ map to the boundaries of R under the mapping defined by $\left(x_1,x_2\right)\mapsto \left[x_1\left(1-x_2\right),x_1^3\left(1-x_2^3\right)/3,x_1^3\left(1-x_2^3\right)/6\right]\times 1/S,$ where S is the sum of the coordinates. We compute the Jacobian of this map explicitly in Macaulay2 (Grayson and Stillman 2002). The result is:

$$-1/6S^3{x}_1^9{\left(x_2-1\right)}^4\left(x_2^2+x_2+1\right)\left(x_2^2+3x_2+1\right).$$

Plainly, this is nowhere zero in our domain. The inverse function theorem then implies that the interior is contained in the image of the boundaries. This accomplishes Step 1 of our proof.

For Step 2, we divide the image into two regions:

1. The triangle defined by vertices $\left(1,0,0\right),$ $\left(2/3,2/9,1/9\right),$ and $\left(1/3,1/3,1/3\right),$ including the two edges $\left[\left(1/3,1/3,1/3\right),\left(2/3,2/9,1/9\right)\right]$ and $\left[\left(2/3,2/9,1/9\right),\left(1,0,0\right)\right].$

2. The remainder of the convex hull of R—explicitly, the interior of the region bounded by ${\gamma }_3$ and the line segment $\left[\left(1/3,1/3,1/3\right),\left(1,0,0\right)\right].$

To show that the triangle is included, let $x_2=\mathit{ϵ}\approx 0$ and let $x_1$ vary. The third column then sits arbitrarily close to $\left(1,0,0\right)$ and the first column traces out ${\gamma }_2.$ Set $y_2\approx 0$ and toggle $y_1$ and $y_3$ to obtain the full span, including the interior of the triangle, and the line segment $\left[\left(1/3,1/3,1/3\right),\left(2/3,2/9,1/9\right)\right].$ Set $x_1=1-\mathit{ϵ}\text{,}$ and the first column sits at $\left(1/3,1/3,1/3\right)$ while the third column traces out ${\gamma }_1.$ This catches the missing line segment.

For the remainder of the convex hull, fix a point P in this region. This point lies on a line segment between $\left(2/3,2/9,1/9\right)$ and some point Q in ${\gamma }_3.$ Suppose it is equal to $\rho \cdot \left(2/3,2/9,1/9\right)+\left(1-\rho \right)\cdot Q.$ Set $x_2=1-\mathit{ϵ}\approx 1.$ We can choose ɛ and $x_1$ so that the second column is arbitrarily close to P. Furthermore, observe that the first column is approximately equal to the point on ${\gamma }_2$ corresponding to $x_1$ and the third column is approximately the point on ${\gamma }_1$ corresponding to $x_1.$ Choosing $y_1=y_3=\rho$ and $y_2=1-\rho$ points us to

$$\rho \cdot \left(\left(\begin{array}{c}|\\ {\gamma }_1\left(x_1\right)\\ |\end{array}\right)+\left(\begin{array}{c}|\\ {\gamma }_2\left(x_1\right)\\ |\end{array}\right)\right)+\left(1-\rho \right)\cdot \left(\begin{array}{c}|\\ {\gamma }_3\left(x_1\right)\\ |\end{array}\right)=\rho \cdot \left(\begin{array}{c}2/3\\ 2/9\\ 1/9\end{array}\right)+(1-\rho )\cdot Q=P.$$

Proof of Proposition 5

This is a direct application of the linear map ${\text{W}}_4,$ computed as in Polanski and Kimmel (2003):

$${\text{W}}_4=\left(\begin{array}{ccc}6/5& 2& 4/5\\ 6/5& 0& -6/5\\ 6/5& -2& 4/5\end{array}\right).$$

Proof of Proposition 6

To prove the result about dimension, we show that ${\mathcal{C}}_{n,k}$ is a relatively open subset of a certain algebraic variety. Because the relevant operations are native to projective geometry, we transport our objects of interest in the obvious way to projective space. The same scaling properties that allow us to focus on the simplex also lead to good behavior in projective space.

Lemma 9. For $k\ge 2,$ the Zariski closure of ${\mathcal{C}}_{n,k}$ is the affine cone over $\mathcal{J}\left({\sigma }_{k-2}\left(C_n,p_n\right)\right),$ where:

1. The symbol $C_n$ denotes the projective curve defined by mapping $\left[s:t\right]$ to

   $$C_n=\left[{\left(\begin{array}{c}2\\ 2\end{array}\right)}^{-1}{s}^{\left(\begin{array}{c}n\\ 2\end{array}\right)-\left(\begin{array}{c}2\\ 2\end{array}\right)}{t}^{\left(\begin{array}{c}2\\ 2\end{array}\right)}:{\left(\begin{array}{c}3\\ 2\end{array}\right)}^{-1}{s}^{\left(\begin{array}{c}n\\ 2\end{array}\right)-\left(\begin{array}{c}3\\ 2\end{array}\right)}{t}^{\left(\begin{array}{c}3\\ 2\end{array}\right)}:\cdots :{\left(\begin{array}{c}n\\ 2\end{array}\right)}^{-1}{t}^{\left(\begin{array}{c}n\\ 2\end{array}\right)}\right].$$

2. The symbol $p_n$ is the projective point $\left[1:\frac{1}{3}:\frac{1}{6}:\cdots :\frac{1}{\left(\begin{array}{c}n\\ 2\end{array}\right)}\right].$

3. The operation $\mathcal{J}$ denotes the join of algebraic varieties.

4. The operation ${\sigma }_i(\cdot )$ denotes the i-th secant variety. Following Harris (2013), the i-th secant variety is the union of i-dimensional planes generated by $i+1$ points in the variety.

Proof of Lemma 9. The variety $\mathcal{J}\left({\sigma }_{k-2}\left(C_n\right),p_n)\right)$ is the image of the following map:

$$\psi \left(\overrightarrow{s},\overrightarrow{t},\overrightarrow{\lambda }\right)=\left(\begin{array}{cccc}1& s_1^{\left(\begin{array}{c}n\\ 2\end{array}\right)-1}{t}_1& \cdots & s_{k-1}^{\left(\begin{array}{c}n\\ 2\end{array}\right)}{t}_{k-1}\\ \frac{1}{3}& \frac{1}{3}{s}_1^{\left(\begin{array}{c}n\\ 2\end{array}\right)-3}{t}_1^3& \cdots & \frac{1}{3}{s}_{k-1}^{\left(\begin{array}{c}n\\ 2\end{array}\right)-3}{t}_{k-1}^3\\ ⋮& ⋮& \ddots & ⋮\\ \frac{1}{\left(\begin{array}{c}n\\ 2\end{array}\right)}& \frac{1}{\left(\begin{array}{c}n\\ 2\end{array}\right)}{t}_1^{\left(\begin{array}{c}n\\ 2\end{array}\right)}& \cdots & \frac{1}{\left(\begin{array}{c}n\\ 2\end{array}\right)}{t}_{k-1}^{\left(\begin{array}{c}n\\ 2\end{array}\right)}\end{array}\right)\left(\begin{array}{c}{\lambda }_0\\ {\lambda }_1\\ ⋮\\ {\lambda }_{k-1}\end{array}\right),$$

where $s_i$ and $t_i$ are not simultaneously zero, and λ is unrestricted.

Define the map $\phi :{\mathrm{ℝ}}^{2k-1}\to {\left({\mathrm{\mathbb{P}}}^1\right)}^{k-1}\times {\mathrm{ℝ}}^k$ sending $\left(x_1,\dots ,x_{k-1},y_1,\dots ,y_k\right)$ to

$$\left(\left[1:x_1\right],\left[1:x_1{x}_2\right],\dots ,\left[1:{\displaystyle \prod _{i=1}^{k-1}{x}_i}\right],y_1,y_1+y_2,\dots ,{\displaystyle \sum _{i=1}^k{y}_i}\right).$$

We can recast the expression in (3) as the composition $\psi \circ \phi .$ Based on this formulation, the set ${\mathcal{C}}_{n,k}$ is clearly contained in $\mathcal{J}\left({\sigma }_{k-2}\left(C_n\right),p\right).$ To demonstrate the equality of the Zariski closures, we only need to show that the dimensions match and that the variety is irreducible. Both joins and secants have the property that irreducible inputs yield irreducible outputs, so the variety of interest is irreducible. The image of ϕ is open in ${\left({\mathrm{\mathbb{P}}}^1\right)}^{k-1}\times {\mathrm{\mathbb{P}}}^{k-2},$ and the map ψ has deficient rank on a set of positive codimension. Therefore, the composition of $\psi \circ \phi$ has full dimension. This proves the Lemma. □

The i-th secant variety of an irreducible nondegenerate curve in ${\mathrm{\mathbb{P}}}^n$ has projective dimension given by $\min \left(2i+1,n\right)$ (Harris 2013, exercise 16.16). The curve $C_n$ is a toric transformation of a coordinate projection of the rational normal curve. The rational normal curve is nondegenerate and both of these operations preserve that property. This means our secant variety has projective dimension $\min \left(2\left(k-2\right)+1,n-2\right)=\min \left(2k-3,n-2\right).$ The join with a point adds 1 to the dimension of the variety, while the operation of passing to the affine cone adds 1 to the dimension of the variety and the ambient space. However, normalizing to the $\left(n-2\right)$-simplex subtracts 1 from both variety and ambient space again. This means that $\dim {\widehat{\mathcal{C}}}_{n,k}=\min \left(2k-2,n-2\right),$ assuming that $k\ge 2.$

Proof of Upper Bound in Theorem 8

Suppose a point $c$ is in ${\mathcal{C}}_{n,q}.$ By definition, this implies that there is a point $\left(x_1,\dots ,x_{q-1},y_1,\dots ,y_q\right)$ such that (2) yields

$$\left[\begin{array}{cccc}1-x_1& x_1\left(1-x_2\right)& \cdots & {\displaystyle \prod _{i=1}^{q-1}{x}_i}\\ \frac{1}{3}\left(1-x_1^3\right)& \frac{1}{3}{x}_1^3\left(1-x_2^3\right)& \cdots & \frac{1}{3}{\displaystyle \prod _{i=1}^{q-1}{x}_i^3}\\ \frac{1}{\left(\begin{array}{c}n\\ 2\end{array}\right)}\left(1-x_1^{\left(\begin{array}{c}n\\ 2\end{array}\right)}\right)& \frac{1}{\left(\begin{array}{c}n\\ 2\end{array}\right)}{x}_1^{\left(\begin{array}{c}n\\ 2\end{array}\right)}\left(1-x_2^{\left(\begin{array}{c}n\\ 2\end{array}\right)}\right)& \cdots & \frac{1}{\left(\begin{array}{c}n\\ 2\end{array}\right)}{\displaystyle \prod _{i=1}^{q-1}{x}_i^{\left(\begin{array}{c}n\\ 2\end{array}\right)}}\end{array}\right]\left[\begin{array}{c}{y}_1\\ ⋮\\ y_q\end{array}\right]=\left[\begin{array}{c}{c}_2\\ ⋮\\ c_n\end{array}\right].$$

Since the point $c$ is in the cone over the q columns of the matrix, Carathéodory’s theorem implies that it is also in the cone over some $n-1$ of the columns. Therefore, we can replace the vector $y_1,\dots ,y_q$ with ${y^{\prime }}_1,\dots ,{y^{\prime }}_q$ so that all but $n-1$ (or fewer) are zero.

Passing to the expression in (3), this gives us:

$$\left[\begin{array}{cccc}1& x_1& \cdots & {\displaystyle \prod _{i=1}^{q-1}{x}_i}\\ \frac{1}{3}& \frac{1}{3}{x}_1^3& \cdots & \frac{1}{3}{\displaystyle \prod _{i=1}^{q-1}{x}_i^3}\\ ⋮& ⋮& ⋮& ⋮\\ \frac{1}{\left(\begin{array}{c}n\\ 2\end{array}\right)}& \frac{1}{\left(\begin{array}{c}n\\ 2\end{array}\right)}{x}_1^{\left(\begin{array}{c}n\\ 2\end{array}\right)}& \cdots & \frac{1}{\left(\begin{array}{c}n\\ 2\end{array}\right)}{\displaystyle \prod _{i=1}^{q-1}{x}_i^{\left(\begin{array}{c}n\\ 2\end{array}\right)}}\end{array}\right]\left[\begin{array}{c}{y}_1^{\prime }\\ y_2^{\prime }-y_1^{\prime }\\ ⋮\\ y_q^{\prime }-y_{q-1}^{\prime }\end{array}\right]=\left[\begin{array}{c}{c}_2\\ ⋮\\ c_n\end{array}\right].$$

Since at most $n-1$ of the $y_i^{\prime }$ are nonzero, at most $2n-2$ of the entries of the vector at right are nonzero. We delete the columns of the X matrix corresponding to zero entries except the first column. A new sequence $\left(x_1^{\prime },\dots ,x_{2n-2}^{\prime }\right)$ may then be obtained from the ratio between the first entries in adjacent columns. The new sequence $y_1^{″},\dots ,y_{2n-1}^{″}$ is obtained by taking the sequence of partial sums of the vector.

Proof of Nonconvexity in Theorem 8

To prove this final result, we combine two properties already proven:

1. The manifold ${\mathcal{C}}_{n,k}$ is a proper subset of ${\mathcal{C}}_{n,k+1}$ for all $k<{\kappa }_n$ (from Proposition 6).

2. The manifold ${\mathcal{C}}_{n,{\kappa }_n}$ is contained in the convex hull of ${\mathcal{C}}_{n,2}.$ (This follows from Equation 2.)

Since ${\mathcal{C}}_{n,k}$ contains ${\mathcal{C}}_{n,2}$ and is properly contained in the convex hull of ${\mathcal{C}}_{n,2},$ it cannot be convex.