Efficiently inferring the demographic history of many populations with allele count data

Abstract

The sample frequency spectrum (SFS), or histogram of allele counts, is an important summary statistic in evolutionary biology, and is often used to infer the history of population size changes, migrations, and other demographic events affecting a set of populations. The expected multipopulation SFS under a given demographic model can be efficiently computed when the populations in the model are related by a tree, scaling to hundreds of populations. Admixture, back-migration, and introgression are common natural processes that violate the assumption of a tree-like population history, however, and until now the expected SFS could be computed for only a handful of populations when the demographic history is not a tree. In this article, we present a new method for efficiently computing the expected SFS and linear functionals of it, for demographies described by general directed acyclic graphs. This method can scale to more populations than p reviously possible for complex demographic histories including admixture. We apply our method to an 8-population SFS to estimate the timing and strength of a proposed “basal Eurasian” admixture event in human history. We implement and release our method in a new open-source software package momi2.

1. Introduction

All natural populations undergo evolutionary processes of migration, size changes, and divergence, and the history of these demographic events shape their present genetic diversity. Thus, inferring demographic history is of central concern in evolutionary and population genetics, both for its intrinsic interest (e.g., in dating the out-of-Africa migration of modern humans (Schaffner et al., 2005; Gutenkunst et al., 2009)) and also for biological applications (such as distinguishing the effects of natural selection from demography (Beaumont and Nichols, 1996; Boyko et al., 2008)). However, genetic sequence data and the space of possible demographic models are both very high dimensional objects, leading to numerous statistical and computational challenges when inferring demographic history from genetic data.

The joint sample frequency spectrum (SFS) is the multidimensional histogram of mutant allele counts in a sample of DNA sequences, and is a popular summary statistic which lies at the core of hundreds of empirical studies in population genetics; e.g., see Wakeley and Hey (1997); Griffiths and Tavaré (1998); Nielsen (2000); Gutenkunst et al. (2009); Coventry et al. (2010); Gazave et al. (2014); Gravel et al. (2011); Nelson et al. (2012); Excoffier et al. (2013); Jenkins et al. (2014); Bhaskar et al. (2015); Jouganous et al. (2017). Recently, progress has also been made on theoretical fronts to characterize statistical properties of SFS-based inference. In particular, studies of identifiability (Myers et al., 2008; Bhaskar and Song, 2014) and the rate of convergence (Terhorst and Song, 2015; Baharian and Gravel, 2018) have been carried out.

Demographic history can be inferred by fitting the observed value of the SFS to its expected value in a composite likelihood framework. The expected SFS can be efficiently computed when the demographic history is a tree, and in previous work we developed a method momi to compute the SFS of hundreds of populations related by a tree (Kamm et al., 2017). However, natural populations are often related by a more complex history that is not tree-like, as gene flow (the exchange of migrants between populations) adds extra edges to the topology associated with the demographic history. In this case, computing the expected SFS is much more computationally demanding, and existing methods for computing the exact expected SFS can scale to only a handful of populations (Gutenkunst et al., 2009; Jouganous et al., 2017).

In this article, we extend our previous algorithm momi to handle discrete (or pulse) migration events between populations, in which case demographies are described by general directed acyclic graphs (DAGs). Our new method momi2 can compute the exact expected SFS with admixture for more populations than previously possible, and uses novel insights from a stochastic process known as the Lookdown Construction (Donnelly and Kurtz, 1996; Donnelly et al., 1999). In addition, momi2 utilizes automatic differentiation (Corliss et al., 2002; Bhaskar et al., 2015) to compute gradients of the SFS, which we use to efficiently search the parameter space during optimization. Finally, momi2 can efficiently compute linear functionals of the SFS, which we exploit to compute the expected values of a number of standard population genetic summary statistics under complex demography.

The rest of this paper as organized as follows. In Section 2 we provide some background and survey related work. Section 3 describes our method, first with an illustrative example in Section 3.1, and then with formulas and pseudocode in Section 3.2. Finally, in Section 4, we apply our method to an 8-population SFS, including ancient and contemporary human populations, to estimate the timing and strength of a proposed “basal Eurasian” admixture event in human history. The Appendix contains all proofs, an analysis of the computational complexity of our method, and additional details of the application to ancient DNA.

2. Background

Suppose a sample of $n=\left(n_1,\dots ,n_{\mathcal{D}}\right)$ haploid genomes have been sampled from $\mathcal{D}$ “demes” or populations. The positions in the genome where the samples are not all identical are called segregating sites. In most organisms mutations are rare; most sites are not segregating. It is therefore reasonable to assume, as we do from now on, that each position in the genome has experienced at most a single mutation in its history, and that each individual can be labeled as having the “ancestral” or “derived” (mutant) allele at each segregating site. In population genetics, this simplifying assumption is known as the infinite sites model.

The sample frequency spectrum (SFS) is a $\mathcal{D}$-dimensional array $\left[{\text{f}}_x\right]\in {\mathbb{Z}}^{\left(n_1+1\right)\times \cdots \times \left(n_{\mathcal{D}}+1\right)}$ whose entry f_x counts the number of segregating sites with exactly x copies of the derived allele and n – x copies of the ancestral allele, where $x=\left(x_1,\dots ,x_{\mathcal{D}}\right)\in {\mathbb{N}}_0^{\mathcal{D}}$ with 0 ≤ x_d ≤ n_d for each $d=1,\dots ,\mathcal{D}$. Note we only consider segregating sites with 2 alleles, so f₀ = f_n = 0 by definition. Compared to the full data set (i.e., the complete genetic sequences of all n = n₁ +⋯+ n_d genomes), the SFS [f_x] is a compressed, low-dimensional summary which nevertheless preserves much of the signal about the various population size changes, divergence times, and admixture events that occurred over the course of the populations’ history.

2.1. Demographic events

The expected multipopulation SFS can be obtained by integrating over random genealogies formed by a backwards-in-time stochastic process known as the structured coalescent (Kingman, 1982; Takahata, 1988; Notohara, 1990). Before getting to the technical details, we first review the basic dynamics of this process in order to build intuition for how the data have power to infer population splits, size changes, and gene flow events. See Durrett (2008) for a more detailed introduction.

Informally, the topology and branch lengths of genealogies are affected by a demographic history in two ways:

1. Two lineages may not coalesce into a common ancestor until they reside in the same population, and the time until this occurs is affected by migration patterns and population split times.

2. At any particular point in time, two members within the same population are more likely to have a common parent if the population size is small; so, for example, residents of a small village will typically be more closely related than residents of a large city.

Regarding the second point, we define the scaled effective population size η(t) such that the rate at which any two lineages find a common ancestor at time t is 1/η(t). Under the simplest random mating model (the Wright Fisher model, cf. Durrett (2008)), the census population size exactly equals Tη, where T is the number of generations per unit time; more generally, η scales with the number of breeding individuals in the population. Thus, estimating η allows us to infer size change events such as bottlenecks, exponential growth, and population crashes.

If we could observe the true distribution of genealogies, then we could directly infer demographic history from the waiting times between coalescence events, following the principles listed in items 1 and 2 above. However, since genealogies are never directly observed, we must make inferences about demographic history indirectly using mutation data.

2.2. Likelihoods and the site frequency spectrum

Consider a genome with L positions and mutation rate $\frac{\theta }{L}$ per position per generation. So at any given position in the genome, mutations arise on the tree there as a Poisson point process with rate $\frac{\theta }{L}$. The chance of 2 or more mutations at a single position is $O\left(\frac{1}{L^2}\right)$, and taking the limit L → ∞ we arrive at the aforementioned infinite sites approximation (Kimura, 1969; Durrett, 2008), which assumes that each segregating site was caused by a single mutation, so that each allele may be labeled as ancestral or derived.

The observed segregating sites are not independent, because trees at neighboring positions are correlated. Unfortunately, even in the simplest case of a single population with constant size, an analytic expression for the likelihood of mutation data at a set of linked (non-independent) sites is not known (Bhaskar et al., 2015). Therefore, SFS data are generally used with composite likelihood methods. Recall that f_x Is the total number of segregating sites with derived allele count pattern $x=\left(x_1,\dots ,x_{\mathcal{D}}\right)$. Define ${\varphi }_x=\frac{1}{\theta }\mathbb{E}\left[{\text{f}}_x\right]$, i.e., ϕ_x is the expected frequency of x per unit mutation rate. Equivalently, ϕ_x is the expected branch length subtending x leaves in a random coalescent tree (i.e., with x₁ descendants in population 1, x₂ in population 2, and so on). A commonly used composite likelihood is the Poisson random field model (Sawyer and Hartl, 1992), which assumes that the total number of segregating sites is Poisson with rate $\theta \sum _x{\varphi }_x$, and that the patterns at the observed sites are independent with sampling probabilities proportional to ϕ_x; this yields a log-likelihood of

$$\mathcal{L}\propto \Vert \text{f}{\Vert }_1\text{log}\left(\theta \Vert \varphi {\Vert }_1\right)-\theta \Vert \varphi {\Vert }_1+\sum _x{\text{f}}_x\text{log}\frac{{\varphi }_{\mathbf{x}}}{\Vert \varphi {\Vert }_1},$$ (1)

where $\Vert \mathbf{f}{\Vert }_1≔\sum _x{\text{f}}_x$ and $\Vert \varphi {\Vert }_1≔\sum _x{\varphi }_x$. Demographic history can then be inferred by searching for the parameter values that maximize $\mathcal{L}$.

2.3. Existing work and our contribution

The composite log-likelihood $\mathcal{L}$ in (1) requires us to compute ϕ_x, the expected branch length subtending x. Let G be a random genealogical tree sampled under the demography, and L_x(G) be the total length of all branches in G subtending x, so that

$${\varphi }_x=\mathbb{E}\left[L_x(G)\right]={\int }_G{L}_x(G)d\mathbb{P}(G).$$ (2)

The integral (2) is difficult since the support of G is at least as large as the number of labeled binary trees with n leaves, a quantity which grows faster than exponentially in the sample size n. Consequently, a number of different methods have been proposed for evaluating (or approximating) the integral (2). Sampling-based methods include Markov chain Monte Carlo (Griffiths and Tavaré, 1997; Nielsen, 2000), importance sampling (Stephens and Donnelly, 2000; De Iorio and Griffiths, 2004), simulation (Excoffier and Foll, 2011; Excoffier et al., 2013), and ABC (Wegmann et al., 2010). The main advantage of these methods is their flexibility: since, as mentioned above, computing the likelihood of the data is trivial conditional on G, these methods can be used on a rich class of models. However, the high dimension of ϕ^x makes it impractical to compute by sampling unless $\mathcal{D}$ is small. In particular, since the support of x grows like $O\left(n^{\mathcal{D}}\right)$, Monte Carlo methods will assign zero mass to configurations that are actually observed in the data.

A second approach, implemented in the software ∂a∂i (Gutenkunst et al., 2009), computes ϕ_x by numerically solving PDEs arising from the Wright-Fisher diffusion (Ewens, 2004), which is dual to the coalescent process described above. For $\mathcal{D}$ populations, this involves numerically solving a $\mathcal{D}$-dimensional integral. The initial ∂a∂i method in Gutenkunst et al. (2009) could handle up to $\mathcal{D}=3$ populations; subsequent improvements (Lukić and Hey, 2012; Jouganous et al., 2017) extended this to $\mathcal{D}=4$ and then $\mathcal{D}=5$ populations by using spectral representations or alternative basis functions for solving the PDEs.

The third approach for computing ϕ_x, which includes our method, integrates over the sample allele frequencies “backwards-in-time”, exploiting conditional independence relationships to reduce computation. This involves considering the alleles of the sample’s ancestors at different points in the demographic history, and integrating out these random variables via inference algorithms for probabilistic graphical models (Pearl, 1982; Felsenstein, 1981; Lauritzen and Spiegelhalter, 1988; Koller and Friedman, 2009). Bryant et al. (2012) and Chen (2012) computed the SFS for finite- and infinite-sites models using this backward-in-time approach under the coalescent. De Maio et al. (2015) and Kamm et al. (2017) substantially lowered the computational burden of this approach by replacing the coalescent with the continuous-time Moran model (Durrett, 2008), a stochastic process which induces the same sampling distribution as the coalescent, but using a much smaller state space. However, until now these Moran-based approaches have been limited to analyzing tree-shaped demographies without admixture between populations.¹

The main contribution of this paper is to extend our previous Moran-based method (Kamm et al., 2017) to allow for demographies defined on general directed acyclic graphs (DAGs), thus allowing for admixture between populations. We describe and implement an algorithm for computing the expected infinite-sites SFS under the multipopulation Moran model with size changes, exponential growth, population splits, and point admixture events (i.e., instantaneous migration “pulses”). This substantially enlarges the space of demographies for which the expected SFS can be accurately computed.

Additionally, our algorithm computes not only individual SFS entries, but also linear functionals of the expected SFS. Specifically, our method computes rank-1 tensor products of the SFS in the same time as a single entry (general linear functionals are sums of these rank-1 products). A number of widelyused statistics in population genetics can be expressed as SFS functionals, and to our knowledge our method is the first to compute expectations of these statistics under complex demography.

We demonstrate our method by using it to infer the history of eight human subpopulations that have undergone multiple admixture events. We complement our theoretical contributions with an open-source, user-friendly software implementation that will enable practitioners to deploy our method. The software uses automatic differentiation (Corliss et al., 2002; Bhaskar et al., 2015; Maclaurin et al., 2015) to compute derivatives of the SFS, leading to efficient optimization and parameter inference. Our package, called momi2, is available for download at https://github.com/popgenmethods/momi2.

3. Method

In this section, we describe the algorithm implemented in momi2 for computing the expected SFS under complex demographies. We begin in Subsection 3.1 with an illustrative example that highlights the novel aspects of our work. Then in Subsection 3.2 we provide pseudocode for our algorithm, and state the formulas used by our algorithm as Propositions. The proofs of these Propositions, and the proof for the correctness of our algorithm, requires substantial additional notation, and we defer this to Appendix A.2. For ease of reference, the symbols we use are summarized in Table 1.

Table 1.

Notation. Rows in the second part of the table are only used for proofs in the Appendix and may be ignored in the main text.

Symbol	Description	Reference
$\mathcal{D}$	Sampled populations are $\{1,\dots ,\mathcal{D}\}$	§2
n	# of samples per population $\left(n_1,\dots ,n_{\mathcal{D}}\right)$	”
fx	# of sites with x derived and n − x ancestral alleles	”
θ	Mutation rate	§2.2
ϕx	Expected branch length with x descendants, $\frac{1}{\theta }\mathbb{E}\left[{\text{f}}_x\right]$	”
$\mathcal{G}$	Population graph	Figure 2(a)
v	A vertex in $\mathcal{G}$, corresponding to a population	§3.2
Tv	time between top and bottom events of v	”
ηv (t)	scaled population size of v at t ∈[0, τv]	”
nv	# of samples with ancestry in v	”
$X_v^{(t)}$	# of derived alleles in v at time t	”
$X_S^{(t)}$	Vector of allele counts $\left(X_{v_1}^{\left(t_1\right)},\dots ,X_{v_k}^{\left(t_k\right)}\right)$ at S = {v1,…,vk}	”
Xv, XS	Shorthand for $X_v^{(0)}$, $X_S^{(0)}$ respectively	”
$\mathcal{T}$	Event tree	Figure 2(b)
E	A join, split or leaf event in $\mathcal{T}$	”
KE	Populations we are keeping track of at E	”
${\ell }_{x,z}^{E,t}$	Conditional likelihood $\mathbb{P}\left(X_{\text{Leaves}(E)}=z|X_{K_E}^{(t)}=x\right)$	Eq. (4)
${\varphi }_z^E$	$\mathbb{E}\left[\text{branch length atbelow }{K}_E\text{ with }z\text{ descendants}\right]$	Eq. (5)
${\ell }_x^{E,t}$, ϕE	Shorthand for ${\ell }_{x,z}^{E,t}$, ${\varphi }_z^E$ respectively when z fixed	”
ntot	The total number of sampled lineages $\sum _{d=1}^{\mathcal{D}}{n}_d$	Appendix A.1
${\mathcal{M}}_{v,t,(i)}$	Label and allele of the ith lineage in v at t ∈[0, τv]	”
${\mathcal{M}}_{v,t}$	$\left({\mathcal{M}}_{v,t,(1)},{\mathcal{M}}_{v,t,(2)},\dots \right)$	”
${\mathcal{M}}_{E,t}$	$\left({\mathcal{M}}_{v_1,t_1},\dots ,{\mathcal{M}}_{v_k,t_k}\right)$ where KE = {v1,…,vk} and t = (t1,…,tk)	”
$X_{v,m}^{(t)}$	the # derived among the 1st m lineages in v, t. (Note: $X_v^{(t)}\equiv X_{v,n_v}^{(t)}$)	”

3.1. Example

Consider the model depicted in Figure 1. This model has 3 sampled (leaf) populations, related as follows. Populations 1 and 2 are sisters, with Population 3 an outgroup to them; however a pulse of migrants from 3 to 2 occurs after the split between populations 1 and 2.

Fig. 1.

An example of a 3-population Moran model. The bottom of the graph corresponds to the present and the top to the past. Population 2 receives admixture from population 3 after splitting from population 1. Other features of the demography include archaic samples in population 1, and various size changes along the edges of this demography. Lineages bearing the ancestral (derived) allele are shown in red (blue). The derived allele arises as the blue star (⋆) in the population immediately ancestral to population 4. The red circles (•) and blue stars represent one potential configuration of the allelic states of the ancestors to the sample; our method integrates over all potential configurations in order to calculate the likelihood. Arrows pointing to the right (→) represent reproduction events in the forward-time Moran model. The black arrow pointing to the left (←) at event 6 is a migration event from the population at node 7 to the population at node 5 (again in the forward-time sense).

At the leaves, we observe (n_1, n_2, n₃) = (2,2,1) samples, with (X_1, X₂, X₃) = (1,1,0) copies of the derived (blue star) allele. We wish to compute the expected number of mutations with pattern (X₁, X₂, X₃); to do this, we will integrate over the unobserved variables (X₄, X₅, X₆, X₇, X₈) which represent allele counts at certain internal positions within the demography. The random variables X₁,…,X₈ are related to each other by the DAG $\mathcal{G}$ in Figure 2(a), with an edge from population v to w if alleles may pass directly from v into w.

Fig. 2.

The population DAG $\mathcal{G}$ (a) and event tree $\mathcal{T}$ (b) corresponding to the demography shown in Figure 1. (a) Each vertex in $\mathcal{G}$ corresponds to a collection of alleles in Figure 1. (b) The corresponding event tree $\mathcal{T}$ is a junction tree of the DAG in $\mathcal{G}$; each vertex of $\mathcal{T}$ corresponds to a set of vertices from $\mathcal{G}$.

In the realization of Figure 1, the hidden blue allele counts are (X₄, X₅, X₆, X₇, X₈) = (3,2,0,0,0). The blue mutation occurs in the edge above X₄, spreading to 3 of the lineages. One copy of the blue allele moves on to X₁, while 2 copies move on to X₅. However, due to the admixture, X₂ only inherits 1 blue allele from X₅, inheriting a red allele from the 2 red alleles at X₆.

Under the infinite-sites assumption described above, the mutation observed at this site arose at a single point in the genealogical tree depicted in Figure 1. To compute the expected number of mutations with (X₁, X₂, X₃) = (1,1,0), we may condition on the population (i.e., the edge in Figure 1) on which it arose:

$$\begin{gathered} {\varphi }_{1,1,0}=\frac{1}{\theta }\mathbb{E}\left[\#\text{ mutations with }\left(X_1,X_2,X_3\right)=(1,1,0)\right] \\ =\frac{1}{\theta }\sum _{v=1}^8\sum _{i=1}^{n_v}\mathbb{E}\left[\#\text{mutations in population }v\text{ yielding }{X}_v=i\right] \\ \times \mathbb{P}\left(X_1=1,X_2=1,X_3=0|\text{ mutation at }v\text{ with }{X}_v=i\right). \end{gathered}$$ (3)

We call $\frac{1}{\theta }\mathbb{E}\left[\#\text{mutations at }v\text{ with }{X}_v=i\right]$, the “truncated SFS”; this only concerns events within a single population and can be computed using the method momi1. We refer the interested reader to Kamm et al. (2017) for further details. The second term $\mathbb{P}\left(X_1=1,X_2=1,X_3=0|\text{ mutation at }v\text{ with }{X}_v=i\right)$ gives the conditional likelihood of observing the data given a mutation and its allele count at population v. In momi1, we computed this term in the case where $\mathcal{G}$ is a tree without admixture using the sum-product (also known as belief propagation) algorithm (Felsenstein, 1981; Koller and Friedman, 2009).

The main result of this work is to extend our previous dynamic program to the case where $\mathcal{G}$ is given by a DAG due to admixture. We will use a dynamic program that is essentially a kind of junction tree algorithm (Koller and Friedman, 2009). This algorithm works by decomposing a DAG graph into a tree in such a way that vertices in the tree correspond to collections of nodes in the original graph. Belief propagation is then applied to the tree decomposition.

We illustrate our algorithm using the example demography in Figure 2(b). We call the tree decomposition $\mathcal{T}$ an event tree, because each internal node corresponds to either an admixture or split event.² We construct the event tree $\mathcal{T}$, and compute the conditional likelihoods $\mathbb{P}\left(X_1=1,X_2=1,X_3=0|\text{ mutation at }v\text{ with }{X}_v=i\right)$ , as follows:

1. We initially start with a collection of 3 singleton sets of leaf populations: {{1},{2},{3}}. For v = 1, 2, 3, we also keep track of conditional likelihoods of the data beneath v; since we are at the leaves, these are simply the Kronecker delta functions

   $$\mathbb{P}\left(X_v=i|X_v=j\right)={\delta }_{i,j}=\{\begin{array}{ll}1,\hfill & \text{ if }i=j,\hfill \\ 0,\hfill & \text{ if }i\ne j.\hfill \end{array}$$

2. Going back in time, the first event is the admixture of populations 5 and 6 into 2. To process this event, we remove the set {2} and replace it with {5, 6}; the current collection of sets becomes {{1},{3},{5,6}}. We make the new set {5, 6} the parent of the removed set {2} in $\mathcal{T}$.

   In addition, we compute ${\left[\mathbb{P}\left(X_2|X_5=i,X_6=j\right)\right]}_{i,j}$, the conditional likelihood of the data beneath {5, 6} given the allele counts X₅, X₆. We obtain this by applying Lemmas 1 and 2 to the previous conditional likelihood ${\left[\mathbb{P}\left(X_2|X_2=i\right)\right]}_i$. Specifically, we apply Lemma 1 to “lift” the conditional likelihood at population 2 to the time point immediately below the admixture event; then, we apply Lemma 2 to obtain $\mathbb{P}\left(X_2|X_5,X_6\right)$, the conditional likelihood at {5, 6} immediately above the admixture event.

3. The next event has the alleles from 6 splitting off from population 3. To process this event, we merge the clusters containing the relevant populations ({5, 6} and {3}); then we remove 3,6 and replace them with their parent population 7, to obtain {5, 7} as the parent cluster of {5, 6} and {3} in $\mathcal{T}$. After this stage, our collection of sets becomes {{1},{5,7}}.

   To obtain $\mathbb{P}\left(X_2,X_3|X_5,X_7\right)$, the conditional likelihood of the data at the leaves beneath {5, 7}, we apply Lemma 3 to $\mathbb{P}\left(X_3|X_3\right)$ and $\mathbb{P}\left(X_2|X_5,X_6\right)$. Lemma 3 computes the conditional likelihood at a split event when the children fall into separate clusters beneath the split (in this case, the child clusters are {3} and {5, 6}).

4. The next event is similar, with populations 1 and 5 merging into population 4. After combining the clusters {1} and {5, 7}, removing the merged populations 1 and 5, and adding in their parent 4, we are left with the collection {{4,7}}.

   Similar to previous steps, we compute $\mathbb{P}\left(X_1,X_2,X_3|X_4,X_7\right)$, the conditional likelihood of the data at the leaves beneath {4, 7}, by applying Lemma 1 to “lift” the conditional likelihoods $\mathbb{P}\left(X_1|X_1\right)$ and $\mathbb{P}\left(X_2,X_3|X_5,X_7\right)$ to the point immediately below the split event, and then apply Lemma 3 to compute the conditional likelihood at a split event above two independent clusters ({1},{5,7}).

5. The final event has populations 4 and 7 merging into population 8. We replace the cluster {4, 7} with its parent cluster {8} at the root of $\mathcal{T}$.

   To compute $\mathbb{P}\left(X_1,X_2,X_3|X_8\right)$ from the child likelihoods $\mathbb{P}\left(X_1,X_2,X_3|X_4,X_7\right)$, we first apply Lemma 1 to lift the likelihoods immediately below the split event, and then apply Lemma 4, which computes the conditional likelihood at the parent of a split when the children belong to the same cluster ({4, 7} in this case).

Finally, at each cluster {ν₁, ν₂,…} in $\mathcal{T}$ with leaves {ι₁, ι₂…} we have computed $\mathbb{P}\left(X_{l_1},X_{l_2},\dots |X_{v_1},X_{v_2},\dots \right)$, the conditional likelihoods at the leaves beneath {ν₁, ν₂,…}. But to apply equation (3), we need $\mathbb{P}\left(\left(X_1,X_2,X_3\right)=(1,1,0)|\text{ mutation at }v\text{ with }{X}_v=i\right)$ mutation at ν with X_ν = i) This is given by

$$\mathbb{P}\left(\left(X_1,X_2,X_3\right)=(1,1,0)|\text{ mutation at }v\text{ with }{X}_v=i\right)=\{\begin{array}{ll}\mathbb{P}\left(\left(X_1,X_2,X_3\right)=(1,1,0)|X_4=i,X_7=0\right),\hfill & \text{if }v=4,\hfill \\ \mathbb{P}\left(\left(X_1,X_2,X_3\right)=(1,1,0)|X_8=i\right),\hfill & \text{if }v=8,\hfill \\ 0,\hfill & \text{else,}\hfill \end{array}$$

since we assume that each site experiences at most a single mutation in its history, and the only way derived alleles are observed in leaf populations 1,2 is if the corresponding mutation occurs in population 4 or 8.

Remark. The observant reader may have noticed that in Figure 1, population 8 has only n₈ = 5 alleles, despite its children having n₄ + n₇ = 7 > 5 alleles in total. This is due to the fact that there are only 5 alleles at the leaves, so there are at most 5 ancestors in any population at any point; thus, when populations 4 and 7 merge into population 8, we can drop two of the tracked lineages. To show this formally, we use a stochastic process called the lookdown construction, which is a version of the Moran model with a countably infinite number of lineages. However, this analysis requires a great deal of additional notation and is not essential to the remainder of the text, so we defer it to Appendix A.1.

3.2. Algorithms and formulas

We now describe the algorithm to construct the event tree $\mathcal{T}$. We assume that the population graph $\mathcal{G}$ has two types of topological events:

1. Population split: two child populations u, v split from each other; their parent population is w. Looking backward in time, u, v merge and become the population w.

2. Population admixture: a single child population u, inherits from exactly two parent populations v, w, with the probability that an allele comes from v(w, respectively) being p(1−P, respectively).

Note that more complicated events, such as trifurcating splits or symmetric pulse migrations, may be expressed as a succession of these 2-way split and admixture events.

We provide pseudocode to construct the event tree $\mathcal{T}$ in Algorithm 1. In words, we initially start with $\mathcal{K}$ equal to a collection of singleton sets corresponding to the leaves of $\mathcal{G}$. Processing each split or admixture event E back in time, we merge all blocks containing the child population(s) of E Then we remove the child population(s) from this merged block, and add in the parent population(s). Within $\mathcal{T}$, the new merged block is the parent of the blocks removed at this stage.

Algorithm 1.

Construct event tree $\mathcal{T}$

1: procedure EventTree($\mathcal{G}$)

2: $\mathcal{K}\leftarrow \left\{\left\{l\right\}:l\in \text{Leaves}(\mathcal{G})\right\}$ ⊳ State variable containing current set of tracked populations

3: $V(\mathcal{T})\leftarrow \left\{\right\}$ ⊳ Vertices of $\mathcal{T}$

4: $\mathcal{E}(\mathcal{T})\leftarrow \left\{\right\}$ ⊳ Edges of $\mathcal{T}$

5: for events E in BackInTimeEvents $(\mathcal{G})$ do

6: if E is split then

7: w1, w2 ← ChildPopulations(E)

8: ν ← ParentPopulation(E)

9: K1 ← block in $\mathcal{K}$ containing w1

10: K2 ← block in $\mathcal{K}$ containing w2

11: KE ← K1 ∪ K2 \{w1, w2} ∪ {ν}

12: $\mathcal{K}\leftarrow \mathcal{K}\backslash \left\{K_1,K_2\right\}\cup \left\{K_E\right\}$

13: $V(\mathcal{T})\leftarrow V(\mathcal{T})\cup \left\{K_E\right\}$

14: if K1 ≠ K2 then

15: $\mathcal{E}(\mathcal{T})\leftarrow \mathcal{E}(\mathcal{T})\cup \left\{K_E\to K_1,K_E\to K_2\right\}$

16: else

17: $\mathcal{E}(\mathcal{T})\leftarrow \mathcal{E}(\mathcal{T})\cup \left\{K_E\to K_1\right\}$

18: end if

19: else if E is admixture then

20: w ← ChildPopulation(E)

21: ν1, ν2 ← ParentPopulations(E)

22: K′ ← block in $\mathcal{K}$ containing w

23: KE ← K′\{w}∪{ν1, ν2}

24: $\mathcal{K}\leftarrow \mathcal{K}\backslash \left\{K^{\prime }\right\}\cup \left\{K_E\right\}$

25: $V(\mathcal{T})\leftarrow V(\mathcal{T})\cup \left\{K_E\right\}$

26: $\mathcal{E}(\mathcal{T})\leftarrow \mathcal{E}(\mathcal{T})\cup \left\{K_E\to K\prime \right\}$

27: end if

28: end for

29: return $V(\mathcal{T}),\mathcal{E}(\mathcal{T})$

30: end procedure

We now describe Algorithm 2, the dynamic program to compute the joint SFS. We need a bit more notation. For population v, let n_v be the number of samples with ancestry in v, we will be keeping track of n_v lineages within population v. Let T_v denote the amount of time between the top and bottom events of v, and let η_ν(t) denote the scaled population size of v at time t ∈ [0, τ_ν] above the base of v, Let $0\le X_v^{(t)}\le n_v$ be the allele count within the n_v lineages of v at time t above its bottom, so $X_v^{(0)}=X_v$ in our earlier notation.

At event E in $\mathcal{T}$, let $K_E=\left\{v_1,\dots ,v_{|K_E|}\right\}$ be the corresponding block of populations in $\mathcal{G}$. We define the conditional likelihood at E as

$${\ell }_{x,z}^{E,t}=\mathbb{P}\left(X_{\text{Leaves}(E)}=z|X_{K_E}^{(t)}=x\right)$$ (4)

where $X_{K_E}^{(t)}=\left(X_{v_1}^{\left(t_1\right)},\dots ,X_{v_{|K_E|}}^{t_{|K_E|}}\right)$ is the vector of allele counts in populations K_E at times t, and X_Leaves(E) the observed data at the leaves beneath E.

In addition, we define the “partial SFS”

$${\varphi }_{\mathbf{z}}^E=\sum _{v\text{ descended from }{K}_E}\mathbb{E}\left[\text{branch length in }v\text{ with }{X}_{\text{Leaves}(E)}=z\right]$$ (5)

as the expected branch length at or below K_E subtending z leaves. (Here, the expectation is with respect to branch lengths in the coalescent tree relating the samples.) Note that ${\varphi }_z^{Root(\mathcal{T})}$ gives the desired final result, and corresponds to equation (3) in the previous subsection.

Algorithm 2.

Dynamic program to compute the SFS ϕ

1: procedure $\text{DP}\left({\ell }^1,\dots ,{\ell }^{\mathcal{D}}\right)⊳{\ell }^i=e_{X_i}\in {ℝ}^{n_i+1}$

2: for event E in DepthFirstSearch $(\mathcal{T})$ do

3: if KE = {d} is leaf then

4: ℓE,0 ← ℓd

5: else if E is split event then

6: ℓE,0 ← Lemmas 1 and 2

7: else if E is join event then

8: if |ChildEvents(E)|=1 then

9: ℓE,0 ← Lemmas 1 and 4

10: else if |ChildEvents(E) |= 2 then

11: ℓE,0 ← Lemmas 1 and 3

12: end if

13: end if⊳ Computed the conditional likelihood ℓE,0

14: ϕE ← (10) ⊳ Computes ϕE the partial SFS

15: end for

16: return ${\varphi }^{\text{Root}(\mathcal{T})}$ ⊳ Return partial SFS at the root event

17: end procedure

For the remainder of the subsection, we will fix Leaved $X_{\text{Leaves}(\mathcal{G})}=z$, and drop the dependence on z in ${\ell }_{x,z}^{E,t}$ and ${\varphi }_z^E$. Algorithm 2 defines a dynamic program (DP) over the conditional likelihoods ${\ell }_x^{E,0}$ and partial SFS ϕ^E. The DP takes input vectors ${\ell }^1,\dots ,{\ell }^{\mathcal{D}}$ corresponding to the leaf populations Leaves $(\mathcal{G})=\left\{1,\dots ,\mathcal{D}\right\}$. If the inputs ${\ell }^1,\dots ,{\ell }^{\mathcal{D}}$ are set to indicator vectors corresponding to the observed counts $X_1,\dots ,X_{\mathcal{D}}$, the DP of Algorithm 2 will return the corresponding SFS entry, as stated in the theorem below:

Theorem 1. If $\left(X_1,\dots ,X_{\mathcal{D}}\right)\ne 0,n$ then

$$\varphi =\text{DP}\left(e_{X_1},\dots ,e_{X_{\mathcal{D}}}\right),$$

where $e_{X_i}=(0,\dots ,1,\dots ,0)\in {ℝ}^{n_i+1}$ denotes the vector with 1 at coordinate X_i and 0 elsewhere.

We now present the formulas used by Algorithm 2, in a series of lemmas also used to prove Theorem 1. We start with a formula to “lift” $\mathbb{P}\left(\dots |\dots ,X_v^{(0)},\dots \right)$ up to $\mathbb{P}\left(\dots |\dots ,X_v^{\left({\tau }_v\right)},\dots \right)$. That is, this formula transforms a likelihood conditioned on $X_v^{(0)}$, the allele count at the bottom of v, into a likelihood conditioned on $X_v^{\left({\tau }_v\right)}$, the allele count at the top of v.

Lemma 1 (Lifting), thmadmixlift Let E be a split or admixture event with corresponding block K_E, ν ∈ K_E be a population within this block, and x, t be vectors of allele counts and times for the populations in K_E. Then, for x_(k) = x − ke_ν the conditional likelihood of E is

$${\ell }_x^{E,t}=\sum _{j=0}^{n_v}{\left[e^{Q^{\left(n_v\right)}{\int }_0^{{\tau }_v}\frac{1}{{\eta }_v(t)}dt}\right]}_{kj}{\mathcal{l}}_{x_{(j)}+j{e}_v}^{E,t-{\tau }_v{e}_v},$$ (6)

where $Q^{(n)}\in {ℝ}^{(n+1)\times (n+1)}$ is the transition rate matrix of the Moran model with n lineages; in particular, $Q^{(n)}={\left(q_{ij}^{(n)}\right)}_{0\le i,j\le n}$ and

$$q_{ij}^{(n)}=\{\begin{array}{ll}-i(n-i),\hfill & \text{if }i=j,\hfill \\ \frac{1}{2}i(n-i),\hfill & \text{if }|j-i|=1,\hfill \\ 0,\hfill & \text{else.}\hfill \end{array}$$

To process event E, we first use Lemma 1 to lift up the conditional likelihoods at the child populations, up to the time of E We then apply one of Lemma 2, Lemma 3, or Lemma 4, to obtain the conditional likelihood at E from the lifted child likelihoods, depending on whether E is an admixture or split event, and whether the child populations of E fall into a single cluster or two independent clusters.

We first consider admixture events; Lemma 2 describes how to compute the conditional likelihood in this case. Let the child population be w and the parent populations be v₁, v₂ Each of the n_w lineages in w independently inherits from v₁ with probability q₁, or from v₂ with probability q₂ =1 − q₁. So the number of lineages inheriting from v₁ is Binomial (n_w, q₁). Then, given that m₁ alleles are inherited from v₁ and m₂ = n_w = m₁ inherited from v₂, the particular alleles inherited from v₁ or v₂ are chosen by sampling without replacement.

Lemma 2 (Admixture event).

Let E be an admixture event, with child population w and parent populations v₁, v₂. Let E′ be the child event in $\mathcal{T}$. Suppose each lineage in w comes from v₁ with probability q₁, and from v₂ with probability q₂ = 1 − q₁ For K_E the population cluster at E, let x_∩ be a vector of allele counts on K_E\ {ν₁, ν₂}. Then the conditional likelihood of allele counts $x_{\cap }+x_1{e}_{v_1}+x_2{e}_{v_2}$ at E is given by

$${\ell }_{x_{\cap }+x_1{e}_{v_1}+x_2{e}_{v_2}}^{E,0}=\sum _{x_w=0}^{n_w}{\ell }_{x_{\cap }+x_w{e}_w}^{E^{\prime },{\tau }_w{e}_w}\sum _{\stackrel{m_1,m_2:}{m_1+m_2=n_w}}\left(\begin{array}{c}{n}_w\\ m_1\end{array}\right)q_1^{m_1}{q}_2^{m_2}\sum _{\stackrel{j_1,j_2:}{j_1+j_2=x_w}}\frac{\left(\begin{array}{c}{x}_1\\ j_1\end{array}\right)\left(\begin{array}{c}{n}_{v_1}-x_1\\ m_1-j_1\end{array}\right)}{\left(\begin{array}{c}{n}_{v_1}\\ m_1\end{array}\right)}\frac{\left(\begin{array}{c}{x}_2\\ j_2\end{array}\right)\left(\begin{array}{c}{n}_{v_2}-x_2\\ m_2-j_2\end{array}\right)}{\left(\begin{array}{c}{n}_{v_2}\\ m_2\end{array}\right)}.$$ (7)

We next consider a split event E, with parent population v and child populations w₁, w₂. We first consider the case where E has 2 distinct children in $\mathcal{T}$, i.e., w₁, w₂ fall into 2 distinct blocks beneath E. Denote the corresponding child events as E₁′, E₂′ respectively. Then the conditional likelihood at E is given by a convolution of the conditional likelihoods at E₁′, E₂′, as described in Lemma 3:

Lemma 3 (Population split, 2 clusters). Let E be a split event with parent population v and child populations w₁, w₂. Suppose E has 2 child events E₁′, E₂′, with corresponding blocks $K_{E_1{}^{\prime }},K_{E_2{}^{\prime }}$, where $w_1\in K_{E_1{}^{\prime }},w_2\in K_{E_2{}^{\prime }}$. Let x₋₁ be allele counts on $K_{E_1{}^{\prime }}\backslash \left\{w_1\right\}$ and x₋₂ be allele counts on $K_{E_2{}^{\prime }}\backslash \left\{w_2\right\}$, Then the conditional likelihood at E is

$${\ell }_{x_{-1}+x_{-2}+x_v{e}_{v_2}}^{E,0}=\sum _{\stackrel{x_1,x_2:}{x_1+x_2=x_v}}\frac{\left(\begin{array}{c}{n}_{w_1}\\ x_1\end{array}\right)\left(\begin{array}{c}{n}_{w_2}\\ x_2\end{array}\right)}{\left(\begin{array}{c}{n}_v\\ x_v\end{array}\right)}{\ell }_{x_1{e}_{w_1}+x_{-1}}^{E_1{}^{\prime },{\tau }_{w_1}{e}_{w_1}}{\ell }_{x_2{e}_{w_2}+x_{-2}}^{E_2{}^{\prime },{\tau }_{w_2}{e}_{w_2}}.$$ (8)

Next, consider the case where the child populations w₁, w₂ fall into the same cluster beneath E, so that E has just one child event, say E′. The next lemma describes how to obtain the conditional likelihood at E from the conditional likelihood at E′. This involves summing over the dimensions corresponding to w₁, w₂ within the conditional likelihood ${\ell }^{E^{\prime },{\tau }_{w_1}}{}^{e_{w_1}+{\tau }_{w_2}{e}_{w_2}}$. In addition, note that we may have $n_v<n_{w_1}+n_{w_2}$ (recall n_v is the number of samples with ancestry in v). That is, after merging w₁, w₂ backwards in time, we may be keeping track of more alleles than originally sampled, allowing us to “drop” some extraneous non-ancestral lineages, as illustrated in the root population of Figure 1. Formally, let $y^{x_{\cap }}$ be the vector whose i-th entry is the likelihood at E conditional on there being a total of i derived alleles in the child populations w₁ and w₂, and also conditional on the vector of allele counts x_∩ in the other populations. If $B\in {ℝ}^{\left(n_{w_1}+n_{w_2}+1\right)\times \left(n_v+1\right)}$ is a matrix whose (i, j)th entry is the probability of sampling without replacement j alleles in the parent population v out of the total of i alleles segregating in child populations w₁ and w₂, then the conditional likelihood ℓ^E at E satisfies the equation

$$y^{x_{\cap }}=B{\ell }^E,$$

leading to the following lemma.

Lemma 4 (Population split, 1 cluster). Let E be a population split with exactly one event E′. Denote the corresponding population clusters as K_E and K_E′. Denote the parent population as v, the child populations as w₁, w₂. Let x_∩, $x_{\cap },y^{x_{\cap }}$ and B be defined as in the preceding paragraph:

$$y_i^{x_{\cap }}=\sum _{\stackrel{j,k:}{j+k=i}}\frac{\left(\begin{array}{c}{n}_{w_1}\\ j\end{array}\right)\left(\begin{array}{c}{n}_{w_2}\\ k\end{array}\right)}{\left(\begin{array}{c}{n}_{w_1}+n_{w_2}\\ i\end{array}\right)}{\ell }_{j{e}_{w_1}+k{e}_{w2}+x_{\cap }}^{E^{\prime },{\tau }_{w_1}{e}_{w_1}+{\tau }_{w_2}{e}_{w_2}}=\mathbb{P}\left(X_{\text{Leaves}(E)}|X_{w_1}^{\left({\tau }_{w_1}\right)}+X_{w_2}^{\left({\tau }_{w_2}\right)}=i,X_{\cap }=x_{\cap }\right)$$

$$B_{i,j}=\frac{\left(\begin{array}{c}{n}_v\\ j\end{array}\right)\left(\begin{array}{c}{n}_{w_1}+n_{w_2}-n_v\\ i-j\end{array}\right)}{\left(\begin{array}{c}{n}_{w_1}+n_{w_2}\\ i\end{array}\right)}.$$

Then the conditional likelihood at E is given by

$${\ell }_{k{e}_v+x_{\cap }}^{E,0}={\left[B^{+}{y}^{{\mathbf{x}}_{\cap }}\right]}_k,$$ (9)

with B⁺ denoting the Moore-Penrose pseudoinverse of B.

Finally, having computed the conditional likelihoods at event E, we wish to compute the partial SFS ϕ^E at the event. This is given by the recursive formula in Lemma 5, which involves the conditional likelihood at E, the expected number of mutations arising within each parent population v, and the partial SFS at the child events.

Lemma 5. For an event E, define the partial ordering E′ ≺ E if E′ occurs beneath E in the event tree, and let $K_E^{\text{ new }}=K_E\backslash \left({\cup }_{E^{\prime }\prec E}{K}_{E^{\prime }}\right)$ be the populations newly formed at E (i.e., formed by a population split or admixture at E). For $v\in K_E^{\text{new}}$, let f_ν(k) be the truncated SFS in population v,

$$f_v(k)=\frac{1}{\theta }\mathbb{E}\left[\#\text{mutations at }v\text{ with }{X}_v=k\right],$$

which can be computed by the formulas in Kamm et al. (2017). Then the partial SFS at E is given by

$${\varphi }^E=\sum _{v\in K_E^{\text{new}}}\sum _{k=1}^{n_v}{f}_v(k){\ell }_{k{e}_v}^{E,0}+\{\begin{array}{ll}0\hfill & \text{if}E\text{is}\text{leaf}\text{event},\hfill \\ {\varphi }^{E^{\prime }},\hfill & \text{if ChildEvents}(E)=\left\{E^{\prime }\right\},\hfill \\ {\varphi }^{E_1{}^{\prime }}\prod _{d\in \text{Leaves }\left(E_2{}^{\prime }\right)}{\ell }_0^d+{\varphi }^{E_2{}^{\prime }}\prod _{d\in Leaves\left(E_1{}^{\prime }\right)}{\ell }_0^d,\hfill & \text{if ChildEvents}(E)=\left\{E_1{}^{\prime },E_2{}^{\prime }\right\}.\hfill \end{array}$$ (10)

3.3. Normalizing constant and other linear functionals

To compute the probability $\frac{{\varphi }_z}{\Vert \varphi {\Vert }_1}$ of a mutation having observed allele counts z, we need not just ϕ_z, but also the normalizing constant $\Vert \varphi {\Vert }_1=\sum _z{\varphi }_z$ the expected total branch length.

Computing ‖ϕ‖₁ directly is inefficient because of the $O\left(\prod _{d=1}^{\mathcal{D}}{n}_d\right)$ possible entries z. Instead, we can use Algorithm 2 to compute ‖ϕ‖₁, and many more statistics of the SFS, in the same time as a single entry:

Theorem 2. For ${\pi }^d\in {ℝ}^{n_d+1},d\in \left\{1,\dots ,\mathcal{D}\right\}$, the tensor dot product of the SFS ϕ against ${\pi }^1\otimes \cdots \otimes {\pi }^{\mathcal{D}}={\left[{\pi }_{z_1}^1\cdots {\pi }_{z_{\mathcal{D}}}^{\mathcal{D}}\right]}_{z_1,\dots ,z_{\mathcal{D}}}$ is

$$\begin{gathered} \varphi \odot \left({\pi }^1\otimes \cdots \otimes {\pi }^{\mathcal{D}}\right)=\sum _{z_1,\dots ,z_{\mathcal{D}}}{\varphi }_{z_1,\dots ,z_{\mathcal{D}}}{\pi }_{z_1}^1\cdots {\pi }_{z_{\mathcal{D}}}^{\mathcal{D}} \\ =\text{DP}\left({\pi }^1,\dots ,{\pi }^{\mathcal{D}}\right)-\left(\prod _{d=1}^{\mathcal{D}}{\pi }_0^d\right)\text{DP}\left(e_0,\dots ,e_0\right)-\left(\prod _{d=1}^{\mathcal{D}}{\pi }_{n_d}^d\right)\text{\hspace{0.05em}DP\hspace{0.05em}}\left(e_{n_1},\dots ,e_{n_{\mathcal{D}}}\right). \end{gathered}$$

Theorem 2 says that for any rank-K tensor $A\in {ℝ}^{\left(n_1+1\right)\times \cdots \times \left(n_{\mathcal{D}}+1\right)}$ with

$$A=\sum _{k=1}^K{a}_1^{(k)}\otimes \cdots \otimes a_{\mathcal{D}}^{(k)},$$

$$\varphi \odot A=\sum _z{\varphi }_{\mathbf{z}}{A}_{\mathbf{z}}=\sum _{k=1}^K\varphi \odot \left(a_1^{(k)}\otimes \cdots \otimes a_{\mathcal{D}}^{(k)}\right)$$

can be computed in K calls to DP $\left({\pi }^1,\dots ,{\pi }^{\mathcal{D}}\right)$. 3 In particular, the expected total branch length ‖ϕ‖₁ is given by

$$\begin{gathered} \Vert \varphi {\Vert }_1=\sum _z{\varphi }_{\mathbf{z}}=\varphi \odot (1\otimes \cdots \otimes 1) \\ =\text{DP}(1,\dots ,1)-\text{DP}\left(e_0,\dots ,e_0\right)-\text{DP}\left(e_{n_1},\dots ,e_{n_{\mathcal{D}}}\right) \end{gathered}$$

with 1 the vector with 1 at every coordinate.

Beyond the applications we explore here, we expect this result to be useful in related settings. A number of population genetic statistics can be expressed as f ⨀ A, including Watterson’s estimator θ_W of the mutation rate (Watterson, 1975), Fay and Wu’s H statistic for positive selection (Fay and Wu, 2000), and Patterson’s f₂, f₃, f₄ statistics for assessing topology (Patterson et al., 2012). Theorem 2 allows us to compute their expected values $\mathbb{E}[\text{f}\odot A]=\theta \varphi \odot A$, and to construct test statistics from the deviance f ⨀ A − θϕ⨀A under an appropriate null model.

An even wider class of population genetic statistics can be written as nonlinear functions of SFS-tensor products like $g\left(\frac{1}{\theta }\text{f}\odot A_1,\frac{1}{\theta }\text{f}\odot A_2,\dots \right)$; this class includes Tajima’s D statistic for selection (Tajima, 1989), the F_ST statistic for population structure (Holsinger and Weir, 2009), and Patterson’s D statistic for introgression (Patterson et al., 2012). These statistics may be viewed as plug-in estimators for g(ϕ⨀ A₁, ϕ⨀A₂,…) which we can compute with Theorem 2. Note that these estimators are biased due to the nonlinear function g, but the bias can be estimated via block jackknife, and will typically be small since $\frac{1}{\theta }\text{f}\odot A\to \varphi \odot A$ almost surely as the number of independent SNPs grows.

Another interesting linear statistic of the SFS that can be computed with Theorem 2 is $\mathbb{E}\left[T_{\text{MRCA}}\right]$, the expected time of the most recent common ancestor. In particular, let d be any leaf population; for simplicity assume d is sampled at the present (i.e. d is not archaic). Then

$$\begin{gathered} \mathbb{E}\left[T_{\text{MRCA}}\right]=\sum _{z_1,\dots ,z_{\mathcal{D}}}{\varphi }_{z_1,\dots ,z_{\mathcal{D}}}\frac{z_d}{n_d} \\ =\varphi \odot \left(1\otimes \cdots \otimes {\left(\frac{z_d}{n_d}\right)}_{z_d\in \left\{0,\dots ,n_d\right\}}\otimes \cdots \otimes 1\right). \end{gathered}$$

To see this, note that T_MRCA is proportional to the expected number of mutations hitting an arbitrary lineage in d, and if a mutation has configuration $z=\left(z_1,\dots ,z_{\mathcal{D}}\right)$ derived copies, then the chance of hitting the lineage is $\frac{z_d}{n_d}$ (Zeng et al., 2006).

4. Application

We tested our method on a demographic inference problem in human genetics that is currently of interest. Lazaridis et al. (2014) showed that genetic variation in present-day Europeans suggests an admixture model involving three ancestral meta-populations: Ancient North Eurasian (ANE), Western Hunter Gatherers (WHG), and Early European Farmers (EEF). They also showed that EEF contains ancestry from a source that is an outgroup to all non-African populations, and yet shares much of the genetic drift common to non-African populations; they dubbed this ancestry component as “Basal Eurasian” ancestry. Later work (Lazaridis et al., 2016) showed that Basal Eurasian ancestry is shared by ancient and contemporary Middle Eastern populations, and is correlated with a decrease in Neanderthal ancestry, implying that Basal Eurasian ancestry contains lower levels of Neanderthal admixture when compared with non-Basal ancestry. The results from Lazaridis et al. (2014, 2016) were based on several related methods for modeling covariances in population allele frequencies, most notably qpGraph and qpAdm (Patterson et al., 2012; Haak et al., 2015). These methods are computationally efficient, and are robust to ascertainment bias and misspecification of the population size history; however, they are unable to infer the timing of demographic events.

We applied momi2 to estimate the strength and timing of basal Eurasian admixture into early European farmers, and the split time of the basal Eurasian lineage. To do this, we built a demographic model relating 12 samples from 8 populations, shown in Figure 3. These samples consisted of the Altai Neanderthal (Prüfer et al., 2014); the 45,000 year old Ust’lshim man from Siberia (Fu et al., 2014); 3 present-day populations (Mbuti, Sardinian, Han) with 3, 2, and 2 samples respectively; and 3 ancient samples representing the European ancestry components identified by Lazaridis et al. (2014): a 7,500 year old sample from the Linearbandkeramik (LBK) culture (representing EEF), an 8,000 year old sample from the Loschbour rock shelter in Luxembourg (representing WHG), and the 24,000 year old Mal’ta boy (“MA1”) from Siberia (representing ANE). After data cleaning, our dataset consisted of 2.4 ×10⁶ autosomal transversion SNPs. See Appendix A.4 for more details about the data.

Fig. 3.

Inferred model and bootstraps for the 11 population demography described in Section 4. The y-axis is linear below 5 × 10⁴ then follows a logarithmic scale above 5 × 10⁴ in the foreground (blue branches) is our point estimate from maximum composite likelihood; in the background (gray branches, with transluscent pulse arrows) are 300 nonparametric bootstraps. Population sampling times are indicated by open circles; note demographic events may still involve populations below their sampling times, as in the case of Loschbour. The nonparametric bootstrap estimates were created by splitting the data into 100 equally sized contiguous blocks, resampling these blocks with replacement, and refitting the model.

To construct the topology of the model in Figure 3, we first obtained a tree by neighbor joining (Saitou and Nei, 1987), then added 3 extra admixture events reflecting prior knowledge, as well as a recent Neanderthal population decline starting at the Mbuti-Eurasian split. We inferred split times, population sizes (including the Neanderthal decline), and admixture times and proportions by maximizing a composite likelihood $\mathcal{L}$, given by the product of the likelihoods at every SNP:

$$\mathcal{L}=\prod _{s\in \text{SNPs}}\mathbb{P}\left(z_s\right)$$ (11)

where $\mathbb{P}\left(z_s\right)\propto {\varphi }_{z_s}$ and was computed by momi2. The low coverage of the MA1 sample and the deep divergence of the Neanderthal sample may cause bias in SFS entries where only these samples contain derived alleles; we thus excluded these entries and corrected the normalizing constant appropriately (see Appendix A.4).

We used automatic differentiation to compute the gradient $\nabla \log \mathcal{L}$, which we used to search for the optimum of logℒ. We constructed nonparametric bootstrap confidence intervals by splitting the genome into 100 equally sized blocks, resampling these blocks to create 300 bootstrap datasets, and re-inferring the demography for each bootstrap dataset. We also used 300 parametric bootstraps to assess how well we could infer the demography under simulated data; for each parametric bootstrap dataset, we used msprime (Kelleher et al., 2016) to simulate ten 300 Mb chromosomes from our initial point estimate, and re-inferred the demography. The nonparametric bootstrap was used for all confidence intervals reported below.

Our inferred demography, along with nonparametric bootstrap re-estimates, are shown in Figure 3 and Table 2. Our parametric bootstrap estimates are shown in Figure 4. We inferred a pulse of 0.094 (95% Cl of 0.049–0.174) from the ghost Basal Eurasian population to EEF ancestry (LBK), substantially less than the 0.44 inferred by (Lazaridis et al., 2014). This admixture was inferred to occur 33.7 kya (95% Cl of 10.8–41.1 kya), shortly after the Loschbour-LBK split at 37.7 kya (95% Cl of 32.2–42.3 kya). The split time of the ghost Basal Eurasian lineage from other Eurasians was inferred at 79.8 kya (95% Cl of 67.4–101 kya). Other parameters were broadly in line with previous estimates, such as a Mbuti-Eurasian split of 96 kya, a Han-European split of 50 kya, a Neanderthal split of 696 kya, and Eurasians deriving 0.03 of their ancestry from Neanderthal (Terhorst et al., 2017; Green et al., 2010; Meyer et al., 2016).

Table 2.

Estimated parameters of the demography in Figure 3, along with nonparametric bootstrap estimates of the bias and standard deviation, and 95% bootstrap quantiles. We use (A, B) to denote the ancestor of A and B; t_v and N_v to denote the height and size at vertex v, and t_A→B and p_A→B to denote respectively the time and strength of an admixture arrow from A to B.

Parameter	Estimate	Bias	SD	2.5%	97.5%
NLosch	1.92e+03	14.1	185	1.57e+03	2.27e+03
NMbu	1.73e+04	–255	1.86e+03	1.6e+04	1.92e+04
N(Mbu,Losch)	2.91e+04	−270	2.34e+03	2.7e+04	3.21e+04
t(Han,Losch)	9.58e+04	−1.48e+03	9.32e+03	9.19e+04	1.03e+05
NHan	6.3e+03	−17.1	400	5.71e+03	6.91e+03
N(Han,Losch)	2.34e+03	−70.2	372	2.13e+03	2.7e+03
t(Han,Losch)	5.04e+04	−171	2.75e+03	4.71e+04	5.43e+04
t(Ust,Losch)	5.15e+04	−79.4	2.47e+03	4.85e+04	5.47e+04
N(Nean,Losch)	1.82e+04	−216	1.59e+03	1.7e+04	1.99e+04
t(Nean,Losch)	6.96e+05	−7.6e+03	5.74e+04	6.49e+05	7.58e+05
tNean→Eurasian	5.68e+04	−279	3.04e+03	5.38e+04	6.01e+04
pNean→Eurasian	0.0296	−2.82e-05	0.00252	0.0251	0.0349
NNean	86.9	−3.33	19.8	76.7	105
t(MA1,Losch)	4.49e+04	98.2	2.36e+03	4.11 e+04	4.87e+04
NLBK	75.7	−685	629	4.12	1.9e+03
t(LBK,Losch)	3.77e+04	543	2.59e+03	3.22e+04	4.23e+04
pBasal→EEF	0.0936	−0.0122	0.0366	0.0485	0.174
tBasal→EEF	3.37e+04	7.46e+03	1.01e+04	1.08e+04	4.11 e+04
t(Basal,Losch)	7.98e+04	−706	1.37e+04	6.74e+04	1.01e+05
NSard	1.5e+04	−1.28e+04	6.53e+04	8.58e+03	8.9e+04
t(Sard,LBK)	7.69e+03	−1.69e+03	1.64e+03	7.51e+03	1.24e+04
N(Sard,LBK)	1.2e+04	1.32e+03	1.99e+03	7.33e+03	1.45e+04
tGhostWHG→Sard	1.23e+03	−2.93e+03	3.1e+03	597	1.06e+04
pGhostWHG→Sard	0.0317	−0.00223	0.0151	0.00631	0.0618
t(GhostWHG,Losch)	1.56e+03	−1.32e+04	1.07e+04	964	3.49e+04

Fig. 4.

Parametric bootstraps for the demography inferred in Figure 3. The inferred demography (blue tree) was used to simulate 300 bootstrap replicates. Re-running our method on each replicate produced a new inferred demography which is plotted in gray.

Inferring the optimal demography from start to finish took 2.5 hours on a laptop with 4 CPU cores, and used 2 GB RAM. The 300 bootstraps were run separately on a high-performance computing cluster. To our knowledge, no other method can infer this demographic model using the full SFS. The moments software package (Jouganous et al., 2017) is capable of computing the SFS for up to 5 populations, less than the 8 populations here, though it can scale to more individuals per population than momi2. While the fastsimcoal2 software package (Excoffier et al., 2013) is capable of handling demographies of this size and larger, it doesn’t compute the full, exact SFS, and also does not include an option for the ascertainment scheme we use here (excluding mutations private to Neanderthal and MA1).

We also used our model to produce estimates of the human mutation rate, which we estimated as 1.22 × 10⁻⁸ per base per generation, with a bootstrap-quantile 95% Cl of (1.12, 1.32) × 10⁻⁸ closely matching previous estimates of the human mutation rate (Scally, 2016). To obtain this estimate, we compared the observed nucleotide diversity with that expected under the inferred demography, adjusting by the empirical transition to transversion ratio (Appendix A.4). Estimating the mutation rate was possible here because we did not use a prespecified mutation rate to estimate the model in Figure 3, instead using the known ages of the Ust’Ishim, LBK, Loschbour, and MA1 samples to calibrate dates.

A. Appendix

A.1. Model and notation

In this section, we formally define the stochastic process underlying our model, and introduce some additional notation needed for the proofs in Section A.2.

The main stochastic process we use is the lookdown construction of the Moran model (Donnelly and Kurtz, 1996; Donnelly et al., 1999), a variant of the Moran model where copying only occurs in one “direction” (as in Figure 5). We will make use of certain conditional independence properties that result from this one-way copying. However, we note that there is a simple coupling between the lookdown and standard Moran models, and these two models generate data with the same distribution.

Fig. 5.

Standard and lookdown Moran models, (a) illustrates the standard Moran model, with copying in both directions; (b) illustrates the lookdown Moran model, with copying always from left to right. The sample genealogy (i.e., the lineages ancestral to present day samples) is in solid. The law of the genealogy is the coalescent under both models, and so the present-day samples of the two models are equal in distribution.

We now describe our lookdown model in more detail. Within each of the $\mathcal{D}$ leaf populations we consider a countably infinite number of lineages, each with a unique label in ${\mathbb{Z}}_{+}$. We arbitrarily assign the $\left(n_1,\dots ,n_{\mathcal{D}}\right)$ sampled lineages to the lowest labels {1,2, …, n_tot}, where $n_{\text{tot}}\equiv \sum _{d=1}^{\mathcal{D}}{n}_d$, and arbitrarily assign the remaining unsampled lineages to labels {n_tot + 1,n_tot + 2, …}. Each lineage extends infinitely backwards in time, and at each admixture event, each lineage randomly chooses the parent population it extends into. In addition, each lineage has an allele, which changes through time due to mutation and copying events.

Similar to the usual Moran model, copying between a pair of lineages in population v occurs at rate $\frac{1}{{\eta }_v(t)}$, where η_ν(t) is the scaled population size of v. However, copying only occurs in one direction, from lineages with lower labels to lineages with higher labels (Figure 5). So if two lineages are labeled i and j respectively, with i < j, then the copying event i → j, occurs at rate $\frac{1}{{\eta }_v(t)}$, whereas the reverse copying event i ← j never happens (has rate 0). By contrast, in the standard (non-lookdown) Moran model, both i → j and i ← j copying events happen at rate $\frac{1}{2{\eta }_v(t)}$. However, under both models, two lineages going backwards in time coalesce at rate $\frac{1}{{\eta }_v(t)}$, as in the coalescent. Thus, the sample genealogies under both models follow the same multipopulation coalescent distribution.

Denote the labeled lineages in population v, and their alleles at height t, by

$${\mathcal{M}}_{v,t}=\left({\mathcal{M}}_{v,t,(1)},{\mathcal{M}}_{v,t,(2)},\dots \right)$$

where ${\mathcal{M}}_{v,t,(i)}=\left({\text{label}}_{v,\left(i\right)},{\text{allele}}_{v,t,\left(i\right)}\right)\in {\mathbb{Z}}_{+}\times \left\{0,1\right\}$ is a pair, consisting of the ith lowest label at v, along with its corresponding allele at height t ∈[0, τ_ν). Note that we have ordered the lineages by their labels, so that label_ν,(i) < label_ν,(i+1). Also note that we measure the time t from the bottom of v, so that allele_ν,0,(i) denotes the ith allele at the bottom of v, and ${\text{allele}}_{v,{\tau }_v,(i)}$ the ith allele at the top of v.

Let $X_{v,m}^{(t)}=\sum _{i=1}^m{\mathbb{I}}_{\left\{{\mathcal{M}}_{v,t,(i)}\text{derived}\right\}}$ denote the number of derived alleles among the lowest m lineages at v, t. Let $n_v=\sum _{d⪰v}{n}_d$ be the number of samples in leaves below v. During computation, we will only need to keep track of the lowest n_v lineages in v, so we denote $X_v^{(t)}\equiv X_{v,n_v}^{(t)}$ to be the number of derived alleles among the first n_v lineages at v, t.

Finally, for an event E with K_E = (ν₁, …, ν_k) the corresponding block of populations in Algorithm 1, let t = (t₁, …, t_k) be a corresponding set of times within each population ν₁, …, ν_k. We define ${\mathcal{M}}_{E,t}=\left({\mathcal{M}}_{v_1,t_1},\dots ,{\mathcal{M}}_{v_k,t_k}\right)$ as the labeled alleles at each of (ν₁,t₁), …, (ν_k, t_k)

A.2. Proofs

For the reader’s convenience, theorem statements are reproduced from the main text before each proof. The following two lemmas will be useful for several of the proofs below:

Lemma 6. The distribution of ${\mathcal{M}}_{E,t}$ is invariant to finite permutations of the labels within any population. ν ∈K_E Furthermore, the labels are independent of the alleles.

Proof. By construction, none of the lineages within the populations in K_E are ancestral to each other. Thus the sample genealogy of any finite subsample of the lineages is the multipopulation coalescent, because going backwards in time, coalescence (copying) between each pair of lineages occurs at rate $\frac{1}{{\eta }_w(s)}$ at vertex w time s. The invariance to permutation, and independence of alleles and labels, follows from the coalescent. □

Lemma 7. For event E and population u ∈ K_E, let $\mathcal{I}\subset \left\{n_u+1,n_u+2,\dots \right\}$ be a (possibly random) collection of integers greater than n_u, and let $X_{u,\mathcal{I}}^{\left(t_u\right)}$ be the number of derived lineages in ${\left\{{\mathcal{M}}_{u,t_u,(i)}\right\}}_{i\in \mathcal{I}}$. Then X_Leaves(E) is conditionally independent of $X_{u,\mathcal{I}}^{\left(t_j\right)}$ the allele counts on $\mathcal{I}$, given $X_{K_E}^{(t)}$ the allele counts on the first $n_{v_1},\dots ,n_{v_k}$ lineages of K_E = {ν₁, …, ν_k}.

Proof. Integrate over ${\mathcal{M}}_{E,t,n_v}\equiv {\left\{{\mathcal{M}}_{v,t_v,(i)}\right\}}_{v\in K_E,1\le i\le n_v}$ the first $\left(n_{v_1},\dots ,n_{v_k}\right)$ labeled alleles within K_E, and ${\mathcal{M}}_{u,t_u,\mathcal{I}}={\left\{{\mathcal{M}}_{u,t_u,(i)}\right\}}_{i\in \mathcal{I}}$ the labeled alleles at $\mathcal{I}$:

$$\begin{gathered} \mathbb{P}\left(X_{\text{Leaves}(E)}|X_{K_E}^{(t)},X_{u,\mathcal{I}}^{t_u}\right)=\mathbb{E}\left[\mathbb{P}\left(X_{\text{Leaves}(E)}|{\mathcal{M}}_{E,t,n_v},{\mathcal{M}}_{u,t_u,\mathcal{I}}\right)|X_{K_E}^{(t)},X_{u,\mathcal{I}}^{\left(t_u\right)}\right] \\ =\mathbb{E}\left[\mathbb{P}\left(X_{\text{Leaves}(E)}|{\mathcal{M}}_{E,t,n_v}\right)|X_{K_E}^{(t)},X_{u,\mathcal{I}}^{\left(t_u\right)}\right] \\ =\mathbb{E}\left[\mathbb{P}\left(X_{\text{Leaves}(E)}|{\mathcal{M}}_{E,t,n_v}\right)|X_{K_E}^{(t)}\right] \\ =\mathbb{P}\left(X_{\text{Leaves}(E)}|X_{K_E}^{(t)}\right) \end{gathered}$$

with the second equality because higher lineages cannot copy to lower lineages (so $X_{\text{Leaves}(E)}\perp {\mathcal{M}}_{u,t_u,\mathcal{I}}|{\mathcal{M}}_{E,t,n_v}$), and the third equality because of the exchangeability and independence from Lemma 6 (so given $X_{K_E}^{(t)}$, the alleles of ${\mathcal{M}}_{E,t,n_v}$ are ordered by a uniform permutation independent of $X_{u,\mathcal{I}}^{\left(t_u\right)}$, and the labels of ${\mathcal{M}}_{E,t,n_v}$ are independent of $X_{K_E}^{(t)}$, $X_{u,\mathcal{I}}^{\left(t_u\right)}$). □

A.2.1. Proof of Theorem 1

Theorem 1. If $\left(X_1,\dots ,X_{\mathcal{D}}\right)\ne 0$,n, then

$$\varphi =\text{DP}\left(e_{X_1},\dots ,e_{X_{\mathcal{D}}}\right),$$

where $e_{X_i}=(0,\dots ,1,\dots ,0)\in {ℝ}^{n_i+1}$ denotes the vector with 1 at coordinate X_i and 0 elsewhere.

Proof. Lemmas 2–4 give formulas for the partial likelihood ${\ell }_x^{E,0}$ using terms like ${\ell }_{x^{\prime }}^{E^{\prime },t^{\prime }}$, where E′ ∈ Children(E) and E is an admixture, 2-cluster split, or 1-cluster split, respectively. The terms ${\ell }_{x^{\prime }}^{E^{\prime },t^{\prime }}$ in turn can be computed from the partial likelihoods ${\ell }_{x^{\prime }}^{E^{\prime },0}$ using Lemma 1; then Lemma 5 provides a formula for ϕ^E in terms of ℓ^E,0 and the partial SFS ϕ^E′.

Algorithm 2 traverses the event tree $\mathcal{T}$ in a depth-first search, applying Lemmas 5,1,2,4,3 to compute ${\ell }_x^{E,0}$, ϕ^E at each event E from their values at the children of E The input to Algorithm 2 are the likelihoods ℓ^{d},0 at the leaves, and the output is the partial SFS ${\varphi }^{\text{Root}(\mathcal{T})}$ at the root.

Thus, since

$${\ell }^{\left\{d\right\},0}={\left[\mathbb{P}\left(X_d|X_d=i\right)\right]}_{0\le i\le n_d}=e_{X_d},$$

it follows that $\text{DP}\left(e_{X_1},\dots ,e_{X_{\mathcal{D}}}\right)={\varphi }^{\text{Root}(\mathcal{T})}=\varphi$. □

A.2.2. Proof of Lemma 5

Lemma 5. For an event E, define the partial ordering E′ ≺ E if E′ occurs beneath E in the event tree, and let $K_E^{\text{new}}=K_E\backslash \left({\cup }_{E^{\prime }\prec E}{K}_{E^{\prime }}\right)$ be the populations newly formed at E (i.e., formed by a population split or admixture at E). For $v\in K_E^{\text{new}}$, let f_v(k) be the truncated SFS in population v,

$$f_v(k)=\frac{1}{\theta }\mathbb{E}\left[\#\text{mutations at }v\text{ with }{X}_v=k\right],$$

which can be computed by the formulas in Kamm et al. (2017). Then the partial SFS at E is given by

$${\varphi }^E=\sum _{v\in K_E^{\text{new}}}\sum _{k=1}^{n_v}{f}_v(k){\ell }_{k{e}_v}^{E,0}+\{\begin{array}{ll}0\hfill & \text{if}E\text{is}\text{leaf}\text{event},\hfill \\ {\varphi }^{E^{\prime }},\hfill & \text{if ChildEvents}(E)=\left\{E^{\prime }\right\},\hfill \\ {\varphi }^{E_1{}^{\prime }}\prod _{d\in \text{Leaves }\left(E_2{}^{\prime }\right)}{\ell }_0^d+{\varphi }^{E_2{}^{\prime }}\prod _{d\in Leaves\left(E_1{}^{\prime }\right)}{\ell }_0^d,\hfill & \text{if ChildEvents}(E)=\left\{E_1{}^{\prime },E_2{}^{\prime }\right\}.\hfill \end{array}$$ (10)

Proof. Without loss of generality assume θ = 1. Then ϕ^E is the expected number of mutations at or below K_E with observed counts X_Leaves(E). We split ϕ^E into two parts: the mutations within $K_E^{\text{new}}=K_E\backslash \left({\cup }_{E^{\prime \prime \prime }\prec E}{K}_{E^{\prime }}\right)$ (i.e. the populations formed by a split or join at E); and the mutations that occur in ∪_E′≺EK_E′, the populations that arise strictly below E.

The first part, of mutations at the new populations of E, is given by

$$\sum _{v\in K_E^{\text{ new }}}\sum _{k=1}^{n_v}{f}_v(k){\ell }_{k{e}_v}^{E,0},$$

since f_v(k) is the expected number of mutations arising at v with X_v = k, and ${\ell }_{k{\mathbf{e}}_v}^{E,0}=\mathbb{P}\left(X_{\text{Leaves}(E)}|X_v=k\right)$.

For the second part, of mutations strictly below E, we split into three cases: either E is a leaf event, or E has a single child event, or E has two child events. In the first case, if E is a leaf event, then no mutations can occur below E. In the second case, if E has a single child event E′, then the expected number of mutations strictly below E is simply ϕ^E′ by definition.

Finally, if E has two child events E₁′, E₂′, then E₁′ and E₂′ share no leaves, so a mutation underneath E₁′ will have no derived alleles in Leaves(E₂′) and vice versa. Thus, the number of mutations strictly below E is

$${\varphi }^{E_1{}^{\prime }}\prod _{d\in \text{L}eaves\left(E_2{}^{\prime }\right)}{\ell }_0^d+{\varphi }^{E_2{}^{\prime }}\prod _{d\in \text{Leades}\left(E_1{}^{\prime }\right)}{\ell }_0^d.$$

□

A.2.3. Proof of Lemma 1

Lemma 1 (Lifting). Let E be a split or admixture event with corresponding block K_E, v ∈ K_E be a population within this block, and x, t be vectors of allele counts and times for the populations in K_E. Then, for x_(k) = x − ke_v, the conditional likelihood of E is

$${\ell }_x^{E,t}=\sum _{j=0}^{n_v}{\left[e^{Q^{\left(n_v\right)}{\int }_0^{{\tau }_v}\frac{1}{{\eta }_v(t)}dt}\right]}_{kj}{\mathcal{l}}_{x_{(j)}+j{e}_v}^{E,t-{\tau }_v{e}_v},$$ (6)

where $Q^{(n)}\in {ℝ}^{(n+1)\times (n+1)}$ is the transition rate matrix of the Moran model with n lineages; in particular, $Q^{(n)}={\left(q_{ij}^{(n)}\right)}_{0\le i,j\le n}$ and

$$q_{ij}^{(n)}=\{\begin{array}{ll}-i(n-i),\hfill & \text{if }i=j,\hfill \\ \frac{1}{2}i(n-i),\hfill & \text{if }|j-i|=1,\hfill \\ 0,\hfill & \text{else.}\hfill \end{array}$$

Proof. Define a “quasi-lookdown” Moran model ${\mathcal{M}}^{*}$, which is identical to $\mathcal{M}$, except within the n_v lowest lineages of v, where we allow copying in both directions at rate $\frac{1}{2{\eta }_v(t)}$ the non-lookdown Moran model).

For an event E with populations K_E = (v₁, v₂,…) and corresponding times t = (t₁,t₂…), let $X_{K_E}^{(t)}=\left(X_{v_1}^{\left(t_1\right)},X_{v_2}^{\left(t_2\right)},\dots \right)$ be the corresponding allele counts. Next, define $X_{E,t}={\left\{X_{K_{E^{\prime }}}^s\right\}}_{\left(E^{\prime },s\right)(E,t)}$ as the sample path of allele counts below E, t, where (E′, s) (E, t) if either E is above E′, or E = E′ and s≤t component-wise. It will suffice to show ${\mathbb{P}}_{\mathcal{M}}\left(X_{E,t}\right)={\mathbb{P}}_{{\mathcal{M}}^{*}}\left(X_{E,t}\right)$, because then for t = τ_ve_v+t_−v,

$$\begin{gathered} {\ell }_{k{e}_v+x_{-v}}^{E,t}=\sum _{j=0}^{n_v}{\mathbb{P}}_{\mathcal{M}}\left(X_{K_E}^{\left(t_{-v}\right)}=j{e}_v+x_{-v}|X_{K_E}^{(t)}=k{e}_v+x_{-v}\right){\mathcal{l}}_{j{e}_v+x_{-v}}^{E,t_{-v}} \\ =\sum _{j=0}^{n_v}{\mathbb{P}}_{{\mathcal{M}}^{*}}\left(X_{K_E}^{\left(t_{-v}\right)}=j{e}_v+x_{-v}|X_{K_E}^{(t)}=k{e}_v+x_{-v}\right){\mathcal{l}}_{j{e}_v+{\mu }_{-v}}^{E,t_{-v}} \\ =\sum _{j=0}^{n_v}{\left[e^{M^{\left(n_v\right)}{\int }_0^{{\tau }_v}\frac{1}{{\eta }_v(t)}dt}\right]}_{kj}{\mathcal{l}}_{j{e}_v+{\mu }_{-v}}^{E,t_{-v}} \end{gathered}$$

as desired.

${\mathbb{P}}_{\mathcal{M}}\left(X_{E,t}\right)={\mathbb{P}}_{{\mathcal{M}}^{*}}\left(X_{E,t}\right)$ follows from a coupling argument. Let ${\mathcal{M}}_{E,t}={\left\{{\mathcal{M}}_{E^{\prime },s}\right\}}_{\left(E^{\prime },s\right)}{}_{(E,t)}$ the sample path of $\mathcal{M}$ below E, t. We can map the partial sample paths of ${\mathcal{M}}^{*}{}_{v,t}$ onto those of ${\mathcal{M}}_{v,t}$ as follows: moving from the bottom to the top of v, whenever a lower label is copied over by a higher label, swap the labels of the lineages above the copying. Then the relabeled sample path has the same distribution as the lookdown construction, since the allele with the higher label is always copied over, and the rate of copying between pairs of lineages is $\frac{1}{{\eta }_v(t)}$. Since this relabeling also leaves $X_v^{(t)}$ unchanged, we have ${\mathbb{P}}_{\mathcal{M}}\left(X_{E,t}\right)={\mathbb{P}}_{{\mathcal{M}}^{*}}\left(X_{E,t}\right)$. □

A.2.4. Proof of Lemma 2

Lemma 2 (Admixture event).

Let E be an admixture event, with child population w and parent populations v₁, v₂. Let E′ be the child event in $\mathcal{T}$. Suppose each lineage in w comes from v₁ with probability q₁ and from v₂ with probability q₂ = 1 − q₁ For K_E the population cluster at E, let x_∩ be a vector of allele counts on K_E \ {v₁, v₂}. Then the conditional likelihood of allele counts $x_{\cap }+x_1{e}_{v_1}+x_2{e}_{v_2}$ at E is given by

$${\ell }_{x_{\cap }+x_1{e}_{v_1}+x_2{e}_{v_2}}^{E,0}=\sum _{x_w=0}^{n_w}{\ell }_{x_{\cap }+x_w{e}_w}^{E^{\prime },{\tau }_w{e}_w}\sum _{\stackrel{m_1,m_2:}{m_1+m_2=n_w}}\left(\begin{array}{c}{n}_w\\ m_1\end{array}\right)q_1^{m_1}{q}_2^{m_2}\sum _{\stackrel{j_1,j_2:}{j_1+j_2=x_w}}\frac{\left(\begin{array}{c}{x}_1\\ j_1\end{array}\right)\left(\begin{array}{c}{n}_{v_1}-x_1\\ m_1-j_1\end{array}\right)}{\left(\begin{array}{c}{n}_{v_1}\\ m_1\end{array}\right)}\frac{\left(\begin{array}{c}{x}_2\\ j_2\end{array}\right)\left(\begin{array}{c}{n}_{v_2}-x_2\\ m_2-j_2\end{array}\right)}{\left(\begin{array}{c}{n}_{v_2}\\ m_2\end{array}\right)}.$$ (7)

Proof. First note that X_Leaves(E) = X_Leaves(E′) and

$$\begin{gathered} {\ell }_{x_{\cap }+x_1{e}_{v_1}+x_2{e}_{v_2}}^{E,0}=\mathbb{P}\left(X_{\text{Leaves}(E)}|X_{K_E}^{(0)}=x_{\cap }+x_1{e}_{v_1}+x_2{e}_{v_2}\right) \\ =\sum _{x_w=0}^{n_w}\mathbb{P}\left(X_{\text{Leaves}(E)}|X_{K_E}^{(0)}=x_{\cap }+x_1{e}_{v_1}+x_2{e}_{v_2},X_w^{\left({\tau }_w\right)}=x_w\right)\mathbb{P}\left(X_w^{\left({\tau }_w\right)}=x_w|X_{K_E}^{(0)}=x\right) \\ =\sum _{x_w=0}^{n_w}\mathbb{P}\left(X_{\text{Leaves}(E)}|X_{K_E}^{(0)}=x_{\cap }+x_1{e}_{v_1}+x_2{e}_{v_2},X_w^{\left({\tau }_w\right)}=x_w\right) \\ \times \sum _{\stackrel{m_1,m_2:}{m_1+m_2=n_w}}\left(\begin{array}{c}{n}_w\\ m_1\end{array}\right)q_1^{m_1}{q}_2^{m_2}\sum _{\stackrel{j_1,j_2:}{j_1+j_2=x_w}}\frac{\left(\begin{array}{c}{x}_1\\ j_1\end{array}\right)\left(\begin{array}{c}{n}_{v_1}-x_1\\ m_1-j_1\end{array}\right)}{\left(\begin{array}{c}{n}_{v_1}\\ m_1\end{array}\right)}\frac{\left(\begin{array}{c}{x}_2\\ j_2\end{array}\right)\left(\begin{array}{c}{n}_{v_2}-x_2\\ m_2-j_2\end{array}\right)}{\left(\begin{array}{c}{n}_{v_2}\\ m_2\end{array}\right)} \end{gathered}$$

by sampling n_w alleles in w from $n_{v_1}$, $n_{v_2}$ alleles in v₁, v₂ which are exchangeable by Lemma 6.

Next, consider the $n_{v_1}+n_{v_2}-n_w$ highest alleles of v₁, v₂, and let $\mathcal{I}\subset \left\{n_w+1,n_w+2,\dots \right\}$ their relative positions within w. Then we conclude the proof by noting that X_Leaves(E) = X_Leaves(E′) and applying Lemma 7, to get

$$\begin{gathered} \mathbb{P}\left(X_{\text{Leaves}(E)}|X_{K_E}^{(0)}=x_{\cap }+x_1{e}_{v_1}+x_2{e}_{v_2},X_w^{\left({\tau }_w\right)}=x_w\right) \\ =\mathbb{P}\left(X_{\text{Leaves}\left(E^{\prime }\right)}|X_{v_1}=x_1,X_{v_2}=x_2,X_{K_E{}^{\prime }}^{\left({\tau }_w{e}_w\right)}=x_{\cap }+x_w{e}_w\right) \\ =\mathbb{P}\left(X_{\text{Leaves}\left(E^{\prime }\right)}|X_{w,\mathcal{I}}^{\left({\tau }_w\right)}=x_1+x_2-x_w,X_{K_E{}^{\prime }}^{\left({\tau }_w,e_w\right)}=x_{\cap }+x_w{e}_w\right) \\ =\mathbb{P}\left(X_{\text{Leaves}\left(E^{\prime }\right)}|X_{E^{\prime }}^{\left({\tau }_w,e_w\right)}=x_{\cap }+x_w{e}_w\right) \\ ={\mathcal{l}}_{x_{c}ap+x_w{e}_w}^{E,0}. \end{gathered}$$

□

A.2.5. Proof of Lemma 4

Lemma 4 (Population split, 1 cluster). Let E be a population split with exactly one event E′. Denote the corresponding population clusters as K_E and K_E′. Denote the parent population as v, the child populations as w₁,w₂. Let x_∩, $y^{x_{\cap }}$ and B be defined as in the preceding paragraph:

$$y_i^{x_{\cap }}=\sum _{\stackrel{j,k:}{j+k=i}}\frac{\left(\begin{array}{c}{n}_{w_1}\\ j\end{array}\right)\left(\begin{array}{c}{n}_{w_2}\\ k\end{array}\right)}{\left(\begin{array}{c}{n}_{w_1}+n_{w_2}\\ i\end{array}\right)}{\ell }_{j{e}_{w_1}+k{e}_{w2}+x_{\cap }}^{E^{\prime },{\tau }_{w_1}{e}_{w_1}+{\tau }_{w_2}{e}_{w_2}}=\mathbb{P}\left(X_{\text{Leaves}(E)}|X_{w_1}^{\left({\tau }_{w_1}\right)}+X_{w_2}^{\left({\tau }_{w_2}\right)}=i,X_{\cap }=x_{\cap }\right)$$

$$B_{i,j}=\frac{\left(\begin{array}{c}{n}_v\\ j\end{array}\right)\left(\begin{array}{c}{n}_{w_1}+n_{w_2}-n_v\\ i-j\end{array}\right)}{\left(\begin{array}{c}{n}_{w_1}+n_{w_2}\\ i\end{array}\right)}.$$

Then the conditional likelihood at E is given by

$${\ell }_{k{e}_v+x_{\cap }}^{E,0}={\left[B^{+}{y}^{{\mathbf{x}}_{\cap }}\right]}_k,$$ (9)

with B⁺ denoting the Moore-Penrose pseudoinverse of B.

Proof.

Let $X_{\cap }=X_{K_E}^{(0)}-X_v^{(0)}{e}_v=X_{K_{E^{\prime }}}^{(0)}-X_{w_1}^{(0)}{e}_{w_1}-X_{w_2}^{(0)}{e}_{w_2}$ be the vector of allele counts on K_E∩K_E′ at times 0. Then note that

$$\begin{gathered} y_i^{x_{\cap }}=\sum _{\begin{array}{c}j,k:\\ j+k=i\end{array}}\frac{\left(\begin{array}{c}{n}_{w_1}\\ j\end{array}\right)\left(\begin{array}{c}{n}_{w_2}\\ k\end{array}\right)}{\left(\begin{array}{c}{n}_{w_1}+n_{w_2}\\ i\end{array}\right)}{\ell }_{j{e}_{w_1}+k{e}_{w_2}+x_{\cap }}^{E^{\prime },{\tau }_{w_1}{e}_{w_1}+{\tau }_{w_2}{e}_{w_2}} \\ =\sum _{\begin{array}{c}j,k:\\ j+k=i\end{array}}\mathbb{P}\left(X_{w_1}^{\left({\tau }_{w_1}\right)}=j,X_{w_2}^{\left({\tau }_{w_2}\right)}=k|X_{w_1}^{\left({\tau }_{w_1}\right)}+X_{w_2}^{\left({\tau }_{w_2}\right)}=i,X_{\cap }=x_{\cap }\right){\ell }_{j{e}_{w_1}+k{e}_{w_2}+x_{\cap }}^{E^{\prime },{\tau }_{w_1}{e}_{w_1}+{\tau }_{w_2}{e}_{w_2}} \\ =\mathbb{P}\left(X_{\text{Leaves}(E)}|X_{w_1}^{\left({\tau }_{w_1}\right)}+X_{w_2}^{\left({\tau }_{w_2}\right)}=i,X_{\cap }=x_{\cap }\right) \end{gathered}$$

with the second equality following from exchangeability (Lemma 6) and the third equality from X_Leaves(E) = X_Leaves(E′).

Next note that

$$\begin{gathered} \mathbb{P}\left(X_{\text{Leaves}(E)}|X_{w_1}^{\left({\tau }_{w_1}\right)}+X_{w_2}^{\left({\tau }_{w_2}\right)}=i,X_{\cap }=x_{\cap }\right) \\ =\sum _{j=0}^{n_v}\mathbb{P}\left(X_{K_E}^{(0)}=j{e}_v+x_{\cap }|X_{w_1}^{\left({\tau }_{w_1}\right)}+X_{w_2}^{\left({\tau }_{w_2}\right)}=i,X_{\cap }=x_{\cap }\right) \\ \times \mathbb{P}\left(X_{\text{Leaves}(E)}|X_{K_E}^{(0)}=j{e}_v+x_{\cap },X_{w_1}^{\left({\tau }_{w_1}\right)}+X_{w_2}^{\left({\tau }_{w_2}\right)}=i\right) \\ =\sum _{j=0}^{n_v}\frac{\left(\begin{array}{c}{n}_v\\ j\end{array}\right)\left(\begin{array}{c}{n}_{w_1}+n_{w_2}-n_v\\ i-j\end{array}\right)}{\left(\begin{array}{c}{n}_{w_1}+n_{w_2}\\ i\end{array}\right)}\mathbb{P}\left(X_{\text{Leaves}(E)}|X_{K_E}^{(0)}=j{e}_v+x_{\cap },X_{w_1}^{\left({\tau }_{w_1}\right)}+X_{w_2}^{\left({\tau }_{w_2}\right)}=i\right) \end{gathered}$$

with the second equality again due to exchangeability (Lemma 6).

Define $\mathcal{I}\subset \left\{n_v+1,n_v+2,\dots \right\}$ so that $\left\{1,\dots ,n_v\right\}\cup \mathcal{I}$ are the indices in v of the first $n_{w_1}$, $n_{w_2}$ alleles in w₁, w₂. Then because $X_{v,\mathcal{I}}^{(0)}=X_{w_1}^{\left({\tau }_{w_1}\right)}+X_{w_2}^{\left({\tau }_{w_2}\right)}-X_v^{(0)}$ and Lemma 7,

$$\mathbb{P}\left(X_{\text{Leaves}(E)}|X_{K_E}^{(0)},X_{w_1}^{\left({\tau }_{w_1}\right)}+X_{w_2}^{\left({\tau }_{w_2}\right)}\right)=\mathbb{P}\left(X_{\text{Leaves}(E)}|X_{K_E}^{(0)},X_{v,\mathcal{I}}^{(0)}\right)=\mathbb{P}\left(X_{\text{Leaves}(E)}|X_{K_E}^{(0)}\right)$$

and thus

$$y_i^{x_{\cap }}=\sum _{j=0}^{n_v}\frac{\left(\begin{array}{c}{n}_v\\ j\end{array}\right)\left(\begin{array}{c}{n}_{w_1}+n_{w_2}-n_v\\ i-j\end{array}\right)}{\left(\begin{array}{c}{n}_{w_1}+n_{w_2}\\ i\end{array}\right)}{\ell }_{j{e}_v+x_{\cap }}^{E,0}=\sum _{j=0}^{n_v}{B}_{ij}{\ell }_{j{e}_v+x_{\cap }}^{E,0}$$

so letting ${\ell }^{\prime }={\left[{\ell }_{j{e}_v+x_{\cap }}^{E,0}\right]}_{0\le j\le n_v}\in {ℝ}^{n_v+1}$, we have $y^{x_{\cap }}=B{\ell }^{\prime }$ and therefore ${\ell }^{\prime }=B^{+}{y}^{x_{\cap }}$. □

A.2.6. Proof of Lemma 3

Lemma 3 (Population split, 2 clusters). Let E be a split event with parent population v and child populations w₁, w₂ Suppose E has 2 child events E₁′, E₂′, with corresponding blocks $K_{E_1{}^{\prime }}$, $K_{E_2{}^{\prime }}$, where $w_1\in K_{E_1{}^{\prime }}$, $w_2\in K_{E_2{}^{\prime }}$. Let x₋₁ be allele counts on $K_{E_1{}^{\prime }}\backslash \left\{w_2\right\}$ and x₋₂ be allele counts on $K_{E_2{}^{\prime }}\backslash \left\{w_2\right\}$. Then the conditional likelihood at E is

$${\ell }_{x_{-1}+x_{-2}+x_v{e}_v}^{E,0}=\sum _{\stackrel{x_1,x_2:}{x_1+x_2=x_v}}\frac{\left(\begin{array}{c}{n}_{w_1}\\ x_1\end{array}\right)\left(\begin{array}{c}{n}_{w_2}\\ x_2\end{array}\right)}{\left(\begin{array}{c}{n}_v\\ x_v\end{array}\right)}{\ell }_{x_1{e}_{w_1}+x_{-1}}^{E_1{}^{\prime },{\tau }_{w_1}{e}_{w_1}}{\ell }_{x_2{e}_{w_2}+x_{-2}}^{E_2{}^{\prime },{\tau }_{w_2}{e}_{w_2}}.$$ (8)

Proof. Notice that

$$\begin{gathered} {\ell }_{x_{-1}+x_{-2}+x_v{e}_v}^{E,0}=\sum _{\begin{array}{c}{x}_1,x_2:\\ x_1+x_2=x_v\end{array}}\mathbb{P}(X_{K_{E^{\prime }{}_1}}^{\left({\tau }_{w_1}{e}_{w_1}\right)}=x_{-1}+x_1{e}_{w_1},X_{K_{E^{\prime }2}}^{\left({\tau }_{w_2}{e}_{w_2}\right)}=x_{-2}+x_2{e}_{w_1}|X_{K_E}^{(0)}=x_{-1}+x_{-2}+x_v{e}_v)\times {\ell }_{x_1{e}_{w_1}+x_{-1}}^{E_1{}^{\prime },{\tau }_{w_1}{e}_{w_1}}{\ell }_{x_2{e}_{w_2}+x_{-2}}^{E_2{}^{\prime },{\tau }_{w_2}{e}_{w_2}} \\ =\sum _{\stackrel{x_1,x_2:}{x_1+x_2=x_v}}\frac{\left(\begin{array}{c}{n}_{w_1}\\ x_1\end{array}\right)\left(\begin{array}{c}{n}_{w_2}\\ x_2\end{array}\right)}{\left(\begin{array}{c}{n}_v\\ x_v\end{array}\right)}{\ell }_{x_1{e}_{w_1}+x_{-1}}^{E_1{}^{\prime },{\tau }_{w_1}{e}_{w_1}}{\ell }_{x_2{e}_{w_2}+x_{-2}}^{E_2{}^{\prime },{\tau }_{w_2}{e}_{w_2}} \end{gathered}$$

with the first equality from the Markov property of the Moran process, and the second equality following from the exchangeability of the n_v alleles at vertex v (Lemma 6). □

A.2.7. Proof of Theorem 2

Theorem 2. For ${\pi }^d\in {ℝ}^{n_d+1},d\in \left\{1,\dots ,\mathcal{D}\right\}$, the tensor dot product of the SFS ϕ against ${\pi }^1\otimes \cdots \otimes {\pi }^{\mathcal{D}}={\left[{\pi }_{z_1}^1\cdots {\pi }_{z_D}^{\mathcal{D}}\right]}_{z_1,\dots ,z_D}$ is

$$\begin{gathered} \varphi \odot \left({\pi }^1\otimes \cdots \otimes {\pi }^{\mathcal{D}}\right)=\sum _{z_1,\dots ,z_{\mathcal{D}}}{\varphi }_{z_1,\dots ,z_{\mathcal{D}}}{\pi }_{z_1}^1\cdots {\pi }_{z_{\mathcal{D}}}^{\mathcal{D}} \\ =\text{DP}\left({\pi }^1,\dots ,{\pi }^{\mathcal{D}}\right)-\left(\prod _{d=1}^{\mathcal{D}}{\pi }_0^d\right)\text{DP}\left(e_0,\dots ,e_0\right)-\left(\prod _{d=1}^{\mathcal{D}}{\pi }_{n_d}^d\right)DP\left(e_{n_1},\dots ,e_{n_{\mathcal{D}}}\right). \end{gathered}$$

Proof. Below, we will prove DP $\left({\ell }^1,\dots ,{\ell }^{\mathcal{D}}\right)$ is a multilinear function of the input vectors ${\ell }^1,\dots ,{\ell }^{\mathcal{D}}$. The result immediately follows from this, because then

$$\begin{gathered} \varphi \odot \left({\pi }^1\otimes \cdots \otimes {\pi }^{\mathcal{D}}\right)=\sum _{\mathbf{z}\mathbf{\ne }\mathbf{0}\mathbf{,}\mathbf{\text{n}}}{\varphi }_z{\pi }_{z_1}^1\cdots {\pi }_{z_{\mathcal{D}}}^{\mathcal{D}} \\ =\sum _{z\ne 0,n}\text{DP}\left({\pi }_{z_1}^1{e}_{z_1},\dots ,{\pi }_{z_{\mathcal{D}}}^{\mathcal{D}}{e}_{z_{\mathcal{D}}}\right) \\ =\text{DP}\left(\sum _{z_1=0}^{n_1}{\pi }_{z_1}^1{e}_{z_1},\dots ,\sum _{z_{\mathcal{D}}=0}^{n_{\mathcal{D}}}{\pi }_{z_{\mathcal{D}}}^{\mathcal{D}}{e}_{z_{\mathcal{D}}}\right)-\text{DP}\left({\pi }_0^1{e}_0,\dots ,{\pi }_0^{\mathcal{D}}{e}_0\right)-\text{DP}\left({\pi }_{n_1}^1{e}_{n_1},\dots ,{\pi }_{n_{\mathcal{D}}}^{\mathcal{D}}{e}_{n_{\mathcal{D}}}\right) \\ =\text{DP}\left({\pi }^1,\dots ,{\pi }^{\mathcal{D}}\right)-\text{DP}\left({\pi }_0^1{e}_0,\dots ,{\pi }_0^{\mathcal{D}}{e}_0\right)-\text{DP}\left({\pi }_{n_1}^1{e}_{n_1},\dots ,{\pi }_{n_{\mathcal{D}}}^{\mathcal{D}}{e}_{n_{\mathcal{D}}}\right). \end{gathered}$$

We now show DP $\left({\ell }^1,\dots ,{\ell }^{\mathcal{D}}\right)$ is a multilinear function of ${\ell }^1,\dots ,{\ell }^{\mathcal{D}}$. We start by showing that if event E has leaf populations Leaves(E)=(d₁,…, d_L), then ℓ^{E, t} is a multilinear functions of ${\ell }^{d_1},\dots ,{\ell }^{d_L}$. We show this by induction over (E,t). The base case, where t = 0 and E is a leaf with K_E = (d_i), is trivially true because ${\ell }^{E,0}={\ell }^{d_i}$.

For the next case, we note that (6), (7), and (9) express ℓ^{E, t} as a tensor product of ℓ^{E′, t′} with a matrix, where (E′, t′) ≺ (E, t) and Leaves(E) = Leaves(E′). ℓ^{E′, t′} is a multilinear function of ${\ell }^{d_1},\dots ,{\ell }^{d_L}$ by induction, hence ℓ^{E, t} is also.

Similarly, (8) expresses ℓ^{E, 0} as a product of ${\ell }^{E_1{}^{\prime },t_1}$ and ${\ell }^{{E^{\prime }}_2,t_2}$, where ${\ell }^{{E^{\prime }}_i,t_i}$ is a multilinear function of (ℓ^k:k ∈Leaves(E′_i)} by induction, and furthermore Leaves(E′₁)∩Leaves(E′₂) = ∅ and Leaves(E′₁)∩Leaves(E′₂) = Leaves(E) Thus ℓ^{E, 0} is a multilinear function of ${\ell }^{d_1},\dots ,{\ell }^{d_L}$.

Thus, each of the operations (6), (7), (8), (9) in Algorithm 2 preserves multilinearity of ℓ^{E, t}, and thus each ℓ^{E, t} is a multilinear function of its leaf vectors by induction. Finally, a similar induction argument shows that each ϕ^E is a multilinear function of {ℓ^k : k ∈ Leaves(E)}, since (10) expresses ϕ^E as a sum of multilinear functions by induction. □

A.3. Computational complexity

Computing the SFS ϕ_z Algorithm 2 involves keeping track of the partial likelihoods ${\ell }_{x,z}^{E,0}$ at each event E. For a dataset with s unique observations z, the array of partial likelihoods for every z and every hidden state x at event E is ${\ell }^{E,0}={\left({\ell }_{x,z}^{E,0}\right)}_{x,z}$ a tensor with $s\prod _{v\in K_E}\left(n_v+1\right)=O\left(s\prod _{v\in K_E}{n}_v\right)=O\left(s{n}^{\left|K_E\right|}\right)$ total entries (where $n=\sum _{d=1}^{\mathcal{D}}{n}_d$).

We now consider the time cost of each operation in Algorithm 2. We start by considering the “universal constants” that depend only on n_v, and not on the parameters of the demography or the number of SNPs. Next, we consider terms that need to be computed once per demography, but don’t depend on the number of SNPs. Finally, we consider those terms that need to be recomputed once for each SNP and each demography.

Proposition 1. For a population graph $\mathcal{G}$ with corresponding event tree $\mathcal{T}$, let |V| be the number of vertices (populations) in the graph, and let M_admix be the number of admixture events. Then, for a dataset with n samples and s unique observations, computing the likelihoods of d different demographies with the same topology {$\mathcal{G}$ and $\mathcal{T}$) but different parameters (sizes, pulse strengths, branch lengths), has a computational complexity of

$$O\left(\left|V\right|n^3+M_{\text{admix}}\left(n^5+d{n}^4\right)+d\left|V\right|n^2+ds\sum _{E\in \mathcal{T}}{n}^{\left|K_E\right|+1}\right).$$

Proof. In Lemma 1, we compute the matrix exponential of (6) as $e^{Q^{\left(n_v\right)}{\int }_0^{{\tau }_v}}{}^{\frac{1}{{\eta }_v(t)}dt}=U{e}^{\Sigma {\int }_0^{{\tau }_v}\frac{1}{{\eta }_v(t)}dt}{U}^{-1}$ where U are the eigenvectors in the decomposition $Q^{(n_v)}=U\mathrm{\Sigma }{U}^{-1}$; this eigenvalue decomposition costs $O\left(n_v^3\right)$. Similarly, in Lemma 4 we compute the pseudoinverse of (9) as B⁺ =VΣ⁻¹U^T for SVD B = UΣV^T; this SVD also costs $O\left(n_v^3\right)$ Both the eigenvalue decomposition and the SVD are universal for all demographic parameters, and computed at most once per population v, which contributes an overall complexity of O(V|n³).

The innermost sum of (7) in Lemma 2 is also universal, and costs $O\left(n_w^5\right)$ to compute for all possible values of x₁,x₂,x_w,m₁ leading to the term O(M_admixn⁵). However, the middle sum of this equation is not universal, since it depends on q₁, q₂. While not universal, this middle sum only needs to be computed once per demography, and does not depend on the number of SNPs; in particular, computing it for all values of x₁,x₂,x_w, costs $O\left(n_w^4\right)$ per demography, contributing an overall complexity of O(M_admixdn⁴) similarly, the term in $f_{n_v}^v(k)$ in (10) of Lemma 5 is computed once per demography, and costs $O\left(n_v^2\right)$ (Kamm et al., 2017), contributing a complexity of O(d|V|n²).

Finally, we consider the terms that need to be recomputed for every SNP and every demography. Equations (6), (7), (8) of Lemmas 1, 2, 3, respectively, each cost $O(n^{\left|K_E\right|+1})$ per SNP, since they express ${\ell }_{x,z}^{E,0}$ as a sum of O(n) terms. Similarly, equation (9) of Lemma 4 costs $O(n^{\left|K_{E\prime }\right|+1})$, since we compute ${\ell }_{x,z}^{E,0}$ as a product of an O(n) × O(n) matrix B⁺ with a tensor $y^{x_{\cap }}$ containing $O(n^{|K_{E\prime }|-1})$ entries; $y^{x_{\cap }}$ is itself computed by summing over all $O(n^{|K_{E\prime }|})$ entries of ${\ell }_{x,z}^{E\prime ,{\tau }_{w_1}{e}_{w_1}+{\tau }_{w_2}{e}_{w_2}}$. Altogether, this contributes a complexity of $ds\sum _E{n}^{|K_E|+1}$. □

A.4. Application supplement

A.4.1. Data

Table 3 gives the populations and samples we used for our example application in Section 4. All samples except for MA1 were taken from the SGDP dataset (Mallick et al., 2016). We added on the additional low-coverage MA1 sample (Raghavan et al., 2014) to represent the Ancient North Eurasian (ANE) component of European ancestry. Due to the low coverage of MA1, we only sampled a single random allele from it at each site, using a GATK pileup (McKenna et al., 2010) and restricting ourselves to reads with quality ≥30.

Table 3.

Populations and samples used for the example application in Section 4. We only used 1 random allele for MA1 due to low coverage. The ages of the samples are given in thousands of years ago (kya).

Population	Individuals	Alleles	kya
Mbuti	3	6	0
Han	2	4	0
Sardinian	2	4	0
Loschbour	1	2	7.5
LBK (Stuttgart)	1	2	8
MA1 (Mal’ta)	1	1	24
Ust’Ishim	1	2	45
Altai Neanderthal	1	2	50

We ascertained SNPs at SGDP filter level 1, then used the genotype calls at filter level 0 at the ascertained SNPs. When ascertaining SNPs, we limited ourselves to sites that were polymorphic among the samples excluding MA1 and Neanderthal. We excluded MA1 during ascertainment due to its low coverage. We also excluded the Neanderthal sample during ascertainment because it had substantially fewer new mutations than expected based on its age; it was unclear whether this was due to changes in the mutation rate on that lineage since its deep split with modern humans, or whether this was an artifact of the SNP calling strategy used by SGDP. We used Theorem 2 to correct the normalizing constant of the SFS due to excluding MA1 and Neanderthal during ascertainment; in particular, we normalized the SFS by the total branch length of the subtree excluding MA1 and Neanderthal.

To avoid biases in ancient DNA caused by deamination (Dabney et al., 2013), we removed all transitions (i.e. A ↔ G and C ↔T mutations), keeping only the transversions. We used Chimp as a proxy for the ancestral allele, removing all sites where the Chimp allele was missing. After data cleaning, we were left with 2,444,888 autosomal transversion SNPs that were segregating among the samples excluding MA1 and Neanderthal.

A.4.2. Model fitting procedure

We fit the model in Figure 3 in an iterative fashion, adding populations in one at a time and re-estimating the parameters. We started with a tree including the 4 populations Mbuti, Loschbour, Han, and Ust’lshim, with no admixture events. This initial model had 8 parameters: the population sizes at the Han, Mbuti, and Loschbour leaves, the population size at the Eurasian and human ancestor (the Ust’lshim leaf was set to the ancestral Eurasian population size), and the times that the Han, Ust’lshim, and Mbuti populations diverged from Loschbour.

We next added in the Neanderthal population, with an admixture event from Neanderthal to the Eurasian ancestor. We added parameters for the Neanderthal-human split time, the Neanderthal-Eurasian introgression tie and strength, the ancestral Neanderthal-human population size, and a Neanderthal population exponential decline rate starting at the Eurasian-Mbuti split, following results from Prüfer et al. (2014).

We followed by adding on the MA1 sample, adding a single parameter for its split time (we fixed its population size to be the ancestral Eurasian population size). We then added the LBK early farmer, adding 4 parameters for its divergence time, Basal Eurasian admixture time and strength, and the split time of the Basal Eurasian lineage. Finally, we added in a Sardinian population, along with parameters for its population size, its split time, and admixture times and strength from the Loschbour WHG.

At each step, we re-estimated all parameters using the L-BFGS-B optimization algorithm, maximizing the multinomial composite likelihood (11). For the final few models (adding in LBK and then Sardinian) we initialized each estimation with a stochastic gradient descent before finishing with L-BFGS-B.

A.4.3. Mutation rate estimation

We used the within-population nucleotide diversity to estimate the mutation rate. The nucleotide diversity is the number of sites where 2 random alleles (drawn without replacement) differ. The empirical value of the nucleotide diversity in population i is

$${\widehat{\pi }}_i=\sum _{s\in \text{SNPs}}2\frac{x_i^{(s)}\left(n_i-x_i^{(s)}\right)}{n_i\left(n_i-1\right)}=\sum _{x}2\frac{x_i\left(n_i-x_i\right)}{n_i\left(n_i-1\right)}{\text{f}}_x,$$

while the expected value of the nucleotide diversity per unit mutation rate is

$${\pi }_i=\frac{1}{\theta }\mathbb{E}\left[{\widehat{\pi }}_i\right]=\sum _{x}2\frac{x_i^{(s)}\left(n_i-x_i^{(s)}\right)}{n_i\left(n_i-1\right)}{\varphi }_x.$$

This yields a mutation rate estimate ${\widehat{\theta }}_i$ for each population, given by ${\widehat{\theta }}_i=\frac{{\widehat{\pi }}_i}{{\pi }_i}$. We then averaged over ${\widehat{\theta }}_i$ to obtain our combined estimate $\widehat{\theta }=\frac{1}{\mathcal{D}}\sum _{i=1}^{\mathcal{D}}{\widehat{\theta }}_i$. To obtain the per-site mutation rate we need to divide by the number of bases L after our data cleaning process. We approximated this as follows:

1. At SGDP filter level 1, there are about 2.13 × 10⁹ sites per individual.

2. Multiply this by 0.93 due to excluding the sex chromosomes.

3. Multiply this by 0.32 due to only using transversions.

4. Multiply by 0.93 to account for excluding sites missing the Chimp allele.

5. Finally, multiply 1.1 to account for using genotype calls at filter level 0. The inflation in the number of sites here is because we are using a stricter filter to ascertain SNPs (SGDP filter level 1) than we are to call genotypes at the SNPs (SGDP filter level 0). In particular, we need to account for SNPs where the sample has a heterozygous genotype at filter level 0, but has a missing genotype at filter level 1 ; these genotypes are counted as heterozygotes in the SFS, and we need to adjust the number of bases L to account for this.

The latter 2 factors (accounting for sites missing the Chimp allele and for adding in genotype calls at filter level 0) we estimated by observing how the number of heterozygotes per individual changed after these data cleaning steps.