Comparing Partition and Mixture Models with Akaike Information Criteria

Abstract

Sophisticated phylogenetic models often include mixture and/or partition model components. It was recently noted that information criteria tend to favour partition models over mixture models even in some cases where the latter are misspecified and give poor topological estimation. We show that this problem arises because partition models and mixture models fundamentally differ in their probability calculations: mixture models calculate likelihood contributions from sites as the marginal probability of the data averaging over parameter vectors that might have arisen at a site whereas partition model likelihood contributions are calculated as the probability of the site pattern conditional upon a fixed assigned parameter vector at that site. These differing probability calculations lead to AIC estimates that are not always comparable. We explore three generally applicable ways of correcting the issue.

For large sequence alignments it has become commonplace in phylogenetic estimation to model the heterogeneity of evolutionary processes over sites. These models are generally either mixture models (Yang 1994; Lartillot and Philippe 2004; Wang et al. 2008; Le et al. 2008; Jayaswal et al. 2014) or partition models (Yang 1996; Lanfear et al. 2012; Wang et al. 2019). In mixture models, several evolutionary processes are modeled but which process is operating at each site is treated as unknown. In partition models, specific processes are assigned to specific groups of sites a priori. Partition models are natural when process variation can be associated with known features of sites, common examples being codon positions and genes. Mixture models are natural as adjustments for heterogeneity of process when specific sites giving rise to such heterogeneity cannot be identified a priori, such as selective constraints due to site-specific structural or functional roles.

In analyses involving partition models or mixtures, model selection is frequently performed to determine which substitution model to use, to determine whether certain partitions can be taken as sharing some or all parameters, or to determine the number of components of the mixture. Common model selection procedures include likelihood ratio tests, cross-validation, or use of information criteria such as the Bayesian information criterion (BIC) or Akaike information criterion (AIC).

Due to uncertainty about whether groups of sites have been partitioned correctly or whether partitioned structure is the major driver of process heterogeneity, a model selection problem of particular interest is choosing between mixture models and partition models. In a surprising and thought-provoking recent paper, Crotty and Holland (2022) used some simple mixture and partition model comparisons to show that AIC heavily favours partition models over mixture models. Although not considered, the BIC criterion shows a similar bias. Here, as in Crotty and Holland (2022), our focus will be on corrections to AIC.

Liu et al. (2023)followed up on Crotty and Holland (2022), again focusing on comparison of mixture and partition models, but added BIC and investigated whether the explanation may have to do with how well AIC did at estimating its target, what they refer to as relative expected Kullback-Leibler divergence (rEKL). The settings considered were different and included model misspecification. They found that although AIC could give very poor estimates of rEKL with misspecification, there remained a favouring of partition models over mixtures even if the target rEKL quantites were used in place of AIC. Jhwueng et al. (2014) point out that as an estimator of rEKL, AIC has a positive bias due to not taking topology into account; this bias becomes small with very large sequence lengths as in the Crotty and Holland (2022) example. Seo and Thorne (2018) had earlier proposed modified versions of AIC and BIC that penalize partitioned models less severely. Their focus, however, was on using information criteria to compare models with differing numbers of partitions. Such adjustments, if not also made to mixture models, could exacerbate problems comparing mixtures and partition models.

We argue that the reason for the bias that Crotty and Holland (2022) discovered is that two differing types of probability calculations are being conducted. Mixture models assume that some parameter vector in the model (edge-lengths in the main example here) comes from a random distribution and gives as a site-wise likelihood contribution the marginal probability of data, averaging over the parameter vectors that might have arisen. They thus model the marginal probabilities of site patterns or the probability of data at a randomly selected site. In contrast, partition models calculate probabilities conditional upon which parameter vector was assigned to the site. They thus model the conditional probabilities of site patterns or the probability of data at a randomly selected site within a partition. Because the probability calculations are not the same, the target information quantities that AIC approximates are not always comparable.

We show below that this problem explains the bias in favour of partition models observed in Crotty and Holland (2022), and propose a number of solutions to the problem. Since the root of the problem is that an AIC calculated from conditional probabilities (i.e. the AIC calculated from a partitioned model) cannot be compared directly to an AIC calculated from marginal probabilities (i.e. the AIC calculated from a mixture model), our first two solutions address the problem by proposing new probability calculations for each approach which allow direct comparison of AIC scores. A similar problem with AIC has been found in statistical random effects models (Hodges and Sargent 2001; Spiegelhalter et al. 2002; Vaida and Blanchard 2005) with a similar solution. In our case, we first propose that once a partition model has been estimated, the parameters of that model can be used to calculate marginal site probabilities, which can be used to calculate a marginal AIC (mAIC) for the partition model. We show that the mAIC for a partition model can be compared directly to the AIC for a mixture model. Second, we propose that once a mixture model has been estimated, the parameters of that model can be used to calculate conditional site probabilities, which can be used to calculate a conditional AIC (cAIC) for the mixture model. We show that the cAIC for a mixture model can be compared directly to the AIC for a partition model. The third approach that we consider is to nest both the mixture and partition model in a model that allows mixtures within partitions (the MWP model). Below, we explain each of these approaches in more detail, describe the pros and cons of each, and illustrate the approaches using the mixture of edge-lengths example in figure 1 of Crotty and Holland (2022).

Motivating Example

The specific setting that Crotty and Holland (2022) consider in their figure 1, which forms the basis for our running example, is a replication of the Kolaczkowski and Thornton (2008) experiments. The four-taxon nucleotide data sets for these experiments were simulated with a Jukes-Cantor substitution model and topology AB|CD. The data consisted of two partitions each with 5000 nucleotides; so n=10000. The first partition was generated with edge-lengths , and middle edge-length . The second partition was generated with roles reversed:   and . To introduce model misspecification, Crotty and Holland (2022) moved of the sites from Partition 1 to 2 and vice versa; here .

In Crotty and Holland (2022) a mixture-of-edge-lengths model estimated the correct AB|CD topology 100% of the time regardless of ρ. The partition model estimated AB|CD  100% of the time when the partition model was correctly specified (ρ =0), 97% of the time when the partition model was misspecified by moving 5% of the sites between the two partitions (ρ =0.05), but estimated the long-branches together tree AC|BD  100% of the time when 10% or more of the sites were moved between partitions (). Thus, for topological estimation, it would be desirable if the mixture model were to be selected whenever and perhaps even with ρ =0.05. Instead, Crotty and Holland (2022) found that for , AIC is highly likely to select the partition model. Although it has been noted before that improving model fit can make tree estimation worse (Gaut and Lewis 1995; Yang 1997; Shavit Grievink et al. 2010), in all such examples that we are aware of, as sequence length increases, model selection chooses the correct models which have large probability of selecting the correct tree. That did not happen in the Crotty and Holland (2022) example and we argue below that such model inconsistency can be expected to arise more generally when comparing mixture and partition models.

AIC for Mixtures and Partition Models

We consider the comparison of mixture and partition models in generality. Under both partition and mixture models, the data at a site, (its site pattern), are independent across sites i=1,… ,n; is an example site pattern in a four-taxon setting; a site is the index i of a location in the alignment and a site pattern, , is the data at that site. We let \boldsymbol x denote all of the data, .

Both partition and mixture models have class labels associated with each site; partition labels is the more common language for partition models. Both models are of the form where ζ denotes the set of parameters determining all of the class specific distributions. Usually some subset of ζ is used to determine p(x|k;ζ ). For instance, for the Crotty and Holland (2022) example, ζ consists of sets of edge lengths and for each class and a topology τ. The model for the kth class is where gives the probability of site pattern x under the Jukes-Cantor model with topology τ and edge-lengths . We use the notation for some class-specific parameter. We let γ denote all parameters.

We explicitly allow that there may be differing numbers of classes for the two models: R for the partition and K for the mixture model; we use K when these are assumed to be the same. We use the same notation, γ , for all of the parameters in the model, and ζ for the within-class parameters in p(x|k;ζ ); for the partition model but includes additional weight parameters for the mixture model. This allows us to state equations that apply to both models rather than repeating them. It is important to emphasize, however, that in comparing a partition and mixture model, ζ may refer to very different parameters for the two models. In addition, some sets of parameters may be shared by some or all classes. For instance, we might compare a partition model where both the edge-lengths and rate matrix parameters vary across partitions (ζ is the set of edge-length vectors and rate matrices, ) with a mixture model where edge-lengths vary over sites but rate matrix parameters are fixed (ζ is the set of edge-length vectors and includes a shared rate matrix parameter Q).

The partition model assumes the set, , of sites from partition r have the same class label. It uses a log likelihood where probabilities are calculated conditional upon the labels,

The model has where denotes the maximizer of and denotes the number of free parameters in the model. We see that the partition model uses conditional probability calculations approximating , the probability of site pattern x for a randomly selected site from partition r; we use the subscript 0 to emphasize that it is a true distribution and n because later sections will consider limiting properties as (dependence on n is implied by dependence on the number of sites, , in ).

The mixture model does not assume knowledge of what sites share the same class label. Instead, independently, with probability . It uses a log likelihood where probabilities are marginal rather than conditional on class labels.

The model has AIC where and denote the maximizers of and the K-1 accounts for the additional w parameters. By contrast with the partition model, the mixture model uses marginal probability calculations and approximates the probability of site pattern x for a randomly selected site from the entire data set \boldsymbol x.

AIC Approximation

In order to motivate different information criteria for partition and mixture models, it helps to recall that the original motivation for AIC in Akaike (1973) was to determine the model that, for a data set \boldsymbol x used for estimation, gives the best prediction of an independent data set \boldsymbol y generated from the same true distribution, , as Because there are two different data sets, we use the notation , and so forth to explicitly indicate dependence of the maximum likelihood estimators on the estimation data set, \boldsymbol x, alone. We also use and to denote a log likelihood and maximum likelihood estimator that could refer to either the mixture or partition model.

Let denote the entropy of a distribution p and let

be the Kullback-Leibler divergence of p from q, which provides a measure of how distant the distribution q is from p. Akaike (1973) argued that AIC estimates a measure of predictive value, AI, that cannot be calculated directly, since it depends on the true distribution, ; in what follows we add a ‘C’ when referring to the data-based estimation (criterion) of an information quantity of interest.

(1)

Note that is a constant in Equation (1) that does not depend on the model. Thus, if AI is small, then there is a small expected Kullback-Leibler divergence between the model-based distribution and the true distribution. An alternative motivation for AI approximation comes from -AI/2 being the expected log likelihood for new data when evaluated at the maximum likelihood estimate from an independent data set. Thus, when it is large, new data is expected to have high probability or be well predicted by the model. This quantity is referred to as the expected predictive log likelihood (EPLnL) in Susko and Roger (2020).

Information quantities are most commonly presented for scenarios where individual observations (site patterns here) are independent and have identical distributions. Mixture models satisfy this property but partition models give rise to different distributions across different partitions. Moreover, we are interested in cases with misspecification of partition labels. Accordingly, we consider what AI simplifies to in a very general framework where is the true probability of site pattern x at site i; note that although are allowed to differ over sites, they need not. For this distribution and any partition of sites, can be interpreted as the probability that a randomly selected site from partition r has site pattern x. Note that this definition applies to any set of partitions. They could be arbitrary, be such that sites within have the same , or be related by virtue of having some meta property that makes their distributional properties of interest, like being in the third codon position or from a particular gene. The marginal distribution or probability of site pattern x at a randomly selected site is .

For any partition of sites, , if the model is such that is constant as a function of i within partitions, AI in Equation (1) simplifies to

Since , taking and gives

(2)

This is a measure of the ability to predict, on average, the site pattern of a randomly selected site within a partition, or how distant the within-partition distribution, , is to .

For a partition model with partitions , calculations are conditional upon class labels and every site within partition r has label r, so and Equation (2) gives

(3)

For a mixture model, calculations are marginal, so and Equation (2) gives

(4)

For a mixture model, however, a different interpretation is possible. Since Equation (2) applies regardless of the partitioning, if we treat all of the data as a single partition, then Equation (2) gives

(5)

This is a measure of the ability to predict the site pattern of a randomly selected site, or how distant the marginal model is to .

We see from Equation (3) that the AI for a partition model measures how well sites are predicted within partitions and then calculates a weighted average over partitions. Alternatively, it measures the weighted average of the Kullback-Leibler divergences of the within-partition distributions from their estimates under the model.

Equations (4) and (5) show that AI has an additional interpretation for mixture models. Similarly as for partition models, Equation (4) gives that AI for a mixture model can be interpreted as a measure of how well sites are predicted within partitions. Equation (5), however, shows that it can also be interpreted as a measure of how well a randomly selected site is predicted. Equivalently, it measures the Kullback-Leibler divergence of the marginal distribution of site patterns from its estimate under the mixture model.

Which version of AI should be the target of an information criteria depends on the underlying goals. If we are interested in partition specific properties, like the partition-specific rate of substitution, a measure of how well sites are predicted within partitions is appropriate and AI for a mixture and a partition can often be compared. A caveat that is discussed below arises when the mixture model utilizes the partition model assumption that each site in the partition has the same class label (shared class labels). In this case an alternative information criterion for the mixture is needed. If instead we are interested in global properties, like the best tree for the whole alignment, we want to use a measure of how well a randomly selected site is predicted. AI for the mixture gives such a measure but for the partition model we don’t have that and need to develop an alternative measure.

cAIC and mAIC

The first set of solutions that we provide is to compare like with like. If global performance is of interest, comparison should be of estimates of a quantity that measures how well a randomly selected site is predicted or, equivalently, up to a constant, measures divergence of the true marginal distribution, to the model-based estimate of it. In what follows, this will be defined as mAI and its estimate as mAIC; depending on the model, mAI and mAIC may also be AI and AIC for that model. If performance within partitions is of interest, comparison should be of estimates of a quantity that measures how well sites are predicted within partitions or, equivalently, measures divergences of the true conditional within-partition distributions, to the model-based estimates of them. In what follows, this will be defined as cAI and its estimate as cAIC; depending on the model, cAI and cAIC may also be AI and AIC for that model. Table 1 provides a summary.

Table 1.

The information criteria used to approximate information for the conditional within-partition distribution and the marginal distribution .

	cAI ()	mAI ()
Partition	AIC	mAIC
Mixture (Dependence)	cAIC	AIC
Mixture (Posterior Mean)	AIC	AIC
Mixture Within Partitions	AIC	mAIC

Notes: AIC indicates that default AIC is used. Mixture (Dependence) refers to a mixture model where within-partition parameters are estimated by the posterior means of Equation (7) that assume parameters within a partition are shared across sites whereas Mixture (Posterior Mean) refers to a mixture model where posterior means are calculated by Equation (9) allowing each site within a partition to have its own parameter vector.

For the partition model, the natural estimate of is where ; is the number of sites in partition r. Thus, for the partition model,

(6)

where is the estimated parameter vector for the partition model. The natural estimate of mAI for the partition model is then

We see that for the partition model, use of mAIC is like treating the partition model as a method for estimating mixture parameters, a method that utilizes (the potentially incorrect) information about which sites correspond to which mixture classes. For the mixture model, mAI is the AI in Equation (5) and AIC is its corresponding mAIC.

If performance within partitions is of interest, a quantity comparable to Equation (3) is needed. For mixtures these can arise in two ways. One is when the mixture model is used to estimate within-class partition parameters assuming shared labels; utilizing information from other partitions about plausible parameter values may give better estimation properties. As a concrete example, suppose that is a class specific parameter of interest, and let be the data for the rth partition. Then an estimate from a mixture that uses sharing of class labels is

(7)

Here is an estimate of the probability that the class label for partition r is k given the data for the partition. We refer to class-specific estimates, , obtained in this way as Mixture (Dependence) estimates in the Results section since they assume dependent class label assignment within a partition. In this case the in Equation (2) is where include the from Equation (7) in their parameter set.

More generally let denote parameters derived from a mixture model that determine the class-specific distribution Then for a mixture model that uses sharing of class labels within a partition, the comparable measure of how well sites are predicted within partitions to the AI for a partition model is

(8)

For a mixture model that does not utilize sharing of class labels cAI is simply its AI as given in Equation (4) and for a partition model cAI is also its AI as given in Equation (3). One could use a mixture model that does not utilize sharing of class labels to infer within partition parameters as posterior means:

(9)

Here is an estimate of the probability that the class label for site i is k given the data for the site. We refer to class-specific estimates, , obtained in this way as Mixture (Posterior Mean) estimates in the Results section.

To summarize, cAIC for a partition model is just its AIC. For a mixture that does not utilize sharing of labels cAIC is also its AIC; this is alternatively its mAIC; which label we use depends on the context. For a mixture model that does utilize sharing of class labels within a partition,

AIC for A Mixtures within Partitions Model

A final approach to the problem is to consider the usual AIC for a Mixtures Within Partitions (MWP) model that nests the partition and mixture models. The way to do this is to keep the parameter vector, ζ , that determines the within-class distributions, p(x|k;ζ ), the same over partitions but allow the weights of the model, , to depend on the partition r. If for all partitions r, implying that all sites in partition r have class parameter vector , this model corresponds to the partition model. If is independent of the partition r that it comes from, the MWP model corresponds to the usual mixture model. Let ω denote the collection of all weight vectors, .

The log likelihood for this model is

and AIC is . The MWP model provides a direct estimate, of the within-partition distribution, and so its AIC can be considered its cAIC, an estimate of the cAI in Equation (2).

Since the proportion of sites from partition r is , under the MWP model, the chance that a randomly selected site is from class k is estimated as and thus an estimate of is given by . Although the number of parameters estimated by the MWP model is the mAIC is for a marginal model with parameters ζ and K-1 weights. That the K-1 weights are determined by the R(K-1) weights in the MWP model does not affect the arguments leading to mAIC approximation, which depend only on the in the estimate of . Thus mAIC for the MWP model is

In principle, obtaining the AIC for this model requires that maximum likelihood estimation be implemented for an entirely new model. The approach that we take is to use the EM algorithm (Dempster et al. 1977) to update the within-partition weights for 100 iterations, leaving fixed at their estimated values under the mixture model. Starting with the from the mixture model, we update via

Here is the probability that a site is from class k given that its site pattern was x, calculated using the coming from the jth iteration of the EM algorithm. The method requires only the posteriors for each class in the model and requires little computational time. It will give a larger AIC than would have been obtained with full maximum likelihood estimation of all parameters but avoids requiring implementation of a new model. If conditions hold for the mixture model to give consistent estimates of ζ, then the approach using the EM algorithm above will give statistically consistent estimators under mild conditions. We show this rigorously in the supplementary material.

AIC Comparisons

We have presented a number of different information criteria choices. These are summarized in Table 1

Fig. 1 shows the mean AIC and target AI quantities for the motivating example based on 100 simulated data sets per value of ρ. Simulated data sets are available on a Dryad repository. Denote the resulting data sets as The target information for a given ρ, is of the form where the nature of the likelihood depends on the model and whether mAI or cAI is being approximated. It was approximated by

Figure 1.

The information criteria (right panels) and mean estimates (left) with 10th and 90th percentiles (vertical bars). Mixture (Posterior Mean) uses the average posterior mean edge-lengths over sites to estimate the edge-lengths for a partition and Mixture (Dependence) uses the posterior mean assuming all sites within a partition came from the same partition. Horizontal lines indicate the mean mAIC values for the mixture.

(10)

Different expressions are needed for different models, but the inner expectation can be expressed as a weighted sum of expectations, each of which depends on a single site pattern, . This allows exact calculation of the inner-expectation because expectations of functions of a single involve sums over only terms. Relative to the scale of the plots, the cAIC quantities appear to closely approximate the corresponding cAI. The standard errors for the approximations in Equation (10) to information are relatively small for mAI (< 2) and larger for cAI (< 8 except for cAI for the Mixture (Dependence) which increased from 6.1 when ρ =0 to 38.7 when ρ =0.5).

Fig. 1 helps to clarify why comparing AIC values computed from partitioned models and mixture models can lead to poor model selection. Crotty and Holland (2022) compared what we refer to as the cAIC for the partition and Mixture (Posterior Mean) model in Fig. 1C. One can see that, in agreement with Crotty and Holland (2022), cAIC for the partition model is substantially lower than mAIC for the mixture model as long as ; vertical bars indicate 10th and 90th percentiles. Thus, again, as shown in Crotty and Holland (2022), the partition model is favoured in spite of model misspecification and it always gives the wrong topology for ρ >0.05.

If mAIC is used to compare models, we see that the problem largely goes away (Fig. 1A). The mAIC values for the mixture and MWP model are comparable but the mAIC for the poorly performing partition model is much worse except for small values of ρ, those being the values where the partition model is close to correct. Over the 100 simulations, when there is no model misspecification (ρ =0), mAIC selected the mixture model 67 times and the partition model 33 times. It is a little surprising that the mixture model is selected more frequently than the partition model. One might expect the partition model to give slightly better edge-lengths and consequently a better mAIC than the mixture, since it uses knowledge of which edge-lengths correspond to which partitions. For ρ >0, the mixture model was always the model selected by mAIC.

The MWP model was never selected by mAIC which is something that, in theory, should always happen with mAIC as long as the number of classes, K, and within-class models, p(x|k;ζ ), are the same, and even with full maximum likelihood estimation for the MWP model. The reason is that mAIC for the mixture and MWP model use the same marginal log likelihood, , but with different estimates of the weights, and possibly ζ; the ζ are the same as in the mixture for the partial MWP maximum likelihood estimation here but would differ with full MWP maximum likelihood estimation. The w and ζ parameters are chosen to give the largest marginal log likelihood for the mixture and thus will always give a larger marginal log likelihood for the mixture than for the MWP model. Since the mixture model has a larger marginal log likelihood than the MWP model and since it subtracts off the same number of parameters prior to multiplication by -2, it must give a smaller mAIC. When comparison of models have different ζ, however, this need not be the case. This depends on the nature of the base mixture vs partition model comparison that led to an MWP model. For instance, suppose that the mixture model was a mixture model for edge-lengths but the partition model allowed different edge-lengths and rate matrices across partitions. One might add an MWP model to model selection that was more consistent with the partition model. In this case mAIC for the MWP model could be larger than for the mixture.

cAIC for the MWP (triangles) model is usually lower than all other cAIC criteria (all other symbols), regardless of the setting (Fig. 1C). The MWP model was selected 100% of the time for 0.05 < ρ < 0.5 (Table 2). When ρ =0, the partition model was selected 100 times. Although misspecified, it was selected 45 times with ρ =0.05; the MWP model was selected the remaining 55 times. When ρ =0.5, the mixture model was selected 95% of the time. When ρ =0.5, a randomly selected site within a partition has a 50-50 chance of being from one class or the other. Thus the marginal distribution over all partitions, which is what the mixture approximates, is the same as the conditional distribution within a partition; the mixture model gives a correctly specified within-partition site pattern distribution.

Table 2.

The percentage of times each model was selected by a class of AIC measures. The models compared are the partition model (Part), mixture model (Mix) and the MWP model. Reading left to right, results for a given criteria are given (eg. cAIC) followed by the corresponding results using cross validation.

mAIC				Cross Validation
ρ	0	0.05	0.10-0.45	0.5	0	0.05	0.1	0.15	0.2	0.25	0.3	0.35	0.4	0.45	0.5
Part	33	0	0	0	95	0	0	0	0	0	0	0	0	0	0
Mix	67	100	100	100	5	87	67	67	68	53	68	57	55	49	49
MWP	0	0	0	0	0	13	33	33	32	47	32	43	45	51	51
cAIC				Cross Validation
ρ	0	0.05	0.10-0.45	0.5	0	0.05	0.1	0.15	0.2	0.25	0.3	0.35	0.4	0.45	0.5
Part	100	45	0	0	100	43	0	0	0	0	0	0	0	0	0
Mix	0	0	0	95	0	0	0	0	0	0	0	0	0	0	94
MWP	0	55	100	5	0	57	100	100	100	100	100	100	100	100	6

Parameter Estimation Comparisons

The expectation is that good model fit, as measured by information criteria, will give rise to good parameter estimation. However, because different information criteria are designed to measure distances to different distributions, this leads to different expectations about implications for estimation. Models with good mAI are expected to give good estimators of topologies and of the distribution of site patterns at a randomly selected site. Models with good cAI are expected to give good estimators of within-partition parameters.

In our running example, when ρ >0, the model for a partition r is misspecified. Nevertheless, the average expected number of substitutions per site along an edge e for a partition is still well defined. Let denote the edge-length for site and let denote the edge-length for partition r when there is no misspecification. Then the expected average number of substitutions per site for partition r is

where is the other partition: when r=1 or 1 when r=2. To get an overall measure of how well we did at estimating within-partition expected substitutions we use the sum of branch score distances (Kuhner and Felsenstein 1994) over partitions which includes topology estimation in its calculation. For a partition r, it is the Euclidean distance between vector for the rth partition and its estimate, where denote expected numbers of substitutions along terminal edges and entries     gives expected substitutions for splits AB|CD, AC|BD and AD|BC; these are set to the internal edge-length if the split is present and 0 if it is not.

We also compare the resulting marginal distributions of branch scores for a randomly selected site. This is the parameter of interest for the mixture but is not as naturally of interest for partition models and mixture within partition models. It is, however, the type of parameter that one expects a model with good mAI to do well at estimating. For the MWP model, the weights in the distribution are weighted averages over partitions. Let denote the kth branch score. Then its weight is obtained as For the partition model, the come from the individual partitions and have associated weights . As a simple distance measure of estimation performance we consider the Euclidean distance between mixture parameter vectors, . Labeling of classes is arbitrary. To adjust for this, for each estimated mixture, we choose the labeling that gives the smallest Euclidean distance.

Results are given in Fig. 2. The variation for a given setting and method across the 100 simulations as measured by the 10th and 90th percentiles is small relative to the differences in means. Estimation of the marginal mixture distribution (Fig. 2A) is comparable for the mixture model and MWP across settings. These models give much better estimation than the partition model when and comparable choices for ρ =0.05. When ρ =0, the partition model tends to give better estimation as might be expected because the partition model is not misspecified in this case and utilizes the additional information about which sites share parameters. When mAIC selected the mixture model which gave more accurate mixture of branch scores than the partition model but slightly smaller mean mixture of branch scores than the MWP model; taking 10th and 90th percentiles into account estimation performance was virtually tied (Fig. 2A). When ρ =0, however, the opposite is true - mAIC selected the partition model in only 33 of 100 simulations (Table 2), despite the partition model giving the most accurate mixture distributions of edge lengths (Fig. 2A).

Figure 2.

The mean distances between parameter estimates and true parameters with 10th and 90th percentiles (vertical bars) for the marginal distribution of branch scores (A) and the within-partition branch scores (B).

Results for within-partition estimation (Fig. 2B) largely agree with trends in cAIC (Fig. 1C). Branch score distances for a mixture using posterior mean edge-lengths assuming all sites within a partition share the same parameter - Mixture (Dependence) - is very small for small ρ and, in agreement with its cAIC, increases as ρ increases. From Fig. 2A one can see that the mixture model gave accurate estimates of the branch scores in the mixture distributions. The difficulty for estimation of within-partition substitutions is that they are a weighted average, , for partition r. Because the posterior in Equation (7) is constructed using many sites, the posterior mean heavily favours one or the other which does not yield good estimates with substantial misspecification.

Branch score distances for the partition model are very good with small ρ but increase as ρ increases, much like their cAIC. For the other two methods, cAIC trends disagree to some extent with branch score distance trends. The branch score distance for the MWP model remains very low whereas its cAIC increases as ρ increases. By contrast, branch score distances decrease yet cAIC is roughly constant for the mixture using posterior mean edge-lengths assuming independent class assignment across sites within a partition - Mixture (Posterior Mean). The reason for this is that the inherent hardness of prediction increases as ρ increases; it becomes increasingly hard to use that sites are most likely from one class. One sees this in the increase in cAI for the MWP model which has a small KL divergence to the true within-partition distribution across values of ρ. The increase in cAI must be due to an increasing entropy contribution to the target cAI in Equation (2). We see that AI is an indicator of both estimation performance and inherent hardness. Since the inherent hardness remains constant for a fixed parameter setting this suggests relative comparisons of estimation performance should agree with relative comparisons of cAIC.

At a given ρ , relative comparisons of cAIC in Fig. 1C largely agree with those having to do with parameter estimation in Fig. 2B. There are some exceptions, however. For instance, at ρ =0.25, the Mixture (Posterior Mean) edge lengths are clearly better than those from the partition model but the cAIC for the partition model tends to be better. The branch score distance for the correctly specified MWP model is always small. The partition model gives better parameter estimation when ρ =0, as expected; the model is correctly specified and there is no variability due to weight estimation as with the other methods. The Mixture (Posterior Mean) estimates are very good when . When ρ =0.5, the mixture model gives a correctly specified within-partition site pattern distribution.

Estimation of mAI and cAI

In our running examples, the cAIC and mAIC estimates all used the conventional correction of subtracting the number of parameters from the conditional or marginal log likelihood and then multiplying by -2. This is consistent with the derivation of Akaike (1973) but that derivation assumed that some parameter choice for the fitted model gives the true model. When the fitted model and true model differ, the number of parameters might need to be replaced by something that could be quite different. For instance, Figure 1 of Konishi and Kitagawa (1996) considers an example of a two-parameter normal fitted model where in the true model some fraction of observations come from a different distribution. They show that the correct penalty to subtract off prior to multiplying by −2 can be as large as 5 rather than the 2 suggested by a parameter count.

If the correct model is a partitioned model, possibly with some error in site assignment, as in the running example, then the mixture model is not technically the correct model but it turns out that this does not create a problem for the Akaike (1973) result when using AIC. This may not be immediately apparent based on the description of the true simulating model as involving generation of site patterns within partitions, each having different edge-lengths and then swapping of sites from different partitions. For the mixture model, the reason is that the true marginal model for the probability of site pattern y at a randomly selected site, , is a special case of the mixture model with weights . When there is independence over sites, even if this independence holds only conditional upon the classes for sites, and the model includes the true marginal distribution, arguments almost identical to Akaike (1973) can be applied. Similarly, usual AIC corrections apply to AIC, mAIC and cAIC for the MWP model.

For the mixture model using a posterior mean but assuming shared class labels within a partition - Mixture (Dependence) - and cAIC, it is not clear that the corrections of Akaike (1973) apply. The arguments of Akaike (1973) use the assumption that the true model is a special case of the fitted model. Similarly, cAIC and mAIC for the partitioned model become problematic when that model is incorrect, as is the case for the running example when a partition model is used with ρ >0.

In contrast to the assumptions required by the AIC, cross-validation (Stone 1977; Susko and Roger 2020) provides a generally applicable approximation to an information criterion in the case where data are assumed to be independent across sites. We use 10-fold cross-validation. Let denote the log-likelihood of interest for data \boldsymbol y, the one that would be used for the corresponding cAIC or mAIC calculation, which may represent the original, conditional or marginal log likelihoods. Let denote the estimated parameters based on data \boldsymbol x. What represents depends on the context. For instance, when calculating mAIC for the partition model, represents the marginal mixture distribution implied by the partition model. By contrast, for cAIC for a mixture model it might be the posterior mean edge-lengths for each partition.

Ten-fold cross-validation randomly divides the data into subsets where each of the are the data for a set of sites and of roughly the same size; small differences in size might arise if the total number of sites is not divisible by 10. The same set of should be used for each model being compared. For partition/mixture model comparisons, the subsets should be constructed by aggregating, over partitions, the obtained after random subdivision of the data within the partition into 10 equal-size sets.

Let denote data \boldsymbol x but taking out . Because is independent of , has expectation where \boldsymbol y is a data set of size n and \boldsymbol w is a data set of size 9 n/10 that is independent of \boldsymbol y; is used instead of to adjust for the differing sizes of and \boldsymbol y. The cross-validation estimate is then

(11)

Note that Equation (11) estimates , where w is of length 9n/10. The target information replaces with where \boldsymbol x is of length n. Because the predictive value of an estimator usually improves with more data, Equation (11) is expected to overestimate . An alternative approximation from Burman (1989) is where

Fig. 3 gives the mean differences between estimates and the corresponding information. A main observation from Fig. 1 that is reinforced here by the difference in scales between Fig. 1 and Fig. 3 is that the most important determinant of which model will be selected is usually the information used, not how it was estimated. Somewhat surprisingly, simply subtracting 2 times the number of parameters (AIC) gives a good approximation even in cases, such as the results for mAI with partitions, where there is model misspecification when ρ >0 (Fig. 3B) or when cAI is approximated for a mixture model that gave poor estimation (Fig. 3E and F); AIC derivations assume parameters converge upon true parameter values.

Figure 3.

The estimated mean differences between approximations and their target information criteria. Vertical lines give 95% confidence intervals for the mean difference. Panel titles indicate the information quantity being approximated. For Fig. 3A, because a posterior mean estimate of within-partition parameters from the mixture assumes the same mixture model holds for all partitions, the cAI is the same as the mAI.

Table 2 gives the percentages of times each model was selected using either an AIC-based criteria or cross validation (cross-validation sub-table to the immediate right for each criteria); the Burman correction is not shown because it gave similar percentages as cross validation. Cross validation and AIC gave very similar percentages for many settings. The notable exception was the results for mAIC. When ρ =0, mAIC selected the partition model only 33% of the time whereas cross validation selected it 95% of the time. From Fig. 2A we know the estimation of the mixture distribution is a little better under the partition model in this case and, more generally, we expect the partition model to perform better when class labels match partitions exactly. Thus the behaviour of cross validation is more desirable because it selected the better-performing partition model 95% of the time.

When ρ > 0, mAIC selected the mixture model over the MWP model. Cross validation, by contrast, sometimes selected the mixture model and, usually less frequently, selected the MWP model. Recall that mAIC will always select the mixture over the MWP model, even with full maximum likelihood estimation. Cross validation has the advantage of allowing other models to be selected. Both mAIC and cross-validation approximate AI. We see in Fig. 1A that compared to the differences in mAI between the mixture and partition model, the differences between the mixture and MWP models are small. In more detail, across values of ρ, the difference in mAI between the MWP and mixture model was always less than 7 in absolute value; note that mAI is in the range of 93,000 for these models across ρ. Approximate mAI favoured the mixture model when and the MWP model otherwise. These are approximations, however. To check whether results might be explained by the finite (B=100) number of data sets in the approximation given by Equation (10) we conducted paired z-tests of the null hypothesis that mAI for the mixture and MWP model are the same. This test rejected the null for providing evidence for a significant but only slightly better mAI for the mixture model for these settings.

Model Selection Inconsistency

In this section we establish that the problem found in Crotty and Holland (2022) is more general, applies to both conventional AIC and BIC and does not go away with large samples. The latter point was clear for the Crotty and Holland (2022) example as the number of sites within partitions was large.

We consider behaviour as the number of sites within partitions grow large: n and . Accordingly we let and denote the limiting probability of site pattern x for a site selected at random and for a site selected at random from partition r. We let denote the long run proportion of sites from partition r.

We also assume that the partition and mixture models are comparable. If this is not the case then it is not at all clear that it is undesirable that the partition model be selected. Formally, there are K partitions and the partition model for the rth partition is p(x|r;ζ ) whereas the mixture model assigns probability regardless of the partition. Here, by contrast with much of the paper, we are assuming that p(x|r;ζ ) is the same model for both the mixture and partition model.

We assume that the true model has some proportion (possibly 0) of sites in partition r that come from a different class. Formally, for some and . Let denote the limiting proportion of among . Then and where Let ω denote the collection of and let denote the collection of true weights, . We let be the ω () for which the partition model is correct.

We assume that is not the same as for at least one pair r and k. This implies that when the partition model is correctly specified () there is no set of mixture parameters for which the mixture model gives the correct and ; this follows from the fact that the mixture model assigns the same probabilities to site patterns regardless of their partition. Thus the mixture model is misspecified as a model for site pattern probabilities within partitions and, if , so is the partition model. On the other hand, because is a mixture model, the mixture model is always correctly specified as a model for the site pattern at a randomly selected site. In the supplementary material we discuss additional mild conditions that are sufficient for results below to hold.

Our main result is that there exists a range of partition model misspecification parameters (value of near ) such that AIC and BIC will select the partition model with probability 1 for all large n. The result does not imply poor parameter estimation of the selected partition model but, because it is misspecified, that is a realistic possibility; the Crotty and Holland (2022) example illustrates that it occurs. By contrast, because the mixture model gives the correct marginal model, , it can provide statistically consistent estimation of and its parameters. If the model is correctly specified, this implies it should be capable of estimating the correct tree, whereas the misspecified partition model selected by AIC and BIC cannot really be expected to give good estimates. Thus it is important to have alternatives like mAIC to default AIC and BIC that recognize that the mixture model does a better job of estimating the marginal model.

In more detail, AIC/n and BIC/n are both of the form, where is the log likelihood and is the maximum likelihood estimator for whatever model is being considered, and is a penalty term for the number of parameters in the model that satisfies that . As a consequence, whenever converge upon different values for two models, the one with the smaller limiting value will be preferred for large samples by both AIC and BIC; the penalties will eventually be irrelevant in such cases.

In the supplementary material we establish that under mild conditions, the limiting AIC/n and BIC/n () for the partition model is

(12)

and is the parameter vector ζ that minimizes the weighted average Kullback-Leibler divergence, . Since the probabilities of x within partitions, depends on the proportions of sites that are misclassified, which is determined by , we use the notation to indicate that dependence of the minimizer on the . When and thus the partition model is not misspecified, the KL divergences, which are always non-negative, can be set to 0 by setting . So

(13)

Two expressions can be given for the limiting AIC/n and BIC/n for the mixture model. The result in the supplementary material gives

where is the minimizer of the Kullback-Leibler divergence of to . Since the mixture model is never misspecified marginally, no matter how sites are misspecified to partitions, the Kullback-Leibler divergence is minimized by choosing so that which gives a Kullback-Leibler divergence of 0. To make the expression more comparable to Equation (12) we express it as

(14)

We see by comparing Equation (13) and Equation  (14) that when the partition model is correctly specified, the difference in limiting AIC/n and BIC/n (mixture - partition) is

Note that this number could be very large if the show a lot of variation over partitions. It isn’t hard to see that is a continuous function of . Since p(x|r;ζ ) is also assumed a continuous function of ζ, it follows that the limiting AIC/n and BIC/n for the partition model is a continuous function of as well. The limiting AIC/n and BIC/n for the mixture model is constant which implies that the difference is continuous as well. Since this difference converges upon as converges upon , it follows that there exists ρ >0 such that for , with probability 1, for all large n, AIC and BIC will select the partition model.

In summary, no matter how classes are misspecified to partitions, the mixture model is not misspecified as a model for . Equation (14) emphasizes that the mixture model is, however, misspecified as a model for when the partition model is correctly specified and may be badly misspecified. This leads to the partition model being selected when partitions are not correctly specified, even with infinite data. We expect that if we correctly specify the marginal model, then global parameters, including trees will be correct. Thus we want model selection procedures that allow us to recognize correct marginal specifications.

Discussion

We have provided three solutions to the problems of comparing mixture and partition models. The first two solutions are to compare like with like. Either estimate the ability of the model to predict site patterns for a randomly selected site (mAIC) or the ability of the model to predict site patterns within partitions (cAIC). Which to use depends on what the end goal is. If it is to have good estimates of within-partition parameters, cAIC is natural. If the goal is to estimate a good marginal model, use mAIC. Knowing that a competing model has good or even better properties at a marginal level is valuable even if within-partition properties are of primary interest, and this cannot be discerned from cAIC alone. We thus generally recommend a cross-check with mAIC.

One challenge for cAIC is comparison of models with differing partitions. For instance one model under consideration might use the three codon positions specifying an amino acid as 3 partitions and another model might use genes as partitions. What our analysis indicates is that default AIC for partition models is comparing the ability of differing models to approximate different choices of the probability that a randomly selected site from partition r has site pattern x. One could still use cAIC if interest is in within-partition performance. In that case, what definition of partition one uses in cAIC depends on what partitioning scheme one is interested in. Use of mAIC bypasses these problems because it is partition independent.

Assuming that the MWP and mixture model are comparable, in the sense that the number of classes and within-class models, p(x|k;ζ ), are the same, that a MWP model is better within partitions than a mixture model can be deduced from cAIC. However, the mAIC will always prefer the mixture model over the MWP model, even with full maximum likelihood estimation. This is because the mixture model maximizes the marginal log likelihood, , that is used in mAIC calculation by both MWP and the mixture models.

Theorem 3.1 in Konishi and Kitigawa (1996) considers properties of usual AIC in a setting where the model is correctly specified. They show that AIC approximates AI in Equation (1) even if the estimator is not the maximum likelihood estimator. Since the maximum likelihood estimator always gives a smaller AIC, it is the estimator that gives the best predictions, up to the error in approximation of AI by AIC. Estimators can have better AI than the maximum likelihood estimator, even with large samples and correct models. But in order for this to occur, it must be the case that their difference in AICs (or equivalently log likelihoods) is smaller in magnitude than the errors in approximating AI by AIC. Since those approximation errors are expected to decrease to 0 as sample sizes increase, their difference in log likelihoods must also be decreasing to 0. In order for this to occur, the two estimators must usually be very similar to each other with large samples, more similar to each other than the true parameter, since it has a log likelihood difference to the maximum likelihood estimator that does not converge to 0; it usually converges to a chi-square distribution.

The relevance of this to the comparison of MWP and the mixture models is that although mAIC will always favour the MWP model due to its larger marginal likelihood, it is possible that the MWP model may have smaller mAI in some settings. The difference in approximations to mAI (Mixture - MWP) in Equation (10) was always less than 7 in absolute value for mAI values that were in the neighbourhood of 93,000. These approximations were obtained using a partial optimization strategy for MWP and so it is possible better mAI might have been found with full optimization. The results for AIC alluded to above suggest that even if this is the case, with large samples, the estimates of parameters in the marginal model from the MWP model are likely to be similar to those from mixture. It is possible that more substantial mAI differences could arise with smaller sample sizes, harder estimation problems or more complex models. In such cases, full implementation of the MWP model could be valuable.

More generally, with smaller samples, when the model is misspecified or to adjust for the error in approximation of mAIC and cAIC, in order to recognize that better cAI or mAI is achievable from some other estimator, an alternative estimate of cAI or mAI is needed. Cross validation provides one approach and showed some advantages when estimating mAI (Table 2). Cross validation is, however, computationally intensive. Future work could be directed towards asymptotic approximations to the correction term to be subtracted from the log likelihood in obtaining cAIC or mAIC, or towards more efficient resampling schemes. For the experiments considered here, which involved large samples, correction was not a crucial issue. The enormous variation in the target cAI quantity in Fig. 1D relative to the errors in estimation in Fig. 3 indicate that, for most settings the main factor causing one model to be preferred over another is that it is the model with the best cAI.

A question that arises in the case of mAIC is whether those marginal likelihoods should be used for downstream inferences such as topology estimation. If model selection using mAIC selects the partitioned model, should the tree be inferred using the mixture model implied by the partition parameters (as used in the mAIC), or is it safe enough to infer a topology with a partitioned model? This has important implications for empirical work, because tree searching under mixture models is typically slower and more compute-intensive than tree searching under partitioned models. For mixtures there are strategies that can be used to reduce search times. These include using the Posterior Mean Site Frequency Profiles (PMSF) model of Wang et al. (2018) which uses a more computationally efficient single frequency profile for each site but allows these to vary over sites; this approach could be used to generate a set of candidate topologies that could be used with a fixed mixture. In terms of computational efficiency, it should be pointed out that mixtures often use fixed frequency profiles (Le et al. 2008), which improves computational efficiency. In any case, we believe that the conditional likelihood can be used with a partition model if it is selected. The main goal here is to make comparison of the models fairer. Using AIC directly with whatever likelihood was natural for the model being fitted would be more natural but is simply not a feasible option.

Similar difficulties using information criteria have been noted in the linear mixed model literature (Hodges and Sargent 2001; Spiegelhalter et al. 2002; Vaida and Blanchard 2005). Vaida and Blanchard (2005) consider comparison of a dose-response regression model with a linear mixed model. In the regression model, which is akin to a partition model, separate slopes and intercepts are fit to different individuals. Individuals play a role akin to partitions and responses at a given dose play a role akin to sites. In the linear mixed model, slopes and intercepts are modelled as coming from a normal distribution (akin to mixtures here). For their illustrative example they find that the separate slopes model gives a much better AIC (−47.1) than the mixed model (11.0). Their solution is to compare approximation of cAI, as done here, which gives a more comparable value of −44.5 for the mixed model. Vaida and Blanchard (2005) do not consider mAIC for the regression model as their main concern is which model is better for inference about individual dose-response relationships.

The cAIC considered in Vaida and Blanchard (2005) is specific to linear mixed models and, prior to multiplication by −2, subtracts a quantity from the log likelihood that need not match the number of parameters. In contrast, in our approach, the quantities subtracted from the log likelihood in cAIC and mAIC are the number of parameters requiring estimation. Subtracting the number of parameters isn’t generally justified by theory because the true distributions in the information need not be special cases of the fitted model, which is needed for the Akaike (1973) argument to work. In addition to these issues, Susko and Roger (2020) show that there are a number of commonly occurring scenarios in phylogenetic estimation that can cause AIC to be a poor estimate of its target. These scenarios include model misspecification, edge-lengths that are close to 0 and mixture components with negligible weight. Cross-validation provides a generally applicable estimation approach that avoids these problems to some degree. As discussed earlier, future work could be directed towards asymptotic approximations to the correction term to be subtracted from the log likelihood or towards more efficient resampling schemes.

The mixed effects setting considered in Vaida and Blanchard (2005) is not completely analogous because, for a given individual, that model assumes all responses for that individual correspond to the same slope and intercept. By contrast the mixture model here allows different parameters for different sites within a partition. Mixed effects model comparisons with fixed regression models are more analogous to comparison of a partition model with a model where the parameters for each site in a partition are assumed shared but are treated as coming from a distribution. That problems have been noted with such comparisons in the linear mixed model literature suggest that they would arise in phylogenetic settings as well. A phylogenetics example would be gene tree estimation. A partition model approach is to estimate separate gene trees using only the data for a gene. A mixture approach arises if one assumes incomplete lineage sorting is the main reason for incongruence of gene trees (Rannala and Yang 2017); maximum likelihood implementations don’t currently exist to the best of our knowledge. Such an approach can potentially yield better gene trees because it utilizes data from other genes to better estimate the species tree that the gene trees vary around. For model comparison using information criteria, approximations to cAI may provide better ways of comparing partition and mixture models in such cases.

Our third solution was to nest the partition and mixture model as special cases of a within-partition mixture. In the running example, the cAIC for the MWP model was better than that of cAIC for the partition model when ρ >0.05. Thus it appears that there was an avenue for recognizing that it provides the best within-partition branch scores (Fig. 2B). The implementation here that only optimized the within-partition weights is a simple, generally applicable solution when comparing partition models and mixture models. Full implementation of the within-partition model is, in principle, desirable but not a natural avenue for software development given that the MWP model was a device for comparing the models. The partial implementation should, in theory, provide consistent estimators and, for the example, did a very good job estimating mixing distributions and within-partition edge-lengths (Fig. 2).

An additional complication for the MWP model solution is that although only one model needed to be fit in our running example, this need not be the case for more complicated model selection scenarios where there could be multiple partition models and mixture models and where class-specific parameters could vary over these models. The general solution would be to fit a MWP version of each of the models under consideration. The additional computation required for full implementation of the MWP model in this case could be substantial. Using the partial implementation considered here remains feasible and maintains good statistical properties like consistency under mild conditions. Although the within-partition model was a device for comparing the models, its very good estimation performance, particularly of within-partition edge-lengths, suggests that its inclusion could provide a valuable cross-check and expansion of an initial set of models.

The results of Crotty and Holland (2022) are surprising. Our explanation for the lack of comparability of AIC for partition and mixture models is that they are conditioning on different events in their calculations. This would seem to be a general issue to be aware of in comparing models, one of which treats some parameters as fixed and the other as random. The issue may arise in many contexts as partition and mixture models are inter-related and commonly used.

We argued that the problem of model selection inconsistency will not go away with large samples and is expected to occur more broadly. The consistency result leading to that conclusion is that when mixture models and partition models are comparable in the sense that the within-class models, p(x|k;ζ ), are the same, the mixture model parameters will converge upon the parameters of the true marginal distribution with large samples. Usually identifying the correct marginal distribution is sufficient for correct phylogenetic estimation. The consistency result holds when data are generated according to a partition model, possibly with misspecification of true partitions. If there is misspecification of true partitions, the partitioned model need not be consistent for the marginal model but the mixture will be consistent. Thus mixture models enjoy a robustness property where they are expected to work (in the sense of consistent estimation) for partitioned data with or without partition misspecification. Evidence for the challenges associated with partition specification comes from Ren et al. (2025). Using empirical data they found (Figure 2 of Ren et al. 2025) that when attention was restricted to single codon positions within genes (constituting natural partitions) a two parameter mixture was favoured over no mixture. The specified partitions were not sufficient and a mixture or MWP model can be expected to better deal with this additional variation within partitions. Partition models have the advantage that they are more easily implemented and the computational costs of likelihood evaluation are less than for mixtures; strategies such as initial fitting of a PMSF model can reduce the cost of mixture estimation. There may also be scenarios with smaller alignments where knowledge of partition structure aids estimation substantially. Identifying such scenarios requires future work but we expect that they don’t correspond to increasingly common phylogenomic settings. Moreover, as the Crotty and Holland (2022) example illustrated, the effects of partition misspecification can be highly detrimental to phylogenetic estimation.

Conflicts of Interest

There are no conflicts of interest or disclosures.

Funding

This research was supported by Discovery Grants from the Natural Sciences and Engineering Research Council of Canada to A.J.R. and E.S., and the Moore-Simon Project on the Origin of the Eukaryotic Cell, Simons Foundation grant 735923LPI (DOI:https://doi.org/10.46714/735923LPI). R.L. is funded by Australian Research Council Discovery Project (DP200103151).

Data Availability

Simulated data sets are available on Dryad at https://doi.org/10.5061/dryad.3xsj3txrb.