New Methods to Calculate Concordance Factors for Phylogenomic Datasets

Abstract

We implement two measures for quantifying genealogical concordance in phylogenomic data sets: the gene concordance factor (gCF) and the novel site concordance factor (sCF). For every branch of a reference tree, gCF is defined as the percentage of “decisive” gene trees containing that branch. This measure is already in wide usage, but here we introduce a package that calculates it while accounting for variable taxon coverage among gene trees. sCF is a new measure defined as the percentage of decisive sites supporting a branch in the reference tree. gCF and sCF complement classical measures of branch support in phylogenetics by providing a full description of underlying disagreement among loci and sites. An easy to use implementation and tutorial is freely available in the IQ-TREE software package (http://www.iqtree.org/doc/Concordance-Factor, last accessed May 13, 2020).

Measures of branch support such as the bootstrap (Felsenstein 1985; Minh et al. 2013) and Bayesian posterior probability (Ronquist and Huelsenbeck 2003) are important for making robust inferences from phylogenetic trees. However, although these measures provide useful information about the statistical support for a given branch, neither captures the topological variation present in the underlying data (Kumar et al. 2012).

A complementary approach involves calculating the fraction of loci consistent with a particular branch, thus capturing underlying agreement and disagreement in the data. Various approaches have been suggested, including the gene concordance factor (gCF) (Baum 2007)—sometimes also referred to as the “gene support frequency” (Gadagkar et al. 2005; Salichos and Rokas 2013)—and internode certainty (Salichos and Rokas 2013), with the former being the most widely used. The gCF describes for each branch in a reference tree the proportion of inferred single-locus trees that contain that branch. Although intuitive, the gCF suffers from three limitations: first, there are few software implementations (Ané et al. 2007); second, although issues with incomplete taxon sampling of gene trees have been addressed by some authors (Smith et al. 2015; Kobert et al. 2016), these fixes are not available in implementations that allow for the calculation of gCF values; and third, low gCF values are hard to interpret because they may result from strongly supported discordance among individual gene trees or weak phylogenetic signal in individual loci.

Here, we resolve these issues by implementing the calculation of gCF while accounting for unequal taxon sampling in the popular IQ-TREE package (Nguyen et al. 2015; Minh et al. 2020), and by introducing the site concordance factor (sCF), a measure that estimates concordance at the level of individual sites. We argue and show by detailed analyses of ten empirical datasets that concordance factors complement commonly used metrics like the bootstrap by providing additional information and insights about topological variation.

Gene Concordance Factor

For a given unrooted bifurcating reference tree, $T$ (e.g., an estimate of the species tree), and a set of unrooted bifurcating input trees, S = $\left\{T_1,\dots ,T_n\right\}$ (e.g., individual gene trees), we can calculate the gCF for every internal branch $x$ in $T$. Each gene tree, $T_i$, must contain a subset of the taxa in $T$ but need not include all of the same taxa.

Each internal branch $x$ of $T$ is associated with four clades of $T$ representing four taxon subsets, $A,\mathrm{ }B,\mathrm{ }C,\mathrm{ }\text{and}\mathrm{ }D$, such that the bipartition $A\cup B|C\cup D$ corresponds to the branch $x$ (fig. 1). The reference tree $T$ contains branch $x$ by definition, but branch $x$ may or may not be present in a gene tree of the same taxa (or a subset of them). Let $A_i,\mathrm{ }{B}_i,\mathrm{ }{C}_i,\mathrm{ }{\text{and}\mathrm{ }D}_i$ be the set of taxa in $A,\mathrm{ }B,\mathrm{ }C,\mathrm{ }\text{and}\mathrm{ }D$ that are also present in $T_i$. We say $T_i$ is decisive for $x$ if $A_i,\mathrm{ }{B}_i,\mathrm{ }{C}_i,\mathrm{ }{\mathrm{and}\mathrm{ }D}_i$ are nonempty (we use “decisive” in the sense of Sanderson et al. [2011], but note for clarity that Dell’Ampio et al. [2014] use this term in a different way): that is, a gene tree is decisive as long as it contains at least one of the taxa in each of the taxon subsets $A,\mathrm{ }B,\mathrm{ }C,\mathrm{ }\text{and}\mathrm{ }D$. A decisive gene tree can potentially contain the branch $x$. We say $T_i$ is concordant with $x$ if $T_i$ is decisive for $x$ and the bipartition $A_i\cup B_i|C_i\cup D_i$ corresponds to a branch in $T_i$. In other words, a gene tree is concordant with the reference tree if it could have contained branch $x$ (i.e., it is decisive) and it does contain branch $x$. The gCF for branch $x$ is now defined as

$$\text{gCF}\left(x\right)=\frac{\left|\right\{i:T_i \text{is} \text{concordant} \text{with} x\}}{\left|\right\{i:T_i\mathrm{ }\text{is} \text{decisive} \text{for}\mathrm{ }x\}}.$$

Fig. 1.

Schematic view of a bifurcating tree at an internal branch $x$ with four surrounding subtrees containing the taxon sets A, B, C, and D. Branch $x$ corresponds to the bipartition $A\cup B|C\cup D$. In our definition, branch $x$ is always present in the reference tree, but it may or may not be present in each of the input trees. By applying the two nearest neighbor interchanges (denoted as arrows), one can produce the two other trees that contain taxon sets A, B, C, and D, but with internal branches y and z.

In other words, the gCF is the proportion of input trees decisive for $x$ that are concordant with $x$; that is, the proportion of all input trees that could have contained $x$ that do contain $x$. We note that gCF is also used to calculate internode certainty and related measures (Salichos et al. 2014; Kobert et al. 2016).

To further understand concordant and discordant gene trees, we categorize the discordant input trees (i.e., those that do not contain branch x) into three groups and calculate three discordance factors: gDF₁, gDF₂, and gDF_P. The first two groups are obtained by applying a nearest neighbor interchange around branch $x$ to result in two alternative topologies (fig. 1) with two branches not appearing in $T$: $y=A\cup C|B\cup D$ and $z=A\cup D|B\cup C$. Accordingly, we then calculate discordance factors as $\mathrm{gCF}\left(y\right)=\mathrm{gD}{\mathrm{F}}_1\left(x\right)$ and $\mathrm{gCF}\left(z\right)=\mathrm{gD}{\mathrm{F}}_2\left(x\right)$. Some input trees may not contain any of the branches x, y, or z. This will occur for input trees in which one or more of clades A, B, C, or D are not monophyletic. We define the proportion of decisive input trees that fall into this category as $\mathrm{gD}{\mathrm{F}}_{\mathrm{P}}\left(x\right)$, where the “P” stands for “paraphyly.” The four proportions $\mathrm{gCF}\left(x\right)$, $\mathrm{gD}{\mathrm{F}}_1\left(x\right)$, $\mathrm{gD}{\mathrm{F}}_2\left(x\right)$, and $\mathrm{gD}{\mathrm{F}}_{\mathrm{P}}\left(x\right)$ will sum to 1, as every decisive input tree must be included in one of the four categories.

Site Concordance Factor

To calculate the sCF for branch $x$, we randomly sample $m$ quartets of taxa $q=\{a,\mathrm{ }b,\mathrm{ }c,\mathrm{ }d\}$, where $a,\mathrm{ }b,\mathrm{ }c,\mathrm{ }\text{and}\mathrm{ }d$ are in $A,\mathrm{ }B,\mathrm{ }C,\mathrm{ }\text{and}\mathrm{ }D$, respectively. For each quartet $q$, we examine the subalignment of taxa $a,\mathrm{ }b,\mathrm{ }c,\mathrm{ }\mathrm{and}\mathrm{ }d$. For every site $j$ in this alignment, we call $j$decisive for $x$ if the characters $a_j,\mathrm{ }{b}_j,\mathrm{ }{c}_j,\mathrm{ }\text{and}\mathrm{ }{d}_j\mathrm{ }$are all present and $j$ is parsimony informative when restricted to this quartet of taxa. Decisive sites can be concordant or discordant with $x$. We say that site $j$ is concordant with $x$ if $a_j=b_j\ne c_j=d_j$ (i.e., $j$ supports the bipartition $\{a,\mathrm{ }b\}|\{c,\mathrm{ }d\}$). The concordance factor for $q$ is defined as

$$\mathrm{C}{\mathrm{F}}_q\left(x\right)=\frac{\left|\right\{j:\mathrm{ }j\mathrm{ }\text{is} \text{concordant} \text{with}\mathrm{ }x\}}{\left|\right\{j:j\mathrm{ }\text{is} \text{decisive} \text{for}\mathrm{ }x\}}.$$

Since many such quartets exist around branch $x$ (when sampling individual tips from within A, B, C, and D), we define sCF($x$) as the mean $\mathrm{C}{\mathrm{F}}_q\left(x\right)$ over m random quartets:

$$\text{sCF}\left(x\right)=\frac{1}{m}\left[{\sum }_q\mathrm{C}{\mathrm{F}}_q\left(x\right)\right].$$

Thus, the sCF is the average proportion of sites decisive for $x$ that are concordant with $x$. In effect, the sCF is a measure of concordance for sites that is directly comparable to the measure of concordance for single-locus trees provided by the gCF. Unlike related quartet approaches that are based on maximum likelihood (ML) tree inference and that calculate discordance at the level of the whole alignment (Strimmer and von Haeseler 1997; Pease et al. 2018), the sCF uses parsimony criteria to calculate discordance at the level of individual sites. We note that the sCF is closely related to the values derived from spectral analysis (Hendy and Penny 1993; Charleston 1998). Spectral analysis is a tree-independent method of investigating phylogenetic signal. In the simple case of binary data, such that each site corresponds to a bipartition of taxa, the spectral support for a single branch counts the number of sites that correspond to the bipartition defined by that branch. In the case of binary data, this is the same as the value $\mathrm{C}{\mathrm{F}}_q\left(x\right)$ that we define above. The sCF in this case differs from the spectral support insofar as it is calculated by averaging over a large set of $\mathrm{C}{\mathrm{F}}_q\left(x\right)$ values calculated by repeatedly subsampling quartets of taxa from an alignment.

Similarly to gene discordance factors, we also define the site discordance factors $\mathrm{sCF}\left(y\right)=\mathrm{sD}{\mathrm{F}}_1\left(x\right)$ and $\mathrm{sCF}\left(z\right)=\mathrm{sD}{\mathrm{F}}_2\left(x\right)$. There is no sDF_P category, because unlike the gDF values, every decisive site must contribute to one of the three proportions $\mathrm{sCF}\left(x\right)$, $\mathrm{sD}{\mathrm{F}}_1\left(x\right)$, or $\mathrm{sD}{\mathrm{F}}_2\left(x\right)$; any quartet of taxa resolves into exactly three tree topologies shown in figure 1. In other words, the sum of $\mathrm{sCF}\left(x\right)$, $\mathrm{sD}{\mathrm{F}}_1\left(x\right)$, and $\mathrm{sD}{\mathrm{F}}_2\left(x\right)$ will always be 1.

gCF and sCF for Rooted Trees

The above definitions of gCF and sCF apply only to unrooted trees. However, in the case that users have a rooted reference tree as well as rooted gene trees, we can extend the calculation of the gCF to allow us to calculate different gCF values on either side of the root. To do this we first add a virtual root node into the rooted reference tree $T$, resulting in an unrooted tree $T\mathrm{\text{'}}$. Similarly, we convert the rooted gene trees $T_i$ into unrooted trees $T_i^{\mathrm{\text{'}}}$ by adding the same virtual root. This allows us to then follow the same procedure as above for calculating gCF values on unrooted trees, with the sole difference that the output is a rooted instead of an unrooted reference tree. In this case, it is possible to calculate gCF values on a rooted reference tree where the gCF values on either side of the root may differ. In all other cases (i.e., for sCF values, and if either the reference tree or the gene trees are unrooted), it is not possible to calculate different concordance factors on either side of the root.

Implementation in IQ-TREE 2

We provide two new options, –gcf and –scf, in IQ-TREE version 2 (Minh et al. 2020) to compute gCF and sCF, respectively. A tutorial for how to use these options is provided at http://www.iqtree.org/doc/Concordance-Factor (last accessed May 13, 2020). Both options can be combined in a single run, which will calculate the gCF and sCF for every branch in the input reference tree. Although sCFs can be calculated on any alignment, the calculation of gCFs requires individual gene trees. We have therefore implemented a convenient option, -S, to specify a partition file or a directory of single-locus alignments in which IQ-TREE will infer separate trees for each partition or alignment (Minh et al. 2020).

IQ-TREE provides a suite of output files to assist users in understanding and investigating gCF and sCF values. It provides tree files that can be viewed in most tree viewers, which contain information on both the proportional data (gCF, gDF₁, gDF₂, and gDF_P; sCF, sDF₁, and sDF₂) and their corresponding absolute count data (gCF_N, gDF₁_N, gDF₂_N, and gDF_P_N; sCF_N, sDF₁_N, and sDF₂_N), as well as the number of decisive genes and sites for each branch (gN and sN, respectively). It also provides a tree file that combines information on the gCF, sCF, and any bootstrap values that have been calculated (e.g., fig. 2). In addition to this, IQ-TREE provides a “.cf.stat” file that contains all 16 concordance and discordance values listed above for every branch in the reference tree in a machine-readable tabulated format, and through the –cf-verbose option, it provides tabulated files that detail for every branch in the reference tree whether each gene tree was concordant with that branch (the “.cf.stat_tree” file produced from a –gcf analysis), and the average number of sites in each locus that was concordant with that branch (the “.cf.stat_loci” file produced from a –scf analysis). Together these output files provide both a convenient overview of the data, and the opportunity to understand in much more detail the extent to which each locus is concordant or discordant with the reference tree.

Fig. 2.

An example of concordance factors on a dataset of lizards (Rodriguez et al. 2018). A cladogram is shown to facilitate the plotting of concordance factors on branches. Numbers on each branch show the sCF above the branch (e.g., s73) and the gCF below the branch (e.g., g37). Bootstrap values are 100% on every branch, except for the branch leading to Lipinia rouxi and L. pulchella, which has a bootstrap value of 62%. The inset shows a scatter plot of gCF values against sCF values for all branches, revealing the large range of gCF and sCF values as well as the fact that for this data set sCF values are always at least as large as gCF values. This is likely because of the short length of the UCE loci used to infer gene trees which leads to low gCF values (Rodriguez et al. 2018).

Application to Empirical Data Sets

To demonstrate the use of gCF and sCF values, we first analyzed a dataset containing 3,220 ultra-conserved elements (UCEs) from lizards, totaling 1,301,107 bp for 43 species (Rodriguez et al. 2018). To do this, we estimated a concatenated ML tree, 3,220 UCE trees, and the gCF, sCF, and bootstrap values with IQ-TREE (fig. 2).

To investigate the relationships between dataset size, concordance factors, and bootstrap values, we also analyzed a collection of nine additional empirical phylogenomic datasets that represent a range of clades and data types (table 1). For each dataset, we performed 20 analyses across a 20-fold range of dataset sizes: the first analysis included ten randomly selected loci from the complete dataset, and each subsequent analysis added ten more randomly selected loci up to a maximum of 200 loci. For each such analysis, we calculated the gCF, sCF, and bootstrap values for every branch of the concatenated ML tree estimated from the 200-locus dataset, with the same approach as above. For the lizard dataset, we calculated both standard bootstrap (StdBoot) values (Felsenstein 1985) and ultrafast bootstrap (UFBoot) values (Hoang et al. 2018). As expected (Minh et al. 2013), we observed that StdBoot and UFBoot values were very similar (fig. 3B and supplementary fig. 8D, Supplementary Material online); therefore, because of the high computational cost of calculating StdBoot values, we calculated only UFBoot values for the remaining nine datasets. Since the results of all ten analyses are very similar, we focus here on the lizard dataset and present the results for the other nine datasets in the supplementary figures 1–10, Supplementary Material online.

Table 1.

The Five DNA and Five Amino Acid (AA) datasets Analyzed in This Study.

No.	Reference	Type	Clade	Taxa	Loci	Sites
1	Ballesteros and Sharma (2019)	AA	Chelicerata	53	3,534	1,484,206
2	Branstetter et al. (2017)	DNA	Aculeata	187	807	183,747
3	Cannon et al. (2016)	DNA	Metazoa	78	424	89,792
4	Jarvis et al. (2015)	AA	Aves	52	8,295	4,519,041
5	Misof et al. (2014)	AA	Insecta	144	2,868	595,033
6	Ran et al. (2018)	AA	Spermatophyta	38	1,308	432,014
7	Ran et al. (2018)	DNA	Spermatophyta	38	3,924	1,296,042
8	Rodriguez et al. (2018)	DNA	Prasinohaema	43	3,220	1,301,107
9	Wu et al. (2018)	AA	Mammalia	90	5,162	3,050,199
10	Wu et al. (2018)	DNA	Mammalia	90	15,486	9,150,597

Fig. 3.

Concordance factors remain relatively stable as loci are added to an analysis, whereas bootstrap values continue to increase toward 100%. (A) The results of 20 reanalyses of the lizard dataset, each of which adds a further ten loci to the analysis, up to a maximum of 200 loci (x axis). Each colored line represents a different branch in the tree. The dashed line on the sCF panel shows a value of 33%. (B) For each of the four metrics considered here (represented by different colored lines), the number of loci included in the analysis affects the proportion of the branches that have a value of 100%.

Reproducible analyses are provided in the Supplementary Material online available at https://doi.org/10.5281/zenodo.1949287.

Interpretation of gCF and sCF Values

Our analyses highlight a number of important differences between concordance factors and bootstrap values. First, concordance factors of any type tend to be much lower than bootstrap values or other measures of statistical support for branches in the species tree (e.g., fig. 3 and supplementary figs. 1–10, Supplementary Material online). This is because bootstrap values measure the sampling variance in support of a focal branch, whereas gCF and sCF values measure the underlying variance in support of that branch at the gene- and site-level, respectively. In other words, resampled datasets may always return the same tree (i.e., 100% bootstrap support), even though incomplete lineage sorting or other processes that lead to genealogical discordance are at work (e.g., gCF and sCF values ≪100%). Of particular note is that a very high bootstrap value (e.g., of 100%) does not predict or require a similarly high concordance factor. For example, although all but one bootstrap value on the tree in figure 1 is 100%, the smallest gCF and sCF values are 4.5% and 33.2%, respectively. This pattern is repeated across all 200 analyses we performed on all ten empirical datasets (fig. 3 and supplementary figs. 1–10, Supplementary Material online).

Because they measure different things, concordance factors and bootstrap values are affected very differently by the addition of loci to a dataset (fig. 3 and supplementary figs. 1–10). In all of the ten empirical datasets that we analyzed, adding loci to the dataset tended to increase the bootstrap values of individual branches (e.g., fig. 3A, see also supplementary figures, Supplementary Material online) and concomitantly the proportion of branches with a bootstrap value of 100% (fig. 3B, see also Supplementary Material online). This is expected: the bootstrap is effectively measuring the standard error of the mean in a dataset (where the mean represents the tree inferred from the full dataset), and this standard error will go down with more samples. The same is not true of concordance factors, which display some estimation error when datasets are small (e.g., 30 or fewer loci in fig. 3), but subsequently remain almost completely insensitive to the addition of loci to the dataset (fig. 3 and supplementary figs. 1–10, Supplementary Material online). This is also expected: any measure of the underlying variance (or standard deviation) in a distribution will have some estimation error when the sample size is small but should not change monotonically as the sample size increases. These differences highlight the complementary information that can be gained from calculating both bootstrap values and concordance factors for individual branches. One measure is not better or worse than the other, rather, concordance factors provide useful information that bootstraps do not, and vice versa. Note that bootstrap values or posterior probabilities calculated by resampling gene trees (e.g., Sayyari and Mirarab 2016) have the same behavior as the site-wise bootstrap carried out here and are also not equivalent to concordance factors.

In principle, both gCF and sCF values can range from 0% (no genes/sites are concordant with the focal branch) to 100% (all genes/sites are concordant with the focal branch). In practice however, as exemplified in figure 3 and supplementary figures 1–10, Supplementary Material online, empirical gCF values tend to range from 0% to 100%, whereas empirical sCF values are rarely lower than 33% (represented by the dashed line in the sCF panel of fig. 3A). This is due to an important underlying difference in the way that the two values are calculated. The sCF is calculated from quartets, so a single site can only support one of three topologies (fig. 1). Because of this, if there is no consistent information in an alignment (e.g., if a long alignment were generated at random) we expect a roughly equal proportion of sites supporting each of the three trees, leading to an sCF value of ∼33% (for the same reason, sCF values for very long branches will approach 33% due to saturation). The same is not true for gCF values, because a gene tree can support not only the three possible relationships shown in figure 1 but also any other relationship in which one or more of clades A, B, C, or D are not monophyletic. The higher the number of gene trees in this latter group, the closer the gCF value will be to 0%. Because of this, we should expect gCF values to be particularly low when gene trees are estimated from alignments with limited information or where branch x is extremely short; in such cases either technical or biological processes may increase the proportion of gene trees that fail to recover the monophyly of clades A, B, C, or D found in the reference tree. Missing data may also impact gCF and sCF values, however, the relationship here is less clear. The requirement that a gene tree or a site is decisive (i.e., could in principle contain branch x, see above) should limit the impact of missing data on gCF and sCF estimates. Nevertheless, since phylogenetic estimates are known to worsen as the proportion of missing data increases (e.g., Roure et al. 2013; Xi et al. 2016), it is plausible that concordance factors may systematically decrease as the proportion of missing data increases.

Finally, cases where the sCF value is lower than 33% may be of particular interest. These cases are, by definition, those in which maximum parsimony (MP) would favor a different resolution of a split found in the reference tree. If the reference tree was calculated from any method other than MP, there are at least two explanations for an sCF value lower than 33%. First, the branch of interest may be in an area of parameter space in which high levels of incomplete lineage sorting are known to mislead concatenated ML analyses (Kubatko and Degnan 2007) but not MP analyses (Mendes and Hahn 2018). Misleading reference trees may therefore be produced by either concatenated ML of the entire dataset or gene tree methods that use shorter sets of concatenated loci as their input (because most protein-coding genes are themselves made up of multiple topologies). Second, and more generally, there are multiple reasons why likelihood, Bayesian, or gene tree methods for producing a species tree will differ from MP resolutions. For instance, the branch of interest may be unduly affected by a small number of highly influential sites in a concatenated ML analysis (Shen et al. 2017). In this case, the influential sites can have an outsized influence on the ML resolution of a split because they have extreme differences in likelihood between different resolutions of that split. Because MP does not account for likelihood differences—it instead weights all sites equally—MP analyses remain unaffected by such outliers. Thus, cases in which the sCF is much lower than 33% may merit further investigation.

We hope that the user-friendly implementation of gCF and sCF in IQ-TREE will assist researchers in gaining additional insights into their phylogenetic reconstructions. In particular, we encourage phylogeneticists to calculate both bootstrap values and concordance factors for the branches on their trees, as the two measures provide complementary information that may help to improve the accuracy of our interpretations of phylogenetic reconstructions. Indeed, the use of concordance factors may help to alleviate the commonly cited problem in phylogenomics that bootstrap values provide relatively little information when they are all 100% (Kumar et al. 2012).

Data Availability

Data are available at https://doi.org/10.5281/zenodo.1949287.