Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2014 Jun 1.
Published in final edited form as: Methods Ecol Evol. 2013 Apr 18;4(6):566–572. doi: 10.1111/2041-210X.12042

The mean and variance of phylogenetic diversity under rarefaction

David A Nipperess 1,✉, Frederick A Matsen IV 2
PMCID: PMC3699894  NIHMSID: NIHMS444246  PMID: 23833701

Summary

  1. Phylogenetic diversity (PD) depends on sampling depth, which complicates the comparison of PD between samples of different depth. One approach to dealing with differing sample depth for a given diversity statistic is to rarefy, which means to take a random subset of a given size of the original sample. Exact analytical formulae for the mean and variance of species richness under rarefaction have existed for some time but no such solution exists for PD.

  2. We have derived exact formulae for the mean and variance of PD under rarefaction. We confirm that these formulae are correct by comparing exact solution mean and variance to that calculated by repeated random (Monte Carlo) subsampling of a dataset of stem counts of woody shrubs of Toohey Forest, Queensland, Australia. We also demonstrate the application of the method using two examples: identifying hotspots of mammalian diversity in Australasian ecoregions, and characterising the human vaginal microbiome.

  3. There is a very high degree of correspondence between the analytical and random subsampling methods for calculating mean and variance of PD under rarefaction, although the Monte Carlo method requires a large number of random draws to converge on the exact solution for the variance.

  4. Rarefaction of mammalian PD of ecoregions in Australasia to a common standard of 25 species reveals very different rank orderings of ecoregions, indicating quite different hotspots of diversity than those obtained for unrarefied PD. The application of these methods to the vaginal microbiome shows that a classical score used to quantify bacterial vaginosis is correlated with the shape of the rarefaction curve.

  5. The analytical formulae for the mean and variance of PD under rarefaction are both exact and more efficient than repeated subsampling. Rarefaction of PD allows for many applications where comparisons of samples of different depth is required.

Keywords: alpha diversity, phylogenetic diversity, rarefaction, sampling depth

Introduction

Phylogenetic Diversity (PD), the total branch length of a phylogenetic tree, has been extensively used as a measure of biodiversity. Originally conceived of as a method for prioritising regions for conservation (Faith, 1992), PD has seen wider use in other applications such as biogeography (Davies and Buckley, 2011), macroecology (Meynard et al., 2011) and microbial ecology (Lozupone and Knight, 2008; Turnbaugh et al., 2008; Caporaso et al., 2012; Yu et al., 2012; Phillips et al., 2012). This increasing breadth of application can be attributed to a number of desirable properties including: 1) explicitly addressing the non-equivalence of species in their contribution to overall diversity, 2) acting as a surrogate for other aspects of diversity such as functional diversity (Cadotte et al., 2009, but see also Faith, 1996), 3) incorporating information on the evolutionary history of communities and biotas and 4) being robust to problems of species delineation because the relationships between populations and even individuals can be represented by relative branch lengths without the need to establish absolute species identity. Further, the original simple formulation of Faith (1992) has been built on to produce a broader “PD calculus” measuring such aspects of diversity as phylogenetic endemism (Faith et al., 2004; Rosauer et al., 2009), evenness (Hill, 1973; Allen et al., 2009) and resemblance (Ferrier et al., 2007; Lozupone and Knight, 2008; Faith et al., 2009; Nipperess et al., 2010). For the purposes of this paper, when referring to “phylogenetic diversity” and “PD”, we refer explicitly to the definition of Faith (1992), where diversity is measured as the sum of branch lengths of a phylogenetic tree.

Phylogenetic diversity increases with increasing sampling effort just like many other measures of biodiversity. Thus, the comparison of the phylogenetic diversity of communities is not straightforward when sample sizes differ, as is common with real data sets. Unless data are standardised in some sense to account for differences in sample size or effort, the relative diversity of communities can be profoundly misinterpreted (Gotelli and Colwell, 2001).

The established solution to the problem of interpreting diversity estimates with samples of varying size is rarefaction. The rarefaction of a given sample of size n to a level k is simply the uniform random choice of k of the n observations (typically without replacement). The observations are typically of either individual organisms or collections of organisms, giving either individual-based or sample-based rarefaction curves (Gotelli and Colwell, 2001). To consider a given measure of diversity under rarefaction, the measure of diversity is simply applied to the rarefied sample. Researchers are typically interested in the expectation and variance of a measure of diversity under rarefaction.

Rarefaction curves can be used to understand the depth of sampling of a community compared to its total diversity. Additionally, rarefaction curves capture information about evenness (Olszewski, 2004) and beta-diversity (Crist and Veech, 2006), depending on whether observations are of individuals or collections. Rarefaction curves have been computed for phylogenetic diversity (Lozupone and Knight, 2008; Turnbaugh et al., 2008; Caporaso et al., 2012; Yu et al., 2012). In each of these cases, rarefaction was not by counts of individual organisms or collections of such, but was instead based on counts of unique sequences or Operational Taxonomic Units. Rarefaction by such units, including taxonomic species, makes sense in the context of phylogenetic diversity where it might not with other measures of biodiversity. In effect, with these examples, rarefaction is by the tips of the tree and the resulting curve gives an indication of tree shape and distribution of sample observations amongst the tips of the tree.

One way to obtain summary statistics such as expectation and variance under rarefaction is to compute these statistics on samples drawn using a Monte Carlo procedure, that is, calculate the desired statistics on a collection of random draws. On the other hand, there are closed form solutions for the mean of many measures of biodiversity under rarefaction. For example, an analytical solution is well-known for species diversity, can be calculated for rarefaction by individuals and samples, and is much more efficient than resampling (Hurlbert, 1971; Ugland et al., 2003; Chiarucci et al., 2008). However, we are not aware of such a formula for any phylogenetic diversity metrics.

In this paper, we establish analytical formulae for the mean and variance of phylogenetic diversity under rarefaction. We develop these formulas in the setting of a phylogenetic tree with “marks,” which are a simple generalization allowing multiplicity of observations and arbitrary positions of observations along the tree.

Materials and methods

There are two different notions of the induced phylogenetic diversity (PD) of a subset K of the leaves L of a tree T; these notions have been called unrooted and rooted PD (Pardi and Goldman, 2007). Unrooted PD is the total branch length of the smallest unrooted subtree contained in T that has all of the leaves in K. Rooted PD is the total branch length of the smallest rooted tree containing the original root of T as well as the selected leaves K. The rooted definition was that originally intended by Faith (1992): see (Faith, 2006) for a historical discussion. These two need not be the same: for example, any K consisting of a single element will have zero unrooted phylogenetic diversity, but nonzero rooted phylogenetic diversity. It is important to make a distinction between rooted and unrooted PD versus rooted and unrooted trees. In our formulation, we are effectively treating all trees as rooted, by assigning an arbitrary root if necessary, and that unrooted and rooted PD refers specifically to the forced inclusion (or not) of a special root.

The two definitions of PD are useful in different domains of application. For example, for conservation applications keeping a single species has significant value, thus it makes sense to have nonzero PD for a single species. On the other hand when comparing the level of ecological diversity between environments, it may not make sense to keep the root, in which case the diversity between the members of a set of size one is zero.

We will derive formulae for both definitions of PD. However, the description of the variance of unrooted PD will be deferred to the Appendix.

Formulae for rarefaction of phylogenetic diversity can be easily and productively generalized from the notion of a tree to the notion of a tree with marks, which allows more flexibility in abundance weighting and attachment locations. We define a tree with marks as a tree along with a collection of special points on the tree (marked with stars in Fig. 1), which may be present with multiplicity. The induced subtree of a collection of marks on a phylogenetic tree is the smallest connected set that contains all of those marks. The phylogenetic diversity of a (sub)tree is the total branch length of the tree.

Figure 1.

Figure 1

Open in a new tab

A hypothetical phylogenetic tree illustrating key concepts in the formulation of the rarefaction of phylogenetic diversity. The tree is populated with marks (indicated by stars) which represent observations of particular points on the tree in a sample. Marks might commonly be placed only at the leaves (tips) of the tree but allowing marks to occur anywhere provides for more flexible applications. Multiple marks indicate multiple observations: for example, several individuals of a species. The tree can then be broken up into snips, which are the edge segments between marks and/or internal nodes. For each snip i, there are two sets of marks, Ci and Di, which name the set of marks that are on the proximal (towards the root) side of i versus those on the distal (towards the leaves) side of i.

In this setting, marks represent observations. Thus if a certain leaf taxon t is observed x times, x marks are put at t. However, it is just as easy to generalize to the setting where marks appear on the interior of tree edges. The motivation for working in terms of marks is that it formalizes the notion of observation count and affords some extra flexibility for location of observations. In particular, microbial ecologists often census a given community by high-throughput sequencing, and it is not practical to build a phylogenetic tree on all of the sequences thus created. For this reason, sometimes scientists map sequences to trees using either similarity search plus a most recent common ancestor strategy, as in the work of Huson et al. (2007), or “place” the sequences into the tree using a phylogenetic criterion (Berger et al., 2011; Matsen et al., 2010). The attachment point of a mapping of a sequence into a tree is then considered as a mark.

The unrooted phylogenetic diversity of a tree with marks is the total branch length of the tree induced by those marks, that is, the total branch length of the smallest connected subset of the tree containing the marks. The rooted phylogenetic diversity of a rooted tree with marks is equal to the unrooted phylogenetic diversity of the tree with the given marks along with a mark added at the root; in this case the path from the root to the selected leaves is always included in the PD calculation. These are simple generalizations of the corresponding definitions for leaf observations.

The following sections will be concerned with rarefying the collection of marks and computing phylogenetic diversities of the corresponding induced subtrees. We will use proximal to indicate the direction towards the root, and distal to mean the opposite. If T is unrooted, we will still use these terms for descriptive purposes; in this case an arbitrary root can be permanently assigned.

We fix a non-empty collection M of n marks on a tree T, and some number 1 ≤ k < n of marks to sample for our rarefaction. Again, marks can be present multiple times in a collection, enabling the expression of multiplicity of observation of a taxon or sequence.

Definition 1. Define an edge snip to be a maximal segment of an edge with no marks or internal nodes.

Say there are s snips on the tree with marks, and that they are indexed by i. Let ℓi be their length for 1 ≤ i ≤ s. Let Ci be the set of marks that are proximal to snip i, and Di be the set that are distal to snip i (Fig. 1).

Definition 2. For every 1 ≤ i ≤ s, let Xir be the random variable that is equal to one if there is at least one mark on the distal side of snip i after rarefaction, and zero otherwise. Let Xiu be the random variable that is equal to one if there is at least one mark on each side of snip i after rarefaction, and zero otherwise.

The following two statements are true for X ∈ {Xr, Xu} with the corresponding Y ∈ {Yr, Yu}. The phylogenetic diversity Y after rarefaction can be expressed as the random variable

Y=∑iℓiXi. (1)

because the length of a snip i contributes to the PD exactly when the corresponding Xi = 1.

Thus 𝔼 (Y) = Σiℓi(Xi), and

Var(Y)=∑i,jℓiℓjCov(Xi,Xj). (2)

To calculate expectations and covariances of the Xi, the following definition will be useful. Fix an R ⊂ M. Let qk (R) be the probability that nothing in R is selected in a uniform sample of size k from M without replacement. Recalling that n = |M|, note that (from the hypergeometric distribution):

qk(R)={(n−|R|k)/(nk)whenn−|R|≥k0otherwise

with the convention that (x0)=1for allx∈ℕ.

Note that the qk (R) can be calculated for successive k by observing that

qk+1(R)=n−|R|−kn−kqk(R).

Because the qk only depend on the size of R, a computer implementation only needs to calculate the qk (R) once for any R of a given size; the qk (R) notation was chosen for convenience.

Rooted phylogenetic diversity

As described above, rooted phylogenetic diversity does PD calculation while always including the root. By (1) and (2) all that is needed is the mean and the covariance matrix of the Xir’s. Note that Xir is zero unless at least one element of Di is sampled, in which case it is one. Thus

𝔼[Xir]=1−qk(Di). (3)

XirXjr is zero unless the rarefaction samples at least one element of both Di, and Dj, in which case it is one. The probability that one or both of these are empty under rarefaction is qk (Di) + qk (Dj) − qk (Di ∪ Dj), thus

𝔼[XirXjr]=1−qk(Di)−qk(Dj)+qk(Di∪Dj).

By (3),

𝔼[Xir]𝔼[Xjr]=1−qk(Di)−qk(Dj)+qk(Di)qk(Dj),

thus,

Cov(Xir,Xjr)=qk(Di∪Dj)−qk(Di)qk(Dj).

In summary,

𝔼[Yr]=∑iℓi[1−qk(Di)]
Var[Yr]=∑i,jℓiℓj[qk(Di∪Dj)−qk(Di)qk(Dj)].

This solution can be seen to be a generalisation of the analytical formulae for the mean and variance of expected species richness under rarefaction (Hurlbert, 1971; Heck Jr et al., 1975) as follows. Consider the special case of a “star” tree with all tips sharing a single common ancestor, where all marks are located at the tips of the tree (with the exception of one mark placed at the root), and where all branch lengths (and thus all snips) have a length of one. Under these particular circumstances, the species richness and phylogenetic diversity of the collection of marks are equal and the formulae for mean and variance of expected phylogenetic diversity simplify to their equivalents for species richness.

Unrooted phylogenetic diversity

Assume as above that we are sampling k > 0 marks for our rarefaction. It is not possible for the rarefaction samples from both Ci and Di to be empty. Thus these two events are mutually exclusive, and

𝔼[Xiu]=1−qk(Ci)−qk(Di). (4)

Then, by (1) and (4),

𝔼(Yu)=∑iℓi[1−qk(Ci)−qk(Di)]. (5)

The variance of the unrooted case is deferred to the Appendix.

Example applications

We demonstrate our method for calculating the mean and variance of phylogenetic diversity under rarefaction using three examples. In the first, we compare the rarefaction curve generated by Monte Carlo randomisation to that calculated by the exact analytical solution. The data are counts of stems of all woody shrubs in forty plots in Toohey Forest, Queensland, Australia. Within each plot, all plant stems over 0.3 m and under 3.0 m were counted; this figure was used as an index of abundance. All shrubs were identified to species and a composite phylogeny was compiled from multiple published trees; see (Nipperess et al., 2010) for a more detailed description of the data. Stem counts were summed across all plots to produce a single value per species before rarefaction by individual stems. Of the total of 582 stems, rarefied values were calculated for every multiple of 10 stems from 10 to 580. For the Monte Carlo procedure, mean and variance of phylogenetic diversity were calculated from 2,000 random subsamples of size k from the pool of 582 stems.

Our second example demonstrates rarefaction of phylogenetic diversity by units of species. Phylogenetic diversity of extant mammals was calculated for each terrestrial ecoregion on the Australian continental shelf (that is, Australia along with Tasmania, New Guinea and offshore islands). Terrestrial ecoregions are biogeographic units representing distinct species assemblages (Olson et al., 2001). Species lists of mammals for each ecoregion were sourced from the WildFinder database maintained by the World Wildlife Fund (http://www.worldwildlife.org/science/wildfinder/). Evolutionary relationships were sourced from a species-level supertree of the world's mammals (Bininda-Emonds et al., 2007). Because of the strong correlation between species richness and phylogenetic diversity, rarefaction allows for the comparison of ecoregions with the effect of spatial variation in species richness removed. To do this, the expected phylogenetic diversity for a subset of 25 mammal species was calculated for each ecoregion. The value of 25 was chosen because it was the minimum species richness for this set of ecoregions.

Our third example comes from the human microbiome. We reanalyze a pyrosequencing dataset describing bacterial communities from women with bacterial vaginosis (Srinivasan et al., 2012). Bacterial vaginosis (BV) has previously been shown to be associated with increased microbial community diversity (Fredricks et al., 2005). For this study, swabs were taken from 242 women from the Public Health, Seattle and King County Sexually Transmitted Diseases Clinic between September 2006 and June 2010 of which 220 samples resulted in enough material to analyze (data available as Sequence Read Archive submission SRA051298). Vaginal fluid for each specimen was also evaluated according to Nugent score, which provides a diagnostic score for BV which ranges from 0 (BV-negative) to 10 (BV-positive) based on presence and absence of bacterial morphotypes as viewed under a microscope (Nugent et al., 1991). Selection of reference sequences and sequence preprocessing were performed using the methods described by Srinivasan et al. (2012). 452,358 reads passed quality filtering, with a median of 1,779 reads per sample (range: 523–2,366). For this application, we investigated the shape of the rarefaction curves with respect to resampling.

Results

There was a very high degree of correspondence between the analytical and Monte Carlo methods for the expected value and variance of phylogenetic diversity of the Toohey Forest dataset under rarefaction (Fig. 2 and 3; corresponding results for unrooted PD are not shown). In this application the Monte Carlo estimate of the PD variance does not converge quickly to the exact value, as can be seen from the deviations of the points (generated from 2,000 Monte Carlo samples) from the curve in Fig. 3. Such slow convergence provides further motivation for an exact formula.

Figure 2.

Figure 2

Open in a new tab

Comparison of analytical value (curve) with Monte Carlo calculation with 2,000 samples (points) for the mean of rooted PD under rarefaction.

Figure 3.

Figure 3

Open in a new tab

Comparison of analytical value (curve) with Monte Carlo calculation with 2,000 samples (points) for the variance of rooted PD under rarefaction.

Correcting phylogenetic diversity for the number of species present made a substantial difference to the ranking of terrestrial ecoregions in terms of their diversity (Fig. 4). With unrarefied phylogenetic diversity, the three highest ranked ecoregions (Southeastern Papuan rainforests, Southern New Guinea lowland rainforests, Central range montane rainforests) are found in New Guinea. However, when variation in species richness is taken into account by rarefaction, two of the three highest ranked ecoregions (Australian Alps montane grasslands, Naracoorte woodlands) were in temperate Australia. Thus the rarefied version demonstrates high phylogenetic diversity for this data set relative to the number of species present for those regions.

Figure 4.

Figure 4

Open in a new tab

Phylogenetic diversity of mammal faunas for terrestrial ecoregions on the Australian continental shelf. Phylogenetic diversity is calculated for (a) all species present and (b) as an expected value after rarefaction to 25 species. Ecoregions are coloured light blue for low values to dark red for high values. The three highest ranked ecoregions in each case are indicated by number.

The rarefaction curves for the vaginal samples shows a connection between the Nugent score of the sample and the shape of the curve (Fig. 5). The rarefaction curves for low Nugent score samples tend to start low and stay low. The high Nugent score samples typically start higher than low Nugent score samples, and stay high.

Figure 5.

Figure 5

Open in a new tab

Rarefaction curve of samples from (Srinivasan et al., 2012). The Nugent score is a diagnostic score for bacterial vaginosis, with 0 being “normal” and 10 being classified as BV.

Discussion

We have presented exact formulae for the mean and variance of rooted and unrooted phylogenetic diversity under rarefaction. This solution gives results that are indistinguishable from those given by Monte Carlo randomisation. The analytical method is preferred both because its results are exact and can be more efficient than sampling.

Rarefaction of phylogenetic diversity is seeing growing use in a variety of biological disciplines and we highlight two specific applications here. Rarefaction of phylogenetic diversity by units of species allows for the assessment of phylogenetic diversity independent of species richness. Removing the influence of species richness can allow for the fairer comparison of the evolutionary history of fauna and flora. While it possible to make this correction by taking the residuals from a regression between species richness and PD (Forest et al., 2007; Davies et al., 2011), the expected PD for a given species richness has also been determined by repeated subsampling of a species pool (Davies et al., 2006; Forest et al., 2007; Morlon et al., 2011; Yu et al., 2012). This latter method describes the relationship between phylogenetic diversity and species richness as a rarefaction curve. Our example of the mammal faunas of the Australian continental shelf shows that such a correction can now be implemented with an exact analytical solution rather than repeated subsampling. Further, as previously found byForest et al. (2007) for the Cape flora of South Africa, correction for the number of species makes a substantial difference to the rank order of phylogenetic diversity of sites.

The rarefaction curves for the vaginal samples give interesting information about the distribution of phylotypes in the vaginal microbiome. Some of this information recapitulates prior knowledge. For example, samples with low Nugent score are typically dominated by a handful of bacterial species in the Lactobacillus genus. These rarefied curves start low and stay low. If there are also other distantly-related organisms present, but in low abundance, the curve can start low and then curve up to a high level. The high Nugent score samples, that tend to start high and increase rapidly, indicate that there are a considerable number of taxa spread across the tree that appear in the samples with nontrivial count.

Software implementing the exact analytical solution for rarefaction of phylogenetic diversity are already available. The phylorare and phylocurve functions are implemented in the R statistical environment (R Development Core Team, 2010). These functions calculate mean rooted phylogenetic diversity and can be used to standardise a set of samples to a particular level of sampling effort (phylorare) or generate a rarefaction curve for units of individuals, collections or species (phylocurve). These functions can be downloaded from http://davidnipperess.blogspot.com.au/. The pplacer suite of programs (http://matsen.fhcrc.org/pplacer/) is a collection of programs for “phylogenetic placement” and associated analyses. The guppy software is the main binary to perform downstream analysis of collections of placements. It calculates Faith's phylogenetic diversity as well as a number of other phylogenetic diversity measures, including the abundance-weighted qD̄ of Chao et al. (2010), a new one-parameter family of PD metrics (manuscript under preparation), phylogenetic entropy (Allen et al., 2009), and phylogenetic quadratic entropy (Rao, 1982). It also calculates PD rarefaction curves with exact formulae as shown here, as well as those for phylogenetic quadratic entropy.

The work presented in this paper relates to and extends previous work in similar areas.Faller et al. (2008) derived a central limit theorem for phylogenetic diversity under a model of random extinction. In doing so, they also derived the mean and variance of phylogenetic diversity under this model. This model is different than the setting of rarefaction in that the random variable signaling extinction is independent between species, which is not true for rarefaction to a given size considered here.

O'Dwyer et al. (2012) have also independently calculated a mean and variance under sampling, but with a different focus: they consider the distributions that might be achieved through a variety of sampling schemes from the “metacommunity tree” of all extant lineages. They derive the expressions for the mean and variance of phylogenetic diversity that we use as a starting point for our proofs and then apply them to their various sampling distributions, using an approximation to bound the variance above. They consider the binomial, Poisson, and negative binomial distributions, but do not consider the hypergeometric distribution as done here, which corresponds to the case of sampling without replacement. They do not derive exact expressions for the variance, nor do they consider unrooted PD or our more general setting.

Although we would like to extend the mean and variance formulas for PD under rarefaction to variants of PD, doing so may not be simple. For example, it would be interesting to investigate the mean and variance of qD̄, the abundance-weighted PD ofChao et al. (2010), under rarefaction. For q = 0, qD̄ is PD divided by T; for q = 1, qD̄ is exp(Hp/T); for q = 2, qD̄ is 1/(1 – Q/T) where T is the maximum height of the phylogenetic tree, Hp is phylogenetic entropy (Allen et al., 2009), and Q is quadratic entropy (Rao, 1982; Warwick et al., 1995). Because these q = 1 and q = 2 cases are nonlinear functions of other abundance-weighted PD measures, the derivation of their mean and/or variance may be challenging.

Future work will include sensitivity of the PD rarefaction curve to tree shape and the distribution of individuals among species. It would also be interesting to investigate extensions of the present work to the “coverage-based” framework recently proposed by Chao and Jost (2012), as well as an extension to “unconditional variance” formulation ofColwell et al. (2012).

Acknowledgements

DAN would like to thank Daniel Faith and Peter Wilson for discussions on probability and PD, Chris Longson and Peter Wilson for help with R programming, and Kyle Barton, Chris Burwell and Roger Kitching for the Toohey Forest dataset. FAM gratefully acknowledges the programming assistance of Connor McCoy and Aaron Gallagher, and would like to thank David Fredricks, Noah Hoffman, Martin Morgan, and Sujatha Srinivasan at the Fred Hutchinson Cancer Research Center for sharing the bacterial vaginosis data. Dan Faith, Anne Chao, and an anonymous reviewer provided helpful comments that greatly improved the manuscript. DAN was funded by the Australian Research Council (DP0665761 and DP1095200), and FAM was funded by National Institutes of Health (R01 HG005966-01).

Appendix

Here we will calculate the variance of the unrooted phylogenetic diversity Yu via (2). First note that (Xiu)2=XiubecauseXiu only takes the values 1 and 0. Thus if i = j then Cov(Xiu,Xju)=𝔼[Xiu]−𝔼[Xiu]2 which can be calculated using (4).

Now assume i ≠ j. Instead of expressing the variance in terms of functions of Ci's and Di's, we will express them in terms of the following quantities defined for a pair of edges i and j. If i is proximal to j, let Si,j be Di (which is the Same side of i as j), Oi,j be Ci (which is on the Other side of i from j), Sj,i be Cj, and Oj,i be Dj. If j is proximal to i, the roles of i and j are reversed. If the path from i to j traverses the root, then let Si,j be Ci, Oi,j be Di, Sj,i be Cj, and Oj,i be Dj. Since Oi,j and Si,j are just Ci and Di in some order, we can think of Xiu being the random variable equal to one if the rarefaction sample from both Oi,j and Si,j are nonempty and zero otherwise; the corresponding definition is true for Xju. By these definitions, a key point is that Oi,j ⊂ Sj,i and Oj,i ⊂ Si,j.

To calculate Cov(Xiu,Xju)=𝔼[XiuXju]−𝔼[Xiu]𝔼[Xju], note that XiuXju is zero unless the rarefaction samples at least one element of both Oi,j and Oj,i, in which case it is one. The probability that one or both of these are empty under rarefaction is qk(Oi,j)+qk(Oj,i)–qk(Oi,j ∪ Oj,i) thus (for i ≠ j)

𝔼[XiuXju]=1−qk(Oi,j)−qk(Oj,i)+qk(Oi,j∪Oj,i).

By (4),

𝔼[Xiu]𝔼[Xju]=[1−qk(Oi,j)−qk(Si,j)][1−qk(Oj,i)−qk(Sj,i)]=1−qk(Oi,j)−qk(Oj,i)+qk(Oi,j)qk(Oj,i)−[1−qk(Oi,j)]qk(Sj,i)−qk(Si,j)[1−qk(Oj,i)]+qk(Si,j)qk(Sj,i).

Thus (again, for i ≠ j),

Cov(Xiu,Xju)=qk(Oi,j∪Oj,i)−qk(Oi,j)qk(Oj,i)+[1−qk(Oi,j)]qk(Sj,i)+qk(Si,j)[1−qk(Oj,i)]−qk(Si,j)qk(Sj,i).

These expressions can then be substituted back into (2) to get an expression for the variance of phylogenetic diversity.

Contributor Information

David A. Nipperess, Department of Biological Sciences, Faculty of Science, Macquarie University, NSW, 2109, Australia, + 61 2 9850 6950 david.nipperess@mq.edu.au

Frederick A. Matsen, IV, Computational Biology Program, Fred Hutchinson Cancer Research Center, 1100 Fairview Ave. N., Mail stop: M1-B514, Seattle, WA 98109-1024, + 1 206 667 7318.

Bibliography

  1. Allen B, Kon M, Bar-Yam Y. A new phylogenetic diversity measure generalizing the Shannon index and its application to phyllostomid bats. The American Naturalist. 2009;174(2):236–243. doi: 10.1086/600101. [DOI] [PubMed] [Google Scholar]
  2. Berger SA, Krompass D, Stamatakis A. Performance, accuracy, and web server for evolutionary placement of short sequence reads under maximum likelihood. Systematic biology. 2011;60(3):291–302. doi: 10.1093/sysbio/syr010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Bininda-Emonds ORP, Cardillo M, Jones KE, MacPhee R, Beck RMD, Grenyer R, Price S, Vos R, Gittleman JL, Purvis A. The delayed rise of present-day mammals. Nature. 2007;446:507–512. doi: 10.1038/nature05634. [DOI] [PubMed] [Google Scholar]
  4. Cadotte M, Cavender-Bares J, Tilman D, Oakley TH. Using phylogenetic, functional and trait diversity to understand patterns of plant community of productivity. PLOS ONE. 2009 Jan;4(5):e5695. doi: 10.1371/journal.pone.0005695. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Caporaso JG, Paszkiewicz K, Field D, Knight R, Gilbert JA. The Western English Channel contains a persistent microbial seed bank. ISME J. 2012 Jun;6(6):1089–1093. doi: 10.1038/ismej.2011.162. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Chao A, Jost L. Coverage-based rarefaction and extrapolation: standardizing samples by completeness rather than size. Ecology, in press. 2012 doi: 10.1890/11-1952.1. [DOI] [PubMed] [Google Scholar]
  7. Chao A, Chiu CH, Jost L. Phylogenetic diversity measures based on hill numbers. Philosophical Transactions of the Royal Society B: Biological Sciences. 2010;365(1558):3599–3609. doi: 10.1098/rstb.2010.0272. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Chiarucci A, Bacaro G, Rocchini D, Fattorini L. Discovering and rediscovering the sample-based rarefaction formula in the ecological literature. Community Ecology. 2008;9(1):121–123. [Google Scholar]
  9. Colwell RK, Chao A, Gotelli NJ, Lin SY, Mao CX, Chazdon RL, Longino JT. Models and estimators linking individual-based and sample-based rarefaction, extrapolation and comparison of assemblages. Journal of Plant Ecology. 2012;5(1):3–21. [Google Scholar]
  10. Crist TO, Veech JA. Additive partitioning of rarefaction curves and species- area relationships: unifying α-, β-and γ-diversity with sample size and habitat area. Ecology Letters. 2006;9(8):923–932. doi: 10.1111/j.1461-0248.2006.00941.x. [DOI] [PubMed] [Google Scholar]
  11. Davies RG, Orme CDL, Webster AJ, Jones KE, Blackburn TM, Gaston KJ. Environmental predictors of global parrot (aves: Psittaciformes) species richness and phylogenetic diversity. Global Ecology and Biogeography. 2006;16(2):220–233. [Google Scholar]
  12. Davies TJ, Buckley LB. Phylogenetic diversity as a window into the evolutionary and biogeographic histories of present-day richness gradients for mammals. Philosophical Transactions of the Royal Society of London B: Biological Sciences. 2011 Jul;366(1576):2414–2425. doi: 10.1098/rstb.2011.0058. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Davies TJ, Buckley LB, Davies TJ, Buckley LB. Phylogenetic diversity as a window into the evolutionary and biogeographic histories of present-day richness gradients for mammals. Philosophical Transactions of the Royal Society B: Biological Sciences. 2011;366(1576):2414–2425. doi: 10.1098/rstb.2011.0058. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Faith DP. Conservation evaluation and phylogenetic diversity. Biological Conservation. 1992;61:1–10. [Google Scholar]
  15. Faith DP. Conservation priorities and phylogenetic pattern. Conservation Biology. 1996;10(4):1286–1289. [Google Scholar]
  16. Faith DP. The role of the phylogenetic diversity measure, PD, in bio-informatics: Getting the definition right. Evolutionary Bioinformatics Online. 2006 Jan;2:277–283. [PMC free article] [PubMed] [Google Scholar]
  17. Faith DP, Reid CAM, Hunter J. Integrating phylogenetic diversity, complementarity, and endemism for conservation assessment. Conservation Biology. 2004 Feb18(1):255–261. [Google Scholar]
  18. Faith DP, Lozupone CA, Nipperess DA, Knight R. The cladistic basis for the Phylogenetic Diversity (PD) measure links evolutionary features to environmental gradients and supports broad applications of microbial ecology's “phylogenetic beta diversity” framework. International Journal of Molecular Sciences. 2009 Jan;10:4723–4741. doi: 10.3390/ijms10114723. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Faller B, Pardi F, Steel M. Distribution of phylogenetic diversity under random extinction. Journal of Theoretical Biology. 2008;251(2):286–296. doi: 10.1016/j.jtbi.2007.11.034. [DOI] [PubMed] [Google Scholar]
  20. Ferrier S, Manion G, Elith J, Richardson K. Using generalized dissimilarity modelling to analyse and predict patterns of beta diversity in regional biodiversity assessment. Diversity and distributions. 2007;13(3):252–264. [Google Scholar]
  21. Forest F, Grenyer R, Rouget M, Davies TJ, Cowling RM, Faith DP, Balmford A, Manning JC, van der Bank M, Reeves G, et al. Preserving the evolutionary potential of floras in biodiversity hotspots. Nature. 2007;445(7129):757–760. doi: 10.1038/nature05587. [DOI] [PubMed] [Google Scholar]
  22. Fredricks DN, Fiedler TL, Marrazzo JM. Molecular identification of bacteria associated with bacterial vaginosis. New England Journal of Medicine. 2005;353(18):1899–1911. doi: 10.1056/NEJMoa043802. [DOI] [PubMed] [Google Scholar]
  23. Gotelli NJ, Colwell RK. Quantifying biodiversity: procedures and pitfalls in the measurement and comparison of species richness. Ecology Letters. 2001;4:379–391. [Google Scholar]
  24. Heck KL, Jr, van Belle G, Simberloff D. Explicit calculation of the rarefaction diversity measurement and the determination of sufficient sample size. Ecology. 1975;56(6):1459–1461. [Google Scholar]
  25. Hill MO. Diversity and evenness: a unifying notation and its consequences. Ecology. 1973;54(2):427–432. [Google Scholar]
  26. Hurlbert S. The nonconcept of species diversity: a critique and alternative parameters. Ecology. 1971;52(4):577–586. doi: 10.2307/1934145. [DOI] [PubMed] [Google Scholar]
  27. Huson DH, Auch AF, Qi J, Schuster SC. MEGAN analysis of metagenomic data. Genome research. 2007;17(3):377–386. doi: 10.1101/gr.5969107. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Lozupone CA, Knight R. Species divergence and the measurement of microbial diversity. FEMS microbiology reviews. 2008;32(4):557–578. doi: 10.1111/j.1574-6976.2008.00111.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Matsen FA, Kodner RB, Armbrust EV. pplacer: linear time maximum-likelihood and bayesian phylogenetic placement of sequences onto a fixed reference tree. BMC bioinformatics. 2010;11(1):538. doi: 10.1186/1471-2105-11-538. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Meynard CN, Devictor V, Mouillot D, Thuiller W, Jiguet F, Mouquet N. Beyond taxonomic diversity patterns: how do α, β and γ components of bird functional and phylogenetic diversity respond to environmental gradients across France? Global Ecology and Biogeography. 2011 Jan20(6):893–903. [Google Scholar]
  31. Morlon H, Schwilk DW, Bryant JA, Marquet PA, Rebelo AG, Tauss C, Bohannan BJM, Green JL. Spatial patterns of phylogenetic diversity. Ecology Letters. 2011;14(2):141–149. doi: 10.1111/j.1461-0248.2010.01563.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Nipperess DA, Faith DP, Barton K. Resemblance in phylogenetic diversity among ecological assemblages. Journal of Vegetation Science. 2010;21(5):809–820. [Google Scholar]
  33. Nugent RP, Krohn MA, Hillier SL. Reliability of diagnosing bacterial vaginosis is improved by a standardized method of gram stain interpretation. Journal of clinical microbiology. 1991;29(2):297–301. doi: 10.1128/jcm.29.2.297-301.1991. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. O'Dwyer JP, Kembel SW, Green JL. Phylogenetic diversity theory sheds light on the structure of microbial communities. In press, PLOS Computational Biology. 2012 doi: 10.1371/journal.pcbi.1002832. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Olson DM, Dinerstein E, Wikramanayake E, Burgess N, Powell G, Underwood E, D'Amico J, Itoua I, Strand H, Morrison J, Loucks C, Allnutt T, Ricketts TH, Kura Y, Lamoreux J, Wettengel W, Hedao P, Kassem K. Terrestrial ecoregions of the world: a new map of life on Earth. BioScience. 2001;51(11):933–938. [Google Scholar]
  36. Olszewski TD. A unified mathematical framework for the measurement of richness and evenness within and among multiple communities. Oikos. 2004;104(2):377–387. [Google Scholar]
  37. Pardi F, Goldman N. Resource-aware taxon selection for maximizing phylogenetic diversity. Systematic biology. 2007;56(3):431–444. doi: 10.1080/10635150701411279. [DOI] [PubMed] [Google Scholar]
  38. Phillips CD, Phelan G, Dowd SE, MCdonough MM, Ferguson AW, Delton Hanson J, Siles L, Ordóñez-garza N, San Francisco M, Baker RJ. Microbiome analysis among bats describes influences of host phylogeny, life history, physiology and geography. Molecular Ecology. 2012;21:26172627. doi: 10.1111/j.1365-294X.2012.05568.x. [DOI] [PubMed] [Google Scholar]
  39. R Development Core Team. R: A language and environment for statistical computing. 2010 URL http://www.R-project.org/. ISBN 3-900051-07-0.
  40. Rao CR. Diversity and dissimilarity coefficients: a unified approach. Theoretical Population Biology. 1982;21(1):24–43. [Google Scholar]
  41. Rosauer D, Laffan SW, Crisp MD, Donnellan SC, Cook LG. Phylogenetic endemism: a new approach for identifying geographical concentrations of evolutionary history. Molecular Ecology. 2009 Oct;18(19):4061–4072. doi: 10.1111/j.1365-294X.2009.04311.x. [DOI] [PubMed] [Google Scholar]
  42. Srinivasan S, Hoffman NG, Morgan MT, Matsen FA, Fiedler TL, Hall RW, Ross FJ, McCoy CO, Bumgarner R, Marrazzo JM, et al. Bacterial communities in women with bacterial vaginosis: high resolution phylogenetic analyses reveal relationships of microbiota to clinical criteria. PLOS ONE. 2012;7(6):e37818. doi: 10.1371/journal.pone.0037818. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Turnbaugh PJ, Hamady M, Yatsunenko T, Cantarel BL, Duncan A, Ley RE, Sogin ML, Jones WJ, Roe BA, Affourtit JP, et al. A core gut microbiome in obese and lean twins. Nature. 2008;457(7228):480–484. doi: 10.1038/nature07540. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Ugland KI, Gray JS, Ellingsen KE. The species-accumulation curve and estimation of species richness. Journal Of Animal Ecology. 2003 Jan72(5):888–897. [Google Scholar]
  45. Warwick RM, Clarke KR, et al. New ‘biodiversity’ measures reveal a decrease in taxonomic distinctness with increasing stress. Marine Ecology Progress Series. 1995;129(1):301–305. [Google Scholar]
  46. Yu DW, Ji Y, Emerson BC, Wang X. Biodiversity soup: metabarcoding of arthropods for rapid biodiversity assessment and biomonitoring. Methods in Ecology and Evolution. 2012;3(4):613–623. [Google Scholar]

RESOURCES