Figure 3.

Adapted from (Patterson et al. 2012). Expected value of f₃(C; A, B) under various tree topologies. Red lines trace genetic drift between populations C and A; blue lines trace genetic drift between C and B. f₃ measures drift between C and A that is also shared between C and B. Drift is shared along branches where arrows going in the same direction overlap. a and b: expected value of f₃(C; A, B) with no admixture. If C is not a product of admixture between A and B, f₃ is expected to be positive. In the case where C is an outgroup to A and B (a), the value of f₃ is proportional to the distance separating C from A and B, which can also be thought of as the amount of shared history between A and B. c–f: expected value of f₃(C; A, B) when C is a product of admixture between A and B. α is the percent ancestry population C derives from A, and β is the percent derived from B. Distance j represents genetic drift between extant population A its ancestral population that admixed to form the population ancestral to C in the past; distance k is proportional to drift between extant population B and its admixing ancestral population. Computation of f₃(C; A, B) in this case requires tracing multiple paths through the tree, since population C can share drift with population B that it received through admixture with population A and vice versa. The expectation is the sum of all shared drift: E[f₃(C; A, B)] = αβi + α²(i + j) + β²(i + k) + αβ(i — p — q). This has the potential to be negative, although it can also be positive. Given that negative values are impossible if C is not a result of admixture (a and b), a negative result can be taken as evidence of admixture; a positive result, however, cannot be used to reject admixture.