Fig 7. Evaluation of standard and adjusted F_ST estimators.

The convergence values we calculated for the standard kinship plug-in and adjusted F_ST estimators are validated using our admixture simulation. All adjusted estimators are unbiased but are “oracle” methods, since the mean kinship (${\overline{\phi }}^T$), mean coancestry (${\overline{\theta }}^T$), or bias coefficient ($s^T=\frac{{\overline{\theta }}^T}{F_{\text{ST}}}$ for IAFs, replaced by $\frac{{\overline{\phi }}^T}{F_{\text{ST}}}$ for genotypes) are usually unknown. (A) Estimation from individual-specific allele frequencies (IAFs): ${\widehat{F}}_{\text{ST}}^{\text{std}}$ is the standard coancestry plug-in estimator in Eq (26); ${\widehat{F}}_{\text{ST}}^{\prime }$ “Adj. ${\overline{\theta }}^T$” is in Eq (27); ${\widehat{F}}_{\text{ST}}^{\prime \prime }$ “Adj. s” is in Eq (31). (B) For genotypes, ${\widehat{F}}_{\text{ST}}^{\text{std}}$ is given in Eq (25), and the adjusted estimators use ${\overline{\phi }}^T$ rather than ${\overline{\theta }}^T$. Lines: true F_ST (red line), limits of biased estimators ${\widehat{F}}_{\text{ST}}^{\text{std}}$ (green lines, which differ slightly per panel). Estimates (blue) include 95% prediction intervals (too narrow to see) from 39 independently-simulated genotype matrices for our admixture model (Methods, section Prediction intervals).