Selecting SNPs to Identify Ancestry

Abstract

Background/Aims

An individual’s genotypes at a group of Single Nucleotide Polymorphisms (SNPs) can be used to predict that individual’s ethnicity, or ancestry. In medical studies, knowledge of a subject’s ancestry can minimize possible confounding, and in forensic applications, such knowledge can help direct investigations. Our goal is to select a small subset of SNPs, from the millions already identified in the human genome, that can predict ancestry with a minimal error rate.

Methods

The general form for this variable selection procedure is to estimate the expected error rates for sets of SNPs using a training dataset and consider those sets with the lowest error rates given their size. The quality of the estimate for the error rate determines the quality of the resulting SNPs. As the apparent error rate performs poorly when either the number of SNPs or the number of populations is large, we propose a new estimate, the Improved Bayesian Estimate.

Conclusions

We demonstrate that selection procedures based on this estimate produce small sets of SNPs that can accurately predict ancestry. We also provide a list of the 100 optimal SNPs for identifying ancestry.

R functions are available at http://bioinformatics.med.yale.edu/group/josh/index.html.

Introduction

An individual’s genotypes at a group of Single Nucleotide Polymorphisms (SNPs) can be used to predict that individual’s ethnicity, or ancestry [15, 22, 25, 28, 29, 31]. Identifying ancestry through this approach is often useful [2, 17, 27]. For example, subjects in a medical study may be genotyped because adjusting for precise ancestry can minimize one source of confounding [3, 11, 19]. Similarly, a sample from a crime scene may be genotyped so ancestry can be included in the description of a suspect [4, 6, 18]. Although millions of SNPs have been identified, only a small subset needs to be genotyped in order to accurately predict ancestry. Reducing the needed number to the 10’s or 100’s is still useful even in the era of SNP microarrays. First, genotyping only a few hundred SNPs, compared to the 100,000’s of SNPs on a microarray, should be less expensive [21,27]. Second, by basing predictions on only those informative SNPs, we can remove variability caused by considering SNPs with little information. In this article, we aim to describe a method for selecting an ‘optimal’ group of SNPs, or a group that has maximal predictive accuracy given its size.

The first set of methods for marker selection ranked SNPs individually by some measure of their ability to distinguish ancestries, such as the estimated values for F_ST, the allele frequencies, p, or the Informativeness for Assignment, I_n, calculated from a training dataset, and then the top ranked SNPs were selected [24]. See section 5.1 for formal definitions of I_n and F_ST. Obviously, this led to redundant markers, and the majority of markers separated African from European ancestries. Therefore, the next set of methods were more advanced [1,13,30], and included a) Selecting those SNPs that are the strongest contributors to the principal components [22], b) Selecting the 1000 SNPs with the highest F_ST and then, among those, using a genetic algorithm to jointly select the set with the largest I_n [16], and c) Selecting a set of SNPs with a greedy algorithm aimed to minimize the apparent error rate [23]. Method a) can still select redundant SNPs and may rank those SNPs that distinguish closely related populations relatively low. Method b) uses unnecessary surrogates, F_ST and I_n, for predictive accuracy. Method c) is the most promising, as it directly tries to minimize the error rate.

Our goal is to improve the latter method by selecting SNPs that minimize a better estimate of the error rate. A selection procedure based on our new estimate, which introduces an improved form for the error rate and an improved estimate for the allele frequencies, will result in a better group of SNPs. Although our focus is on SNP selection, our discussion should have broader appeal. We will introduce a parametric estimator for the error rate that can be applied to any prediction rule based on genotypes, or, even more generally, any prediction rule using logistic regression to discriminate categories. Unlike the apparent error, this method acknowledges that the true allele frequencies are unknown [5, 8]. Moreover, our new estimate of allele frequencies offer a means to reduce the variance of the maximum likelihood estimates whenever knowledge of the evolutionary tree is available [10, 26]. Instead of estimating the allele frequencies for an ancestry using only subjects from that ancestry, we will now average over all available subjects in the training dataset. The need for these improvements has only arrived as we now attempt to predict ancestry more accurately than continental origin. For a training dataset, we now have access to the Human Genome Diversity Project (HGDP), where 500,000+ SNPs have been genotyped on hundreds of subjects from 54 populations [14].

The order of the paper is as follows. In section 2, we introduce our new selection procedure. In Section 3, we apply our selection procedure to both simulated and HGDP data. In Section 4, we conclude with a short discussion.

Methods

2.1 Introduction to a New Selection Procedure

2.1.1 Overview

As discussed in the introduction, our goal is to define a procedure for choosing a small set of SNPs that, when genotyped, can be used to predict an individual’s ancestry. We focus our search to one specific group of candidate procedures. Each selection procedure considered here will first estimate the expected error rate for every set of SNPs and will then choose a set with the lowest estimated error rate given its size. Obviously, computational limits prevent truly searching over every set of SNPs, but we will deal with that technical issue later. The key point is that we need only define an estimate of the error rate to define a selection procedure.

2.1.2 Notation

Assume we select n individuals from a heterogeneous population containing n_e distinct ancestries or ethnicities and that we denote the ancestry of individual i by Y_i ∈ {1,… n_e}. We let n_k be the total number of subjects from ancestry k, n_k = ∑_i 1(Y_i = k), where 1(Y_i = k) = 1 if Y_i = k and 0 otherwise. In the heterogeneous population, from which our sample was obtained, we denote the proportion of subjects from ethnicity k by ${\pi }_k^{*}\text{ and let }{\overrightarrow{\pi }}^{*}=\{{\pi }_1^{*},\dots ,{\pi }_{n_e}^{*}\}$. We will presume that π⃗* can be accurately estimated and treated as a known, fixed, quantity. Note that the asterisk, *, denotes the true value of a parameter.

Assume a genome for an individual contains N SNPs and denote the three genotypes at each SNP by AA, AB, and BB. Denote the genotype for subject i at SNP j by G_ij ∈ {AA, AB, BB} and the genotype at all SNPs by G⃗_i. When referring to a subset, Ω ⊂ {1,…, N}, of those N SNPs, we denote the genotypes for those specific SNPs by G⃗_i (Ω) ≡ {G_ij : j ∈ Ω}. Let X_i = {G⃗_i,Y_i} be the genotype and ancestry information for subject i, and let X = {X₁,…, X_n} be the training dataset.

The genotype frequencies at these N SNPs vary by population. For ancestry k, we denote the proportion of individuals with genotype g at SNP j by $p_{\mathit{\text{kj}}}^{*}(g),{\overrightarrow{p}}_k^{*}=\{p_{\mathit{\text{kj}}}^{*}(g):j\in (1,\dots ,N),g\in \{\mathit{\text{AA}},\mathit{\text{AB}},\mathit{\text{BB}}\}\},\text{and }{\mathbf{p}}^{*}=[{\overrightarrow{p}}_1^{*},\dots ,{\overrightarrow{p}}_{n_e}^{*}]$.

2.1.3 Maximum Likelihood Estimates

The maximum likelihood estimates, p̂^ML, for p* will assume Hardy-Weinberg Equilibrium. Let

$${\widehat{p}}_{\mathit{\text{kj}}}^{\mathit{\text{ML}}}\equiv \frac{2{\displaystyle {\sum }_{i=1}^n}1(Y_i=k)1(G_{\mathit{\text{ij}}}=\mathit{\text{AA}})+{\displaystyle {\sum }_{i=1}^n}1(Y_i=k)1(G_{\mathit{\text{ij}}}=\mathit{\text{AB}})}{2{\displaystyle {\sum }_{i=1}^n}1(Y_i=k)}$$ (1)

Then

$$\begin{array}{cc}{\widehat{p}}_{\mathit{\text{kj}}}^{\mathit{\text{ML}}}(\mathit{\text{AA}})=\hfill & \hfill {({\widehat{p}}_{\mathit{\text{kj}}}^{\mathit{\text{ML}}})}^2\hfill \\ {\widehat{p}}_{\mathit{\text{kj}}}^{\mathit{\text{ML}}}(\mathit{\text{AB}})=\hfill & \hfill 2{\widehat{p}}_{\mathit{\text{kj}}}^{\mathit{\text{ML}}}(1-{\widehat{p}}_{\mathit{\text{kj}}}^{\mathit{\text{ML}}})\hfill \\ {\widehat{p}}_{\mathit{\text{kj}}}^{\mathit{\text{ML}}}(\mathit{\text{BB}})=\hfill & \hfill {(1-{\widehat{p}}_{\mathit{\text{kj}}}^{\mathit{\text{ML}}})}^2\hfill \end{array}$$ (2)

Again, note that the MLE, p̂^ML, are estimators of the true parameter p*.

2.1.4 Prediction Rule

For an individual i not in the training dataset, we would like to predict his ancestry using his genotype, G⃗_i, and the training data, X. Our prediction rule, Ŷ, will assign the most likely ancestry to individual i assuming that p̂^ML (X) were the true allele frequencies. This estimate is asymptotically optimal, in the sense that as n → ∞, this prediction rule will have the lowest possible error rate [24]. Because of its optimality, we chose this estimate over other options [12, 20].

To define the prediction rule formally, we need to provide equations for calculating the probability individual i has a specific genotype given his ethnicity and p*. The likelihood of the event can be written as

$$P({\overrightarrow{G}}_i(\mathrm{\Omega })|Y_i=k,{\mathbf{p}}^{*})={\displaystyle \prod _{j\in \mathrm{\Omega }}}{p}_{\mathit{\text{kj}}}^{*}{(\mathit{\text{AA}})}^{1(G_{\mathit{\text{ij}}}=\mathit{\text{AA}})}{p}_{\mathit{\text{kj}}}^{*}{(\mathit{\text{AB}})}^{1(G_{\mathit{\text{ij}}}=\mathit{\text{AB}})}{p}_{\mathit{\text{kj}}}^{*}{(\mathit{\text{BB}})}^{1(G_{\mathit{\text{ij}}}=\mathit{\text{BB}})}$$ (3)

Next, we use Bayes theorem to define the probability that individual i is from a specific ancestry given his genotype, p*, and π⃗*,

$$P(Y_i=k|{\overrightarrow{G}}_i(\mathrm{\Omega }),{\mathbf{p}}^{*},{\overrightarrow{\pi }}^{*})=\frac{P({\overrightarrow{G}}_i(\mathrm{\Omega })|Y_i=k,{\mathbf{p}}^{*}){\pi }_k^{*}}{{\displaystyle {\sum }_{t=1}^{n_e}}P({\overrightarrow{G}}_i(\mathrm{\Omega })|Y_i=t,{\mathbf{p}}^{*}){\pi }_t^{*}}$$ (4)

If we knew p*, we would just classify individual i to the ancestry that maximized equation (4). However, without knowing the true value of p*, we replace p* with its estimate p̂^ML. Then we can define our prediction rule, Ŷ, by

$$\widehat{Y}({\overrightarrow{G}}_i(\mathrm{\Omega }),{\widehat{\mathbf{p}}}^{\mathit{\text{ML}}}(\mathbf{X}),{\overrightarrow{\pi }}^{*})\equiv {\mathit{\text{argmax}}}_{k}P(Y_i=k|{\overrightarrow{G}}_i(\mathrm{\Omega }),{\widehat{\mathbf{p}}}^{\mathit{\text{ML}}}(\mathbf{X}),{\overrightarrow{\pi }}^{*})$$ (5)

When clear, we may use one of the two abbreviation, Ŷ_i(X) or Ŷ_i(Ω). Similarly, when notation gets cumbersome, we omit π⃗* from P(Y_i = k|G⃗_i(Ω), p*, π⃗*).

2.1.5 Error Rate

There are two types of error rates for prediction. First, there is the expected error rate when p* is known. Second, there is the expected error rate when p* is unknown and an estimate, p̂^ML, must be used in its place. These two error rates are distinct, and the error rate of interest is the second one. In the genetics literature, we are the first to propose a parametric estimate for this second, realistic, error rate.

Before describing our estimate, let us consider the example where we use known values of p* and π* to predict the ancestry of individual i, and another group aims to estimate our error rate. If this other group also knew the true parameters, they could accurately estimate our error rate by

$$\mathit{\text{err}}=1-{\displaystyle \sum _{k=1}^{n_e}}P(Y_i=k|{\overrightarrow{G}}_i(\mathrm{\Omega }),{\mathbf{p}}^{*})1(\widehat{Y}({\overrightarrow{G}}_i(\mathrm{\Omega }),{\mathbf{p}}^{*})=k)$$ (6)

Note that equation (6) is calculated as one minus the probability of correctly predicting the ancestry. Now, if this different group only knew p̂_ML, their best prediction would be to plug p̂_ML, π*, and G_i(Ω) into that same function. The result is the Apparent Error Rate

$${\widehat{\mathit{\text{err}}}}_{\mathit{\text{AE}}}=1-{\displaystyle \sum _{k=1}^{n_e}}P(Y_i=k|{\overrightarrow{G}}_i(\mathrm{\Omega }),{\widehat{\mathbf{p}}}^{\mathit{\text{ML}}})1(\widehat{Y}({\overrightarrow{G}}_i(\mathrm{\Omega }),{\widehat{\mathbf{p}}}^{\mathit{\text{ML}}})=k)$$ (7)

However, the true scenario, where our predictions are based only on estimates of p* presents a far more difficult challenge. There is no closed form equivalent to equation (6). Even if the other group knew the true parameters, they could not precisely calculate our error rate. In fact, we can only define a function, equation (13), that inputs the true parameters and outputs a consistent approximation of the error rate. The remainder of this section discusses the derivation of this equation.

We start by writing down a formula describing the error rate. Given our prediction rule and p*, the expected error rate for a given genotype, G⃗_i(Ω), averaged over all possible training datasets can be described by equation (8). The probability of a training dataset is the probability of the observed genotypes given the ancestries (i.e. the product of equation 3 across all subjects). Note that in the prediction rule, we consider Ŷ_i to be a function of X and that Ŷ_i(X) is a random variable, as opposed to a fixed value. Here, the terms random and fixed refer to the training data.

$$\begin{array}{cc}\mathit{\text{err}}(G_i(\mathrm{\Omega }),{\mathbf{p}}^{*})\hfill & \equiv E_{Y_i,X}[1({\widehat{Y}}_i(\mathbf{X}))\ne Y_i)|{\overrightarrow{G}}_i(\mathrm{\Omega }),{\mathbf{p}}^{*},{\overrightarrow{\pi }}^{*}]\hfill \\ \hfill & \equiv 1-{\displaystyle \sum _{k=1}^{n_e}}P(Y_i=k|{\overrightarrow{G}}_i(\mathrm{\Omega }),{\mathbf{p}}^{*},{\overrightarrow{\pi }}^{*})P({\widehat{Y}}_i(\mathbf{X})=k|{\overrightarrow{G}}_i(\mathrm{\Omega }),{\mathbf{p}}^{*},{\overrightarrow{\pi }}^{*})\hfill \end{array}$$ (8)

Also, unless stated otherwise, we assume that ancestry is being predicted for an individual not in the training dataset so Ŷ_i(X) and Y_i are independent. The probability of correctly predicting the ancestry is the sum, over all possible k, of the probability that the true ancestry is k multiplied by the probability that the predicted ancestry is k.

Unfortunately, equation (8) does not quite offer a way to calculate err(G_i(Ω), p*). For a subject with a specific genotype, we know how to calculate P(Y_i = k|G⃗_i(Ω), p*, π⃗*) using equation (4). However, we have yet to state a means to calculate P(Ŷ_i(X) = k|p*, π⃗*). Below, we suggest that P(Ŷ_i(X) = k|p*, π⃗*) can be approximated by the probability that a value drawn from one normal distribution is greater than k − 1 values drawn from other normal distributions. Unfortunately, there’s no closed form solution. The details of the derivation are left to the appendix, (section 5.3), and here we offer only a sketch of how to go from equation (8) to equation (12).

Consider the variable p̂^ML(X). We can plug that variable into the function described by equation (4), to create a new variable P(Y = k|G⃗_i(Ω), p̂^ML(X), π⃗*). We can estimate the distribution of this continuous variable by

$$\mathit{\text{log}}(P(Y=k|{\overrightarrow{G}}_i(\mathrm{\Omega }),{\widehat{\mathbf{p}}}^{\mathit{\text{ML}}}(X),{\overrightarrow{\pi }}^{*}))\approx N({\mu }_{k{\overrightarrow{G}}_i}({\mathbf{p}}^{*},{\overrightarrow{\pi }}^{*}),{\sigma }_{k{\overrightarrow{G}}_i}^2({\mathbf{p}}^{*}))$$ (9)

where

$$\begin{array}{cc}\hfill {\mu }_{k{\overrightarrow{G}}_i}({\mathbf{p}}^{*},{\overrightarrow{\pi }}^{*})=\hfill & {\displaystyle \sum _{j\in \mathrm{\Omega }}}1(G_{\mathit{\text{ij}}}=\mathit{\text{AA}})\mathit{\text{log}}(p_{\mathit{\text{kj}}}^{*2})+1(G_{\mathit{\text{ij}}}=\mathit{\text{AB}})\mathit{\text{log}}(2p_{\mathit{\text{kj}}}^{*}(1-p_{\mathit{\text{kj}}}^{*}))+1(G_{\mathit{\text{ij}}}=\mathit{\text{BB}})\mathit{\text{log}}{((1-p_{\mathit{\text{kj}}}^{*})}^2+\mathit{\text{log}}({\pi }_k^{*})+C\hfill \\ \hfill {\sigma }_{k{\overrightarrow{G}}_i}^2({\mathbf{p}}^{*})=\hfill & \frac{1}{n_k}{\displaystyle \sum _{j\in \mathrm{\Omega }}}{\mathbf{V}}_{\mathit{\text{kj}}}[{\kappa }_{{\overrightarrow{G}}_{\mathit{\text{ij}}}},{\kappa }_{{\overrightarrow{G}}_{\mathit{\text{ij}}}}]\hfill \\ \hfill {\mathbf{V}}_{\mathit{\text{kj}}}\equiv \hfill & 4\left[\begin{array}{ccc}\hfill \frac{1-p_{\mathit{\text{kj}}}^{*}}{p_{\mathit{\text{kj}}}^{*}}\hfill & \hfill \frac{1-2p_{\mathit{\text{kj}}}^{*}}{2p_{\mathit{\text{kj}}}^{*}}\hfill & \hfill \frac{1-2p_{\mathit{\text{kj}}}^{*}}{1-p_{\mathit{\text{kj}}}^{*}}\hfill \\ \hfill \frac{1-2p_{\mathit{\text{kj}}}^{*}}{2p_{\mathit{\text{kj}}}^{*}}\hfill & \hfill \frac{{(1-2p_{\mathit{\text{kj}}}^{*})}^2}{4p_{\mathit{\text{kj}}}^{*}(1-p_{\mathit{\text{kj}}}^{*})}\hfill & \hfill -\frac{2p_{\mathit{\text{kj}}}^{*}-1}{2(1-p_{\mathit{\text{kj}}}^{*})}\hfill \\ \hfill \frac{1-2p_{\mathit{\text{kj}}}^{*}}{1-p_{\mathit{\text{kj}}}^{*}}\hfill & \hfill -\frac{2p_{\mathit{\text{kj}}}^{*}-1}{2(1-p_{\mathit{\text{kj}}}^{*})}\hfill & \hfill \frac{p_{\mathit{\text{kj}}}^{*}}{1-p_{\mathit{\text{kj}}}^{*}}\hfill \end{array}\right]\hfill \end{array}$$ (10)

where κ_G⃗j = 1, 2, and 3 when G_ij = AA, AB, and BB respectively and C is a constant independent of ancestry.

The prediction rule will output ancestry k, Ŷ_i = k, when P(Y = k|G⃗_i(Ω), p̂^ML(X)) is the largest among the probabilities for all ancestries. The probability of this event is the same as the probability that Z_kG⃗i(p*) is greatest among {Z_υG⃗i(p*)} where

$$Z_{\upsilon {\overrightarrow{G}}_i}({\mathbf{p}}^{*})\sim N({\mu }_{\upsilon {\overrightarrow{G}}_i}({\mathbf{p}}^{*}),{\sigma }_{\upsilon {\overrightarrow{G}}_i}^2({\mathbf{p}}^{*}))$$ (11)

Again, we exclude π⃗* to simplify notation. The Z_υ are independent as the MLE for population k are derived only from the subjects within population k. Therefore, we take the following to be a satisfactory approximation of the error rate. Let

$$\widehat{\mathit{\text{err}}}(G_i(\mathrm{\Omega }),\mathbf{p})\equiv 1-{\displaystyle \sum _{k=1}^{n_e}}P(Y_i=k|\overrightarrow{G}(\mathrm{\Omega }),{\mathbf{p}}^{*})P({\mathit{\text{max}}}_{\upsilon }\{Z_{\upsilon {\overrightarrow{G}}_i}({\mathbf{p}}^{*})\}=Z_{k{\overrightarrow{G}}_i}({\mathbf{p}}^{*}))$$ (12)

and as an approximation for the overall error rate, let

$$\widehat{\mathit{\text{err}}}(\mathrm{\Omega },{\mathbf{p}}^{*})\equiv {\displaystyle \sum _{\mathbf{G}}}\widehat{\mathit{\text{err}}}({\overrightarrow{G}}_i(\mathrm{\Omega }),{\mathbf{p}}^{*})P({\overrightarrow{G}}_i(\mathrm{\Omega })|{\mathbf{p}}^{*})$$ (13)

The foundation of this estimate is the ‘Prediction Focused Information Criteria’ described by Claeskens et. al. and Efron, et. al. for other scenarios [5, 8].

2.1.6 Estimated Error Rate and the Selection Procedure

We could estimate the error rate for a set of SNPs, Ω, by replacing p* with p̂^ML in equation (13)

$${\widehat{\mathit{\text{err}}}}_{\mathit{\text{I AE}}}\equiv \widehat{\mathit{\text{err}}}(\mathrm{\Omega },{\widehat{\mathbf{p}}}^{\mathit{\text{ML}}})$$ (14)

However, the maximum likelihood estimates, p̂^ML, use only a small subset of the training data to estimate any given allele frequency. Estimates of ${\overrightarrow{p}}_k^{*}$ are based only on subjects from ancestry k. We believe this to be wasteful because ancestries located near each other on an evolutionary tree should have similar allele frequencies. Therefore, we suggest estimating ${\overrightarrow{p}}_k^{*}$ by an appropriate average of all p̂^ML. Obviously, population k and its evolutionary neighbors will have the greatest weights in this average. In other words, we permit our estimates to be slightly biased when n is finite, and in return, our estimates will have much smaller variances. The details will be postponed to later, but we will create a Bayesian averaged estimate, p̂^B, of p* and will ultimately suggest estimating the error rate by

$${\widehat{\mathit{\text{err}}}}_{\mathit{\text{I BE}}}\equiv \widehat{\mathit{\text{err}}}(\mathrm{\Omega },{\widehat{\mathbf{p}}}^B)$$ (15)

Therefore our ideal selection procedure would be to estimate the error for every given set of SNPs, and then choose a set with the lowest error rate for its size. In practice, because it is computationally infeasible to search through all sets, we suggest using the greedy algorithm described in the appendix (section 5.4). Because our method does not naturally incorporate linkage disequilibrium (LD), we amend the standard greedy algorithm so that new SNPs cannot be added to the selected set if they are within approximately 75kb of any previously selected SNP.

2.2 Alternatives to ${\widehat{\mathit{\text{err}}}}_{\mathit{\text{IBE}}}$

Instead of using the Improved Error Rates, we could have used the apparent error rate, ${\widehat{\mathit{\text{err}}}}_{\mathit{\text{AE}}}$, defined in equation (7). As the apparent error rate has been used for SNP selection previously [23], this will be one of the selection procedures discussed in our simulations and examples. As we will see, this estimate always underestimates the true error rate.

Because the apparent error performs poorly when dealing with 100,000’s of SNPs, we offer another selection procedure. This selection procedure uses the non-parametric 0.632+ bootstrap estimate for the error rate. We tried other non-parametric methods, including various methods of cross-validation, but in our simulations, we always found the 0.632+ bootstrap estimate to be the most accurate and precise. In general, the 0.632+ estimate outperforms other non-parametric estimates [7, 9]. Here, we briefly explain how to calculate this estimate.

A bootstrap sample, $X_b^{*},b\in \{1,\dots ,B\}$, is a randomly selected sample of n pairs of observations from {(G⃗₁, Y₁), (G⃗₂, Y₂),…, (G⃗_n, Y_n)}, with replacement. By chance, each bootstrap sample excludes some sets of observations. If we create our prediction rule based on $X_b^{*}$, we can calculate the error rate for those excluded observations, leading to the leave-one-out estimate, ${\widehat{\mathit{\text{err}}}}_{(1)}$

$${\widehat{\mathit{\text{err}}}}_{(1)}=\frac{{\displaystyle {\sum }_{b=1}^B}{\displaystyle {\sum }_{i=1}^n}1(\widehat{Y}(G_i(\mathrm{\Omega }),X_b^{*})\ne Y_i)1(({\overrightarrow{G}}_i,Y_i)\notin X_b^{*})}{{\displaystyle {\sum }_{b=1}^B}{\displaystyle {\sum }_{i=1}^n}1(({\overrightarrow{G}}_i,Y_i)\notin X_b^{*})}$$ (16)

The bootstrap 0.632 estimator [7] combines ${\widehat{\mathit{\text{err}}}}_{(1)}$ and the observed error, ${\widehat{\mathit{\text{err}}}}_{\mathit{\text{obs}}}$,

$${\widehat{\mathit{\text{err}}}}_{\mathit{\text{obs}}}=\frac{{\displaystyle {\sum }_{i=1}^n}1(\widehat{Y}(G_i(\mathrm{\Omega }),X)\ne Y_i)}{n}$$ (17)

Note that ${\widehat{\mathit{\text{err}}}}_{\mathit{\text{obs}}}$ uses the training data as test data as well. The bootstrap 0.632 estimate, ${\widehat{\mathit{\text{err}}}}_{0.632}$, is defined as

$${\widehat{\mathit{\text{err}}}}_{0.632}=0.632{\widehat{\mathit{\text{err}}}}_{(1)}+0.368{\widehat{\mathit{\text{err}}}}_{\mathit{\text{obs}}}$$ (18)

The ${\widehat{\mathit{\text{err}}}}_{0.632+}$ is a slight variation of this estimate, but for brevity we omit the details here and refer the reader elsewhere [9].

2.3 Calculating p̂^B

As promised in section 2.1.6, we now describe p̂^B. We found it best to propose a mathematical model that describes the development of the allele frequencies over time. We started with a single set of allele frequencies in one historical population, and then allowed these allele frequencies to change, in steps, as individuals spread around the globe and formed distinct populations. Given this model, we then calculate Bayesian estimates, p̂^B, of the allele frequencies. We essentially obtain prior distributions for p* based on the evolutionary tree and then update these priors given the maximum likelihood estimates obtained in the training dataset. Therefore, our proposed estimate, given in equation (21), is E[p|p̂^ML]. The remainder of this section shows the derivation of this estimate.

The new estimate, p̂^B, takes advantage of a known evolutionary tree. Assume the tree has n_n nodes (see figures 1, 2, and 3 for examples). There are n_e terminal nodes, each representing an observed population, and n_n − n_e interior nodes, each representing a historical, combined, population. Label the nodes 1, 2, ‥, n_n. Label the edges by the attached node, 2,…, n_n, where the edge acquires the label of the larger of the two attached nodes.

Figure 1.

evolutionary Tree - 25 nodes. An example of an evolutionary tree with 25 nodes (n_n = 25) and 20 populations (n_e = 20). Next to each population node is the population identifier (k).

Figure 2.

evolutionary Tree - 24 nodes. An example of an evolutionary tree with 46 nodes (n_n = 46) and 24 populations (n_e = 24). Next to each population node is the population identifier (k). See appendix for population names.

Figure 3.

evolutionary Tree - 13 nodes. An example of an evolutionary tree with 24 nodes (n_n = 24) and 13 populations (n_e = 13). Next to each node is both the node identifier (node) and the population identifier (k), listed as k/node. Internal nodes are listed –/node. Edge 17 is labeled for discussion in the text.

In the actual model, we will assume that groups of populations share a common allele frequency at each SNP. We will introduce a vector S⃗ which identifies those ancestries sharing the same allele frequency. For SNP j, 1 ≤ j ≤ N we create an n_n − 1 length vector of binary variables, S⃗_j ≡ {S_2j,…, S_nnj}. If S⃗_j = 0⃗ then p_kj is the same for all populations. If S_υ1j = 1, but S_υj = 0 for all other υ ≠ υ₁, then the populations prior to edge υ₁ will share a common allele frequency, and the populations following edge υ₁ will share a different common allele frequency. In figure 3 we label edge 17 for an example. If S_17j = 1, but S_υj = 0 for all other υ, then populations 15 and 16 would share a common allele frequency, and all other populations would share a different frequency. Since S_υj = 1 allows allele frequencies to vary, we refer to it as a ‘bottleneck event’. In general, if two nodes, or populations, k₁ and k₂, can be connected by a set of edges, V, and S_υj = 0 ∀ υ ∈ V, then p_k1j = p_k2j. We place a prior distribution on S_υj, P(S_υj = 1) = α_υ, where α_υ can be an increasing function of the distance between the nodes adjacent to edge υ.

The number of unique allele frequencies, N_Sj, will be much smaller than the total number of ancestries, n_e.

$$N_{\mathbf{S}j}=1+{\displaystyle \sum _{k=2}^{n_e}}1(p_{\mathit{\text{kj}}}\notin \{p_{1j},\dots ,p_{k-1j}\}$$ (19)

We denote the N_Sj unique allele frequencies by p_(1)j,…, p_(NSj)j, and we denote the number of subjects in populations with each of those N_Sj unique allele frequencies by n_(1)j,…, n_(NSj)j.

$$n_{(\kappa )j}={\displaystyle \sum _k}{n}_{k}1(p_{\mathit{\text{kj}}}=p_{(\kappa )j})$$ (20)

The Bayesian hierarchical model can be described as follows. We place a uniform prior on S. Given S, the distribution of p is f(p|S) = 1 for any set of allele frequencies consistent with S. Given p, we know the distributions of the training dataset. After, we perform the appropriate integrations, we can calculate the posterior means for p.

$${\widehat{p}}_{\mathit{\text{kj}}}^B\equiv E[p_{\mathit{\text{kj}}}|{\widehat{p}}^{\mathit{\text{ML}}}]={\displaystyle \sum _S}\frac{C_1{C}_2}{C_3}{\displaystyle \prod _{\upsilon }}{\alpha }_{\upsilon }^{S_{\upsilon j}}{(1-{\alpha }_{\upsilon })}^{(1-S_{\upsilon j})}{\displaystyle \sum _{\kappa =1}^{N_{\mathit{\text{Sj}}}}}\frac{n_{(\kappa )j1}+1}{n_{(\kappa )j1}+n_{(\kappa )j0}+2}1(p_{\kappa j}=p_{(\kappa )j})$$ (21)

where n_kj1 is the number of A alleles in population k, n_(κ)j1 is the number of A alleles in populations with the κ^th unique allele frequency, n_kj0 and n_(κ)j0 are the respective quantities for the a allele. To minimize confusion, we note that although n_kj and n_(κ)j are numbers of subjects, n_kj0, n_(κ)j0, n_kj1 and n_(κ)j1 are numbers of alleles. Furthermore,

$$\begin{array}{c}{C}_1={\displaystyle \prod _{k=1}^{n_e}}\left(\begin{array}{c}2n_k\hfill \\ n_{\mathit{\text{kj}}1}\hfill \end{array}\right)\hfill \\ C_2={\displaystyle \prod _{\kappa =1}^{N_{\mathit{\text{Sj}}}}}B(n_{(\kappa )j1}+1,n_{(\kappa )j0}+1)\hfill \\ C_3={\displaystyle \sum _S}{C}_1{C}_2{\displaystyle \prod _{\upsilon }}{\alpha }_{\upsilon }^{S_{\upsilon j}}{(1-{\alpha }_{\upsilon })}^{(1-S_{\upsilon j})}\hfill \end{array}$$ (22)

Note that p̂^B requires specification of the hyperparameter {α₂,…, α_nn}. Derivation of this equation is in the appendix.

The model is an obvious simplification for the development of allele frequencies. In history, bottlenecks are actually rare events and allele frequencies change gradually over evolutionary development. Therefore, allele frequencies for neighboring populations should be highly correlated. For any given S_·j, the abrupt bottlenecks may distort the estimated allele frequencies. However, by averaging over multiple S_·j, we observe allele frequencies that vary smoothly across the evolutionary tree. Therefore, although we could try to incorporate a correlation structure into f(p|S) and allow for more events, these additions did not improve our estimates. We found our proposed model to perform as well as any of the more complex models examined.

2.4 Data and Simulation

2.4.1 Simulations: Aims

Our main objective is to understand and compare the selection procedures. We start with three sets of simulations, designed to answer three questions:

1. Which is a better estimate of p: p̂^B or p̂^ML?

2. Which is a better estimate of the error rate: ${\widehat{\mathit{\text{err}}}}_{\mathit{\text{AE}}},{\widehat{\mathit{\text{err}}}}_{0.632+},\text{or}{\widehat{\mathit{\text{err}}}}_{\mathit{\text{IBE}}}$?

3. Which error rate is best for our selection procedure: ${\widehat{\mathit{\text{err}}}}_{\mathit{\text{AE}}},{\widehat{\mathit{\text{err}}}}_{0.632+},\text{or}{\widehat{\mathit{\text{err}}}}_{\mathit{\text{IBE}}}$?

2.4.2 Simulations: Common Framework

The three sets of simulations examining these questions share a common framework. There are n_e ancestries and these ancestries are related by an evolutionary tree with all edges of equal length. In these simulations, n_e ∈ {13, 20, 24} and the possible evolutionary trees are illustrated in figures 1, 2, and 3. The evolutionary trees with 13 and 24 populations were trimmed versions of the tree that described the relationships among the HGDP populations [14]. Details of the tree follow in section 2.4.4. We assume that all populations are equally common and let the training dataset contain an equal number of subjects, 5, 10, or 20, from each population. Simulation results were always based 10,000 datasets.

For simulation sets 2 and 3, we introduce a new type of error. Recall that the expected error rate, err(G_i(Ω), p*) (equation (8)), is averaged over all possible training datasets. Now, we let err_X(G_i(Ω), p*) be the error that would be observed given a specific value of p̂^ML or a specific training dataset, X.

2.4.3 Simulations: Description

Simulation 1

General: For each dataset containing a group of subjects with one genotyped SNP, calculate p̂^B and p̂^ML and compare them to p*.

Specifics: Let n_e = 20 and N = 1. To fairly compare p̂^B and p̂^ML, we examine three possible sets of allele frequencies, (1) No variation: $p_k^{*}=0.5$ for all k, (2) Inter-continental variation: $p_k^{*}=0.5-1.5d_1$ for k ∈ {1,…, 5}, $p_k^{*}=0.5-0.5d_1$ for k ∈ {6,…, 10}, $p_k^{*}=0.5+0.5d_1$ for k ∈ {11,…, 15}, and $p_k^{*}=0.5+1.5d_1$ for k ∈ {16,…, 20}, and (3) Intra-continental variation: $p_1^{*}=0.5-1.5d_1-2d_2,p_2^{*}=0.5-1.5d_1-1d_2,p_3^{*}=0.5-1.5d_1,p_4^{*}=0.5-1.5d_1+1d_2,p_5^{*}=0.5-1.5d_1+2d_2,\dots$, where d₁ = 0.2 and d₂ = 0.067.

Simulation 2

General: For each dataset, calculate ${\widehat{\mathit{\text{err}}}}_{\mathit{\text{AE}}},{\widehat{\mathit{\text{err}}}}_{0.632+},\text{and}{\widehat{\mathit{\text{err}}}}_{\mathit{\text{IBE}}}$. With the additional use of a test set containing 100,000 individuals, calculate err_X. Compare the three estimated error rates with err_X.

Specifics: Let n_e ∈ {13, 24} and N ∈ {10, 40, 80}. We generate allele frequencies from the more complex evolutionary trees according to the Bayesian model described in section 2.3. For each SNP, we first generate S⃗_j = {S_2j,…, S_{V j}}. For n_e = 13, P(∑_υ S_υj = t) = 1/3 for t ∈ {1,…, 3}. For n_e = 24, P(∑_υ S_υj = t) = 1/4 for t ∈ {1,…, 4}. S_υj are iid ∀υ. Allele frequencies for each connected set of populations are generated from a uniform[0.05,0.95] distribution.

Simulation 3

General: For each dataset, select the top 40 SNPs according to ${\widehat{\mathit{\text{err}}}}_{\mathit{\text{AE}}},{\widehat{\mathit{\text{err}}}}_{632+},\text{and}{\widehat{\mathit{\text{err}}}}_{\mathit{\text{IBE}}}$. Then using those SNPs and a training dataset, calculate ${\mathit{\text{err}}}_X({\mathrm{\Omega }}_{\mathit{\text{AE}}}^{*},{\mathbf{p}}^{*}),{\mathit{\text{err}}}_X({\mathrm{\Omega }}_{632+}^{*},{\mathbf{p}}^{*}),\text{and}{\mathit{\text{err}}}_X({\mathrm{\Omega }}_{\mathit{\text{IBE}}}^{*},{\mathbf{p}}^{*})$. We compare these three error rates to see which is the lowest.

Specifics: Let n_e ∈ {13, 20, 24} and N ∈ {1000, 10000}. Here, P(∑_υ S_υj = 0) = 0.9 and the remaining probability is split evenly over ∑_υ S_υj ∈ {1,…, 3} events when n_e = 13 and ∑_υ S_υj ∈ {1,…, 4} events when n_e ∈ {20, 24}, where S_υj are iid ∀υ.

2.4.4 HGDP Data

Data 1

The Human Genome Diversity Project (HGDP) data set is more than an example. As it is the data set that will likely be used for selecting SNPs, the performance of the three possible selection procedures on this specific data set is of primary importance. As the HGDP grows and changes, the rankings of the three methods will need to be reevaluated. For this comparison, we use only a subset of the data, containing 400 subjects in 24 populations, from the HGDP (population names given in appendix). We limit our focus to those subjects with easily available data [14]. The evolutionary tree for these groups was based on pairwise allele-sharing distance among populations and had been previously estimated by Jakobsson [14]. We split the data into 50 sets of 10,000 SNPs. For each set of SNPs, we select the top 40 using the greedy algorithm with either ${\widehat{\mathit{\text{err}}}}_{\mathit{\text{AE}}},{\widehat{\mathit{\text{err}}}}_{632+}\text{ or }{\widehat{\mathit{\text{err}}}}_{\mathit{\text{IBE}}}$ on 80% of the data. Then we estimate the true error rate using using the remaining 20%. These error rates are then averaged over all 50 sets of data. Splitting the data into smaller sets was a necessity to decide whether the improvement in the set of SNPs selected by ${\widehat{\mathit{\text{err}}}}_{\mathit{\text{IBE}}}$ is statistically significant. In the supplementary material, we show the results from selecting SNPs according to a different set of methods. In these methods, the top ranked SNPs, where rankings are by F_ST, I_n, or the Optimal Rate of Correct Assignment (ORCA), are selected.

Data 2

We use the entire HGDP data set to select an optimal group of 100 SNPs for distinguishing ancestry. We start by selecting a candidate group of 5,000 SNPs. This group includes the 2000 SNPs (40 SNPs × 50 test sets) chosen from our initial 10,000-SNP searches. We then repeat the analysis described for data set 1 focusing on populations within each continent separately. Here, we select the top 20, as opposed to the top 40 SNPs. These chosen SNPs comprise the remaining 3,000 SNPs (3 continental regions × 20 SNPs × 50 data sets). The top 100 SNPs are selected from this set of 5,000 SNPs and listed in the supplementary material.

Results

3.1 Simulations

Simulation 1

The MLE, ${\widehat{p}}_j^{\mathit{\text{ML}}}$, are the most commonly used approximations for ${\overrightarrow{p}}_j^{*}$. The Bayesian estimates, ${\widehat{p}}_j^B$, shrink the MLE toward the average value from neighboring populations. Therefore, if the truth is that neighboring populations share a common ‘A’ allele frequency, $p_{\mathit{\text{kj}}}^{*}$, at SNP j, then the mean square error, MSE^ML, for the MLE, should be larger than the MSE^B for the Bayesian estimates, where ${\mathit{\text{MSE}}}^{\mathit{\text{ML}}}\equiv n_e^{-1}{\displaystyle {\sum }_k}{({\widehat{p}}_{\mathit{\text{kj}}}^{\mathit{\text{ML}}}-p_{\mathit{\text{kj}}}^{*})}^2\text{ and }{\mathit{\text{MSE}}}^B=n_e^{-1}{\displaystyle {\sum }_k}{({\widehat{p}}_{\mathit{\text{kj}}}^{\mathit{\text{IBE}}}-p_{\mathit{\text{kj}}}^{*})}^2$. The first two columns in table 1 show that the improvement can be quite high when all populations in the study share a single $p_{\mathit{\text{kj}}}^{*}$. When populations can have different allele frequencies, the extent of the advantage or disadvantage depends on the evolutionary tree. The tradeoff between maximum likelihood and Bayesian estimates is a tradeoff between variance and bias. p̂^B can be biased, but will have lower variance. In general, as the the number of subjects per population decreases, the MSE^B : MSE^ML decreases, favoring estimation by p̂^B (table 1).

Table 1.

The mean-square error $(\text{MSE}),k^{-1}{\displaystyle {\sum }_k}{({\widehat{p}}_k^{\mathit{\text{ML}}}-p_k)}^2$, between the MLE and the truth, and the $\text{MSE},k^{-1}{\displaystyle {\sum }_k}{({\widehat{p}}_k^{\mathit{\text{B}}}-p_k)}^2$, between the Bayesian estimate and the truth. These MSE, averaged over all simulations, are listed for different sets of p, where a) all populations share a common p (No Var) b) all populations within a continent share a common p (Inter-Continental Var) c) all populations have a unique p (Intra-Continental Var)

	No Var		Inter-Continental Var		Intra-Continental Var
	MLE	Bayes	MLE	Bayes	MLE	Bayes
5	0.025	0.002	0.02	0.016	0.019	0.013
10	0.013	0.001	0.01	0.011	0.01	0.008
15	0.008	0.001	0.007	0.009	0.007	0.007

Simulation 2

We compared three options, ${\widehat{\mathit{\text{err}}}}_{\mathit{\text{AE}}},{\widehat{\mathit{\text{err}}}}_{0.632+,}\text{ and }{\widehat{\mathit{\text{err}}}}_{\mathit{\text{IBE}}}$ for the 13 and 24 population examples (table 2). Clearly, ${\widehat{\mathit{\text{err}}}}_{\mathit{\text{AE}}}$ greatly underestimates the true error, and the $\mathit{\text{MSE}}({\widehat{\mathit{\text{err}}}}_{\mathit{\text{AE}}})=n_{\mathit{\text{sim}}}^{-1}{\displaystyle {\sum }_i^{n_{\mathit{\text{sim}}}}}{({\mathit{\text{err}}}_{\mathit{\text{AE}}}-{\mathit{\text{err}}}_X)}^2$, where n_sim is the number of simulations, is an order of magnitude larger than the MSE for either of the other estimates. The ratios, $\mathit{\text{MSE}}({\widehat{\mathit{\text{err}}}}_{\mathit{\text{AE}}}):\mathit{\text{MSE}}({\widehat{\mathit{\text{err}}}}_{0.632+})\text{and}\mathit{\text{MSE}}({\widehat{\mathit{\text{err}}}}_{\mathit{\text{AE}}}):\mathit{\text{MSE}}({\widehat{\mathit{\text{err}}}}_{\mathit{\text{IBE}}})$ increase as the number of informative SNPs or the number of populations increases. In these simulations, $\mathit{\text{MSE}}({\widehat{\mathit{\text{err}}}}_{\mathit{\text{IBE}}})$ tends to be lower than $\mathit{\text{MSE}}({\widehat{\mathit{\text{err}}}}_{0.632+})$, but the order reverses as N grows large. The ${\widehat{\mathit{\text{err}}}}_{\mathit{\text{IBE}}}$, with its default settings for α⃗, slightly overestimates the true value, but when calculating the MSE, this bias is offset by lower variance and a higher correlation between $\widehat{\mathit{\text{err}}}$ and err_X.

Table 2.

A comparison of the three methods for estimating error rates ${\widehat{\mathit{\text{err}}}}_{\mathit{\text{AE}}},{\widehat{\mathit{\text{err}}}}_{632+,}\text{ and }{\widehat{\mathit{\text{err}}}}_{\mathit{\text{IBE}}}$. The first results column, err_X, is the true unconditional error. The second set of results’ columns is the MSE from comparing the estimated errors with err_x̂. The third set of columns is the standard deviation of each estimate. The fourth set of columns is the correlation between each estimate and err_x̂.

13 Populations
		errX	MSE			$\widehat{\mathit{\text{err}}}$			$\text{SD}(\widehat{\mathit{\text{err}}})$			$\text{cor}(\widehat{\mathit{\text{err}}},{\mathit{\text{err}}}_{\widehat{X}})$
nk	N		AE	0.632+	IBE	AE	0.632+	IBE	AE	0.632+	IBE	AE	0.632+	IBE
5	10	0.776	0.1311	0.0269	0.0204	0.6481	0.7662	0.786	0.0488	0.0477	0.0449	0.805	0.851	0.9178
5	40	0.4489	0.2669	0.0311	0.0278	0.1844	0.4461	0.4548	0.0403	0.0653	0.0684	0.8212	0.8815	0.9185
5	80	0.2357	0.2038	0.0287	0.023	0.0352	0.2486	0.2361	0.0144	0.0477	0.048	0.591	0.8456	0.8776
10	10	0.7797	0.133	0.0274	0.0197	0.6499	0.7684	0.7876	0.0505	0.0502	0.0467	0.8146	0.8674	0.9219
10	40	0.4426	0.263	0.03	0.0282	0.1819	0.4386	0.4476	0.0383	0.0625	0.0643	0.7828	0.8796	0.9027
10	80	0.2284	0.1987	0.0274	0.0221	0.0332	0.2415	0.2286	0.0134	0.0482	0.0494	0.5882	0.8676	0.8945
24 Populations
		errX	MSE			$\widehat{\mathit{\text{err}}}$			$\text{SD}(\widehat{\mathit{\text{err}}})$			$\text{cor}(\widehat{\mathit{\text{err}}},{\mathit{\text{err}}}_{\widehat{X}})$
nk	N		AE	0.632+	IBE	AE	0.632+	IBE	AE	0.632+	IBE	AE	0.632+	IBE
5	10	0.9037	0.1033	0.016	0.0099	0.8021	0.8962	0.9084	0.0306	0.0265	0.0219	0.802	0.8474	0.9178
5	40	0.7792	0.3432	0.0226	0.0207	0.4368	0.7704	0.7928	0.046	0.0455	0.0384	0.8553	0.8907	0.9174
5	80	0.6601	0.4851	0.0227	0.0257	0.1758	0.6573	0.6775	0.0307	0.0507	0.0484	0.8149	0.8952	0.9214
10	10	0.9044	0.1017	0.0154	0.0096	0.8044	0.8985	0.9081	0.0302	0.0267	0.0216	0.8032	0.852	0.9109
10	40	0.7814	0.3436	0.0215	0.0207	0.4386	0.7722	0.7949	0.0459	0.046	0.0413	0.863	0.907	0.9267
10	80	0.6542	0.4816	0.0227	0.0259	0.1734	0.6508	0.6699	0.0305	0.0488	0.0483	0.8016	0.8879	0.9043

Simulation 3

SNPs were selected by the greedy algorithm aimed to minimize either ${\widehat{\mathit{\text{err}}}}_{0.632+},{\widehat{\mathit{\text{err}}}}_{\mathit{\text{IBE}}},\text{or}{\widehat{\mathit{\text{err}}}}_{\mathit{\text{AE}}}$. For each group, the selected SNPs were ordered by the step in which they were added. Therefore, SNP 1 is essentially the most informative and SNP 40 is the least informative. For each group, the error rate was calculated (via simulation) when the top T SNPs were used, T ∈ {1,…, 40} and is illustrated in figure 4. The main point is that when more than 3 SNPs were used, the SNPs in Ω_AE (the set chosen using the apparent error) proved to be poor predictors of the true ancestries. Selection based on ${\widehat{\mathit{\text{err}}}}_{0.632+}$ resulted in lower error rates, and selection based on ${\widehat{\mathit{\text{err}}}}_{\mathit{\text{IBE}}}$ resulted in the lowest error rates. Therefore, these simulations clearly suggest that the use of ${\widehat{\mathit{\text{err}}}}_{\mathit{\text{AE}}}$ is extremely inefficient and the use of ${\widehat{\mathit{\text{err}}}}_{\mathit{\text{IBE}}}$ can be the most efficient. However, as the simulation model unfairly favors ${\widehat{\mathit{\text{err}}}}_{\mathit{\text{IBE}}}$, we hold off general statements about the ${\widehat{\mathit{\text{err}}}}_{\mathit{\text{IBE}}}$-based selection procedure until we see the results for the HGDP data.

Figure 4.

Comparison of the error rates when the markers are chosen by er̂r₆₃₂₊, er̂r_IBE, and er̂r_AE using simulated data.

3.2 Data

Data 1

Selecting from groups of 10,000 SNPs, we denoted the resulting sets of 40 SNPs by ${\mathrm{\Omega }}_{\mathit{\text{AE}}}^{*},{\mathrm{\Omega }}_{632+}^{*},\text{and }{\mathrm{\Omega }}_{\mathit{\text{IBE}}}^{*}$. These SNPs and their corresponding p̂^ML were then used to predict the ancestry for the 80 subjects in the separate test dataset, resulting in three sets of error rates err_AE, err_0.632+, and err_IBE. These error rates were then averaged over all 50 sets of 10,000 SNPs to produce figure 5. The results are similar to those from the simulations, showing that SNP selection by ${\widehat{\mathit{\text{err}}}}_{\mathit{\text{IBE}}}$ outperformed both of the other selection procedures so long as there were more than 8 SNPs. Using only 8 SNPs, 77% of the subjects were assigned to a population in the correct continental region. Additional SNPs were selected to distinguish intra-continental populations. At this stage in the selection procedure, differences in allele frequencies due to random chance could rival informative differences, and because ${\widehat{\mathit{\text{err}}}}_{\mathit{\text{IBE}}}$ is designed to remove those that occur by chance, it starts to perform better.

Figure 5.

Comparison of the error rates when the markers are chosen by er̂r₆₃₂₊, er̂r_IBE and er̂r_AE using HGDP data.

We then compared the 0.632+ and IBE methods to see whether selection by ${\widehat{\mathit{\text{err}}}}_{\mathit{\text{IBE}}}$ produced a statistically significantly better set of SNPs than selection by ${\widehat{\mathit{\text{err}}}}_{0.632+}$. Figure 6 shows the difference in error rate, ${\widehat{\mathit{\text{err}}}}_{0.632+}-{\widehat{\mathit{\text{err}}}}_{\mathit{\text{IBE}}}$, and a point-wise 95% CI, using the sample variance of the 50 values and assuming normality. The improvement was statistically significant. Although training sets contained only 80% of the data, we presume this benefit persists when selecting SNPs using all individuals. For future studies, we recommend splitting the data into training and test sets or using a cross-validation approach to choose the optimal method for selecting SNPs and, when desirable, to tune the hyper-parameter α⃗. Here, using simulations as our guide, we let (n_n − 1)α_υ = 7 ∀ υ.

Figure 6.

The difference between the error rates when the markers are chosen by er̂r₆₃₂₊ and er̂r_IBE. The solid line is the difference, error rate using er̂r₆₃₂₊ - error rate using er̂r_IBE, and the dotted lines are the point-wise 95% confidence intervals.

The error rate is still near 50% when Ω* includes 40 markers. However, a more detailed analysis of P(Y|G⃗(Ω), p̂^ML, π̂) shows that the majority of errors involve classifying a subject from population k₁ to population k₂, where k₁ and k₂ are close to each other on the evolutionary tree. Figure 7, created by superStruct (available on author’s website), is similar to the output from STRUCTURE and shows that using 2000 markers reduces the error rate to near 0%. Each point on the axis corresponds to one of the subjects from one of the test data sets, and above that point, is a series of 24 stacked bars. Each bar has a unique color and represents a single population. The height of the colored bar corresponding to population k is proportional to the posterior probability, P(k|G⃗(Ω), p̂^ML, π̂). Populations within the same continental region are different shades of the same color. The total number of subjects described by figure 7a is 3650 (= 73 subjects × 50 datasets). As for the overall potential for SNPs, we examined the predictive accuracy of all 2000 SNPs (40 SNPs × 50 datasets) and found near perfect identification (i.e. P(Ŷ_i = Y_i|G⃗_i(Ω), p̂^ML, π̂) ≈ 1) for the majority of the 73 subjects (figure 7b). The 6 predictions that disagreed with the self-identification, (i.e. Ŷ_i ≠ Y_i) were neighboring populations. This figure shows that we can do better than predicting continental origin.

Figure 7.

A graph to summarize the ancestry information for each individual. The x-axis indicates subject. For each subject, 24 bars, corresponding to the 24 populations, are stacked. The height of a bar k is the estimated probability that the individual is from that population. Each bar is a different color. Populations from one continent are varying shades of a single color: Red = Africa, Orange = America, Green = S.E. Asia, Blue = EuroAsia. Black lines separate populations. Top (a) and Bottom (b) images show the expected results using 40 and 2000 markers respectively. The total number of subjects described in the top half of the figure is 3650 (= 73 subjects × 50 datasets), whereas the bottom half of the figure describes only 73 subjects.

Data 2

We used ${\widehat{\mathit{\text{err}}}}_{\mathit{\text{IBE}}}$ to select an optimal set of 100 SNPs. Those SNPs are listed in the supplementary material.

Discussion

This article has introduced two ideas with an influence that should extend beyond SNP selection procedures. First, we offer an improved method for estimating the population specific allele frequencies. Second, we offer an improved method for estimating the error rate for prediction rules using genotypes. In fact, this latter method can be applied to any classification problem based on logistic regression. Focusing on the SNP selection procedures, we have demonstrated that selecting SNPs to minimize ${\widehat{\mathit{\text{err}}}}_{\mathit{\text{IBE}}}$, instead of ${\widehat{\mathit{\text{err}}}}_{\mathit{\text{AE}}}$ can lead to a group of SNPs that can predict ancestry with high accuracy.

The apparent error rate and maximum likelihood estimates have been successfully used in the past for selecting SNPs [23]. However, here the apparent error performed poorly. The main difference is that the number of populations has increased. With more populations, SNPs that truly separate a small group of populations no longer stand out. The population differences in p̂^ML caused by sample selection are relatively small, but, in terms of the overall importance of a SNP, these differences are additive. Also, as the number of populations increases, we need more SNPs. As the number of needed SNPs increases, the improvement due to each additional SNP decreases, and it is more likely that a non-informative SNP can appear to be the best candidate. More general limitations of the apparent error rate are that it estimates the wrong quantity and cannot account for the fact that estimates of allele frequencies for some populations (i.e. those with more subjects in the training dataset) should be more accurate than others.

In this manuscript, we never actually considered using ${\widehat{\mathit{\text{err}}}}_{\mathit{\text{IAE}}}$, and here we discuss one of its limitations and our reason for avoiding it. Although it has gone unstated in the literature, ${\widehat{\mathit{\text{err}}}}_{\mathit{\text{IAE}}}$ can be heavily biased because using p̂^ML will exaggerate the true accuracy of the prediction rule. The following, simple, example illustrates that $E[{\widehat{\mathit{\text{err}}}}_{\mathit{\text{IAE}}}]<\mathit{\text{err}}$. Let there be one gene and two populations, where the allele frequencies in the populations are p₁ and p₂. As an extremely rough approximation, suitable only for illustration, consider the error to be a function of the difference log(p₁) − log(p₂) ≡ log(p₁/p₂). The true error generally increases as the distance between p₁ and p₂ decreases, with err attaining its maximum of 0.5 when log(p₁/p₂) = 0, or when the allele frequencies are the same in both populations. Now, assume we are unlucky, and the truth happens to be log(p₁/p₂) = 0. The estimate, $\mathit{\text{log}}({\widehat{p}}_1^{\mathit{\text{ML}}}/{\widehat{p}}_2^{\mathit{\text{ML}}})$ is distributed around its true value, resulting in ${\widehat{\mathit{\text{err}}}}_{\mathit{\text{IAE}}}<0.5$.

This bias discussed for ${\widehat{\mathit{\text{err}}}}_{\mathit{\text{IAE}}}$ is absent from ${\widehat{\mathit{\text{err}}}}_{0.632+}$, and therefore without the additional information from the evolutionary tree, ${\widehat{\mathit{\text{err}}}}_{0.632+}$ would be the preferred method for estimating the error rate. However, we did find that ${\widehat{\mathit{\text{err}}}}_{\mathit{\text{IBE}}}$ performed favorably when compared to ${\widehat{\mathit{\text{err}}}}_{0.632+}$ in the results section. Because of the nature of non-parametric estimates, it would be difficult to introduce the information from the evolutionary tree into ${\widehat{\mathit{\text{err}}}}_{0.632+}$.

Our study focused on individuals with only a single ancestry. However, our general conclusions about the SNP selection procedure and the selected SNPs will be valid when our goal is to identify the multiple ancestries of admixed individuals. Obviously the selected group of SNPs will need to be expanded to attain similar error rates. We suspect that the total number of SNPs needed to identify one of the admixed ancestries will be inversely proportional to the percentage of an individual’s genome originating with that ancestry. Instead of looking for ancestries of an individual, we would now be looking for ancestries of sections of the chromosomes. Admixture therefore requires a selection procedure that assumes only a random subset of the chosen SNPs will actually be available to identify a given ancestry. Therefore, the selected set should include some redundancy. This will also safeguard against genotyping error. We are currently exploring solutions for our two objectives in admixtures.

The next goal, already under examination, is how to incorporate the HGDP data and the knowledge of the optimal set of SNPs in identifying population substructure in Genome Wide Association Studies. First, most GWAS are large enough to contribute their own information about allele frequencies in populations. Second, GWAS are often more influenced by large population substructure, and may not need to identify populations that are not greatly present in the study. However, this focus is likely to change as we start searching for rare disease causing mutations.

Appendix

5.1 Definition of F_ST and I_n

The fixation index (F_ST) is a measure of population differentiation, and is usually the correlation of randomly chosen alleles within the same sub-population relative to that found in the entire population. To define F_ST, let $p_k^{*}$ be the frequency of allele “A” in population k, k ∈ {1,…, n_e}, where n_e is the number of ancestries and $q_k^{*}=1-p_k^{*}$. Then

$$F_{\mathit{\text{ST}}}=1-\frac{H_S}{H_T}$$ (23)

$$H_S={\displaystyle \sum _{k=1}^{n_e}}2w_k{p}_k^{*}{q}_k^{*}$$ (24)

$$H_T=2({\displaystyle \sum _{k=1}^{n_e}}{w}_k{p}_k^{*})({\displaystyle \sum _{k=1}^{n_e}}{w}_k{q}_k^{*})$$ (25)

where w_k is the relative size of the k^th subpopulation and ∑_k w_k = 1. In our definitions of these quantities, w_k is the relative size of each subpopulation in some larger, natural, population, not among the individuals sampled.

Information, I, is based on the idea of statistical entropy as described in [24]. The information, or Informativeness of Assignment is

$$I=-p^{*}\mathit{\text{log}}(p^{*})+{\displaystyle \sum _{k=1}^{n_e}}\frac{p_k^{*}}{n_e}\mathit{\text{log}}{p}_k^{*}-q^{*}\mathit{\text{log}}(q^{*})+{\displaystyle \sum _{k=1}^{n_e}}\frac{q_k^{*}}{n_e}\mathit{\text{log}}{q}_k^{*}$$ (26)

where $p^{*}={\displaystyle {\sum }_k}{p}_k^{*}/n_e$ and q* = 1 − p*.

5.2 Calculating p^B

Recall, for each SNP j, we defined a set of indicator variables, S⃗_j ≡ {S_2j,…, S_nnj}, representing bottleneck events. Formally, a bottleneck is an evolutionary event in which only a small subset of individuals survives to the next generation, leading to a dramatic change in population allele frequencies. We use the term more liberally to imply any event that allows two neighboring populations to have different allele frequencies. Recall that N_Sj is the number of unique allele frequencies, n_(κ)j is the number of subjects with the κ^th unique allele frequency, and n_e is the total number of ancestries. Central to this calculation is the fact that the density of the estimated parameters, p̂, conditional on the true parameters, p, can be defined by

$$f(\widehat{p}|p)=C_1{\displaystyle \prod _{k=1}^{n_e}}{p}_k^{2n_k{\widehat{p}}_k}{(1-p_k)}^{2n_k(1-{\widehat{p}}_k)}=C_1{\displaystyle \prod _{t=1}^{N_S}}{p}_{(\kappa )}^{n_{(\kappa )1}}{(1-p_{(\kappa )})}^{n_{(\kappa )0}}$$ (27)

where

$$C_1={\displaystyle \prod _{k=1}^{n_e}}\left(\begin{array}{c}\hfill 2n_k\hfill \\ \hfill n_{k1}\hfill \end{array}\right)$$ (28)

Note that f(·|·) represents a generic conditional density function, with the exact form depending on the variables in that function. Clearly, as f(p|S) is a constant when non-zero,

$$f(p|S,\widehat{p})\propto f(\widehat{p}|p,S)f(p|S)\propto f(\widehat{p}|p)$$ (29)

Because we know that f(p|S, p̂) is a density function, we know that f(p|S, p̂) must have the form ${\displaystyle {\prod }_{t=1}^{N_s}}{f}_{\beta }(p_{(t)}|n_{(t)1}+1,n_{(t)0}+1)$ for all non-zero values, where f_β(·|α, β) is the β density.

We know E[p|p̂] = ∑_S E[p|S, p̂]f(S|p̂). We start by calculating f(S|p̂),

$$f(S|\widehat{p})\propto f(\widehat{p}|S)f(S)={\displaystyle {\int }_p}f(\widehat{p}|S,p)f(p|S)f(S)\mathit{\text{dp}}=C_1{\displaystyle \prod _{\kappa =1}^{N_S}}{\displaystyle {\int }_{p|f(p|S)\ne 0}}{p}_{(\kappa )}^{n_{(\kappa )1}}{(1-p_{(\kappa )})}^{n_{(\kappa )0}}\mathit{\text{dp}}{\displaystyle \prod _{t=2}^{n_n}}{\alpha }_t^{S_t}{(1-{\alpha }_t)}^{(1-S_t)}=C_1{C}_2{\displaystyle \prod _{t=2}^{n_n}}{\alpha }_t^{S_t}{(1-{\alpha }_t)}^{(1-S_t)}$$ (30)

where we calculate C₂ by noting that $p_{(\kappa )}^{n_{(\kappa )1}}{(1-p_{(\kappa )})}^{n_{(\kappa )0}}$ is a multiple of f β(p_(κ)|n_(κ)1 + 1, n_(κ)0 + 1),

$$C_2={\displaystyle \prod _{\kappa =1}^{N_S}}B(n_{(\kappa )1}+1,n_{(\kappa )0}+1)$$ (31)

where B is beta function. Because we know that f(S|p̂) is a density function, we know that we can define $C_3={\displaystyle {\sum }_S}{C}_1{C}_2{\displaystyle {\prod }_t}{\alpha }_t^{S_t}{(1-{\alpha }_t)}^{(1-S_t)}$ and conclude that

$$f(S|\widehat{p})=\frac{C_1{C}_2}{C_3}{\displaystyle \prod _t}{\alpha }_t^{S_t}{(1-{\alpha }_t)}^{(1-S_t)}$$ (32)

To calculate E[p|S, p̂], note that for each value of S, we now know

$$E[p_j|S,\widehat{p}]=\frac{n_{(\kappa )1}+1}{n_{(\kappa )1}+n_{(\kappa )0}+2}1(p_j=p_{(\kappa )})$$ (33)

where 1(p_j = p_(κ)) indicates whether population j shares the κ^th unique allele frequency. Therefore, we have arrived at

$$E[p_j|\widehat{p}]={\displaystyle \sum _S}\frac{C_1{C}_2}{C_3}{\displaystyle \prod _t}{\alpha }_t^{S_t}{(1-{\alpha }_t)}^{(1-S_t)}{\displaystyle \sum _{\kappa =1}^{N_S}}\frac{n_{(\kappa )1}+1}{n_{(\kappa )1}+n_{(\kappa )0}+2}1(p_j=p_{(\kappa )})$$ (34)

5.3 Estimating Error

Here, we show that log(P̂(Y_i = k| G⃗_i(Ω), p̂^ML)) is asymptotically normal with mean μ_ki and variance ${\sigma }_{\mathit{\text{ki}}}^2$ defined in equation (10). Start by focusing on SNP j in population k. Recall, n_k is the number of subjects in population k and n_e is the total number of ancestries. Then, our estimate of p_kj is

$${\widehat{p}}_{\mathit{\text{kj}}}\equiv \frac{2{\displaystyle {\sum }_{i=1}^n}1(Y_i=k)1(G_{\mathit{\text{ij}}}=\mathit{\text{AA}})+{\displaystyle {\sum }_{i=1}^n}1(Y_i=k)1(G_{\mathit{\text{ij}}}=\mathit{\text{AB}})}{2{\displaystyle {\sum }_{i=1}^n}1(Y_i=k)}$$ (35)

where by the central limit theorem we know,

$$\sqrt{2n_k}({\widehat{p}}_{\mathit{\text{kj}}}-p_{\mathit{\text{kj}}}){\to }_{d}N(0,p_{\mathit{\text{kj}}}(1-p_{\mathit{\text{kj}}}))$$ (36)

Next, we want to define our estimate for

$${\mathbf{m}}_{\mathit{\text{kj}}}\equiv \mathbf{m}(p_{\mathit{\text{kj}}})\equiv t([\mathit{\text{log}}(p_{\mathit{\text{kj}}}(\mathit{\text{AA}})),\mathit{\text{log}}(p_{\mathit{\text{kj}}}(\mathit{\text{AB}})),\mathit{\text{log}}(p_{\mathit{\text{kj}}}(\mathit{\text{BB}}))])$$ (37)

and

$${\widehat{\mathbf{m}}}_{\mathit{\text{kj}}}\equiv \mathbf{m}({\widehat{p}}_{\mathit{\text{kj}}})\equiv \left[\begin{array}{c}\hfill \mathit{\text{log}}({\widehat{p}}_{\mathit{\text{kj}}}^2)\hfill \\ \hfill \mathit{\text{log}}(2{\widehat{p}}_{\mathit{\text{kj}}}(1-{\widehat{p}}_{\mathit{\text{kj}}}))\hfill \\ \hfill \mathit{\text{log}}({(1-{\widehat{p}}_{\mathit{\text{kj}}})}^2)\hfill \end{array}\right]$$ (38)

We approximate the distribution of m̂_kj by a linear function of p̂_kj, specifically,

$${\widehat{\mathbf{m}}}_{\mathit{\text{kj}}}={\mathbf{m}}_{\mathit{\text{kj}}}+\mathbf{m}\prime (p_{\mathit{\text{kj}}})(\mathbf{m}({\widehat{p}}_{\mathit{\text{kj}}})-\mathbf{m}(p_{\mathit{\text{kj}}}))$$ (39)

where ${\mathbf{m}}_{\mathit{\text{kj}}}^{\prime }\equiv \mathbf{m}\prime (p_{\mathit{\text{kj}}})$ and

$${\mathbf{m}}_{\mathit{\text{kj}}}^{\prime }\equiv \left[\begin{array}{c}\hfill \frac{\partial \mathit{\text{log}}(p_{\mathit{\text{kj}}}^2)}{\partial p}\hfill \\ \hfill \frac{\partial \mathit{\text{log}}(2p_{\mathit{\text{kj}}}(1-p_{\mathit{\text{kj}}}))}{\partial p}\hfill \\ \hfill \frac{\partial \mathit{\text{log}}({(1-p_{\mathit{\text{kj}}})}^2)}{\partial p}\hfill \end{array}\right]=2\left[\begin{array}{c}\hfill \frac{1}{p_{\mathit{\text{kj}}}}\hfill \\ \hfill \frac{4(2p_{\mathit{\text{kj}}}-1)p_{\mathit{\text{kj}}}(1-p_{\mathit{\text{kj}}})+{(2p_{\mathit{\text{kj}}}-1)}^3}{4p_{\mathit{\text{kj}}}^2{(1-p_{\mathit{\text{kj}}})}^2}\hfill \\ \hfill \frac{p_{\mathit{\text{kj}}}}{1-p_{\mathit{\text{kj}}}}\hfill \end{array}\right]$$ (40)

Then, we have the following approximation

$$\sqrt{2n}({\widehat{\mathbf{m}}}_{\mathit{\text{kj}}}-{\mathbf{m}}_{\mathit{\text{kj}}})\approx N(0,{\mathbf{m}}_{\mathit{\text{kj}}}^{\prime }{p}_{\mathit{\text{kj}}}(1-p_{\mathit{\text{kj}}})t({\mathbf{m}}_{\mathit{\text{kj}}}^{\prime }))\equiv N(0,{\mathbf{v}}_{1j})$$ (41)

where

$${\mathbf{v}}_{1j}\equiv 4\left[\begin{array}{ccc}\hfill \frac{1-p_{1j}}{p_{1j}}\hfill & \hfill \frac{1-2p_{1j}}{2p_{1j}}\hfill & \hfill \frac{1-2p_{1j}}{1-p_{1j}}\hfill \\ \hfill \frac{1-2p_{1j}}{2p_{1j}}\hfill & \hfill \frac{{(1-2p_{1j})}^2}{4p_{1j}(1-p_{1j})}\hfill & \hfill -\frac{2p_{1j}-1}{2(1-p_{1j})}\hfill \\ \hfill \frac{1-2p_{1j}}{1-p_{1j}}\hfill & \hfill -\frac{2p_{1j}-1}{2(1-p_{1j})}\hfill & \hfill \frac{p_{1j}}{1-p_{1j}}\hfill \end{array}\right]$$ (42)

5.4 Greedy Algorithm

If the total of number of SNPs available is around 1,000,000, the number of possible groups grows at a rate of 1, 000, 000^N_S. It is computationally infeasible to search such a large space, making solution 2a impractical. Therefore, we propose using a greedy algorithm with N_S steps.

• Step 1: Select the single SNP j that minimizes the expected error rate: j₁ = argmin_j err({j}, p*, π⃗*, p*, π⃗*).

• Step 2…N_s: Given a set of n − 1 SNPs, {j₁, j₂,…, j_{n − 1}}, select the SNP that when added to that current set minimizes the error rate: j_n = argmin_j err({j₁, j₂,…, j_n−1, j}, p*, π⃗*, p*, π⃗*).

Although {j₁,…, j_Ns}, the set chosen by the greedy algorithm, is not guaranteed to be the optimal set, the set should perform satisfactorily, in that the resulting error rate should be similar to the true minimum.

5.5 Population Names

(Continental Region 1) 1: Yoruba (25), 2: Mandeka (20), 3: Bantu (8), 4: San (7), 5: Biaka Pygmy (32), 6: Mbuti Pygmy (15) (Continental Region 2) 7: Papuan (16), 8: Melanesian (17), 9: Pima (11), 10: Maya (13), 11: Columbian (7), 12: Yakut (15), 13: Mongola (9), 14: Daur (10), 15: Cambodian (10), 16: Yi (10) (Continental Region 3) 17: Burusho (7), 18:Kalash (18), 19: Balochi (15), 20: Russian (13), 21: Druze (43), 22: Beduin (47), 23: Palestinian (26), 24: Mozabite 96) Population Number: Population Name (Number of subjects in population)