Training Population Optimization for Genomic Selection

Abstract

Core Ideas

• Training populations can be optimized for specific testing populations.

• Optimized training populations are smaller, more related, and more predictive.

• Stratified sampling with a relationship matrix weighted by marker effect is optimal.

The effectiveness of genomic selection in breeding programs depends on the phenotypic quality and depth, the prediction model, the number and type of molecular markers, and the size and composition of the training population (TR). Furthermore, population structure and diversity have a key role in the composition of the optimal training sets. Our goal was to compare strategies for optimizing the TR for specific testing populations (TE). A total of 1353 wheat (Triticum aestivum L.) and 644 rice (Oryza sativa L.) advanced lines were evaluated for grain yield in multiple environments. Several within‐TR optimization strategies were compared to identify groups of individuals with increased predictive ability. Additionally, optimization strategies to choose individuals from the TR with higher predictive ability for a specific TE were compared. There is a benefit in considering both the population structure and the relationship between the TR and the TE when designing an optimal TR for genomic selection. A weighted relationship matrix with stratified sampling is the best strategy for forward predictions of quantitative traits in populations several generations apart.

Abbreviations

AYT

advanced yield trial

BLUE

best linear unbiased estimate

CDmean

average coefficient of determination

EYT

elite yield trial

GBLUP

mixed model best linear unbiased prediction

GBS

genotyping‐by‐sequencing

GS

genomic selection

GY

grain yield

INIA

Instituto Nacional de Investigación Agropecuaria (National Agricultural Research Institute)

IRBP

INIA rice breeding program

IWBP

INIA wheat breeding program

OTR

optimized training population

PEV

mean, prediction error variance

PYT

preliminary yield trial

sBLUP

super best linear unbiased prediction

SNP

single nucleotide polymorphism

taBLUP

trait‐specific relationship matrix best linear unbiased prediction

TE

testing population

TR

training population

W_C

weighted additive relationship matrix with a stratified sampling accounting for genetic cluster

Genomic selection (GS) consists of selecting individuals from a TE on the basis of genotypic values predicted from their genome‐wide molecular marker scores and a statistical model adjusted with individuals that have phenotypic and genotypic information (Meuwissen et al., 2001). The group of individuals that were phenotyped and genotyped is called the TR (Heffner et al. 2009).

Genomic selection is preferred over marker‐assisted selection approaches for complex traits (Habier et al., 2007; Lorenz et al., 2011) because it includes all molecular markers in the prediction model and because it considers the quantitative trait loci of both major and minor effects (Xu, 2003; Jannink et al., 2010; Poland and Rife, 2012; Smith et al., 2018). Simulated and empirical cross‐validation studies in plants show that GS can accelerate progress in plant breeding compared with marker‐assisted selection, resulting in higher genetic gains (Hayes et al., 2009; Crossa et al., 2010; Heffner et al. 2011), higher profit per unit of cost, and superior progenies (Bernardo and Yu, 2007; Heffner et al., 2009, 2011; Heslot et al., 2012; Poland and Rife, 2012; Lado et al., 2017). Genomic selection is also useful when phenotypic information is not available, is not reliable because of low heritability, is costly or labor‐intensive (Lado et al., 2018), or when there is a strong genotype × environment interaction with the target environment.

The most widely used methods of estimating the genotypic values are ridge regression, which estimates marker effects by regressing the trait values on marker genotypes, and the mixed model best linear unbiased prediction (GBLUP), which uses the genomic‐estimated relationship matrix to model the correlation among individuals (VanRaden, 2008). The genotypic values from ridge regression and GBLUP are equivalent (Habier et al., 2009; de los Campos et al., 2012).

The main factors affecting the predictive abilities of GS models are the number and quality of phenotypic observations of the TR (Heffner et al. 2009; Jannink et al. 2010; Cooper et al., 2014; Lado et al., 2018), the genome coverage of molecular markers in both the TR and the TE (Solberg et al., 2008; Poland and Rife, 2012), the size and composition of the TR (Jannink et al., 2010; Heffner et al., 2011; Wientjes et al., 2013), and the relationship between the TR and the TE (Pszczola et al., 2012; Wientjes et al., 2013; Crossa et al., 2014; Hickey et al., 2015). The amount of phenotypic information used in GS has increased over recent years with the development of high‐throughput phenotyping methods and the integration of phenotypic information from different breeding programs in the same geographical region. However, obtaining more phenotypic information of better quality remains a challenge for improving the selection of superior individuals (Araus and Cairns, 2014; Cooper et al., 2014). Studies have shown that the higher the heritability, the higher the prediction accuracy (Combs and Bernardo, 2013; Isidro et al., 2015). In addition, the more complex the trait, the poorer the prediction accuracy regardless of the heritability (Combs and Bernardo, 2013). The number and genome‐wide coverage of molecular markers affect the prediction accuracy, with more molecular markers providing higher predictive ability until a plateau is reached that depends on the population size, structure, and diversity (Lorenzana and Bernardo, 2009; Gorjanc et al., 2017). More molecular markers are generally needed to maintain the prediction accuracy over cycles of selection through a better representation of the linkage disequilibrium structure (Asoro et al., 2011; Habier et al., 2007; Heffner et al. 2011; Lorenzana and Bernardo, 2009; Norman et al., 2018). Additionally, a large number of molecular markers is required to capture the effect of all quantitative trait loci when the TR used is genetically diverse (Norman et al., 2018).

The diversity of the TR affects the accuracy of the predictions in many ways. Several studies have shown that the larger the TR, the higher the predictive ability (Habier et al., 2007; Hayes et al., 2009; Lorenzana and Bernardo, 2009; Asoro et al., 2011; Crossa et al., 2014). Furthermore, low prediction accuracies are obtained when the TR has narrow genetic diversity because it is not possible to accurately estimate all the genotypic effects that explain the variation in the phenotype (Norman et al., 2018). On the other hand, increasing the diversity of the TR by using individuals that are genetically distant from the TE decreases the prediction accuracy (Crossa et al., 2014).

Therefore, one of the most challenging aspects of effective GS is the design of an optimal TR. After a good marker system and high‐quality phenotyping have been established, choosing an optimal TR is not trivial. On the one hand, increasing the TR population size increases the prediction accuracy (Lorenzana and Bernardo, 2009; Asoro et al., 2011). However, because GS models rely on linkage disequilibrium, some studies have found that smaller, more related populations might be optimal (Crossa et al., 2010), whereas other studies have argued that increasing the population size, even at the expense of genetic relatedness, might be optimal (Asoro et al., 2011) for predicting a new population (i.e. the TE). On the other hand, genetic diversity within the TR is fundamental for estimating marker effects appropriately (Norman et al., 2018). Finally, the region in the genome where individuals are more similar also affects genomic predictions, especially for oligogenic traits (Zhang et al., 2010; Wang et al., 2018). Some strategies have been designed to weight genetic relationship matrices on the basis of their marker effects regardless of their position or linkage disequilibrium [trait‐specific relationship matrix best linear unbiased prediction (taBLUP)] (Zhang et al., 2010), or weighting only bin‐selected quantitative trait nucleotides [super best linear unbiased prediction (sBLUP)] (Wang et al., 2014). Several attempts have been made to optimize the TR for a given TE based on maximizing the accuracy of the prediction. Rincent et al. (2012) proposed an iterative process of exchanging individuals to maximize a function derived from the generalized average coefficient of determination (CDmean), defined as the squared correlation between the true and the predicted contrast of genetic values (Laloë, 1993) that maximized prediction accuracy between the TR and a TE, and the use of the prediction error variance (PEVmean). Isidro et al. (2015) used these criteria and three different TR optimization methods based on stratified sampling and showed that population structure has a large effect on optimizing the TR and the best methods depend on it. When the population structure effect was small, the CDmean performed adequately. However, with a strong population structure, the best strategy for optimizing the TR was a stratified sampling based on the population structure (Isidro et al., 2015; Rincent et al., 2017). Because these methods rely on GBLUP theory, they work well with highly polygenic traits but low accuracy would be found in oligogenic traits determined by a few large‐effect quantitative trait loci (Rincent et al., 2017; Wang et al., 2018). A method that would be superior across populations and traits could not be found (Rincent et al., 2012; Isidro et al., 2015). In summary, the composition of the TR should have a balance between the within‐population genetic diversity, the relatedness with the TE, and the regions of the genome where individuals are more similar.

The goal of this study was to compare strategies for optimizing the TR for genomic prediction models in a rice and a wheat breeding program. First, we evaluated how the size, genetic relationships among individuals, and population structure affect the prediction ability within the TR. Second, we proposed some strategies for optimizing the TR based on different methods that account for the genetic relationship between the TR and the TE and we compared these to other methods proposed in the literature.

MATERIALS AND METHODS

Wheat Population

Plant Material

A total of 1353 spring bread wheat advanced inbred lines from the Wheat Breeding Program of the Instituto Nacional de Investigación Agropecuaria (INIA, National Agricultural Research Institute) of Uruguay (IWBP) were used. The IWBP lines consisted of all the advanced inbred lines from the preliminary yield trials (PYT) from 2010, 2011, and 2013, as well as the lines from the advanced yield trials (AYT) and elite yield trials (EYT) from 2010 (Fig. 1).

Fig. 1.

Structure of the advanced wheat inbred lines from the Instituto Nacional de Investigación Agropecuaria Wheat Breeding Program and their genetic relationships. (A) The composition of the training population (TR) and the testing population (TE), with elite yield trials (EYT), advanced yield trials (AYT) and preliminary (PYT) yield trials from 2010 and 2011 comprising the TR. The PYT from 2013 was the forward TE. (B) Dendrogram of the training population constructed with the additive relationship matrix (K) where five groups were identified.

Phenotyping

Grain yield (GY) evaluations were conducted in five locations in Uruguay from 2010 to 2014, including one location with four sowing dates. Locations used to evaluate the genotypes were Dolores (33°50′S, 58°14′W; 15 m a.s.l.), Durazno (33°33′S, 56°31′W; 91 m a.s.l.), La Estanzuela (34°20′S, 57°42′W; 81 m a.s.l.), Young (32°76′S, 57°57′W; 85 m a.s.l.), and Ruta2 (33°45′S, 57°90′W; 95 m a.s.l.). For a full description of the number of lines evaluated in each location and years, see Lado et al. (2016).

Genotyping

Genotyping‐by‐sequencing (GBS) data were obtained for all 1353 IWBP lines. Tissue was collected from plants grown in either the field or the greenhouse. The cetyl trimethylammonium bromide method (Saghai‐Maroof et al., 1984) was used to isolate DNA for the GBS protocol as in Poland and Rife (2012). The TASSEL‐GBS pipeline (Glaubitz et al., 2014) was run with a modification for nonreference genomes (Poland and Rife, 2012). Briefly, markers with a minor allele frequency below 1% or with more than 80% missing data were discarded. Marker–data imputation was conducted via the realized relationship matrix through the multivariate normal expectation maximization method of the rrBLUP package (Endelman, 2011) in R software (R Development Core Team, 2018). We identified 81,999 single nucleotide polymorphisms (SNPs).

Phenotypic Data Analysis

Best linear unbiased estimates (BLUEs) of GY were obtained for all genotypes present in each trial by the nlme package (Pinheiro et al., 2007) in R software (R Development Core Team, 2018). Field analysis was conducted according to the experimental design. Since the PYT consisted of a series of smaller alpha‐design trials grouped by heading date and connected through common checks, the following model [Eq. 1] was used to estimate genotypic means for each heading date group in each environment (i.e. combination of location and year):

$$y_{\mathit{\text{ijkl}}}=\mu +{\alpha }_i+{\lambda }_j+{\gamma }_{k\left(j\right)}+{\beta }_{l\left(kj\right)}+{\epsilon }_{\mathit{\text{ijkl}}},$$ (1)

where μ is the overall mean, α_i is the effect of the i ^th genotype, λ_j is the effect of the j ^th trial, γ _k(j) is the effect of the k ^th replication within the j ^th trial, β _l(kj) is the effect of the l ^th incomplete block within the k ^th replication and the j ^th trial, and ε _ijkl is the residual error from the i ^th genotype in the l ^th block within the k ^th replication in the j ^th trial, with λ _j , β _l(jk), and ε _ijkl as the random variables λ _j ∼ N(0,σ_λ ²), β _l(jk) ∼ N(0,σ_β ²), and ε _ijkl ∼ N(0, σ_ε ²), all of which are independent. The AYT and EYT consisted of alpha‐designs grouped by heading date; therefore, the following model [Eq. 2] was used to estimate genotypic means for each heading date group in each environment (i.e., combination of location and year):

$$y_{\mathit{\text{ijk}}}=\mu +{\alpha }_i+{\gamma }_j+{\beta }_{k\left(j\right)}+{\epsilon }_{\mathit{\text{ijk}}},$$ (2)

where μ, α _i , γ _j , β _k(j), and ε _ijk, were defined as in Eq [1], with β _k(j) and ε _ijk as the random variables β _k(j) ∼ N(0,σ_β ²) and ε _ijk ∼ N(0, σ_ε ²), both of which are independent.

To decrease the genotype × environment interactions, Mega‐Environments 1 and 2, as identified in Lado et al. (2016) were used for this study. We did not use Mega‐Environment 3 because a large genotype × environment interaction exists between Mega‐Environment 3 and Mega‐Environments 1 and 2. The mega‐environments were constructed as groups of environments with low within genotype × environment interaction and high among genotype × environment interaction. Mega‐Environments 1 and 2 include all locations from 2010, 2011, and 2013 and one of the La Estanzuela environments from 2014. They have a high correlation between environments. The final model used to obtain genotypic means across environments was:

$$y_{\mathit{\text{ijk}}}=\mu +{\alpha }_i+{\delta }_j+{\gamma }_{k\left(j\right)}+{\epsilon }_{\mathit{\text{ijk}}},$$ (3)

where μ, α _i , and γ _k(j), were defined as in Eq. [1]; δ _j is the j ^th environment (i.e., location and year) within a mega‐environments, with γ _k(j), and ε _ijk being the random variables γ _k(j) ∼ N(0,σ² _γ) and ε _ijk ∼ N(0, σ_ε ²), both of which are independent. The BLUEs were estimated for each genotype by the nlme package (Pinheiro et al., 2007) in R software (R Development Core Team, 2018).

Rice Population

Plant Material

A total of 644 lines from the INIA Rice Breeding Program of Uruguay (IRBP) were used. The population consisted of 325 lines from the O. sativa ssp. indica, 314 lines from O. sativa ssp. japonica (tropical japonica), two indica cultivars [El Paso 144 (Yan et al., 2007) and INIA Olimar (Blanco et al., 1993; Instituto Nacional de Semillas, 2017) that are the most widely grown indica cultivars in Uruguay, and three tropical japonica cultivars [INIA Parao (Molina et al., 2011), INIA Tacuarí (Blanco et al.,1993), and INIA Caraguatá (Blanco et al., 1993)]. All cultivars were used as checks in all phenotyping experiments.

Phenotyping

Rice lines were evaluated for GY in the Experimental Unit of Paso de la Laguna (33°16′S, 54°10′W), Treinta y Tres, Uruguay, during three growing seasons (October–March): 2010–2011, 2011–2012, and 2012–2013. For a full description, see Monteverde et al. (2018).

Genotyping

Genotyping‐by‐sequencing data were obtained for the 644 advanced inbred lines and cultivars from the IRBP. DNA was extracted from young leaf tissue from plants grown at the Biotechnology Unit in Las Brujas, Canelones, Uruguay. The extraction was done with the Qiagen Dneasy kit (www.qiagen.com, accessed 19 Oct. 2019). The GBS libraries and sequencing were done at the Biotechnology Resource Center in the Genomic Diversity Facility at Cornell University in Ithaca, New York. Libraries were prepared according to the protocol of Elshire et al. (2011). Because of the strong population structure present within the lines, three datasets were obtained: an indica set, a tropical japonica set, and a combined indica and tropical japonica set. For both subspecies, indica and tropical japonica, SNPs were called from fastq files via the TASSEL version 3.0 GBS pipeline (Bradbury et al., 2007) as described in Spindel et al. (2013). Alignment to the Michigan State University Nipponbare rice reference genome version 7.0 was performed with Bowtie 2 (Langmead and Salzberg, 2012). Imputation of missing data for each of the indica (SNP_I) and tropical japonica (SNP_TJ) genotypes was performed with the FILLIN algorithm implemented in TASSEL version 5.0 (Bradbury et al., 2007). The average imputation accuracy was approximately 94% for both the indica and tropical japonica datasets. Single nucleotide polymorphism markers that had more than 50% missing data after the imputation along with monomorphic SNPs and SNPs with a minor allele frequency below 5% were removed from the datasets, as reported in Quero et al. (2018). The remaining missing data were imputed by the mean to perform principal component analysis. The final indica dataset contained 92,430 markers and the tropical japonica dataset had 44,598 markers. For the combined indica and tropical japonica set (SNP_ALL), SNPs were called from fastq files via the TASSEL version 3.0 GBS pipeline (Bradbury et al. 2007); the final dataset had 15,545 markers (Fig. 2).

Fig. 2.

Structure of the advanced rice inbred lines from the Instituto Nacional de Investigación Agropecuaria Rice Breeding Program and their genetic relationships. (A) Three datasets were used: indica with 92,430 single nucleotide polymorphisms (SNPs) and 327 advanced inbred lines, tropical japonica with 44,598 SNPs and 317 advanced inbred lines, and a combined indica and tropical japonica set with 15,545 SNPs and 644 individuals. (B) The first two principal components discriminating indica (purple, n = 327) and tropical japonica (green, n = 317) individuals in the combined data set.

Phenotypic Data Analysis

The BLUEs of GY were obtained for all genotypes present in each trial via the lme4 package (Bates and Sarkar, 2010) in R software (R Development Core Team, 2018). Field analyses were conducted according to experimental design, which consisted of a series of smaller trials with randomized complete block designs connected through common checks. The following model, which used spatial correction for rows and columns, was used to estimate GY genotypic means for each environment (i.e., year) and subspecies (i.e., indica and tropical japonica):

$$y_{\mathit{\text{ijklm}}}=\mu +{\alpha }_i+{\lambda }_j+{\gamma }_{k\left(j\right)}+{\eta }_{l\left(j\right)}+{\kappa }_{m\left(j\right)}+{\epsilon }_{\mathit{\text{ijklm}}},$$ (4)

where μ, α _i , λ _j , and γ _k(j) were defined as in Eq. [1]; η _l(j) is the random effect associated with the l ^th row in the j ^th trial; κ _l(j) is the random effect associated with the l ^th column in the j ^th trial, with λ _j , η _l(j), κ _m(j), and ε _ijklm as the random variables λ _j ∼ N(0,σ_λ ²), η _l(j) ∼ N(0,σ_η ²), κ _m(j) ∼ N(0,σ_κ ²), and ε _ijklm ∼ N(0, σ_ε ²), all of which are independent. The GY in tropical japonica in 2011 was not improved by the spatial corrections based on Akaike information criterion values and therefore we used Eq. [4] without η _l(j) and κ _m(j) for this trait. A final model with all the environments (i.e., years) was used to obtain overall genotypic means via the lm function in R software (R Development Core Team, 2018):

$$y_{ij}=\mu +{\alpha }_i+{\delta }_j{+\epsilon }_{ij},$$ (5)

where μ is the overall mean, α _i is the effect of the i ^th genotype, δ _j is the j ^th environment (i.e., year), and ε _ij is the residual error, ε _{ij ∼} N (0, σ² _ε).

Overall GBLUP Model

Genotypic values were predicted via an overall GBLUP model following de los Campos and Pérez (2010):

$$y=1\mu +u+\epsilon ,$$ (6)

where y _{(N × 1)} is the vector of mean yield for each genotype in all environments (i.e., the BLUEs from a model accounting for field design and environment) of length N (N = population size or number of genotypes in the set), 1_{(N × 1)} is a vector of ones of length N, μ is the overall mean, and u _{(N × 1)} is a random vector of genotypic predictors with u ∼ N(0, K _{(N × N)} σ² _g), where K is the realized additive relationship matrix estimated as the cross‐product of the centered and standardized marker states divided by the number of markers and estimated with the rrBLUP package (Endelman, 2011) in R software (R Development Core Team, 2018); ε is the vector of residuals with ε ∼ N(0, I _{(N × N)} σ² _ε) where I is an identity matrix of size N.

Structured GBLUP Model

We used a GBLUP approach where subgroup‐specific marker effects were estimated. The marker effects are described as the sum of common effects for all groups and group‐specific random deviation. The structured GBLUP model can be represented as:

$$y=1\mu +u_0+u_1+\epsilon ,$$ (7)

where y _{(N × 1)} is the vector of mean yield for each genotype in all the environments (i.e., the BLUEs from a model accounting for field design and environment) of length N (N = population size or number of genotypes in the set); 1_{(N × 1)} is a vector of ones of length N; μ is the overall mean; u _{0(N × 1)} is a random vector of genotypic predictors with u ₀ ∼ N(0,Kσ² _u0), where K _{(N × N)} is the realized additive relationship matrix estimated as in the overall GBLUP model; u _{1(N × 1)} is a random vector for the group‐specific genotypic predictors; and u ₁ ∼ N(0, K ₁ ), where K ₁ is a block diagonal matrix, with each block being the realized additive relationship matrix estimated for each specific group with a group variance and off‐diagonal zeros. Assuming two groups, K ₁ is:

$$K_1=\frac{\left(\begin{array}{cc}{\sigma }_{u_1}^2{X}_1{X^{\prime }}_1& 0\\ 0& {\sigma }_{u_2}^2{X}_2{X^{\prime }}_2\end{array}\right)}{p},$$ (8)

where ${\sigma }_{u_1}^2\text{ and }{\sigma }_{u_2}^2$ are the genetic variance specific to Groups 1 and 2, respectively, and p is the number of markers. In this model, u ₀ allows information to be shared between groups, whereas u ₁ captures group‐specific effects. For this model, we used the BGLR package (de los Campos and Pérez, 2010) in R software (R Development Core Team, 2018).

Within‐Population Optimization

Three strategies were used to optimize the TR. For each strategy, a GBLUP model was fitted by the rrBLUP package (Endelman, 2011) in R software (R Development Core Team, 2018). The predictive ability was estimated as the correlation between predicted and observed genotypic values obtained via cross‐validation (Burgueño et al., 2012; De Leon et al., 2016). Models were compared in terms of their predictive ability to random samples of the same size and to using all the individuals in the TR.

Strategy 1: Grouping Based on Genetic Relationship

Wheat Population. A clustering algorithm with the realized additive relationship matrix was used to group individuals in the IWBP TR population with the cluster package (Kaufman and Rousseeuw, 1990) in R software (R Development Core Team, 2018). The Ward hierarchical agglomerative method with the pseudo‐F statistic was used to group similar individuals. The overall GBLUP model and the structured GBLUP model with groups as classes were used to predict genotypic values in the whole population.

Rice Population. The IRBP was grouped by subspecies: indica or tropical japonica. Given the high level of allelic phasing in the rice population, several dataset models were compared: (i) SNP_ALL to predict all individuals, (ii) SNP_ALL to predict indica individuals, (iii) SNP_ALL to predict tropical japonica individuals, (iv) SNP_TJ to predict tropical japonica individuals, and (v) SNP_I to predict indica individuals. The overall GBLUP model was used to predict genotypic values in the rice population.

Strategy 2: Grouping Based on Trials (Wheat Only)

The wheat population was evaluated in multiple trials, each having different phenotypic data quality. The overall GBLUP model and the structured GBLUP model that had field trials (EYT, AYT, and PYT) as classes were compared to predict genotypic values in the whole population (Fig. 1A).

Strategy 3: Grouping Based on Maturity (Wheat Only)

The advanced inbred lines were routinely grouped into early or late maturity lines on the basis of their heading date. Different breeding objectives for GY were then pursued for each group: early‐maturity lines (short‐cycle lines) were selected for high yield based on a large number of grains per spike, whereas late‐maturity lines (long‐cycle lines) were selected for high yield based on a large number of spikes. Since different selection pressures have been imposed on the two groups, the overall GBLUP model and the structured GBLUP model that used maturity as classes were compared to predict breeding values in the whole population.

Model Validation

Predictive ability was estimated as the correlation between the predicted and observed genotypic values via Type 1 cross‐validation (Legarra et al., 2008; Burgeño et al., 2012; De Leon et al., 2016). A k‐fold cross validation within TR was used, following Burgueño et al. (2012). Briefly, the observations were randomly divided into k nonoverlapping subsets. Next, k –1 groups were used as training sets and the remaining group was used as the validation set (i.e., genotypic values were obtained for each individual in the validation set). This procedure was followed until the genotypic values of individuals in all k subsets had been predicted. One hundred replications of the cross‐validation with k = 7 were performed and the correlation between the predicted genotypic values and observed genotypic means was used to estimate the predictive ability.

Optimization of the TR to a Specific TE

For optimization of the TR to predict a specific TE, we used 984 wheat lines of the EYT, AYT, PYT 2010, and PYT 2011 as the TR and 369 PYT 2013 lines as the TE (Fig. 1A). Four strategies were evaluated for the selection of the training population; random selection, the genetic relationships between the TR and the TE, or two optimization criteria: the CDmean proposed by Rincent et al. (2012) and PEVmean.

Strategy 4: Selection Based on the Estimated Additive Genetic Relationship

The optimized TR (OTR) was constructed by choosing individuals from the TR on the basis of their high similarity to the TE. Two criteria were used to define similarity for each individual: the average relationship with the TE $\left(\overline{K}\right)$ and the median relationship with the TE (K _0.5). The predictive ability of the OTR was evaluated for selecting the top 15, 20, …, 100% individuals from the TR.

Strategy 5: Selection Based on the Weighted Estimated Additive Genetic Relationship

Similar to Strategy 4, the OTR was constructed by choosing individuals from the TR on the basis of their similarity to the TE, but the weighted additive relationship matrix (W) was used instead of the kinship matrix. The weighted relationship matrix is used in a similar manner to the taBLUP of Zhang et al. (2010) or the sBLUP of Wang et al. (2018), with the difference being that we are using this matrix to identify the most related individuals, but not in the prediction model. The genetic distance between the TR lines and the TE lines was calculated as the mean marker distance weighted by the estimated marker effects with a modification of the variance estimate of Endelman (2011):

$$\mathit{\text{Wij}}=\sqrt{\sum _{k=1}^{k=M}\left(\frac{\left|x_{.TR\left[i,k\right]}-x_{.TE\left[j,k\right]}\right|}{2}\right)\times \widehat{u}{\left[k\right]}^2},$$ (9)

where i is the i ^th line of the TR, j is the j ^th line of the TE, k is the k ^th marker, X._{TR(n.TR ′ m)} and X._{TE(n.TE′ m)} are the genotypic matrices with the marker state (–1, 1), M is the number of markers, and u is the vector of the estimated marker effects. The mixed.solve function of the rrBLUP package (Endelman, 2011) in R (R Development Core Team, 2018) was used to estimate u. Three criteria were used to define similarity: (i) the average relationship to the TE $\left(\overline{W}\right)$, (ii) the median relationship to the TE (W _0.5), and (iii) a proportional stratified sampling based on the average relationship to the TE and the groups defined in Strategy 1 (W _C ). The predictive ability of the OTR was evaluated for selecting the top 15, 20, and 100% individuals from the TR via the GBLUP models.

Strategy 6: Selection Based on the CDmean and PEVmean Genetic Algorithms

The OTR was constructed by choosing individuals from the TR according to a genetic algorithm that maximized the precision of the prediction of the difference between the values of each nonphenotyped individual (TE) and the mean of the population of candidate individuals (TR). We used the CDmean, defined as the squared correlation between the true and the predicted contrast of genetic values, and the PEVmean, defined as the variance of the distance between the true and the predicted contrast of genetic values, as optimization criteria. The optimization algorithm code was adapted from code provided by R. Rincent (pers. comm., 29 May 2017) and was implemented in R (R Development Core Team, 2018). The code is available as Supplemental File S1. For each sample size and each criterion, 50 repetitions of the algorithm with 800 iterations were used. The predictive ability of the OTR was evaluated for selecting the top 15, 20, and 100% individuals from the TR.

Random Selection

Finally, a random selection of TR lines was used to predict the TE by taking subgroups of different sizes. For subgroups with 15, 20, and 100% of the lines, GBLUP models were trained and used to predict the TE.

RESULTS

Within‐Population TR Optimization

Wheat Population

The predictive ability increased with the number of individuals used in the TR (Fig. 3); however, after 600 individuals, the increase in predictive ability became marginal. Some groups had higher predictive ability than those obtained with either a random sample of the same size or the entire TR (Fig. 3). Group 1 obtained from the cluster analysis was better predicted than the other cluster groups, AYT was better predicted than the other trials groups, and the short‐cycle lines were better than the long‐cycle lines. The highly predictive groups had high within‐population structure (Fig. 4) that was not associated with yield (data not shown). The model that includes the structure of the trials or cycle groupings showed higher predictive ability than random samples of the same size (Fig. 3).

Fig. 3.

Predictive ability of three clustering strategies [genetic groups based on the realized additive relationship matrix: Group to Group 5 (C1–C5); field trials: elite yield trials (yet), advanced yield trials (AYT), preliminary yield trials from 2010 (PYT10), and preliminary yield trials from 2011 (PYT11); and maturity: short‐cycle lines (SC) and long‐cycle lines (LC)], and the structured mixed model best linear unbiased prediction (GBLUP) model used for the within‐population optimization for the wheat dataset.

Fig. 4.

Within‐group population structure for the highly predictive groups: (A) Group 1, (B) advanced yield trials (AYT), and (C) short‐cycle lines for the wheat dataset represented by heatmaps of the realized additive relationship matrix, with the parental contribution given below (i.e., each color represents a different parent).

Rice Population

Using both indica and tropical japonica subspecies as the TR population to predict all individuals resulted in a high predictive ability (r = 0.8, Fig. 5A). Group membership (i.e., indica or tropical japonica) was highly predictable but within‐group predictions were low (Fig. 5B). Furthermore, when individuals from both subspecies were used to predict either indica (r = 0.53) or tropical japonica (r = 0.37), the predictive ability was low. Finally, when using only a subspecies to predict its own performance, the predictive ability of both indica (r = 0.69) and tropical japonica (r = 0.54, Fig. 5A) increased. There was a strong population structure among rice subspecies that was highly associated with GY (Fig. 5C).

Fig. 5.

(A) Predictive ability of different rice training population (TR) datasets to predict specific testing population (TE) datasets: ALL = all individuals; I = indica individuals; TJ = tropical japonica individuals. (B) One realization of a scatterplot of observed vs. predicted values for the dataset that used ALL individuals to predict ALL. (C) Heatmap of the realized additive relationship matrix of the rice population. (D) Box‐plot of grain yield by rice subspecies.

Optimization of the TR to a Specific TE

The best OTR strategy was to use the average of the weighted additive relationship matrix with a stratified sampling accounting for genetic cluster (W _C , Fig. 6). The K mean and K median were the worst optimization criteria (Fig.6). The CDmean and PEVmean stragtegies were no better than random selection but were better than K (Fig. 6).

Fig. 6.

Optimization of the training population (TR) through three overall strategies. (A) Strategy based on the average $\left(\overline{K}\right)$ or median (K _0,5 ) realized genetic relationship of TR individuals to the testing population (TE). (B) Strategy based on the average $\left(\overline{W}\right)$ or median (W _0,5 ) weighted genetic relationship of TR individuals to the TE or the average W with a stratified sampling considering genetic cluster (W _C , and C) on the basis of the the average coefficient of determination (CDmean) and average prediction error variance (PEVmean) optimization criteria. All optimized TRs are compared with random samples of the same size (the mean performance of 100 random samples is in black and the range is shaded).

DISCUSSION

Our study was able to first characterize the structure of small, highly predictive training sets to design an optimization strategy for identifying training sets for forward prediction of specific testing sets. We propose the use of a weighted relationship matrix in combination with stratified sampling to optimize the TR. This strategy is superior to random samples of the same size, the use of all available individuals, and other optimization strategies proposed in the literature. We discuss our findings in the context of population sizes, the diversity and structure of the TR, and relationship between the TR and the TE and propose conditions (i.e., traits and populations) where we believe our strategy would be superior.

Size of the TR

We found an increase in predictive ability with larger population sizes up to 600 individuals, which is similar to other studies (Lorenzana and Bernardo, 2009, Asoro et al., 2011). It has been widely established that predictive ability is higher when larger TRs are used (Muir, 2007; Meuwissen, 2009; Asoro et al., 2011; Lorenz, 2013; Isidro et al., 2015; Edwards et al., 2019); however, the relationship between TR size and other factors such as genetic relationship and population structure is less understood. We were able to identify smaller groups of individuals with high predictive ability (i.e., Group 1, AYT, short‐cycle lines, and the W _C optimization strategy). These groups made better predictions than random samples of the same size and better predictions than all available individuals in the population. Therefore, there is a trade‐off between population size and other factors that should be considered when optimizing the TR.

Diversity and Population Structure of the TR

There are a few processes that are relevant to how diversity affects the predictive ability of a TR. Diversity is a necessary condition for marker effect estimations (Norman et al., 2018) but how diversity is structured in the population is also relevant (Isidro et al., 2015). The genetic relationships among individuals are required for accurate genomic predictions (Habier et al, 2007; Asoro et al., 2011; Clark et al., 2012; Isidro et al., 2015; Edwards et al., 2019) because a relationship matrix is used to borrow information from relatives for prediction. Therefore, unrelated individuals can only be predicted by the mean performance of the population in a GBLUP context. In a broader interpretation, marker effects are genetic‐background‐dependent (Asoro et al., 2011; Toosi et al., 2010). Different levels of genetic relationship such as those created by family relationships can also create population structure (Würschum et al., 2017; Schmidt et al., 2016). The level of population structure determines the optimal prediction strategy (Isidro et al., 2015) and different strategies have been proposed to deal with population structure such as prediction within families (Würschum et al., 2017) or groups (Norman et al., 2018), prediction of group performance [i.e., compressed BLUP, (Wang et al., 2018)] or prediction of group performance and within‐group deviation [structured GBLUP (de los Campos and Pérez, 2010)]. We evaluated these processes in two species with different levels of population structure and then proposed a new strategy for dealing with population structure that is more effective for forward prediction.

We found strong population structure in opposite phases in rice that was associated with the phenotype. This population structure overestimated the overall prediction accuracy (i.e., r = 0.85) by being accurate in predicting group membership and performance but being a poor predictor of within‐group performance (i.e., r < 0.5). We show the effect of strong population structure on overall, group, and within‐group predictions in a constructed example (Fig. 7). When population structure is associated with the phenotypic trait of interest, the model might be accurate at predicting group membership and group performance but extremely poor at predicting within‐group performance. Group performance is relevant for predicting traits with low heritability that are hard to predict in general and where individuals within a group may act as pseudo‐replications of family or groups, improving overall prediction accuracy (Wang et al., 2018). When the groups are families, average family performance can be easily predicted as the average performance of both parents and is therefore less relevant (Würschum et al., 2017). For example, in a broader case, identifying indica and tropical japonica individuals and predicting all individuals in each subspecies as the mean performance of their group had no practical relevance. Because population structure can create a challenge for predictions when markers are in opposite phases among subpopulations (Toosi et al., 2010; Asoro et al., 2011; Lopez‐Cruz et al., 2015), within‐family or within‐group prediction might be more effective. Overestimation of the predictive accuracy was also observed by Schmidt et al. (2016), who combined spring and winter barley (Hordeum vulgare L.). Therefore, overall predictions with a strong population structure that is associated with the phenotype are challenging.

Fig. 7.

Example constructed to show the consequences of overestimation when the phenotypic trait is associated with population structure, showing the case where population structure is associated with the phenotype (left panels) and a case where population structure is not associated with the phenotype (right panels). Boxplots for phenotypic values for a hypothetical trait are shown in the upper panels and the correlations between predicted and observed values are shown in the lower panels. Overall predictions might be high when population structure is associated with the phenotypic traits as the result of an artifact of mean performance for each group. This model will excel at predicting group membership and group performance (i.e., which population every individual belongs to and what that group's overall performance is) but might be a very poor predictor of within‐group performance (i.e., which individual within that group or family will be superior). On the other hand, when population structure is not associated with the phenotype, overall predictions will reflect within‐group predictions, with the advantage that larger populations are used.

On the other hand, all the small groups that were highly predictive in our study (i.e., Group 1, AYT, and short‐cycle lines) shared a low level of within‐group population structure that was created by family relationships and was not associated with the phenotype. The presence of low levels of population structure that is not associated to the phenotype creates optimal prediction conditions (Fig. 7), where enough diversity is present to estimate marker effects and there is a high likelihood of each individual in the TE having a relative represented in the TR. Overall predictions in these cases can benefit from larger population sizes.

Furthermore, structured GBLUP models outperformed overall predictions in most of our situations by including a group effect as well as within‐group deviations. Marker‐independent groups are more effective than groups based on marker‐only information, probably because these groups provide redundant information that is already accessible from the relationship matrix. Other studies have shown that considering population structure improves prediction ability (Isidro et al., 2015; Rincent et al., 2017; Norman et al., 2018); therefore, population structure should play a key role in the strategies used to build OTRs for genomic selection (Asoro et al., 2011; Crossa et al., 2014; Isidro et al., 2015, Lorenz and Nice, 2017; Rincent et al., 2017).

The Relationship between TR and TE

One of the main factors driving the trade‐offs among population size, diversity, and population structure is the relationship between the TR and the TE (Habier et al., 2007, 2013; Crossa et al., 2010, 2014; Lorenz et al., 2012; Pszczola et al., 2012). Training sets that are more related to the testing sets have higher predictive ability (Crossa et al., 2014; Lorenz and Smith, 2015; Riedelsheimer et al., 2013). Furthermore, Isidro et al. (2015) concluded that an optimal design for the TR should minimize the relationships among the genotypes in the TR (i.e., the high diversity within the TR) while maximizing the relationship between the TE and the TR. Several methods have been proposed to optimize the TR in this context (Rincent et al., 2012, 2017; Isidro et al., 2015) with trait‐ and population‐structure‐dependent results. We used both a within‐TR cross‐validation and an independent forward TE approach to evaluate these effects and found that genetic relationship between TR and TE is one of the most relevant properties of the TR for increasing predictive ability. We proposed an optimization strategy that outperformed previous methods. The best optimization strategy was the weighted relationship matrix with stratified sampling. The CDmean and PEVmean methods outperformed the relationship matrix (K) method, as in Rincent et al. (2012). However, the weighted relationship matrix with stratified sampling was superior to CDmean and PEVmean. Both CDmean and PEVmean have shown limited success when population structure is present (Isidro et al., 2015). We believe that proper modeling of the population structure is one of the main reasons for the success of our strategy. Estimations of the predictive ability of the CDmean strategy can be biased in the optimization phase if the search space is not large enough in terms of the number of starting points and iterations. We believe that our CDmean estimations are appropriate, the number of iterations used were based on achieving a plateau in the predictive ability for each population size, and the number of starting points was increased until consistent estimations of the predictive ability were found for each population size (Supplemental Fig. S1).

The region of the genome where individuals are more similar is more relevant than being similar across the genome. We found that a weighted genetic relationship matrix (W) outperformed the genetic relationship matrix (K). This shows that the kinship matrix per se is not a good indicator of groups of individuals with different marker effect responses for relevant phenotypic traits. Zhang et al. (2010) proposed and explored the idea of a weighted relationship matrix but in a different context. They found that the use of a trait‐specific marker information matrix (taBLUP) for predictions improved the predictive ability over the GBLUP model. We used the same general idea of the taBLUP but with a few modifications. Genome‐wide estimations of marker effects were used instead of grouping markers through previous identification of the relevant markers by genome‐wide association studies. Furthermore, instead of using the weighted matrix for the prediction model, we used it in the sampling strategy of the TR to decide which individuals are the most related to the TE (i.e., W strategy). This OTR has higher predictive ability than random samples of the same size and higher predictive ability than including all available individuals from the TR. This strategy is probably better suited for complex quantitative traits because it uses a genome‐wide approach for marker effect estimation (Zhang et al., 2010). Other strategies for weighting the relationship matrix, such as taBLUP and sBLUP will be more effective for oligogenic traits (Wang et al., 2018). Furthermore, because bin‐optimized quantitative trait nucleotides from genome‐wide association studies are optimized to account for linkage disequilibrium structures in sBLUP, sBLUP will perform better than taBLUP (Wang et al., 2018). The W TR optimization strategy that we proposed could be used with the sBLUP weighted matrix instead of the genomic‐weighted matrix to predict oligogenic traits in future studies.

The best optimization strategy in our study included a weighted matrix to evaluate relationship among individuals, along with stratified sampling (W _C ). We used the general ideas of the compressed BLUP (Wang et al., 2018) and structured GBLUP (de los Campos and Pérez, 2010) approaches but modified them to optimize the TRs. Instead of using the clusters to predict group performance, as in the compressed BLUP (Wang et al., 2018), or group and within‐group performance, as in the structured GBLUP (de los Campos and Pérez, 2010), we used a clustering strategy to obtain a stratified sample of the TR that was more similar to Isidro et al (2015). The stratified sampling approach is superior to sampling the most related individuals overall because it produces a better representation of all the linkage disequilibrium structures in the TR that are relevant for the TE. However, when the TR and TE are conceptually the same as in x‐fold validations or replaced phenotyping strategies, the W _C method performs similar to the PEVmean and CDmean methods. We showed the effect of different sampling strategies for forward and x‐fold validation strategies (Supplemental Fig. S2). When the TE is conceptually different from the TR or has a lower level of relationship with the TR (i.e., similar to when forward predictions are used in GS to predict future populations), the W _C is the best strategy (Supplemental Fig. S2A). However, when an x‐fold validation is used, such as in replaced phenotyping strategies, the W _C strategy has no advantage over the other methods (Supplemental Fig. 2B). Stratified sampling with the weighted relationship matrix (W _C ) is therefore especially relevant when forward predictions are attempted with future TEs, as in our case. In our study, we optimized the TR for a forward prediction three generations apart (i.e., a future PYT). This strategy was able to capture the relevant genomic regions and better estimate the marker effects increasing predictive ability. Therefore, smaller, OTR can be used to improve predictive ability. This indicates that neither the general relationship nor the population size per se are the best indicators of predictive ability. Optimized populations perform better than random samples of the population of the same size but are more relevant and they have higher predictive ability than the use of all individuals available from the TR.

CONCLUSIONS

In summary, we proposed a new strategy to optimize TRs to predict specific TEs in a forward approach. Our strategy of using a weighted relationship matrix in combination with a stratified sampling approach was the best approach for optimizing the TR. This strategy performed better than a random sample of the population of equal size, use of all the individuals available in the training population, or use of the relationship matrix, the CDmean, or PEVmean to choose the individuals to make up the TR. This strategy will be superior for complex quantitative traits and when small levels of population structure are present, such as with the familial relationship structures common in plant breeding populations. For oligogenic traits, a similar sampling strategy could be implemented but with the sBLUP weighted matrix instead of genome‐wide marker effects. Our strategy will perform similar to the CDmean or PEVmean when population structure is absent or if the TE has conceptually the same structure as the TRs.

Supplemental Information

Supplemental File S1. The optimization algorithm code used for the CDmean and PEVmean optimization. This code was adapted from a code provided by R. Rincent (pers. comm., 29 May 2017) and was implemented in R (R Development Core Team, 2018).

Supplemental Fig. S1. (A) Coefficients of determination in each iteration and by training population size. (B) Predicted ability for CDmean, PEVmean, and random selection for each training population size.

Supplemental Fig. S2. Optimization of the training population using two overall strategies for two partitions of the TR: (i) predicting the least related individuals and (ii) predicting individuals from the same conceptual population as in an x‐fold validation or replaced phenotyping. (A) Principal component of the TR, highlighting the partition strategies. (B) Predictions based on the average $\left(\overline{K}\right)$ or median (K _0.5) realized genetic relationship of TR individuals to the TE. (C) Predictions based on the average ($\left(\overline{W}\right)$ or median (W _0.5) weighted genetic relationship of TR individuals to the TE, or the average W with a stratified sampling considering genetic cluster (W _C ). All OTRs are compared with random samples of the same size (the mean performance of 100 random samples is in black and the range is shaded).

Conflict of Interest Disclosure

The authors declare that there is no conflict of interest.

Author Contributions

IB, BL, RN and LG conducted statistical analyses. MQ designed the phenotyping experiments. BL performed genotyping analysis. IB and LG wrote the paper. LG designed the study and hypothesis. All authors read and approved the final manuscript.

Supporting information

Supplemental File 1

Supplemental File 2

Supplemental File 3

Supplemental File 4