Constructing training sets for genomic selection to identify superior genotypes in candidate populations

Abstract

Key message

Approaches for constructing training sets in genomic selection are proposed to efficiently identify top-performing genotypes from a breeding population.

Abstract

Identifying superior genotypes from a candidate population is a key objective in plant breeding programs. This study evaluates various methods for the training set optimization in genomic selection, with the goal of enhancing efficiency in discovering top-performing genotypes from a breeding population. Additionally, two approaches, inspired by classical optimal design criteria, are proposed to expand the search space for the best genotypes and compared with methods focusing on maximizing accuracy in breeding value prediction. Evaluation metrics such as normalized discounted cumulative gain, Spearman’s rank correlation, and Pearson’s correlation are employed to assess performance in both simulation studies and real trait analyses. Overall, for candidate populations lacking a strong subpopulation structure, a ridge regression-based method, referred to as ${\text{MSPE}}^{\text{Ridge}},$ is recommended. For candidate populations with a strong subpopulation structure, a heuristic-based version of generalized coefficient of determination $\left({\text{CD}}_{\text{mean}(\text{v}2)}\right)$ and a D-optimality-like method that maximizes overall genomic variation $({\text{GV}}_{\text{overall}})$ are preferred approaches for the primary objective of plant breeding. For populations with a large number of candidates, a proposed ranking method (${\text{GV}}_{\text{average}}$) can first be used to down-scale the candidate population, after which a heuristic-based method is employed to identify the best genotypes. Notably, the proposed ${\text{CD}}_{\text{mean}(\text{v}2)}$ has been verified to be equivalent to the original version, known as ${\text{CD}}_{\text{mean}}$, but its implementation is much more computationally efficient.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00122-024-04766-y.

Introduction

Genomic selection (GS) is a powerful strategy for selecting quantitative traits in plant breeding, whose primary concept is to capture quantitative trait loci using high-density molecular markers over an entire genome (Meuwissen et al. 2001). A GS-based approach predicts genomic estimated breeding values (GEBVs) by building a prediction model using the phenotype and genotype data of a training set. After the model training, the GEBV of any individual can be readily available by plugging its genotype data into the prediction model. In practice, breeders conduct breeding selection according to the resulting GEBVs. Model training plays a crucial role in GS, and its prediction accuracy is highly dependent upon the data quality of the training set. Genotyping costs have decreased considerably; while, phenotyping costs have stayed relatively expensive. Moreover, space and time constraints are usually inevitable in an agricultural experiment for collecting phenotype data. Therefore, a cost-effective optimal training set for selective phenotyping is highly desired to increase the probability of success in GS (Wu et al. 2023).

In the context of GS, the expected genetic gain per year for measuring breeding progress can be defined as $\mathrm{\Delta }G=ir{\sigma }_A/t$, where $\mathrm{\Delta }G$ is the response to selection, $i$ is the intensity of selection,$r$ is the Pearson’s correlation between true breeding values (TBVs) and GEBVs, ${\sigma }_A$ is the square root of the additive genetic variance, and $t$ is the breeding cycle time (Heffner et al. 2010). This could partially explain why Pearson’s correlation is usually used as an index to measure prediction accuracy in GS. The training set optimization for maximizing Pearson’s correlation has drawn much attention. Based on linear mixed effects models, Rincent et al. (2012) first promoted the generalized coefficient of determination (CD), initially proposed by Laloë (1993), to determine an optimal training set. Isidro et al. (2015) and Rincent et al. (2017) extended CD-based optimization for highly structured populations. Alternatively, based on ridge regression models, Akdemir et al. (2015) considered variation of prediction errors, and Ou and Liao (2019) proposed a criterion derived directly from Pearson’s correlation to optimize a training set. Akdemir and Isidro-Sánchez (2019) discussed classical optimal design criteria such as A- and D-optimality in training set optimization. Atanda et al. (2021a) proposed an algorithm that maximizes the relationship between training and testing sets. Most recently, Fernández-González et al. (2023) made a comprehensive comparison among these training set optimization methods.

Even though Pearson’s correlation between TBVs and GEBVs is essential in genetic gain, a breeding program often aims to select a few top individuals to serve as commercial varieties or parents for future breeding cycles. For this purpose, it would be sufficient to correctly rank candidates from the most favorable to the least favorable. Blondel et al. (2015) promoted the use of normalized discounted cumulative gain (NDCG) to measure the ability of a selection strategy and evaluated some ranking approaches based on real trait analyses. NDCG can quantify the efficiency of the selection strategy for identifying the top genotypes from a candidate population, which has been commonly used to measure the ability of search engines to retrieve highly relevant documents in the top search results (Järelin and Kekäläinen 2000).

Tanaka and Iwata (2018) proposed a Bayesian optimization approach for identifying the best genotypes from a candidate population. Tsai et al. (2021) further developed improved versions of their approach. Bayesian optimization approaches are required to sequentially select individuals for phenotyping. In practice, multi-stage approaches are infeasible in plant breeding mainly due to the time constraint that breeders can usually collect phenotype data only after a whole growing season. A positive linear relationship exists between TBVs and GEBVs if their Pearson’s correlation coefficient r equals 1. That is, $\text{TBV}=a+b\times \text{GEBV}$ for some $b>0$ if $r=1$. Therefore, Pearson’s correlation can serve as a ranking measure across all individuals of interest, suggesting that training set optimization methods aimed at enhancing r can be utilized to identify the most promising genotypes within a breeding population. It would be highly beneficial to assess their effectiveness in achieving this practical objective in plant breeding.

To identify the best genotypes, this study assessed both existing and newly developed training set optimization methods, and two approaches inspired by classical optimal design criteria of A- and D-optimality. We comprehensively compared the methods based on simulation studies using the genotype data of four genome datasets as templates and analyses of phenotype data of real traits in the genome datasets. NDCG, together with Spearman’s rank correlation (SRC) (Spearman 1904) and Pearson’s correlation, were used to measure the ability to discover the best genotypes from a candidate population. The four real genome datasets under study included one rice (Oryza sativa L.), one wheat (Triticum aestivum L.), one sorghum (Sorghum bicolor L.), and one soybean (Glycine Max).

Materials and methods

Genome datasets

The phenotype and genotype data analyzed in this study are described as follows.

Tropical rice dataset

This dataset, presented by Spindel et al. (2015), contained 73,147 single nucleotide polymorphism (SNP) markers and 363 elite breeding lines belonging to an indica or indica-admixed group. Phenotypic observations were conducted eight times in 2009–2012, once in the dry season and once in the wet season each year, on grain yield (YLD), flowering time (FT), and plant height (PH), although PH data were not available for the wet season of 2009. Phenotypic values for 35 of the 363 individuals were missing. The adjusted least squares mean (ls_means) presented in Spindel et al. (2015) of the 328 individuals was used in this example. One SNP marker was randomly chosen per 0.1-cM interval over each chromosome. This resulted in 10,772 of the 73,147 SNP markers used in this example. The SNP at each locus was coded as − 1, 0, or 1 for the homozygote of the minor allele, the heterozygote, and the homozygote of the major allele, respectively. After SNP coding, any missing locus in an individual was imputed by 1 (the coding for the homozygote of the major allele).

Wheat dataset

This dataset, presented by Kristensen et al. (2019), contained 13,006 SNP markers and 635 F₆ winter wheat lines from two breeding cycles. The first breeding cycle had 321 individuals harvested in 2014; while, the second had 314 in 2015. The ls_means were obtained on the four alveograph quality traits: flour yield (FYD), dough tenacity (DT), dough extensibility, and dough strength (DS). Only 313 wheat lines from the second breeding cycle, with no missing data on the four quality traits, were used in this study. The SNPs were filtered at a missing rate > 0.1 and a minor allele frequency (MAF) < 0.05, leaving 11,214 retained for further study. SNP coding was performed as described above for the tropical rice dataset.

Sorghum dataset

This dataset was used to compare methods for training set optimization in genomic selection by Fernández-González et al. (2023), originally presented in Fernandes et al. (2018). The filtered dataset contained 56,299 SNP markers and 451 diverse sorghum lines with the best linear unbiased prediction (BLUP) values of PH, moisture content (MC), and biomass yield (BYD). The BLUP values for each individual were calculated to account for variations resulting from spatial and year effects. This dataset featured a strong subpopulation structure, classifying individuals into four clusters.

Soybean dataset

This dataset, presented by Stewart-Brown et al. (2019), contained 2647 SNP markers and 483 recombinant inbred lines with the BLUP values of yield (YLD), protein content (PRC), and oil content (OC). The BLUP values for each genotype were calculated to account for environmental factors and maturity variations. Individuals were classified into four subpopulations and one admixed group, where the admixed group was composed of individuals in sets 9–11 and 12–14 (see Table 1 in Stewart-Brown et al. 2019). SNPs with a missing rate > 0.1 and a MAF < 0.05 were filtered out, leaving 2425 SNPs for 483 individuals retained for further analyses. SNP coding was the same as in the above tropical rice dataset.

Training set construction methods

We first introduced the training set optimization methods, and then proposed the two approaches, inspired by the optimal design criteria, for searching the top-ranked genotypes.

Training set optimization methods

The methods for training set optimization were derived mainly from two kinds of statistical models: whole genome regression (WGR) models and genomic BLUP (GBLUP) models.

Whole genome regression model-based methods

The additive effect WGR model used in this study can be described as follows:

$$\mathit{y}={\beta }_0{\mathbf{1}}_n+\mathit{X}\mathit{\beta }+\mathit{\epsilon },$$ 1

where $\mathit{y}$ is the vector of phenotypic values of length n,${\beta }_0$ is the constant term, $1_n$ is the vector of length n with all elements equal to 1, $\mathit{X}$ is a marker-associated matrix of order $n\times p$, $\mathit{\beta }$ is the vector of marker-associated effects of length p, and $\mathit{\epsilon }$ is the vector of random errors. Here, n is the number of individuals, and p is the number of marker-associated components. Furthermore, let ${\mathit{X}}_c$ and ${\mathit{X}}_t$ denote the marker-associated matrices for the candidate population and the training set, respectively. ${\mathit{X}}_c$ and ${\mathit{X}}_t$ are of order $n_c\times p$ and $n_t\times p$, where $n_c$ and $n_t$ are the respective numbers of individuals in the two sets. Two methods for predicting the entire candidate population were evaluated, both of them were based on ridge regression.

r-score

From Ou and Liao (2019), the r-score, derived from Pearson’s correlation between phenotypic values and GEBVs of the whole candidate population, is given by:

$$r\text{-}score=\frac{q_{12}}{\sqrt{q_1{q}_2}},$$ 2

where

$$q_{12}=Tr\left[{\mathit{X}}_c^{\top },\left({\mathit{I}}_{n_c},-,{\overline{\mathit{J}}}_{n_c}\right),{\mathit{X}}_c,\mathit{A},{\mathit{X}}_t\right],$$

$$q_1=\left(n_c,-,1\right)+Tr\left[{\mathit{X}}_c^{\top },\left({\mathit{I}}_{n_c},-,{\overline{\mathit{J}}}_{n_c}\right),{\mathit{X}}_c\right],$$

$$q_2=Tr\left[{\mathit{A}}^{\top },{\mathit{X}}_c^{\top },\left({\mathit{I}}_{n_c},-,{\overline{\mathit{J}}}_{n_c}\right),{\mathit{X}}_c,\mathit{A}\right]+Tr\left[{{\mathit{X}}_t^{\top }\mathit{A}}^{\top },{\mathit{X}}_c^{\top },\left({\mathit{I}}_{n_c},-,{\overline{\mathit{J}}}_{n_c}\right),{\mathit{X}}_c,\mathit{A},{\mathit{X}}_t\right]$$

Here, $Tr[·]$ denotes the trace of a square matrix, $\mathit{A}={\mathit{X}}_t^{\top }{\left({\mathit{X}}_t,{\mathit{X}}_t^{\top },+,\lambda ,{\mathit{I}}_{n_t}\right)}^{-1}$, ${\overline{\mathit{J}}}_{n_c}$ is the square matrix with all elements equal to $\frac{1}{n_c}$ and $\lambda$ is a shrinkage parameter in the ridge regression.

Mean of squared prediction errors

Let ${\mathit{y}}_c$ be the vector of phenotypic values in the candidate population, which is predicted by ${\widehat{\mathit{y}}}_c={\mathit{X}}_c\mathit{A}{\mathit{y}}_t$ with ${\mathit{y}}_t$ being the vector of phenotypic values in the training set. Then, the mean of squared prediction errors (MSPE) between ${\mathit{y}}_c$ and ${\widehat{\mathit{y}}}_c$ is given as follows:

$$\begin{array}{cc}\hfill \text{MSPE}& =\frac{1}{n_c}{\left({\mathit{y}}_c,-,{\widehat{\mathit{y}}}_c\right)}^T\left({\mathit{y}}_c,-,{\widehat{\mathit{y}}}_c\right),\hfill \\ \hfill & =\frac{1}{n_c}{\left({\mathit{y}}_c,-,{\mathit{X}}_c,\mathit{A},{\mathit{y}}_t\right)}^T\left({\mathit{y}}_c,-,{\mathit{X}}_c,\mathit{A},{\mathit{y}}_t\right).\hfill \end{array}$$ 3

An approximation proportional to the expectation of MSPE is given as:

$${\text{MSPE}}^{\text{Ridge}}=1+\frac{1}{n_c}Tr\left[{\mathit{X}}_c,\mathit{A},{\mathit{A}}^T,{\mathit{X}}_c^T\right]+Tr[{\left({\mathit{X}}_c,-,{\mathit{X}}_c,\mathit{A},{\mathit{X}}_t\right)}^T\left({\mathit{X}}_c,-,{\mathit{X}}_c,\mathit{A},{\mathit{X}}_t\right)].$$ 4

More details regarding the derivation of ${\text{MSPE}}^{\text{Ridge}}$ are presented in Appendix A.

Let ${\mathit{X}}_0$ be the original marker score matrix in the candidate population, and $\mathit{W}$ be the standardized marker score matrix. That is, $w_{\mathit{ij}}=\frac{x_{\mathit{ij}}-{\overline{x}}_j}{s_j}$, where $w_{\mathit{ij}}$ and $x_{\mathit{ij}}$ are the $\left(i,j\right)th$ elements of $\mathit{W}$ and ${\mathit{X}}_0$, and ${\overline{x}}_j$ and $s_j$ are the sample mean and the sample standard deviation for column j in ${\mathit{X}}_0$ for $i=1,2,\cdots ,n_c$ and $j=1,2,\cdots ,p_c$, where $p_c$ is the number of markers. Under the assumption that $n_c<p_c$, the spectral decomposition was performed on ${\mathit{W}}^T\mathit{W}$, producing ${\mathit{W}}^T\mathit{W}={\sum }_{i=1}^c{u}_i{\mathit{q}}_i{\mathit{q}}_i^T$, where $u_i$ is a nonzero eigenvalue in decreasing order $u_1\ge u_2\ge \cdots \ge u_{n_c}>0$, and ${\mathit{q}}_i$ is the eigenvector of length $p_c$. The principal component (PC) score matrix was then obtained as $\mathit{R}=\mathit{W}\mathit{Q}$ with$\mathit{Q}=[{\mathit{q}}_1⋮{\mathit{q}}_2⋮\cdots ⋮{\mathit{q}}_{n_c}]$. The first PCs that could explain about 99% of the total variation were used for the marker-associated matrices. The shrinkage parameter $\lambda$ was fixed at 1 in the two WGR model-based methods.

Genomic best linear unbiased prediction model-based methods

An additive effect GBLUP model used in this study is given by:

$$\mathit{y}=\mathit{X}\mathit{\beta }+\mathit{Z}\mathit{g}+\mathit{e},$$ 5

where $\mathit{X}$ is the design matrix for the fixed effects $\mathit{\beta },$ and $\mathit{Z}$ is the incidence matrix for the vector of random genotypic values $\mathit{g}$. It is assumed that $\mathit{g}$ and $\mathit{e}$ are mutually independent and follow multivariate normal distributions, denoted by $\mathit{g}\sim MVN\left(0,,,{\sigma }_A^2,\mathit{K}\right),$ and $\mathit{e}\sim MVN\left(0,,,{{\sigma }_e^2\mathit{I}}_n\right)$. Here, $0$ is a zero vector, ${\sigma }_A^2$ is the additive genetic variance, $\mathit{K}$ stands for a genomic relationship matrix and ${\sigma }_e^2$ is the random error variance. The genomic relationship matrix was calculated by $\mathit{K}=\frac{1}{p_c}(\mathit{W}{\mathit{W}}^T)$. Recall that $\mathit{W}$ is the standardized marker score matrix and $p_c$ is the number of markers. The variance–covariance matrix for $\mathit{g}-\widehat{\mathit{g}},$ usually called the prediction error variance (PEV) matrix in the context of GS, is given by:

$$\text{PEV}=\text{Var}\left(\mathit{g},-,\widehat{\mathit{g}}\right)={\left({\mathit{Z}}^T,\mathit{M},\mathit{Z},+,\lambda ,{\mathit{K}}^{-1}\right)}^{-1}{\mathrm{\sigma }}_e^2,$$ 6

where $\lambda =\frac{{\sigma }_e^2}{{\sigma }_A^2}$ and $\mathit{M}={\mathit{I}}_n-\mathit{X}{({\mathit{X}}^T\mathit{X})}^{-}{\mathit{X}}^T$ is a projection matrix orthogonal to the vector space spanned by the columns of $\mathit{X}$. Here, ${({\mathit{X}}^T\mathit{X})}^{-}$ is a generalized inverse of ${\mathit{X}}^T\mathit{X}$. Furthermore, the variance–covariance matrix for $\widehat{\mathit{g}}$ is equal to the covariance matrix between $\mathit{g}$ and $\widehat{\mathit{g}}$, which is given as:

$$\text{Var}\left(\widehat{\mathit{g}}\right)=\text{Cov}\left(\mathit{g},,,\widehat{\mathit{g}}\right)=\left(\mathit{K},-,\lambda ,{\left({\mathit{Z}}^T,\mathit{M},\mathit{Z},+,\lambda ,{\mathit{K}}^{-1}\right)}^{-1}\right){\sigma }_A^2.$$ 7

The CD with a coefficient vector ${\mathit{c}}_i$ is then obtained as:

$$\begin{array}{cc}\hfill \text{CD}\left({\mathit{c}}_i\right)& =\frac{{\mathit{c}}_i^T\left[\text{Var},\left(\mathit{g}\right),-,\text{PEV}\right]{\mathit{c}}_i}{{\mathit{c}}_i^T\left[\text{Var},\left(\mathit{g}\right)\right]{\mathit{c}}_i}\hfill \\ \hfill & =\frac{{({\mathit{c}}_i^T\left[,\text{Cov},\left(\mathit{g},,,\widehat{\mathit{g}}\right)\right]{\mathit{c}}_i)}^2}{{(\mathit{c}}_i^T[\text{Var}\left(\mathit{g}\right)]{\mathit{c}}_i)\times {(\mathit{c}}_i^T[\text{Var}\left(\widehat{\mathit{g}}\right)]{\mathit{c}}_i)}\hfill \\ \hfill & =\frac{{\mathit{c}}_i^T[\mathit{K}-\lambda {\left({\mathit{Z}}^T,\mathit{M},\mathit{Z},+,\lambda ,{\mathit{K}}^{-1}\right)}^{-1}]{\mathit{c}}_i}{{\mathit{c}}_i^T\mathit{K}{\mathit{c}}_i}.\hfill \end{array}$$ 8

Clearly, the CD in Eq. (8) is the squared correlation between ${\mathit{c}}_i^T\mathit{g}$ and ${\mathit{c}}_i^T\widehat{\mathit{g}}$.

A ranking approach

Ou and Liao (2019) specified the coefficient vectors in Eq. (8) as

$${\mathit{c}}_i={\mathit{e}}_i-\frac{1}{n_c}{1}_{n_c},$$

where ${\mathit{e}}_i$ is the vector of length $n_c$ with all elements equal to 0, except the ith element, which is equal to 1, for $i=1,2,\cdots ,n_c$. Furthermore, considering the situation where the model is trained by the entire candidate population, $\mathit{Z}={\mathit{I}}_{n_c}$. In the current study, the vector of fixed effects $\mathit{\beta }$ consists only of the general mean $\mu$, hence $\mathit{M}={\mathit{I}}_{n_c}-{\overline{\mathit{J}}}_{n_c}$. Genotypes are then sorted according to the individually calculated $CD\left({\mathit{c}}_i\right)$, and the top-ranked candidates are selected to form the required training set. This ranking approach is called as ${\text{CD}}_{\text{ranking}}$, which was also described in Atanda et al. (2021b).

A heuristic-based approach

Based on the GBLUP model in Eq. (5) with $\mathit{\beta }$ consisting only of the general mean $\mu$, the BLUP for the true genotypic values of a training set, denoted by ${\widehat{\mathit{g}}}_t$, can be written as:

$${\widehat{\mathit{g}}}_t={({\mathit{M}}_t+\lambda {\mathit{K}}_t^{-1})}^{-1}{\mathit{M}}_t{\mathit{y}}_t$$ 9

where ${\mathit{M}}_t={\mathit{I}}_{n_t}-{\overline{\mathit{J}}}_{n_t}$, ${\mathit{K}}_t$ is the genomic relationship matrix for the training set and ${\mathit{y}}_t$ denotes the phenotypic values in the training set. From Henderson (1977), the BLUP for the true genotypic values in the candidate population, denoted by ${\widehat{\mathit{g}}}_c$, is obtained as:

$${\widehat{\mathit{g}}}_c={\mathit{K}}_{\mathit{ct}}{({\mathit{M}}_t{\mathit{K}}_t+\lambda {\mathit{I}}_{n_t})}^{-1}{\mathit{M}}_t{\mathit{y}}_t$$ 10

where ${\mathit{K}}_{\mathit{ct}}$ with order $n_c\times n_t$ is the genomic relationship matrix between the candidate population and the training set. It can be verified that the variance–covariance matrix for ${\widehat{\mathit{g}}}_c$ is the same as the covariance matrix between ${\mathit{g}}_c$ and ${\widehat{\mathit{g}}}_c$, which is given as:

$$\text{Var}\left({\widehat{\mathit{g}}}_c\right)=\text{Cov}\left({\mathit{g}}_c,,,{\widehat{\mathit{g}}}_c\right)={(\mathit{K}}_{\mathit{ct}}{({\mathit{M}}_t{\mathit{K}}_t+\lambda {\mathit{I}}_{n_t})}^{-1}{\mathit{M}}_t{\mathit{K}}_{\mathit{ct}}^T{)\mathrm{\sigma }}_A^2.$$ 11

Let $A_i$ denote the $i\text{th}$ diagonal element of ${\mathit{K}}_{\mathit{ct}}{({\mathit{M}}_t{\mathit{K}}_t+\lambda {\mathit{I}}_{n_t})}^{-1}{\mathit{M}}_t{\mathit{K}}_{\mathit{ct}}^T$ in Eq. (11), and $B_i$ denote the corresponding element in ${\mathit{K}}_c$, for $i=1,2,\cdots ,n_c$. A heuristic-based version of CD is given by:

$${\text{CD}}_{\text{mean}(\text{v}2)}=\sum _{i=1}^{n_c}\frac{{[\text{cov}\left(g_{\mathit{ci}},,,{\widehat{g}}_{\mathit{ci}}\right)]}^2}{\text{var}\left(g_{\mathit{ci}}\right)\times \text{var}({\widehat{g}}_{\mathit{ci}})}=\sum _{i=1}^{n_c}\left(\frac{{[A_i]}^2}{B_i\times A_i}\right)=\sum _{i=1}^{n_c}\left(\frac{A_i}{B_i}\right).$$ 12

Maximizing ${\text{CD}}_{\text{mean}(\text{v}2)}$ is equivalent to maximizing the sum (or mean) of squared correlations between the true and estimated genotypic values in the candidate population. More details regarding the derivation of ${\text{CD}}_{\text{mean}(\text{v}2)}$ are presented in Appendix B.

The shrinkage parameter $\lambda$ was fixed at 1 when searching the optimal training sets for both ${\text{CD}}_{\text{ranking}}$ and ${\text{CD}}_{\text{mean}(\text{v}2)}$. All the above optimization methods based on the WGR and GBLUP models are untargeted as they do not consider the genomic information of a test set.

Two approaches inspired by classical optimal design criteria

The probability of correctly identifying the best genotypes from a candidate population can be enhanced, if the training set can explain the variation of genotypic values as much as possible (Tanaka and Iwata 2018). The genomic relationship matrix in the GBLUP model, described as Eq. (5), serves as a metric for the genomic variation among genotypes, i.e., $\mathit{g}\sim \text{MVN}\left(0,,,{\sigma }_A^2,\mathit{K}\right)$. Two approaches inspired by A- and D-optimality were employed on the genomic relationship matrix to construct training sets with maximal genomic variation. Unlike in the field of classical optimal designs, these criteria do not involve an information matrix for estimating parameters. For a fixed training set size $n_t$, the A- and D-optimality-like criteria were utilized to construct training sets that maximize the trace and the determinant over all possible variance–covariance matrices corresponding to subsets with $n_t$ genotypes chosen from the candidate population. Thus, they were used here to measure the average (${\text{GV}}_{\text{average}}$) and the overall (${\text{GV}}_{\text{overall}}$) genomic variation of the training sets, respectively. They can be described as:

$${\text{GV}}_{\text{average}}=\text{arg}_{d_{}^{\ast }\in S_d}^{}\text{max}\left(T,r,\left[{\mathit{K}}_d\right]\right),$$ 13

and

$${\text{GV}}_{\text{overall}}=\text{arg}_{d_{}^{\ast }\in S_d}^{}\text{max}\left(\text{det},\left[{\mathit{K}}_d\right]\right),$$ 14

where $S_d$ denotes the set composed of all possible subsets with $n_t$ candidates, ${\mathit{K}}_d$ is the submatrix of $\mathit{K}$ corresponding to $d$, $Tr\left[{\mathit{K}}_d\right]$ is the trace of ${\mathit{K}}_d$, and $det\left[{\mathit{K}}_d\right]$ is the determinant of ${\mathit{K}}_d$.

A genetic algorithm (GA) implemented in the R package TSDFGS (Ou 2022) was utilized to optimize training sets for individually maximizing the r-score, ${\text{CD}}_{\text{mean}(\text{v}2)}$, and ${\text{GV}}_{\text{overall}},$ or minimizing ${\text{MSPE}}^{\text{Ridge}}$. It is worth noting that both the ${\text{CD}}_{\text{ranking}}$ and ${\text{GV}}_{\text{average}}$ ranks all candidates based on their individually calculated metrics and select the top-ranked ones, eliminating the need for computationally intensive algorithms.

Evaluation metrics

Three indices were applied to measure the ability of a training set to determine the best k genotypes with the most favorable TBVs from a candidate population.

NDCG

Let $v_{\left(1\right)}\ge v_{\left(2\right)}\ge \cdots \ge v_{\left(n_c\right)}$ be the TBVs of all candidates arranged in descending order. Moreover, let ${\widehat{v}}_{\left(1\right)}$, ${\widehat{v}}_{\left(2\right)}$, …, ${\widehat{v}}_{\left(n_c\right)}$ be their corresponding GEBVs based on the training set. By reordering these GEBVs, it follows that ${\widehat{v}}_{\left({\pi }_1\right)}\ge {\widehat{v}}_{\left({\pi }_2\right)}\cdots \ge {\widehat{v}}_{\left({\pi }_{n_c}\right)},$ where $\pi =\left({\pi }_1,,,{\pi }_2,,,\cdots ,,,{\pi }_{n_c}\right)$ is a permutation of ${\pi }_0=\left(1,,,2,,,\cdots ,,,n_c\right).$ Note that ${\pi }_1$, ${\pi }_2$, …, ${\pi }_{n_c}$ here denote the ranking of ${\widehat{v}}_{\left(1\right)}$, ${\widehat{v}}_{\left(2\right)}$, …, ${\widehat{v}}_{\left(n_c\right)}$. From Blondel et al. (2015), the discounted cumulative gain (DCG) score at position k of the predicted ranking using the training set was defined as follows:

$$\text{DCG}@k\left(\mathit{v},,,\pi ,\left(\widehat{\mathit{v}}\right)\right)=\sum _{i=1}^{k}f\left(v_{\left({\pi }_i\right)}\right)d\left(i\right),$$ 15

the corresponding DCG score at position k of the ideal ranking was also given as:

$$\text{DCG}@k\left(\mathit{v},,,{\pi }_0,\left(\mathit{v}\right)\right)=\sum _{i=1}^{k}f\left(v_{\left(i\right)}\right)d\left(i\right),$$ 16

where $f(v)$ is a monotonically increasing gain function, and $d\left(i\right)$ is a monotonically decreasing discount function. The linear gain function of $f\left(v\right)=v$, and the discount function of $d\left(i\right)=\frac{1}{{\text{log}}_2\left(i,+,1\right)}$ are used in Eq. (15) and (16). Finally, the normalized DCG score at position k was defined as follows:

$$\text{NDCG}@k\left(\mathit{v},,,\widehat{\mathit{v}}\right)=\frac{DCG@k\left(\mathit{v},,,\pi ,(,\widehat{\mathit{v}},)\right)}{DCG@k\left(\mathit{v},,,{\pi }_0,(\mathit{v})\right)}.$$ 17

Spearman’s rank correlation

SRC is a nonparametric rank correlation measure used to assess the monotonic relationship between TBVs and GEBVs among the best k genotypes with the highest TBVs. The SRC can be defined as follows:

$$SRC@k\left({\pi }_0,\left(\mathit{v}\right),,,\pi ,\left(\widehat{\mathit{v}}\right)\right)=\frac{\text{cov}@k({\pi }_0\left(\mathit{v}\right),\pi \left(\widehat{\mathit{v}}\right))}{\sqrt{\text{var}@k({\pi }_0\left(\mathit{v}\right))\times \text{var}@k(\pi \left(\widehat{\mathit{v}}\right))}},$$ 18

where $\text{cov}@k({\pi }_0\left(\mathit{v}\right),\pi \left(\widehat{\mathit{v}}\right))$ is the covariance between the rankings of TBVs and GEBVs among the best k genotypes, and $\text{var}@k({\pi }_0\left(\mathit{v}\right))$ and $\text{var}@k(\pi \left(\widehat{\mathit{v}}\right))$ are the corresponding variances.

Pearson’s correlation

Pearson’s correlation was also used to measure the linear relationship between TBVs and GEBVs among the best k genotypes with the highest TBVs, which can be described as follows:

$$r@k\left(\mathit{v},,,\widehat{\mathit{v}}\right)=\frac{\text{cov}@k\left(\mathit{v},,,\widehat{\mathit{v}}\right)}{\sqrt{\text{var}@k\left(\mathit{v}\right)\times \text{var}@k\left(\widehat{\mathit{v}}\right)}},$$ 19

where $cov@k(\mathit{v},\widehat{\mathit{v}})$ is the covariance between TBVs and GEBVs among the best k genotypes, and $\text{var}@k(\mathit{v})$ and $\text{var}@k(\widehat{\mathit{v}})$ are the corresponding variances.

Simulation studies

The above proposed methods were employed for a given dataset to construct training sets of 50, 75, 100, 150, and 200 individuals. For the two datasets (sorghum and soybean) with a strong subpopulation structure, the number of genotypes selected from each cluster was proportional to the number of genotypes of the cluster in the candidate population.

Based on the GBLUP model in Eq. (5), phenotype data were simulated to evaluate the performance of the aforementioned methods. For a given genome dataset, the genomic relationship matrix $\mathit{K}$ was first calculated from its genotype data. The model parameters were fixed at $\mu =10,{\sigma }_A^2=25,$ and ${\sigma }_e^2=\frac{{\sigma }_A^2\left(1,-,h^2\right)}{h^2}$ with $h^2=0.2,$ 0.5, and 0.8 for genomic heritability. Accordingly, 1000 datasets were generated for each setting of parameters, i.e., $\mathit{g}\sim \text{MVN}\left(0,,,{\sigma }_A^2,\mathit{K}\right)$ and $\mathit{e}\sim \text{MVN}\left(0,,,{{\sigma }_e^2\mathit{I}}_n\right)$. Simulated TBVs were obtained as the constant $\mu$ plus the simulated $\mathit{g}$, and simulated phenotypic values in $\mathit{y}$ were obtained as the simulated TBVs plus the simulated $\mathit{e}$. To mitigate the potential vulnerability to stochastic processes in the heuristic algorithm of TSDFGS during training set construction, we evaluated 10 different optimal training sets using the r-score, ${\text{MSPE}}^{\text{Ridge}}$, ${\text{CD}}_{\text{mean}(\text{v}2)}$, and ${\text{GV}}_{\text{overall}}$ methods for each fixed size across all simulation scenarios. It is worth noting that both the ${\text{CD}}_{\text{ranking}}$ and ${\text{GV}}_{\text{average}}$ generated consistent simulation results as they do not suffer from vulnerability to stochastic processes. The mean performances (calculated from 10 training sets) for the three metrics of $\text{NDCG}@k,$ $\text{SRC}@k,$ and $r@k$ with $k=20$ over the 1000 simulated datasets were compared among the training sets generated from each method. Additionally, random sampling was performed as the baseline method. The Bayesian reproducing kernel Hilbert space method in the R package BGLR (Pérez and de los Campos 2014) was used to perform GEBV prediction.

Real trait analyses

The following approach was proposed to compare the methods based on real trait values.

Step 1: For a given dataset, 250 individuals were randomly selected as the candidate population.

Step 2: For the chosen candidate population, the required training sets with sizes equal to 50, 100, 150, and 200 were generated according to each method.

Step 3: For each trait in the dataset, phenotypic values were treated as TBVs, and $NDCG@k,$ $SRC@k,$ $r@k$ with $k=20$ were then calculated based on each training set generated from Step 2.

Step 4: Steps 1 to 3 were repeated 50 times to obtain the $NDCG@k,$ $SRC@k,$ and $r@k$ for all of the training sets, and the averages of the three sets consisting of the 50 resulting values were reported to make a comparison among the methods.

Results

Simulation studies

The averages and standard deviations of the mean performances for the three metrics over the 1000 simulation runs for the tropical rice dataset are displayed in Figs. 1, 2, 3. The corresponding results for the remaining three datasets are shown in Figures S1–S3, S4–S6, and S7–S9 of the Supplementary Materials.

Fig. 1.

Averages and standard deviations of NDCG values among the best 20 genotypes over the 1000 simulated datasets using the proposed methods in the tropical rice dataset

Fig. 2.

Averages and standard deviations of SRC values among the best 20 genotypes over the 1000 simulated datasets using the proposed methods in the tropical rice dataset

Fig. 3.

Averages and standard deviations of Pearson’s correlation values among the best 20 genotypes over the 1000 simulated datasets using the proposed methods in the tropical rice dataset

The summarized results are as follows. (i) All of the proposed methods for constructing training sets outperformed the baseline method of random sampling across various datasets and evaluation metrics. However, the ${\text{CD}}_{\text{ranking}}$ exhibited inferior performance, even worse than the random sampling method, in the two datasets (sorghum and soybean) characterized by a strong subpopulation structure; (ii) For a combination of evaluation metrics and $h^2,$ the performance of any construction method increased as the training set size became larger; (iii) For a combination of evaluation metrics and training set sizes, any construction method improved its performance with increasing $h^2$; (iv) For a particular dataset, the results for $SRC@k=20$ and those for $r@k=20$ appeared to have very similar patterns, e.g., see Figs. 2 and 3 for the tropical rice dataset; (v) For a specific dataset, the discrepancies among the construction methods in $NDCG@k=20$ were relatively small compared to those in $SRC@k=20$ and $r@k=20,$, e.g., see Figs. 1, 2, 3 for the tropical rice dataset; (vi) From Figs. 1, 2, 3 and S1–S3, the r-score, ${\text{MSPE}}^{\text{Ridge}},$ and ${\text{GV}}_{\text{average}}$ methods had slightly better performance in most cases for the two datasets (tropical rice and wheat) lacking a strong subpopulation structure; (vii) For the two datasets with a strong subpopulation structure (Figures S4–S6 for sorghum and S7–S9 for soybean), ${\text{CD}}_{\text{mean}(\text{v}2)},$ and ${\text{GV}}_{\text{overall}}$ methods were found to slightly outperform the others in most cases.

Real trait analyses

The averages of NDCG, SRC, and Pearson’s correlation values among the top 20 genotypes over the 50 sampled candidate populations for the real trait analysis are displayed in Fig. 4 for the tropical rice dataset, and Figures S10–S12 in the Supplementary Materials for the remaining three datasets.

Fig. 4.

Averages of NDCG, SRC, and Pearson’s correlation values among the top 20 genotypes over the 50 re-sampled candidate populations using the proposed methods for the real traits in the tropical rice dataset. YLD: grain yield; FT: flowering time; PH: plant height

There were some results summarized as follows. (i) Construction methods exhibited varying performances across different trait–dataset combinations; (ii) For most trait–dataset combinations, the performance patterns among the proposed methods in the evaluation indices of SRC and Pearson’s correlation were consistent; (iii) No dominant method was found for all the traits in a dataset. Overall, the r-score and ${\text{CD}}_{\text{mean}(\text{v}2)}$ showed relatively satisfactory performance in the tropical rice dataset (Fig. 4); while, the ${\text{MSPE}}^{\text{Ridge}}$ and ${\text{CD}}_{\text{ranking}}$ can be recommended for the wheat dataset (Figure S10). In the sorghum dataset (Figure S11), the ${\text{GV}}_{\text{overall}},$ ${\text{CD}}_{\text{ranking}}$, and ${\text{GV}}_{\text{average}}$ had better performance at PH, MC, and BYD, respectively. In the soybean dataset (Figure S12), the r-score or ${\text{MSPE}}^{\text{Ridge}}$ demonstrated relatively superior performance at YLD, while ${\text{GV}}_{\text{average}}$ outperformed the others at PRC and OC; (iv) The baseline method of random sampling sometimes outperformed a few construction methods in certain trait–dataset combinations. For instance, the random sampling method showed better performance than ${\text{MSPE}}^{\text{Ridge}}$, ${\text{CD}}_{\text{ranking}}$, and ${\text{GV}}_{\text{average}}$ at FT in the tropical rice dataset (Fig. 4).

Discussion

The two WGR model-based methods of r-score and ${\text{MSPE}}^{\text{Ridge}}$ perform equally well in most of the simulation scenarios. Theoretically, both of them consider the variance and bias of errors in GEBV prediction, but ${\text{MSPE}}^{\text{Ridge}}$ has a simpler form compared to r-score. Specifically, the derivation of ${\text{MSPE}}^{\text{Ridge}}$ involves the approximation of a single quadratic form ${\mathit{\beta }}^{\top }\mathit{L}\mathit{\beta }$, as described in Appendix A, while that of r-score requires the approximation of three different quadratic forms (Ou and Liao 2019). Therefore, ${\text{MSPE}}^{\text{Ridge}}$ can serve as an alternative to r-score.

In regard to the two GBLUP model-based methods of ${\text{CD}}_{\text{ranking}}$ and ${\text{CD}}_{\text{mean}(\text{v}2)}$, neither involves any approximation in their derivations, unlike the r-score and ${\text{MSPE}}^{\text{Ridge}}$. Theoretically, an optimal training set constructed based on ${\text{CD}}_{\text{mean}(\text{v}2)}$ maximizes the mean squared correlations between the true and predicted genotypic values in the candidate population. Fernández-González et al. (2023) showed that the originally heuristic-based CD method, known as CD_mean, excelled in GEBV prediction, albeit with high computational intensity. A detailed illustration exemplifying CD_mean can be found in Alemu et al. (2024). Most interestingly, CD_mean implementing the covariance matrix in Eq. (7), aims to maximize the same mean squared correlations as the proposed ${\text{CD}}_{\text{mean}(\text{v}2)}$, implementing the covariance matrix in Eq. (11). A proof for the equivalence between ${\text{CD}}_{\text{mean}(\text{v}2)}$ and CD_mean is presented in Appendix C. According to Eq. (7), CD_mean involves two times matrix inversions of $n_c$ dimensions. Therefore, it usually requires extremely intensive computational time due to the cubic time complexity of matrix multiplication and inversion (Alemu et al. 2024). However, according to Eq. (11), ${\text{CD}}_{\text{mean}(\text{v}2)}$ involves only one time matrix inversion of $n_t$ dimensions. Since $n_t$ (the training set size) is usually much smaller than $n_c$ (the candidate population size), ${\text{CD}}_{\text{mean}(\text{v}2)}$ can be an alternative to CD_mean due to its significant advantage of saving computational cost. Note that ${\text{CD}}_{\text{mean}(\text{v}2)}$ can also use the same contrast vectors of ${\mathit{c}}_i$ as CD_ranking; however, this cannot have a large impact on training set optimization (Alemu et al. 2024). Additionally, ${\text{CD}}_{\text{mean}(\text{v}2)}$ can be easily modified to be the targeted method by replacing the part of candidate population in ${\mathit{K}}_{\mathit{ct}}$ and ${\mathit{K}}_c$ with a target test set.

The CD_ranking method employs a ranking approach to select the $n_t$ genotypes with the highest $CD\left({\mathit{c}}_i\right)$ values, as described in Eq. (8), when trained on the entire candidate population. However, this method tends to favor individuals that can be better predicted rather than necessarily the best ones for the training set. Consequently, individuals with high reliability often share similarities, resulting in a lack of diversity in the training set. In an extreme scenario, if the best genotype is duplicated within the candidate population, CD_ranking will include both copies in the training set. This highlights a potential drawback, as the ranking approach may select highly reductant and inefficient candidates (Alemu et al. 2024). This observation could partially account for the notably poor performance of the CD_ranking method in the simulation study involving the sorghum and soybean datasets.

Regarding the A- and D-optimality-like methods of ${\text{GV}}_{\text{average}}$ and ${\text{GV}}_{\text{overall}}$, although the former had competitive performance in the tropical rice, wheat and soybean datasets, it exhibited more significantly inferior performance in the SRC and Pearson’s correlation in the sorghum dataset. From a theoretical standpoint, ${\text{GV}}_{\text{overall}}$ appears to make more sense than ${\text{GV}}_{\text{average}},$ because it takes into account the genomic covariance among the candidates. ${\text{GV}}_{\text{average}}$ solely focuses on individuals with the highest diagonal elements in the genomic relationship matrix, thereby risking the selection of individuals with large covariances (strong relationship). Similar to the ranking approach of CD_ranking, ${\text{GV}}_{\text{average}}$ fails to maximize the training set diversity due to the absence of a heuristic. However, ${\text{GV}}_{\text{average}}$ outperformed CD_ranking in most simulation scenarios. Hence, in practical terms, ${\text{GV}}_{\text{average}}$ can be recommended over CD_ranking for identifying superior genotypes within candidate populations. Alternatively, ${\text{GV}}_{\text{overall}}$, by maximizing the determinant, aims to select individuals with small covariances (weak relationship). Interestingly, ${\text{GV}}_{\text{overall}}$ aligns with the concept of $\text{Avg}\_\text{GRM}\_\text{self}$ as mentioned in Fernández-González et al. (2023) , wherein $\text{Avg}\_\text{GRM}\_\text{self}$ was identified as the best untargeted method in GEBV prediction. Unlike ${\text{GV}}_{\text{overall}},$ which maximizes the determinant of genomic relationship matrix of the training set, $\text{Avg}\_\text{GRM}\_\text{self}$ aims to minimize the average of all elements in it. Evidently, $\text{Avg}\_\text{GRM}\_\text{self}$ may cost less computational time than ${\text{GV}}_{\text{overall}}$. A performance comparison between ${\text{GV}}_{\text{overall}}$ and $\text{Avg}\_\text{GRM}\_\text{self}$ in identifying the best genotypes would be beneficial.

The relative performance of the construction methods in the simulation results was not consistently observed in the real trait analysis. Several factors could account for this inconsistency. Firstly, the empirical phenotype data may not precisely align with the assumptions of the GBLUP model used to generate the simulated data. Secondly, the models used to derive the construction methods may not fully account for the sources of variability in the phenotype data. Additionally, the number of the re-sampled candidate populations in the real data analysis may be insufficient to corroborate the results demonstrated in the simulation study.

From the simulation results, the performance of any construction method generally improved with an increase in the training set size. This consistent behavior held in most cases during the real trait analysis, but there were still some exceptions, particularly occurring more frequently with the two heuristic methods of ${\text{GV}}_{\text{average}}$ and ${\text{GV}}_{\text{overall}}$. For instance, ${\text{GV}}_{\text{average}}$ showed discrepancies for $\text{SRC}@k=20$ and $r@k=20$ at YLD in the tropical rice dataset (Fig. 4), while ${\text{GV}}_{\text{overall}}$ exhibited discrepancies for $\text{SRC}@k=20$ and $r@k=20$ at PH in the sorghum dataset (Figure S11), as well as at YLD in the soybean dataset (Figure S12). The two methods inspired by the optimal design criteria, focusing solely on the genomic variation, may be more susceptible to negative effects from an increase in training set size during real trait analyses.

The sampling rule of selecting a proportion of candidates within every cluster was employed in the two datasets (sorghum and soybean) with a strong subpopulation structure. In a recent study by Fernández-González et al. (2023), it was found that the inclusion of the stratified sampling rule did not lead to the expected improvement in GEBV prediction. We reran the simulation study at $h^2=0.5$ without considering the sampling rule in the two datasets to explore how including subpopulation structure information impacts the ability to identify the best genotypes. The original results, combined with these simulation outcomes without considering the sampling rule, are displayed in Tables 1 and 2. In both the sorghum and soybean datasets, implementing the sampling rule did not yield a significant difference when employing the methods of r-score, ${\text{MSPE}}^{\text{Ridge}}$, ${\text{GV}}_{\text{overall}}$, and ${\text{CD}}_{\text{mean}(\text{v}2)}$. However, the two ranking approaches, ${\text{GV}}_{\text{average}}$ and ${\text{CD}}_{\text{ranking}}$, showed a noticeable advantage when the sampling rule was applied in the sorghum dataset (Table 1). Conversely, ${\text{CD}}_{\text{ranking}}$ exhibited a disadvantage when the sampling rule was applied in all cases, except for $NDCG@k=20$ at a sample size of 50, in the soybean dataset (Table 2). Scatter plots of the genotypes according to the first two PCs for the two datasets are displayed in Fig. 5. There were different patterns between the two datasets. Further study is needed to investigate how subpopulation structures affect the identification of the best genotypes. An R package called TSODBG is available from GitHub (https://github.com/spcspin/TSODBG) for conducting the construction methods discussed in this study.

Table 1.

Averages of 1000 resulting NDCG, SRC, and Pearson’s correlation values using the construction methods with and without employment of the sampling rule in the sorghum dataset in the simulation study at $h^2$ = 0.5

Evaluation metrics	Sample sizes	r-score	MSPERidge	CDranking	CDmean(v2)	GVaverage	GVoverall
NDCG@k = 20	50	0.558	0.559	0.514	0.561	0.529	0.495
		(0.561)	(0.563)	(0.445)	(0.557)	(0.437)	(0.495)
	75	0.602	0.602	0.560	0.602	0.580	0.552
		(0.604)	(0.605)	(0.526)	(0.599)	(0.534)	(0.549)
	100	0.635	0.634	0.603	0.634	0.623	0.594
		(0.635)	(0.639)	(0.584)	(0.624)	(0.599)	(0.591)
	150	0.678	0.680	0.656	0.677	0.676	0.649
		(0.678)	(0.681)	(0.644)	(0.659)	(0.667)	(0.641)
	200	0.710	0.710	0.698	0.704	0.708	0.685
		(0.709)	(0.711)	(0.683)	(0.688)	(0.700)	(0.679)
SRC@k = 20	50	0.102	0.101	0.057	0.097	0.092	0.103
		(0.100)	(0.100)	(0.029)	(0.095)	(0.059)	(0.102)
	75	0.124	0.124	0.072	0.112	0.106	0.129
		(0.12)	(0.123)	(0.035)	(0.118)	(0.081)	(0.128)
	100	0.147	0.147	0.087	0.141	0.129	0.147
		(0.138)	(0.144)	(0.038)	(0.134)	(0.109)	(0.146)
	150	0.177	0.176	0.108	0.183	0.158	0.172
		(0.172)	(0.177)	(0.073)	(0.168)	(0.136)	(0.178)
	200	0.205	0.205	0.150	0.211	0.189	0.205
		(0.203)	(0.205)	(0.106)	(0.202)	(0.161)	(0.209)
r@k = 20	50	0.110	0.109	0.055	0.107	0.103	0.123
		(0.108)	(0.11)	(0.013)	(0.102)	(0.074)	(0.13)
	75	0.136	0.135	0.077	0.125	0.126	0.153
		(0.133)	(0.137)	(0.025)	(0.128)	(0.095)	(0.161)
	100	0.167	0.167	0.091	0.160	0.152	0.176
		(0.154)	(0.159)	(0.032)	(0.149)	(0.126)	(0.176)
	150	0.202	0.202	0.116	0.210	0.180	0.205
		(0.193)	(0.200)	(0.071)	(0.193)	(0.155)	(0.212)
	200	0.230	0.231	0.161	0.242	0.211	0.237
		(0.227)	(0.231)	(0.107)	(0.229)	(0.179)	(0.242)

The results for the situation without the sampling rule are displayed in parentheses

Table 2.

Averages of 1000 resulting NDCG, SRC, and Pearson’s correlation values using the construction methods with and without employment of the sampling rule in the soybean dataset in the simulation study at $h^2$ = 0.5

Evaluation metrics	Sample sizes	r-score	MSPERidge	CDranking	CDmean(v2)	GVaverage	GVoverall
NDCG@k = 20	50	0.594	0.594	0.560	0.597	0.568	0.574
		(0.606)	(0.608)	(0.545)	(0.596)	(0.568)	(0.585)
	75	0.644	0.646	0.608	0.648	0.627	0.627
		(0.655)	(0.656)	(0.620)	(0.644)	(0.626)	(0.642)
	100	0.682	0.682	0.638	0.680	0.662	0.664
		(0.689)	(0.690)	(0.661)	(0.679)	(0.673)	(0.678)
	150	0.731	0.731	0.693	0.727	0.716	0.718
		(0.737)	(0.742)	(0.713)	(0.722)	(0.734)	(0.727)
	200	0.766	0.765	0.744	0.763	0.758	0.752
		(0.769)	(0.774)	(0.760)	(0.753)	(0.773)	(0.760)
SRC@k = 20	50	0.162	0.161	0.099	0.154	0.143	0.167
		(0.174)	(0.174)	(0.102)	(0.155)	(0.176)	(0.191)
	75	0.186	0.188	0.118	0.186	0.178	0.199
		(0.206)	(0.207)	(0.159)	(0.188)	(0.206)	(0.225)
	100	0.216	0.215	0.131	0.218	0.204	0.231
		(0.236)	(0.237)	(0.183)	(0.220)	(0.241)	(0.248)
	150	0.265	0.264	0.187	0.268	0.255	0.280
		(0.277)	(0.288)	(0.224)	(0.268)	(0.293)	(0.29)
	200	0.304	0.303	0.257	0.310	0.299	0.311
		(0.314)	(0.322)	(0.286)	(0.305)	(0.337)	(0.325)
r@k = 20	50	0.173	0.173	0.087	0.164	0.155	0.190
		(0.187)	(0.188)	(0.102)	(0.166)	(0.197)	(0.218)
	75	0.202	0.205	0.113	0.202	0.198	0.224
		(0.226)	(0.229)	(0.171)	(0.206)	(0.233)	(0.253)
	100	0.237	0.236	0.131	0.238	0.231	0.259
		(0.259)	(0.261)	(0.197)	(0.244)	(0.271)	(0.280)
	150	0.294	0.294	0.191	0.296	0.283	0.313
		(0.306)	(0.319)	(0.243)	(0.295)	(0.324)	(0.327)
	200	0.334	0.333	0.270	0.346	0.324	0.347
		(0.347)	(0.357)	(0.311)	(0.335)	(0.366)	(0.363)

The results for the situation without the sampling rule are displayed in parentheses

Fig. 5.

Scatter plots of the genotypes according to the first two principal components for the two datasets (sorghum and soybean) with a strong subpopulation structure.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Appendix A: The derivation of ${\mathbf{M}\mathbf{S}\mathbf{P}\mathbf{E}}^{\mathbf{R}\mathbf{i}\mathbf{d}\mathbf{g}\mathbf{e}}$

According to the identity (Searle et al. 1982, p.355),

$$E\left({\mathit{v}}_1^{\top },{\mathit{v}}_2\right)=Tr\left[C,o,v,\left({\mathit{v}}_1,,,{\mathit{v}}_2\right)\right]+E\left({\mathit{v}}_1^{\top }\right)E\left({\mathit{v}}_2\right)$$

for two random variables vectors ${\mathit{v}}_1$ and ${\mathit{v}}_2$ of same order, where $E(\bullet )$ denotes the expectation operator and $Cov\left({\mathit{v}}_1,,,{\mathit{v}}_2\right)$ denotes the covariance matrix between ${\mathit{v}}_1$ and ${\mathit{v}}_2$. The expectation of MSPE in Eq. (3) is therefore given as:

$$E\left(\text{MSPE}\right)=\frac{1}{n_c}\{Tr[{\mathit{I}}_{n_c}+{\mathit{X}}_c\mathit{A}{\mathit{A}}^T{\mathit{X}}_c^T]{\sigma }_e^2+{\mathit{\beta }}^T{\left({\mathit{X}}_c,-,{\mathit{X}}_c,\mathit{A},{\mathit{X}}_t\right)}^T\left({\mathit{X}}_c,-,{\mathit{X}}_c,\mathit{A},{\mathit{X}}_t\right)\mathit{\beta }\}.$$ 20

Rewrite ${\mathit{\beta }}^T{\left({\mathit{X}}_c,-,{\mathit{X}}_c,\mathit{A},{\mathit{X}}_t\right)}^T\left({\mathit{X}}_c,-,{\mathit{X}}_c,\mathit{A},{\mathit{X}}_t\right)\mathit{\beta }$ as:

$${\mathit{\beta }}^{\top }\mathit{L}\mathit{\beta }=\sum _{i=1}^{n_c}{l}_{\mathit{ii}}{\beta }_i^2+\sum _{i=1}^{n_c}\sum _{j\ne i}^{n_c}{l}_{\mathit{ij}}{\beta }_{\mathit{ij}}.$$ 21

The quadratic form ${\mathit{\beta }}^{\top }\mathit{L}\mathit{\beta }$ equals ${\sum }_{i=1}^c{l}_{\mathit{ii}}{\beta }_i^2$ when the off-diagonal elements of the matrix $\mathit{L}$ are assumed to be zero. Under this assumption, ${\mathit{\beta }}^{\top }\mathit{L}\mathit{\beta }$ is proportional to $\sum l_{\mathit{ii}}{\beta }_i^2$ since $l_{\mathit{ii}}\ge 0$ for all $i$. Hence, the second term on the right-hand side of Eq. (20) is proportional to $Tr[{\left({\mathit{X}}_c,-,{\mathit{X}}_c,\mathit{A},{\mathit{X}}_t\right)}^T\left({\mathit{X}}_c,-,{\mathit{X}}_c,\mathit{A},{\mathit{X}}_t\right)]$ if the off-diagonal elements are disregarded. Given that $E\left(\text{MSPE}\right)$ is also proportional to $Tr[{\mathit{I}}_{n_c}+{\mathit{X}}_c\mathit{A}{\mathit{A}}^T{\mathit{X}}_c^T]$ since ${\sigma }_e^2>0$, the derivation of ${\text{MSPE}}^{\text{Ridge}}$ in Eq. (4) is completed.

Appendix B: The derivation of ${\mathbf{C}\mathbf{D}}_{\mathbf{m}\mathbf{e}\mathbf{a}\mathbf{n}(\mathbf{v}2)}$

Consider the GBLUP model in Eq. (5) with $\mathit{\beta }$ consisting solely of $\mu$ for a training set, described as:

$${\mathit{y}}_t={\mu 1}_{n_t}+{\mathit{g}}_t+{\mathit{e}}_t.$$ 22

The projection matrix orthogonal to the vector space spanned by $1_{n_t}$ is given by:

${\mathit{M}}_t={\mathit{I}}_{n_t}-1_{n_t}{\left(1_{n_t}^T,1_{n_t}\right)}^{-}{1}_{n_t}^T={\mathit{I}}_{n_t}-{\overline{\mathit{J}}}_{n_t},$ satisfying that ${{\mathit{M}}_{t}1}_{n_t}=0$. Henderson’s mixed model equations (Henderson 1975) lead to $\left({\mathit{M}}_t,+,\lambda ,{\mathit{K}}_t^{-1}\right){\widehat{\mathit{g}}}_t={\mathit{M}}_t{\mathit{y}}_t.$ Hence,

$${\widehat{\mathit{g}}}_t={\left({\mathit{M}}_t,+,\lambda ,{\mathit{K}}_t^{-1}\right)}^{-1}{{\mathit{M}}_t\mathit{y}}_t.$$ 23

From Henderson (1977), we have that:

$$\begin{array}{cc}\hfill {\widehat{\mathit{g}}}_c& ={\mathit{K}}_{\mathit{ct}}{\mathit{K}}_t^{-1}{\widehat{\mathit{g}}}_t={\mathit{K}}_{\mathit{ct}}{\mathit{K}}_t^{-1}{\left({\mathit{M}}_t,+,\lambda ,{\mathit{K}}_t^{-1}\right)}^{-1}{\mathit{M}}_t{\mathit{y}}_t\hfill \\ \hfill & ={\mathit{K}}_{\mathit{ct}}{\left(\left({\mathit{M}}_t,+,\lambda ,{\mathit{K}}_t^{-1}\right),{\mathit{K}}_t\right)}^{-1}{\mathit{M}}_t{\mathit{y}}_t\hfill \\ \hfill & ={\mathit{K}}_{\mathit{ct}}{({\mathit{M}}_t{\mathit{K}}_t+\lambda {\mathit{I}}_{n_t})}^{-1}{\mathit{M}}_t{\mathit{y}}_t.\hfill \end{array}$$ 24

Then, the identity in Eq. (11) can be obtained as follows:

$$\begin{array}{cc}\hfill Var\left({\widehat{\mathit{g}}}_c\right)& ={[{\mathit{K}}_{\mathit{ct}}{\left({\mathit{M}}_t,{\mathit{K}}_t,+,\lambda ,{\mathit{I}}_{n_t}\right)}^{-1}{\mathit{M}}_t[Var(\mathit{y}}_t){\mathit{M}}_t]{({{\mathit{K}}_t\mathit{M}}_t+\lambda {\mathit{I}}_{n_t})}^{-1}{\mathit{K}}_{\mathit{ct}}^T]{\sigma }_A^2\hfill \\ \hfill & =[{\mathit{K}}_{\mathit{ct}}{\left({\mathit{M}}_t,{\mathit{K}}_t,+,\lambda ,{\mathit{I}}_{n_t}\right)}^{-1}{\mathit{M}}_t[({{\mathit{K}}_t\mathit{M}}_t+\lambda {\mathit{I}}_{n_t})-\lambda {\overline{\mathit{J}}}_{n_t}]{({{\mathit{K}}_t\mathit{M}}_t+\lambda {\mathit{I}}_{n_t})}^{-1}{\mathit{K}}_{\mathit{ct}}^T]{\sigma }_A^2\hfill \\ \hfill & =[{\mathit{K}}_{\mathit{ct}}{\left({\mathit{M}}_t,{\mathit{K}}_t,+,\lambda ,{\mathit{I}}_{n_t}\right)}^{-1}{\mathit{M}}_t{\mathit{K}}_{\mathit{ct}}^T]{\sigma }_A^2\hfill \end{array}$$ 25

since that ${\mathit{M}}_t{\overline{\mathit{J}}}_{n_t}=0$.

$$\begin{array}{cc}\hfill Cov\left({\mathit{g}}_c,,,{\widehat{\mathit{g}}}_c\right)& =Cov\left({\mathit{g}}_c,,,{{\mathit{K}}_{\mathit{ct}}{({\mathit{M}}_t{\mathit{K}}_t+\lambda {\mathit{I}}_{n_t})}^{-1}{\mathit{M}}_t\mathit{y}}_t\right)\hfill \\ \hfill & =Cov\left({\mathit{g}}_c,,,{{\mathit{K}}_{\mathit{ct}}{({\mathit{M}}_t{\mathit{K}}_t+\lambda {\mathit{I}}_{n_t})}^{-1}{\mathit{M}}_t\mathit{g}}_t\right)\hfill \\ \hfill & ={[\mathit{K}}_{\mathit{ct}}{\left({\mathit{M}}_t,{\mathit{K}}_t,+,\lambda ,{\mathit{I}}_{n_t}\right)}^{-1}{\mathit{M}}_t]Cov\left({\mathit{g}}_t,,,{\mathit{g}}_c\right)\hfill \\ \hfill & ={[\mathit{K}}_{\mathit{ct}}{({\mathit{M}}_t{\mathit{K}}_t+\lambda {\mathit{I}}_{n_t})}^{-1}{\mathit{M}}_t{\mathit{K}}_{\mathit{ct}}^T{]\sigma }_A^2.\hfill \end{array}$$ 26

Therefore, $Var\left({\widehat{\mathit{g}}}_c\right)$ in Eq. (25) equals $Cov\left({\mathit{g}}_c,,,{\widehat{\mathit{g}}}_c\right)$ in Eq. (26), and ${\text{CD}}_{\text{mean}(\text{v}2)}$ in Eq. (12) follows directly from the definition.

Appendix C: A proof for the equivalence between ${\mathbf{C}\mathbf{D}}_{\mathbf{m}\mathbf{e}\mathbf{a}\mathbf{n}(\mathbf{v}2)}$ and ${\mathbf{C}\mathbf{D}}_{\mathbf{m}\mathbf{e}\mathbf{a}\mathbf{n}}$

${\text{CD}}_{\text{mean}}$ select an optimal training set through the incidence matrix $\mathit{Z}$ in Eq. (7). Without loss of generality, the first $n_1$ genotypes in the candidate population with $n$ individuals are assumed to be the training set. According to Alemu et al. (2024) (Supplementary file 1), this framework leads to that the incidence matrix $\mathit{Z}=[{\mathit{I}}_{n_1}0]$ of order $n_1\times n$ and the projection matrix $\mathit{M}={\mathit{I}}_{n_1}-{\overline{\mathit{J}}}_{n_1}$ of order $n_1\times n_1$ in Eq. (7). Fixing $\lambda =1$ and ignoring ${\sigma }_A^2$, Eq. (7) can be rewritten as:

$$\mathit{K}-{\left({\mathit{K}}^{-1},+,{(\mathit{Z}}^T,),{\mathit{I}}_{n_1},(\mathit{M}\mathit{Z})\right)}^{-1}$$ 27

From the Woodbury matrix identity (Searle 1982, p261):

$${(\mathit{A}+\mathit{B}{\mathit{D}}^{-1}\mathit{C})}^{-1}={\mathit{A}}^{-1}-{\mathit{A}}^{-1}\mathit{B}{(\mathit{D}+\mathit{C}{\mathit{A}}^{-1}\mathit{B})}^{-1}\mathit{C}{\mathit{A}}^{-1,}$$

And let $\mathit{A}$ =${\mathit{K}}^{-1}$; $\mathit{B}={\mathit{Z}}^T$;${\mathit{D}}^{-1}={\mathit{I}}_{n_1}$; $\mathit{C}$ = $\mathit{M}\mathit{Z}$, then Eq. (27) can be equivalently written as:

$$\mathit{K}-\left(\mathit{K},-,\mathit{K},{\mathit{Z}}^T,{\left({\mathit{I}}_{n_1},+,\mathit{M},\mathit{Z},\mathit{K},{\mathit{Z}}^T\right)}^{-1},\mathit{M},\mathit{Z},\mathit{K}\right)=\mathit{K}{\mathit{Z}}^T{\left({\mathit{I}}_{n_1},+,\mathit{M},\mathit{Z},\mathit{K},{\mathit{Z}}^T\right)}^{-1}\mathit{M}\mathit{Z}\mathit{K}$$ 28

Partition $\mathit{K}=\left[\begin{array}{cc}{\mathit{K}}_1& {\mathit{K}}_{12}\\ {\mathit{K}}_{21}& {\mathit{K}}_2\end{array}\right]$, then Eq. (28) can be expressed as:

$$\left[\begin{array}{c}{\mathit{K}}_1\\ {\mathit{K}}_{21}\end{array}\right]{\left({\mathit{I}}_{n_1},+,\left({\mathit{I}}_{n_1},-,{\overline{\mathit{J}}}_{n_1}\right),{\mathit{K}}_1\right)}^{-1}({\mathit{I}}_{n_1}-{\overline{\mathit{J}}}_{n_1})\left[\begin{array}{cc}{\mathit{K}}_1& {\mathit{K}}_{12}\end{array}\right].$$ 29

Based on the same framework, plug ${\mathit{K}}_{\mathit{ct}}=\left[\begin{array}{c}{\mathit{K}}_1\\ {\mathit{K}}_{21}\end{array}\right]$; ${\mathit{M}}_t={\mathit{I}}_{n_1}-{\overline{\mathit{J}}}_{n_1}$; ${\mathit{K}}_t={\mathit{K}}_1$ and ${\mathit{I}}_{n_t}={\mathit{I}}_{n_1}$ into Eq. (11) for CD_mean(v2) to yield exactly the same expression as Eq. (29) for CD_mean. This completes the proof.

Author Contribution statement

SPC contributed to data curation; investigation; tables and figures preparation; Software. WHS contributed to data curation; investigation. CTL contributed to conceptualization; project administration; supervision; validation; writing the original draft; review and editing.

Funding

This research was supported by the Ministry of Science and Technology, Taiwan (grant number MOST 112–2118-M-002–003-MY2).

Data availability statement

All phenotype and genotype datasets analyzed in this study are freely accessible and can be downloaded from Figshare: 10.6084/m9.figshare.24425581.

Declarations

Conflict of interest

The authors declare that there is no conflict of interest.