Scaling linear-model breeding values to the liability scale: an application to pig binary traits

Abstract

In commercial pig production, many important traits are recorded as binary phenotypes. For such traits, threshold models offer an appropriate framework but are computationally intensive. Thus, linear models are widely used to obtain genomic estimated breeding values (GEBV); however, these are on the observed scale (phenotypic). This creates the need for a robust method to approximate GEBV from linear models to the liability scale. A recently proposed approximation showed good concordance for low-prevalence traits (<5%) but has not yet been tested for a wider range of prevalence values and for models with more than one random effect. We aimed to evaluate the performance of this approximation for pig binary traits with prevalences ranging from <5% to >86%, in both animal and maternal animal models. Data were available for five fitness traits (FT1–FT5), with up to 233k animals with phenotypes, of which 204k animals were genotyped with a 25k SNP array. Variance component estimates were obtained using threshold models. Classical animal models were used for FT1–FT3, and maternal animal models for FT4 and FT5. Variance components on the observed scale were then obtained by multiplying estimates from a threshold model by the square of the height of the standard normal density evaluated at the threshold. GEBV were predicted using single-step genomic best linear unbiased prediction under both linear and threshold models. The approximation tested involved scaling the GEBV using the height of the ordinate of the standard normal distribution evaluated at the threshold as a scaling factor. The agreement between GEBV from the scaled linear model and the threshold model on the probability scale was evaluated using Pearson and Spearman correlations, mean squared error (MSE), regression parameters, overlapping coefficient (OVL), distribution overlap, and classification accuracy (C_ACC). Correlations between linear and threshold GEBV ranged from 0.94 (low-prevalence traits) to 0.99 (high-prevalence traits) for the direct GEBV and were 0.99 for the maternal GEBV. MSE were close to zero. The OVL exceeded 0.83 for all traits. C_ACC ranged from 95.10% to 98.33% for the direct GEBV and from 92.54% to 97.42% for the maternal GEBV. Regardless of model and trait prevalence, this approximation yielded GEBV that are highly consistent with threshold model GEBV, providing a reliable, practical approach for large-scale pig genetic evaluations for binary traits using linear models.

This study presents the application of a precise and robust method that allows linear model predictions to closely match those from the threshold model, enabling large-scale genetic evaluations of binary traits without compromising selection decisions.

Introduction

Linear mixed models are widely used in genetic evaluations because they assume that phenotypes and estimated breeding values (EBV) follow a multivariate normal distribution, which provides a reasonable description of variability for many livestock traits (Perez-Enciso et al. 1993). However, an increasing number of economically relevant traits are recorded as binary phenotypes. When linear models are applied to binary traits, their discrete distribution is ignored (i.e. predictions are made on the observed phenotypic scale), which can lead to biased estimates of genetic parameters (Gianola 1982). This is particularly relevant in pigs, where several traits, such as reproductive performance (Bloemhof et al. 2012), piglet survival (Leite et al. 2021), and health traits (Cappelloni et al. 2022), are routinely recorded as binary traits.

Threshold models offer a biologically coherent alternative by assuming an underlying, normally distributed variable (liability) that determines the observed binary outcome through a fixed threshold (Gianola and Foulley 1983). This approach appropriately accounts for the binary nature of the trait and yields prevalence-independent estimates (Falconer and Mackay 1996). Despite these advantages, threshold models demand substantially greater computational resources and are more prone to convergence issues in large datasets than linear models (Hidalgo et al. 2024a; Cesarani et al. 2025), especially in the presence of data imperfections such as miscoding (Rekaya et al. 2001) or extreme prevalences (Dempster and Lerner 1950). In contrast, linear models are computationally simpler and have shown similar predictive ability for EBV of pig binary traits (Liu et al. 2023; Cesarani et al. 2025).

Estimated breeding values for binary traits are expressed as probabilities, which is desirable in industry due to their ease of interpretation. For threshold models, obtaining those probabilities is straightforward, as EBV can be directly transformed from the liability to the probability scale (Gianola and Foulley 1983). For linear models, however, EBV must first be approximated to the liability scale before being converted to probabilities. Hidalgo et al. (2023) proposed a different approximation based on a scaling factor derived from the residual variance and the proportion of nonadditive genetic variance, both on the observed scale, which performs well for traits with prevalence above 5% but loses precision when the prevalence is lower than 5% (Hidalgo et al. 2024a).

An alternative approximation was proposed by de Oliveira Padilha et al. (2026), in which the scaling factor is determined by the height of the standard normal density evaluated at the threshold ($z$). This method showed clear advantages for health traits with low prevalence (<5%) in dairy cattle (de Oliveira Padilha et al. 2026) and for morphological defects in beef cattle (Campos et al. 2026). However, these evaluations were limited to simple animal models and did not investigate scenarios including maternal genetic effects or a wider range of trait prevalences. In pigs, maternal effects are highly relevant and may equal or even exceed direct genetic contributions (Leite et al. 2021). Therefore, our objective was to evaluate the performance of this approximation method from observed to liability scale by De Oliveira Padilha et al. (2026) for pig binary traits with prevalences ranging from low to high, using both models with and without maternal genetic effects.

Materials and methods

Animal care and use committee approvals were unnecessary because data were obtained from preexisting databases.

Dataset and prevalence

The dataset used in this study was provided by Topigs Norsvin (’s-Hertogenbosch, the Netherlands). The data comprised up to 450,800 individuals, born from 2003 to 2025, from three purebred lines and two ${\mathrm{F}}_1$ lines. Five fitness reproductive-related traits (FT; recorded as 0 or 1) were analyzed. Due to industry confidentiality demands, the actual trait names cannot be divulged. The number of records for each trait was as follows: FT1 (N = 233,003), FT2 (N = 176,974), FT3 (N = 80,441), FT4 (N = 107,747), and FT5 (N = 91,712), and all traits but FT3 were composed of single records. For FT1 and FT2, the pedigree contained 309,362 animals, from which 204,429 were genotyped; for FT3, the pedigree included 33,881 animals, with 24,315 genotyped; and for FT4 and FT5, the pedigree contained 165,111 animals, of which 74,867 were genotyped. Animals were genotyped for 25k using a custom single-nucleotide polymorphisms (SNP) chip owned by Topigs Norsvin. After quality control (minor allele frequency > 0.01, SNP call rate ≥ 0.95), 19,327 SNP remained for further analyses.

Preliminary edits applied to all traits included: (1) removing contemporary groups (defined as herd-year-season) that did not contain both categories (0 and 1); and (2) excluding levels of fixed effects that contained only a single category or were severely unbalanced, as these provide insufficient information for reliable estimation and may cause complete separation (extreme-category problem). The overall prevalence (α, %) for each trait was determined as the proportion of records with the phenotypic level of interest (i.e. 1 for all traits) from the total number of records analyzed.

Models and variance components estimation

Following Topigs Norsvin pipelines, variance components for all traits were estimated using a pedigree-based single-trait animal threshold model with a probit link function and a binomial distribution, using the ASReml-R package (Butler et al. 2023). All analyses were implemented through in-house R scripts (R Core Team 2024). A classical animal model was applied to FT1 and FT2, a repeatability animal model to FT3, and a maternal animal model to FT4 and FT5. In matrix notation, the general mixed-model structure applied across traits can be expressed as:

$$\begin{array}{c}\mathbf{y}=\mathbf{X}\mathbf{\beta }+{\mathbf{Z}}_1{\mathbf{u}}_d+{\mathbf{Z}}_2{\mathbf{u}}_m+{\mathbf{Z}}_3\mathbf{c}+{\mathbf{Z}}_4\mathbf{p}\mathbf{e}+{\mathbf{Z}}_5\mathbf{s}\mathbf{b}+{\mathbf{Z}}_6\mathbf{p}{\mathbf{e}}_m+\mathbf{e}, \end{array}$$ (1)

where $\mathbf{y}$ is the vector of phenotypic observations (FT1, FT2, FT3, FT4, or FT5); $\mathbf{\beta }$ is the vector of fixed effects (Table S1); ${\mathbf{u}}_d$ is the vector of random direct additive genetic effects; ${\mathbf{u}}_m$ is the vector of random maternal genetic effects (only for FT4 and FT5); $\mathbf{c}$ is the vector of random common litter effects; $\mathbf{pe}$ is the vector of random animal permanent environmental effects (only for FT3); $\mathbf{sb}$ is the vector of random service-boar effects (only for FT3); $\mathbf{p}{\mathbf{e}}_m$ is the vector of random maternal permanent environmental effects (only for FT4 and FT5); $\mathbf{e}$ is the vector of random residuals; $\mathbf{X}\mathbf{ }$ and $\mathbf{Z}$ are the incidence matrices relating levels of β, ${\mathbf{u}}_d$, ${\mathbf{u}}_m$, $\mathbf{c}$, $\mathbf{pe}$, $\mathbf{sb}$, and $\mathbf{p}{\mathbf{e}}_m$ to $\mathbf{y}$, respectively. For all the analyses, multivariate normal distributions with zero mean were assumed for the random effects (${\mathbf{u}}_d$, ${\mathbf{u}}_m$, $\mathbf{c}$, $\mathbf{pe}$, $\mathbf{sb}$, $\mathbf{p}{\mathbf{e}}_m$, and e). Thus, assuming multivariate normality, all random effects but ${\mathbf{u}}_{\mathbf{d}}$ and ${\mathbf{u}}_{\mathbf{m}}$ were independently distributed:

$$\begin{array}{c}\mathit{Var}\left[\begin{array}{c}{\mathbf{u}}_d\\ {\mathbf{u}}_m\\ \mathbf{c}\\ \mathbf{p}\mathbf{e}\\ \mathbf{s}\mathbf{b}\\ \mathbf{p}{\mathbf{e}}_m\\ \mathbf{e}\end{array}\right]=\left[\begin{array}{ccccccc}\mathbf{A}{\sigma }_{u_d}^2& \mathbf{A}{\sigma }_{u_d{u}_m}& \mathbf{0}& \mathbf{0}& \mathbf{0}& \mathbf{0}& \mathbf{0}\\ \mathbf{A}{\sigma }_{u_d{u}_m}& \mathbf{A}{\sigma }_{u_m}^2& \mathbf{0}& \mathbf{0}& \mathbf{0}& \mathbf{0}& \mathbf{0}\\ \mathbf{0}& \mathbf{0}& \mathbf{I}{\sigma }_c^2& \mathbf{0}& \mathbf{0}& \mathbf{0}& \mathbf{0}\\ \mathbf{0}& \mathbf{0}& \mathbf{0}& \mathbf{I}{\sigma }_{pe}^2& \mathbf{0}& \mathbf{0}& \mathbf{0}\\ \mathbf{0}& \mathbf{0}& \mathbf{0}& \mathbf{0}& \mathbf{I}{\sigma }_{sb}^2& \mathbf{0}& \mathbf{0}\\ \mathbf{0}& \mathbf{0}& \mathbf{0}& \mathbf{0}& \mathbf{0}& \mathbf{I}{\sigma }_{p{e}_m}^2& \mathbf{0}\\ \mathbf{0}& \mathbf{0}& \mathbf{0}& \mathbf{0}& \mathbf{0}& \mathbf{0}& \mathbf{I}{\sigma }_e^2\end{array}\right], \end{array}$$ (2)

where $\mathbf{A}$ is the numerator relationship matrix and $\mathbf{I}$ is an identity matrix of proper order. Parameters ${\sigma }_{u_d}^2$, ${\sigma }_{u_d{u}_m}$, ${\sigma }_{u_m}^2$, ${\sigma }_c^2$, ${\sigma }_{pe}^2,\mathrm{ }{\sigma }_{sb}^2,\mathrm{ }{\sigma }_{p{e}_m}^2,\mathrm{ }\mathrm{and}\mathrm{ }{\sigma }_e^2$ denote the (co)variance components for direct additive genetic, maternal additive genetic, common litter, animal permanent environmental, service-boar, maternal permanent environmental, and residual effects, respectively.

Transformation of (co)variance components to the observed scale

The (co)variance components from the threshold model were transformed from the liability scale to the observed scale following Hidalgo et al. (2026), who refined the classical method proposed by Dempster and Lerner (1950). The rationale is as follows: heritability is the ratio of additive genetic variance to the phenotypic variance. According to Dempster and Lerner (1950), on the observed scale, the additive genetic variance is $z^2{h}_l^2$, which is the product of the squared height of a standard normal distribution evaluated at the threshold and the heritability on the liability scale. Hidalgo et al. (2026) suggested that $h_l^2$ is the additive genetic variance—as postulated by Dempster and Lerner (1950)—only if the liability has unity variance. Hidalgo et al. (2026) proposed to multiply the variance component on the liability scale by $z^2$ to transform it to the observed scale, i.e. ${\sigma }_{i\left(o\right)}^2=z^2{\sigma }_{i\left(l\right)}^2$, for the ith variance component. Then the residual variance on the observed scale was calculated by subtracting all other variance components from ${\sigma }_y^2$. This reasoning is also applicable to covariances, thus providing a straightforward conversion from the liability to the observed scale: ${\sigma }_{u_d{u}_m\left(o\right)}=z^2{\sigma }_{u_d{u}_m\left(l\right)}$.

Prediction of breeding values

For the threshold model, the binary phenotypes $y_i$ ($i=1,\dots ,\mathrm{n}$) were assumed to follow a Bernoulli distribution: $y_i|p_i\sim \mathit{Bernoulli}(p_i)$, where $p_i$ denotes the probability of observing category 0 ($y_i=0$), and $1-p_i$ is the probability of observing category 1 ($y_i=1$). The observed phenotypes were assumed to be the response of a truncated underlying variable ($l_i$), which, conditional on model parameters, follows a normal distribution with mean $\mathbf{\eta }=\mathbf{X}\mathbf{\beta }+{\mathbf{Z}}_1{\mathbf{u}}_d+{\mathbf{Z}}_2{\mathbf{u}}_m+{\mathbf{Z}}_3\mathbf{c}+{\mathbf{Z}}_4\mathbf{p}\mathbf{e}+{\mathbf{Z}}_5\mathbf{s}\mathbf{b}+{\mathbf{Z}}_6\mathbf{p}{\mathbf{e}}_m$ and variance $\mathbf{I}{\sigma }_e^2$. The unconditional liability variance was defined as ${\sigma }_l^2={\sigma }_{u_d}^2+{\sigma }_{u_m}^2+{\sigma }_c^2+{\sigma }_{pe}^2+{\sigma }_{sb}^2+{\sigma }_{p{e}_m}^2+{\sigma }_e^2$, and ${\sigma }_e^2$ was set to one to make the model identifiable (Sorensen and Gianola 2002). The conditional response of the binary phenotypes on the liability and the threshold was modeled with the following distribution:

$$\begin{array}{c}p\left(\mathbf{y}|\mathbf{l},t\right)=\prod _{i=1}^n\left[I\left(l_i\le t\right)p\left(y_i=0|l_i,t\right)+I\left(l_i>t\right)p\left(y_i=1|l_i,t\right)\right], \end{array}$$ (3)

where $t$ is the threshold defining the categories, $I\left(·\right)$ is an indicator function, taking the value of one if the specified condition is true and zero otherwise.

Genomic estimated breeding values (GEBV) were predicted using single-step genomic best linear unbiased prediction (ssGBLUP) via single-trait models with both a linear (GEBV on the observed scale) and a threshold (GEBV on the liability scale) model. Threshold model GEBV were used as a benchmark to assess the quality of approximations obtained from linear models. All computations for both linear and threshold models were performed with MiXBLUP 3.0 (Vandenplas et al. 2022), following Topigs Norsvin pipelines. In ssGBLUP, $\mathbf{A}$ for equation 2 is replaced by $\mathbf{H}$, which combines pedigree and genomic relationships (Legarra et al. 2009; Christensen and Lund 2010), and its inverse was computed following Aguilar et al. (2010). The genomic relationship matrix ($\mathbf{G}$) was constructed as in VanRaden (2008), method 1, and blended with 10% of $\mathbf{A}$ corresponding to the genotyped animals (i.e. ${\mathbf{A}}_{22}$). Due to the number of genotyped animals, the algorithm for proven and young (Misztal 2016) was used to approximate the inverse of $\mathbf{G}$. Because three different genotype files were used, three sets of core animals were defined: 9,933 for FT1 and FT2, 4,720 for FT3, and 4,859 for FT4 and FT5. The number of core animals was determined as the number of eigenvalues explaining at least 98% of the variation in $\mathbf{G}$, and for all analyses, the core was kept constant.

Approximation from observed to liability scale

The GEBV were approximated from the observed scale to the liability scale following De Oliveira Padilha et al. (2026) as:

$$\begin{array}{c}{\mathit{GEBV}}_l\approx \frac{{\mathit{GEBV}}_o}{\sqrt{h_o^2{h}_l^{-2}\alpha \left(1-\alpha \right)}}, \end{array}$$ (4)

where ${\mathit{GEBV}}_l$ is the GEBV on the liability scale; ${\mathit{GEBV}}_o$ is the GEBV on the observed scale; $h_o^2{h}_l^{-2}$ is the ratio of heritability estimates (observed to liability scale); and $\alpha \left(1-\alpha \right)$ is the phenotypic variance, with $\alpha$ representing the overall prevalence of each trait. Near the population mean, ${\mathit{GEBV}}_o\mathrm{ }$can be viewed as a rescaled version of ${\mathit{GEBV}}_l$, with the scaling factor determined by the height of the standard normal density evaluated at the threshold (Dempster and Lerner 1950).

Transformation to probability scale

The GEBV obtained from linear and threshold models were transformed from the liability to the probability scale following (Gianola and Foulley 1983):

$$\begin{array}{c}{p}_{ij}=1-\Phi \left(t-x_j^{\mathrm{\prime }}\beta -z_j^{\mathrm{\prime }}{u}_d\right), \end{array}$$ (5)

where $p_{ij}$ is the probability that the animal $j$ expresses its phenotype in category i; $\mathit{\Phi }\left(·\right)$ is the cumulative density function (CDF) of a standard normal distribution; $t$ is the threshold (fixed at zero, in our study); $x_j^{\prime }\beta$ is the fixed component, defined as $x_j^{\prime }\beta ={\mathrm{\Phi }}^{-1}\left(\alpha \right)$; $z_j^{\prime }{u}_d$ is the direct GEBV of animal $\mathrm{j}$ (or the maternal GEBV, $z_j^{\prime }{u}_m$).

GEBV concordance between threshold and linear models

To evaluate how the GEBV from a linear model (GEBV_L) resembles the GEBV from a threshold model (GEBV_T), we used several metrics: Pearson and Spearman rank correlations, regression analysis, mean squared error (MSE), overlap of their distributions, and the overlapping coefficient (OVL). It is worth noting that GEBV_L was first approximated to liability (equation 4) and then both GEBV_L and GEBV_T were transformed to probability (equation 5). For the regression analysis, a simple linear model was fitted using GEBV_T as the response variable and GEBV_L as the predictor. The MSE is strictly the mean squared difference between GEBV_T and GEBV_L. The OVL was computed as the integral of the minimum density between the two distributions (GEBV_L and GEBV_T), which represents their shared area (0 = no overlap; 1 = complete overlap), using the overlapping R package (Pastore 2016).

To assess potential selection consequences of using GEBV_L instead of GEBV_T, we ranked all animals by GEBV_T and assumed that the top 10% were selected as parents of the next generation. Using this cutoff, each animal was labeled as selected or not by each model, yielding four classification categories: selected by both models, selected only by the linear model, selected only by the threshold model, or not selected by either model. Using all the animals, the Classification accuracy (C_ACC) was then quantified as the percentage of animals for which the two models produced the same outcome:

$$\begin{array}{c}{C}_{\mathit{ACC}}=\frac{n_S+n_N}{N}\times 100, \end{array}$$ (6)

where $n_S$ is the number of animals selected by both GEBV_L and GEBV_T, $n_N$ is the number of animals not selected by both models, and $N$ is the total number of animals. All GEBV post-processing was performed using in-house R (R Core Team 2024) scripts, and all plots were generated with the ggplot2 package (Wickham 2016).

Results and discussion

Heritability and prevalence

Table 1 presents the number of records, prevalences (%), and genetic parameters for each trait. Overall, prevalence ranged from low (2.2% for FT1) to high (89.6% for FT5). Direct heritability varied from 0.005 (FT4) to 0.057 (FT3) on the liability scale, and from 0.002 (FT1) to 0.024 (FT3) on the observed scale. Maternal heritability ranged from 0.020 (FT4) to 0.055 (FT5) on the liability scale, and from 0.010 (FT4) to 0.023 (FT5) on the observed scale. The genetic correlation between direct and maternal additive genetic effects was negative and low for FT4 (−0.288), whereas it was positive and high for FT5 (0.730).

Table 1.

Number of records, prevalence (%), heritability (direct $h_d^2$ and maternal $h_m^2$), covariances (${\sigma }_{u_{d,}{u}_m}$), and correlation ($r_{u_{d,}{u}_m}$) between direct and maternal effects on the liability and observed scales for fitness traits (FT) in pigs.

Trait	No. of records	Prevalence (%)	Liability			Observed			${\mathit{r}}_{{\mathit{u}}_{\mathit{d},}{\mathit{u}}_{\mathit{m}}}$
			${\mathit{h}}_{\mathit{d}}^2$	${\mathit{h}}_{\mathit{m}}^2$	${\mathit{\sigma }}_{{\mathit{u}}_{\mathit{d},}{\mathit{u}}_{\mathit{m}}}$	${\mathit{h}}_{\mathit{d}}^2$	${\mathit{h}}_{\mathit{m}}^2$	${\mathit{\sigma }}_{{\mathit{u}}_{\mathit{d},}{\mathit{u}}_{\mathit{m}}}$
FT1	233,003	2.2	0.013	…	…	0.002	…	…	…
FT2	176,974	4.7	0.033	…	…	0.008	…	…	…
FT3	80,441	87.7	0.057	…	…	0.024	…	…	…
FT4	107,747	86.0	0.005	0.020	−0.0036	0.003	0.010	−0.0002	−0.288
FT5	91,712	89.6	0.007	0.054	0.0178	0.003	0.023	0.0006	0.730

These results follow the expected behavior for binary traits, particularly the contrast between liability and observed scale estimates. Threshold models are the theoretically appropriate approach for binary traits because they provide estimates that are independent of prevalence (Gianola 1982) and explicitly account for the non-normal distribution of binary phenotypes (Gianola and Foulley 1983) In contrast, linear models do not satisfy these assumptions and typically produce lower heritability on the observed scale than on the liability scale. This difference has been consistently reported in studies with pigs (Cappelloni et al. 2022; Cesarani et al. 2025), dairy cattle (Vukasinovic et al. 2025), and beef cattle (Hidalgo et al. 2024b), reinforcing the hypothesis that estimates on the observed scale are lower than on the liability scale, due to the dependence on the trait prevalence.

For FT4 and FT5, maternal genetic effects were particularly relevant, as indicated by maternal heritability exceeding the corresponding direct estimates and by genetic correlations between direct and maternal effects. Previous evidence shows that maternal effects are an important component of pig fitness traits and may equal or even exceed direct contributions (Leite et al. 2021; Nguyen et al. 2021). This implies that selection for fitness must account for maternal genetic components; otherwise, biased selection decisions could lead to reduced genetic gain and underestimated expected genetic gain.

Correlations and MSE

Correlations and MSE between GEBV_T and the corresponding direct and maternal GEBV_L are shown in Table 2. For the direct GEBV, Pearson and Spearman rank correlations ranged from 0.94 to 0.99, indicating a strong linear association and minimal re-ranking of animals between models. The MSE were generally small, with slightly higher values for FT1 and FT2 (0.13 and 0.30, respectively), i.e. we observed marginally greater GEBV error for traits with lower prevalence. For the maternal GEBV, correlations were also high (0.99 for both Pearson and Spearman), and MSE were close to zero.

Table 2.

Correlations and mean squared errors between estimated breeding values (GEBV; direct and maternal) on the probability scale from the threshold model and the corresponding direct and maternal EBV from the linear model (after approximation) for fitness traits (FT) in pigs.

Trait	Direct effect				Maternal effect
	Correlation		Squared error		Correlation		Squared error
	P1	S2	Mean (SD)3	Min–Max4	P	S	Mean (SD)	Min–Max
FT1	0.95	0.94	0.13 (0.22)	0.00–2.55	…	…	…	…
FT2	0.97	0.96	0.30 (0.63)	0.00–15.63	…	…	…	…
FT3	0.99	0.99	0.00 (0.00)	0.00–0.07	…	…	…	…
FT4	0.99	0.99	0.00 (0.00)	0.00–0.01	0.99	0.99	0.00 (0.00)	0.00–0.02
FT5	0.98	0.98	0.00 (0.00)	0.00–0.02	0.99	0.99	0.01 (0.01)	0.00–0.23

¹ Pearson correlation. ² Spearman correlation. ³ Standard deviation. ⁴ Minimum and maximum differences.

In a multibreed evaluation of Italian pigs (Large White, Landrace, and Duroc), Cappelloni et al. (2022) estimated Pearson and Spearman rank correlations greater than 0.97. Using a maternal animal model for calving ease in the American Angus population, Hidalgo et al. (2024b) reported consistent Spearman rank correlations of 0.99 for direct GEBV and 0.97 for maternal GEBV across different transformation methods from the observed to the liability scale and across different subsets of animals (e.g. full pedigree or sire groups defined by progeny size). This altogether suggests that, regardless of species, GEBV_L is a very good proxy for GEBV_T, implying that we can base selection decisions on GEBV_L without any significant detrimental effect.

Similarly, Cesarani et al. (2025) evaluated two binary traits in pigs (wean-to-finish mortality and leg structure) with prevalences ranging from 9 to 13% and reported rank correlations above 0.98 between GEBV from linear and threshold models. The authors noted that correlation magnitudes tend to increase with larger datasets and with traits that do not have extreme prevalences, reaching their highest values when the distribution of phenotypes is more balanced (i.e. close to 50%). In the same study, Pearson and Spearman rank correlations were unchanged across scales (observed-liability-probability), a result that was also observed in our study. According to De Oliveira Padilha et al. (2026), this occurs because the approximation method is a linear transformation, i.e. GEBV on the liability scale are linearly related to GEBV on the observed scale by a constant factor, which preserves the probability structure.

The slightly higher MSE observed for the low-prevalence traits (FT1 and FT2), compared with the other traits, is consistent with theoretical expectations. Parameter estimation for binary traits becomes more challenging as prevalences approach zero or one because the phenotypic variance on the observed scale decreases in these situations, reducing the amount of information available for inference (Dempster and Lerner 1950). This issue is closely related to a known challenge in genetic evaluations of binary traits: the extreme-category problem. This occurs when all observations within a level of a fixed effect (e.g. a contemporary group) fall into the same category. In this situation, the equations become unstable, causing the solution for that subclass to diverge toward ±∞, affecting also the solutions of the other classes (Misztal et al. 1989). Overall, these results indicate that the approximation method preserves both the scale and the ranking of animals, leading to minimal practical differences in selection decisions when comparing the linear model with the benchmark threshold model.

Overlapping, regression parameters, and classification accuracy between distributions

Figures 1 and 2 present the distribution of direct GEBV for each trait according to the model (linear or threshold) and the prevalence range (low or high), and Figure 3 shows the distribution of maternal GEBV for FT4 and FT5. These figures present the GEBV density distributions on their original scale (observed for the linear model and liability for the threshold model), along with the approximated GEBV on the liability scale and the corresponding values transformed to the probability scale. The desired outcome is a clear overlap between the GEBV distributions on the adjusted liability and probability scales, as well as an OVL value closer to 1 on these scales. For the direct GEBV (Figures 1 and 2), the distributions overlapped effectively across all traits, with slightly higher OVL values for low-prevalence traits and for FT3 (OVL > 0.94) compared with traits that include maternal effects (OVL = 0.83–0.91). For the maternal GEBV (Figure 3), the distributions also showed substantial overlap, with OVL ranging from 0.84 (FT5) to 0.97 (FT4). In general, OVL measures the area shared by two probability density functions (Clemons and Bradley 2000), making it useful for comparing how close these distributions are to each other (Martínez-Camblor 2023). Although the OVL on the unadjusted scale is ≤0.42, this is expected because the GEBV_L and GEBV_T solutions are expressed on different scales and are not directly comparable. The main focus of the agreement lies on the liability and probability scales, which generally showed OVL values close to 1. This means that the distributions have similar shapes, implying comparable variances and means, suggesting that the transformations produce GEBV_L compatible with GEBV_T.

Figure 1.

Distributions of direct estimated breeding values (${\mathrm{GEBV}}_{\mathrm{d}}$) on the observed and liability scales, after approximation to the liability scale, and on the probability scale for fitness traits (FT) with low prevalence (2–5%). OVL, overlapping coefficient.

Figure 2.

Distributions of direct estimated breeding values (${\mathrm{GEBV}}_{\mathrm{d}}$) on the observed and liability scales, after approximation to the liability scale, and on the probability scale for fitness traits (FT) with high prevalence (>86%). OVL, overlapping coefficient.

Figure 3.

Distributions of maternal estimated breeding values (${\mathrm{GEBV}}_{\mathrm{m}}$) on the observed and liability scales, after approximation to the liability scale, and on the probability scale for fitness traits (FT) with high prevalence (>86%). OVL, overlapping coefficient.

Scatterplots comparing the GEBV on the probability scale are shown in Figures 4 and 5 for the direct GEBV, and in Figure 6 for the maternal GEBV. Each figure includes the regression equation obtained by fitting GEBV_T as the response variable and GEBV_L as the predictor. Under ideal circumstances, the expected coefficients are $b_0=0.00$ and $b_1=1.00$, indicating no bias or dispersion issues between GEBV_L and GEBV_T. Overall, the estimated regression coefficients were close to these ideal values across traits for the direct GEBV. However, some traits showed slopes slightly greater than 1, indicating that GEBV_L may overestimate GEBV_T in those cases. For the maternal GEBV (Figure 6), the regression coefficients were also close to the ideal values and, in some traits, aligned even more closely with GEBV_T than the corresponding direct GEBV. For example, in FT4, the direct GEBV yielded $b_0=9.70$ and $b_1=1.10$, whereas the maternal GEBV yielded $b_0=1.60$ and $b_1=1.00$. This pattern suggests that the approximation can reproduce the scale of GEBV_T reasonably well, including models that incorporate maternal genetic effects. Additionally, the agreement was slightly stronger for the direct GEBV of low-prevalence traits (Figure 4), reflecting the ability of the approximation method derived by De Oliveira Padilha et al. (2026) to perform well even in scenarios with extreme prevalence.

Figure 4.

Scatterplots comparing direct estimated breeding values (${\mathrm{GEBV}}_{\mathrm{d}}$) on the probability scale (threshold vs. linear models) for traits with low prevalence (2–5%) and animals commonly selected across models. The dashed lines represent the cutoff used to select the top 10% of animals in both models. Points are colored according to each animal’s selection category. C_ACC, classification accuracy between the threshold and linear models; LIN,  linear model; THR, threshold model.

Figure 5.

Scatterplots comparing direct estimated breeding values (${\mathrm{GEBV}}_{\mathrm{d}}$) on the probability scale (threshold vs. linear models) for traits with high prevalence (>86%) and animals commonly selected across models. The dashed lines represent the cutoff used to select the top 10% of animals in both models. Points are colored according to each animal’s selection category. C_ACC, classification accuracy between the threshold and linear models; LIN,  linear model; THR, threshold model els.

Figure 6.

Scatterplots comparing maternal estimated breeding values (${\mathrm{GEBV}}_{\mathrm{m}}$) on the probability scale (threshold vs. linear models) for traits with high prevalence (>86%) and animals commonly selected across models. The dashed lines represent the cutoff used to select the top 10% of animals in both models. Points are colored according to each animal’s selection category. C_ACC, classification accuracy between the threshold and linear models; LIN,  linear model; THR, threshold model.

Figures 4–6 also show the proportion of animals selected in common by the threshold and linear models, which is used to compute C_ACC. For the direct GEBV, C_ACC ranged from 95.10% (FT3) to 98.33% (FT4), and for the maternal GEBV, it ranged from 92.54% (FT5) to 97.42% (FT4). These results show that the scaling method yields GEBV_L that closely resembles those from the benchmark (GEBV_T), with strong agreement in both ranking and magnitude across traits. A similar pattern was observed for traits analyzed with maternal animal models, indicating that the approximation remains consistent even in more complex model structures. Because C_ACC indicates whether animals are similarly classified as selected or not selected by both models, this metric effectively summarizes how the models agree in ranking near the selection threshold. Thus, C_ACC indicates how consistently animals are prioritized for selection when using GEBV_L rather than GEBV_T. In livestock breeding programs, matching selection decisions could be more important than obtaining identical GEBV values. Since the cutoff was defined based on GEBV_T for both evaluations, the high agreement in C_ACC (consistently above 90%), combined with similar GEBV values, supports the conclusion that the approximation derived by de Oliveira Padilha et al. (2026) can be effectively used to approximate GEBV_L to GEBV_T in terms of expected genetic response.

Although threshold models remain the most appropriate framework for estimating genetic parameters for binary traits (Abdel-Azim and Berger 1999), their routine use in genetic evaluations faces several challenges, including extreme prevalences (Dempster and Lerner 1950), sensitivity to miscoded phenotypes (Rekaya et al. 2001), and increasingly computational demands associated with large datasets (Hidalgo et al. 2024b). It is well documented that threshold models are more computationally demanding than linear models, not only due to the number of iterations required but also because each iteration can take up to twice as long (Hidalgo et al. 2024b). In this context, the possibility of approximating GEBV_T from GEBV_L becomes justified as genomic datasets and model complexity continue to grow.

On the other hand, although de Oliveira Padilha et al. (2026) showed better agreement at very low prevalences compared with the approach by Hidalgo et al. (2023), prevalence still imposes limitations, particularly when heritability and prevalence differ across subpopulations (e.g. contemporary groups or herds). According to Lee et al. (2011), one important source of bias arises when the sample used to estimate genetic parameters has a different prevalence than the full population, as is typically the case in practice. This was observed by Hidalgo et al. (2024b) in an Angus cattle population, where variance components were estimated using a subset with a 5% prevalence for calving ease, compared with 7% in the full dataset. Because the scaling factor proposed by Hidalgo et al. (2023) also depends on heritability, such differences were identified as a potential source of bias. Although in our study we used the same dataset to estimate genetic parameters and predict GEBV, genomic information was not incorporated into variance component estimation, which may introduce bias when selection is based on genomic predictions (Hidalgo et al. 2020).

Furthermore, using a single scaling factor for all animals implicitly assumes constant prevalence across subpopulations, which is not completely true. For instance, Hidalgo et al. (2024a) reported a coefficient of variation of 10% for mastitis prevalence in contemporary groups in a Holstein population. As expected, in our study, the weighted coefficient of variation for FT1 was 7.82%, ranging from 2.66% to 13.96%, indicating that prevalence was not homogeneous across contemporary groups. Finally, we worked with the assumption that the GEBV is near the population mean, which allows us to generalize the transformation to all individuals. If the GEBV is far from the population mean, the relationship between the observed and the liability scale is non-linear, in which case any approximation will lose statistical power.

Overall, this study demonstrates that the approximation proposed by de Oliveira Padilha et al. (2026) is a robust approach for applying linear models to pig binary traits with varying prevalence and model complexity. By producing GEBV_L that closely match GEBV_T in distribution, correlation, and selection outcomes, this approximation offers a practical way to leverage linear mixed models for binary traits. Future studies could extend this approximation to more complex (e.g. multi-trait) models and refine the scaling factor to incorporate trait-specific prevalences and heritability across subpopulations. Another avenue of research is the approximation of reliabilities for threshold traits modeled as linear traits. To date, the implementation of threshold models has not allowed for the approximation of reliabilities, and our approximation methods still need to be extended for this purpose. Recently, however, threshold models have been implemented more efficiently, enabling the future development of approximate reliabilities using the mixed-model equations (Bermann et al. 2026).

Conclusions

The approximation tested in this study allows the GEBV to be scaled from the observed to the liability scale in agreement with threshold models for binary fitness traits in pigs, regardless of trait prevalence (low or high) or model complexity, with consistent performance even when maternal genetic effects are included. By preserving both the scale of the distributions and the ranking of animals, the tested method represents a practical and robust alternative for large-scale genetic evaluations of binary traits using linear mixed models.

Glossary

Abbreviations

C_ACC

classification accuracy

CDF

cumulative density function

EBV

estimated breeding value

FT

fitness trait

GEBV

genomic estimated breeding value

G

genomic relationship matrix

H

combined pedigree and genomic relationship matrix

MSE

mean squared error

OVL

overlapping coefficient

SNP

single-nucleotide polymorphisms

ssGBLUP

single-step genomic best linear unbiased prediction

VCE

variance components estimate

Author contributions

Denyus Augusto de Oliveira Padilha (Conceptualization, Data curation, Formal analysis, Methodology, Writing—original draft, Writing—review & editing), Natalia Galoro Leite (Conceptualization, Data curation, Project administration, Supervision, Writing—review & editing), Egiel Hanenberg (Conceptualization, Supervision, Writing—review & editing), Dianne Van der Spek (Data curation, Writing—review & editing), Tomas Stevens (Data curation, Writing—review & editing), Fernando Bussiman (Conceptualization, Methodology, Writing—review & editing), Daniela Lourenco (Conceptualization, Funding acquisition, Writing—review & editing), Ignacy Misztal (Funding acquisition, Writing—review & editing), and Jorge Hidalgo (Conceptualization, Funding acquisition, Methodology, Project administration, Supervision, Writing—review & editing)

Supplementary data

Supplementary data is available at Journal of Animal Science online.

Conflict of interest statement. The authors declare no real or perceived conflicts of interest.