Relatedness coefficients and their applications for triplets and quartets of genetic markers

Abstract

Relatedness coefficients which seek the identity-by-descent of genetic markers are described. The markers are in groups of two, three or four, and if four, can consist of two pairs. It is essential to use cumulants (not moments) for four-marker-gene probabilities, as the covariance of homozygosity, used in four-marker applications, can only be described with cumulants. A covariance of homozygosity between pairs of markers arises when populations follow a mixture distribution. Also, the probability of four markers all identical-by-descent equals the normalized fourth cumulant. In this article, a “genetic marker” generally represents either a gene locus or an allele at a locus. Applications of three marker coefficients mainly involve conditional regression, and applications of four marker coefficients can involve identity disequilibrium. Estimation of relatedness using genetic marker data is discussed. However, three- and four-marker estimators suffer from statistical and numerical problems, including higher statistical variance, complexity of estimation formula, and singularity at some intermediate allele frequencies.

Introduction

Relatedness is a general term for the level of genetic similarity between individuals and is measured by the sharing alleles identical-by-descent (Malécot 1948; Pamilo and Crozier 1982). Relatedness is quantified with gene identity coefficients, which characterize both the pattern and the frequency of identity-by-descent. The unit of observation is normally a pair of marker loci, and the object of estimation is the kinship coefficient or the coefficient of relationship. In this article, the unit of observation is extended to triplets and quartets of genes, allowing the opportunity to characterize additional parameters of population structure.

Relatedness may be estimated with genetic markers (Morton et al. 1971) and for pairs of marker genes, many computer programs are available for estimation of relatedness (Wang 2014), in particular for “pairwise relationship”, such as the r of Queller and Goodnight (1989). However, the equations for pairwise relationship are not extendable to three or four genes, as the covariances and higher moments need to be defined in new ways.

With more than two markers the situation becomes much more complex. Estimators for three- and four-marker-gene measures of relatedness have recently been proposed. Samanta et al (2009) provided the first estimator for three genes. Ackerman et al. (2017) examined all four genes and described the estimation of seven of eight coefficients of relatedness. Multiallelic data have information about all eight coefficients, but they used a biallelic model which provides just seven degrees of freedom, constraining their space of estimates. In addition, they did not use cumulants and their moments are normalized differently than will be here.

Cumulants are of use in certain problems in quantitative genetics (Burger 1991; Turelli and Barton 1994) and compared to cumulants have more useful theoretical properties (Kendall et al. 1977), but ordinary moments are sufficient for two and three-gene fixation indices. Ritland (1987) found cumulants instead of moments were an essential component of four-gene fixation indices. Fourth-order cumulants are needed to specify the probability of gene identity for all four genes and also to describe identity disequilibrium as the “covariance of covariances”. As an example of the necessity of cumulants, for population gene frequency p, the fourth central moment for four genes, denoted $(X_1,X_2,X_3\mathrm{and}{X}_4)$, is $E[(X_1-p)(X_2-p)(X_3-p)(X_4-p)]={\sigma }_{12}^2{\sigma }_{34}^2+$  ${\sigma }_{13}^2{\sigma }_{24}^2+{\sigma }_{23}^2{\sigma }_{13}^2+{\kappa }_4$, where ${\sigma }_{ij}^2$ is the covariance of $X_i$ and $X_j$ and ${\kappa }_4$ is a fourth-order cumulant which does not appear in moment-based treatments.

The probabilities developed in the article are used to estimate relatedness in a population with polymorphic genetic markers. Any numbers of alleles at a single locus are allowed in the equations, but practically, one has two and at most three alleles with SNP types of data, and for tractability, we use a three-allele model to examine a special situation with four genes. These models and estimation procedures are readily applicable to the emerging mountains of genome data.

Applications of markers

Definitions

Between pairs of genetic markers, the coefficient of relationship measures the degree of consanguinity (e.g. the probability that markers are identical by descent, termed ibd). The coefficient relationship equals twice the kinship coefficient. The inbreeding coefficient is the probability that a pair of markers within one individual are ibd. With more than two markers, the coefficient of relationship is more broadly defined with groups of markers (two, three, or four). With four markers, there are nine modes of Jacquard's gene identity, with ibd genes connected by lines. The normalized central moment gives the probability of ibd of all markers. At the level of four markers, cumulants are necessary to describe identity disequilibrium (the excess of identity between marker pairs). The covariance of cumulants forms the machinery of higher order interactions.

Relatedness and two markers

The two-marker coefficient of relatedness is used for many inferences with genetic markers, mainly involving pairs of genes sampled between two individuals (“coefficient of relationship”) or pairs of genes sampled within one individual (the “inbreeding coefficient”). Analysis of data with two marker measures are ubiquitous (Wang 2014) and the two-marker probabilities are often incorporated into probabilities of groups of three and four genes.

Regression and three markers

The three-marker relationship coefficient is the probability that the three marker loci have alleles all identical-by-descent (Fig. 1a). This coefficient, G, is usually combined with two-allele coefficients for biological interpretable parameters, useful at least for problems involving mating systems or kin selection. In the theory of mating system estimation, the “effective selfing rate” is the genetically equivalent rate of selfing caused by all types of biparental inbreeding (Ritland 1985); the effective selfing rate of individual A equals 2R-G, where R is the relatedness between mates and G the third moment involving the two maternal and single paternal allele (Fig. 1b). In the theory of kin selection, the regression coefficient of relatedness is used and, properly, a three-gene model is needed (Fig. 1c), as shown by Michod and Hamilton (1980), where their Equation 18 depends upon whether the reference genotype is homozygous (18a) vs heterozygous (18b). Note that both the effective selfing rate and the regression coefficients of relationship can be asymmetrical when inbreeding coefficients differ between the two relatives.

Fig. 1.

Three cases where three-gene modes of gene identity are used. a) Effective selfing model involves two genes of one parent, b) progeny-pair model, and c) Altruist-recipient. Identical genes are linked by lines. In the effective selfing model, two of the genes are from the maternal parent and the other gene is the paternal contribution.

Identity disequilibrium and four markers

Between two diploid individuals, there are 15 patterns of gene identity (Liu and Weir 2005). A pair of individuals can share two, three, or four markers, and at each level, allelic similarities can describe aspects of relatedness. After Jacquard (1966, 2012), for four marker genes and two individuals, there are nine condensed identity modes, denoted as Δ_i (Fig. 2a). There are eight independent parameters of relatedness: three pairwise measures (F_A, F_B, R), two three-way measures (G_A, G_B), and three four-marker measures (F_AB, R_AB, H). At the highest level, the measures are much different as the four-marker measures F_AB, R_AB are covariances (not identities). The four-marker parameter, H, is the probability that four markers are identical. As well, the quantities must be defined as cumulants. Cumulants equal moments up to order three, but fourth-order moments do not equal fourth-order cumulants.

Fig. 2.

Two cases where four-gene modes of identity are used. a) The general case where each of the nine identity modes are inferred. b) A model to fit progeny pairs to mating system parameters. c) The inference of heritability in the field, where M is the marker and Q the quantitative trait.

The fourth central moment equals ${\kappa }_4+F_{AB}+2R_{AB}$ (note the covariances between second moments enter this expression) and the normalized cumulant ${\kappa }_4$ (equivalent to H) equals the probability of identity of all four marker genes. While the variance is a measure of the spread of the distribution, kurtosis is a measure of the “peakedness” of the distribution of random variables, and infrequent extreme deviations contributing excessively to this statistic (de la Rosa and Moreno Muñoz 2008).

While applications of three and four gene measures are in their infancy, at least, the skew and kurtosis as measured by higher moments can help remove bias in DNA forensics caused by genotyping error (Weir 1994).

In the four-marker model, many possibilities exist about attaching meaning to each of the _i. One example is the progeny-pair model (Ritland and Leblanc 2004), where A and B are two progeny of the same mother plant (Fig. 2b). At another more abstract level, two of the genes can be markers and two are quantitative trait loci (Fig. 2c). If identity disequilibrium is present, the regression of phenotypic similarity (QTL) on estimated relationship (markers) gives an estimate of heritability “in the field” (Ritland 2000).

Two, three, and four marker probabilities

At any level of comparison, associations are measured as the frequency of a given configuration (allele “state”) divided by the denominators in Table 1. These denominators are termed “normalization constants” and are the maximum possible value of the numerator. Some of these normalized measures of association arise naturally in the derivations below.

Table 1.

Examples of the statistical variances of relationship coefficients when estimates are based upon a single locus and when the true values are zero (“x” denotes not estimable). See Equations 3 and 6 for definitions of G and H.

Array of p	F	GA	GB	${\varphi }_{XY}$	H
0.6,0.3,0.1	2.93	73.5	74.3	12.96	7.63
0.5,0.3,0.2	2.32	X	x	11.66	3.66
0.4,0.3,0.2,0.1	1.58	0.19	7.79	13.05	2.18

Probabilities of two-marker relationship

From Equation (7) of Ritland (1987), which follows Kendall et al. (1977, eq. 13.36), the frequency of gametes with allele i and with allele j is

$$f_{ij}={\mathit{\kappa }}_i{\mathit{\kappa }}_j+{\mathit{\kappa }}_{ij}.$$

The two-marker coefficient of relationship can be estimated from the frequencies of each allele in a sample. For any given allele, say A_i, it derived by equating the observed frequency of homozygotes to that expected by the above equation

$$f_{ii}=\mathrm{E}[A_i{A}_i]=p_i^2+p_i(1-p_i)R$$

$$f_{ij}=\mathrm{E}[A_i{A}_j]=2p_i{p}_j(1-p_i)R.$$

The likelihood of the data, given R, $L(R)=\prod _{ij}{f}_{ij}^{X_{ij}}$. Solving for R gives estimators based upon pairs of alleles A,

$${\hat{R}}_{ii}={\displaystyle \frac{f_{ii}-p_i^2}{p_i(1-p_i)}.}$$

The estimate of R for allele i, estimates are combined across alleles as

$$\hat{R}={\displaystyle \sum _i}{w}_i{\hat{R}}_{ii},$$ (1)

where the weights w_i sum to unity.

These weights are found by finding the w_i that minimize ${\mathit{w}}^{T}Vw,$ where w is an n element vector of weights, and V the n × n variance–covariance matrix of allele-specific estimates (for details, see Ritland (1996)). The weights require prior specification of true relatedness. With zero prior R, the weight for allele A_i is $w_i=\frac{1-p_i}{n-1}$. An m-allele locus receives the weight (n_m − 1), giving the estimator for r given by equation 5 in Ritland (1996).

Probabilities of three-marker relationship

The three-marker relationship coefficient is the probability that three sampled marker genes are all identical-by-descent. From Equation (7) of Ritland (1987), which follows Kendall et al. (1977 eq. 13.36), The joint frequency of markers i, j, and k is

$$f_{ijk}={\mathit{\kappa }}_i{\mathit{\kappa }}_j{\mathit{\kappa }}_k+{\mathit{\kappa }}_i{\mathit{\kappa }}_j+{\mathit{\kappa }}_i{\mathit{\kappa }}_k+{\mathit{\kappa }}_j{\mathit{\kappa }}_k+{\mathit{\kappa }}_{ijk}.$$

This written in conventional population genetic terms as

$$f_{ijk}=p_i{p}_j{p}_k+p_i{p}_j{v}_f+(p_k{p}_l+p_j{p}_k)v_r+w_{ijk}.$$

Where alleles i and j are from one individual and allele k from a second individual. The cumulants are written in bold face to emphasize they have a random component that may covary.

From Equation (7) of Ritland (1987), there are three primary patterns

$$f_{iii}=\mathrm{E}[A_i{A}_i{A}_i]=p_i^3(1-F-2R+2G)+p_i^2(F+2R-3G)+p_{i}G.$$

$$f_{iij}=\mathrm{E}[A_i{A}_i{A}_j]=p_i^2{p}_j(1-F-2R+2G)+p_i{p}_j(F-G).$$

$$f_{ijk}=\mathrm{E}[A_i{A}_j{A}_k]=p_i{p}_j{p}_k(1-F-2R+2G).$$ (2)

Where the order is irrelevant ($A_i{A}_i{A}_j$, $A_j{A}_i{A}_j$, and $A_i{A}_j{A}_j$ are equivalent). The genotype frequencies are mixtures of marker gene identity: G is the probability that all three markers are ibd, R-G is the probability of ibd of one pair of markers, $F+2R-3G$ for two pairs, and 1-F-2R + 2G is the probability of no ibd among the three markers).

Solving for G in Equation (7) gives three probabilities involving G,

$${\hat{G}}_{iii}={\displaystyle \frac{f_{iii}-p_i^2(1-p_i)(F+2R)-p_i^3}{p_i(1-p_i)(1-2p_i)}}$$

$${\hat{G}}_{iij}={\displaystyle \frac{f_{iij}-p_i(1-p_i)p_j(F+2R)-p_i^2{p}_j}{p_i{p}_j(2p_i-3)}}$$

$${\hat{G}}_{ijk}={\displaystyle \frac{f_{ijk}-p_i{p}_j{p}_k(F+2R)}{2p_i{p}_j{p}_k}.}$$ (3)

G is a normalized third central moment and the normalization constant depends upon the pattern of subscript.

Each allele can provide an estimate of G, denoted ${\hat{G}}_i$, and its weighted estimate across possible alleles i is

$$\hat{G}={\displaystyle \sum _i}{w}_i{\hat{G}}_i.$$ (4)

This represents a “linear estimator” of G. The weights are derived in the Appendix. The best alternative to linear estimation is maximum likelihood.

Probabilities of four-marker relationship

From Equation (7) of Ritland (1987), which follows Kendall et al. (1977, eq. 13.36),

$$f_{ijkl}={\mathit{\kappa }}_i{\mathit{\kappa }}_j{\mathit{\kappa }}_k{\mathit{\kappa }}_l+{\mathit{\kappa }}_i{\mathit{\kappa }}_j{\mathit{\kappa }}_{kl}+{\mathit{\kappa }}_i{\mathit{\kappa }}_k{\mathit{\kappa }}_{jl}+{\mathit{\kappa }}_i{\mathit{\kappa }}_l{\mathit{\kappa }}_{jk}+{\mathit{\kappa }}_j{\mathit{\kappa }}_k{\mathit{\kappa }}_{il}+{\mathit{\kappa }}_j{\mathit{\kappa }}_l{\mathit{\kappa }}_{ik}+{\mathit{\kappa }}_k{\mathit{\kappa }}_l{\mathit{\kappa }}_{ij}$$

$$+{\mathit{\kappa }}_i{\mathit{\kappa }}_{jkl}+{\mathit{\kappa }}_j{\mathit{\kappa }}_{ikl}+{\mathit{\kappa }}_k{\mathit{\kappa }}_{ijl}+{\mathit{\kappa }}_l{\mathit{\kappa }}_{ijk}+{\mathit{\kappa }}_{ij}{\mathit{\kappa }}_{kl}+{\mathit{\kappa }}_{ik}{\mathit{\kappa }}_{jl}+{\mathit{\kappa }}_{il}{\mathit{\kappa }}_{jk}+{\mathit{\kappa }}_{ijkl}$$

The cumulants ${\mathit{\kappa }}_i$ are similar to moments and covariances and may have a random component that may covary with other cumulants. The subscripts indicate alleles. The recursion equation is

$$f_{ijkl}=p_i{p}_j{p}_k{p}_l+(p_i{p}_j+p_k{p}_l)v_f+(p_i{p}_l+p_j{p}_k+p_j{p}_l+p_k{p}_l)v_r+p_i{\omega }_{jkl}+p_j{\omega }_{ikl}+p_k{\omega }_{ijl}+p_l{\omega }_{jkl}+v_{ij}^2{v}_{kl}^2+v_{ik}^2{v}_{jl}^2+v_{il}^2{v}_{jk}^2+\mathrm{Cov}(v_{ij},v_{kl})+\mathrm{Cov}(v_{ik}{v}_{jl})+\mathrm{Cov}(v_{il}{v}_{jk})+{\mathit{\kappa }}_{ijkl}.$$ (5)

Where the v terms are second-order covariances. When there is a mixture model (which creates the covariances), each subpopulation m, contributes to the mean cumulant across pooled m. The term ${\kappa }_{i,m}{\kappa }_{j,m}{\kappa }_{k,m}{\kappa }_{l,m}$ contributes to all 18 population level moments, the term ${\kappa }_{i,m}{\kappa }_{j,m}{\kappa }_{kl,m}$ contributes to six population level moments, and so on. However, the quantitative extent of these contributions are complex and beyond treatment here. Regardless, that subpopulation cumulants “distill” to the same assortment of cumulants, albeit with perhaps slightly different values. where the covariance terms are across the mixture terms m. This is a finite mixture model, needed when a single component distribution is inadequate (McLachlan et al. 2019). These can get complex but Withers et al. (2015) does provide the first known expressions for cumulants used available computer technology (an equation solver), not available in 1987 to KR. Cumulants are allowed to vary across the mixture, and that this results in effective covariance between second-order cumulants. The expectations taken across m result in changes to the above expression due to associations among the ijkl across m, that causes the cumulants to be associated in a certain way, since for example, $E[{\kappa }_{i,m}{\kappa }_{j,m}]\ne {\kappa }_i{\kappa }_j$.

The associations between pairs of markers is termed identity disequilibrium. If one pair of alleles is heterozygous, it is more likely the second pair is also heterozygous. This is a four-gene marker measure that has been neglected due to inordinate attention to linkage disequilibrium. Identity disequilibrium has classically been characterized as the excess of homozygosity above that expected from the squared gene frequencies (Hill 1975; Ohta 1980). The identity excess is closely correlated to the expectation of the total squared linkage disequilibrium (Takahata 1982). Some of the problem is that haploid gametes are not directly assayed but rather imputed (Vitalis and Couvet 2001b).

We can add a cumulant to the equation for the probability of identity-by-state. From Equation 3.78 in Kendall et al. (1977), the moments about the mean for two squared random variables (the alleles present at each locus) equals

$$\mathrm{E}[p_i^2{p}_j^2]={\kappa }_{22}+{\kappa }_{20}{\kappa }_{02}+2{\kappa }_{11}^2$$

Whose form corresponds to $3{\sigma }_r^4$ for the fourth central moment with the difference that the cumulant ${\kappa }_{22}$ is added. Vitalis and Couvet (2001a) and others have given estimator for identity disequilibrium which omits this cumulant.

The four-allele case introduces higher-order associations and brings with it new statistical problems. Among four alleles, two new measures arise. The first is termed H and is the probability that all four alleles are identical-by-descent. The other two have not been recognized in the literature, perhaps because they invoke the existence of cumulants, which differ from the corresponding moments with products of four or more variates.

The first, termed $R_{AB}$, is the probability that both alleles in the first relatives are identical-by-descent to both alleles in the second relative. The second, termed $F_{AB}$, is the probability that both individuals have both marker genes identical-by-descent. Thus, the three unique four-allele measures are

$$H$$

$$F_{AB}=F_A{F}_B+\mathrm{Cov}(F_A,F_B)$$

$$R_{AB}=R^2+\mathrm{Cov}(R,R^c)$$ (6)

the covariances between second moments, $\mathrm{Cov}(F_A,F_B)$ and $\mathrm{Cov}(R_{AB},R_A^{\mathrm{\prime }}{B}^{\mathrm{\prime }})$, exist only when the distribution of gene frequency follows a mixture distribution where subpopulations vary for F and R.

We can rewrite Equation (5) as

$$\begin{array}{rl}{f}_{ijkl}=& (2-{\delta }_{ij})(2-{\delta }_{kl})[p_i{p}_j{p}_k{p}_l+({\delta }_{jl}{p}_i{p}_k{p}_j+{\delta }_{jk}{p}_i{p}_l{p}_j+{\delta }_{il}{p}_j{p}_k{p}_i\\ & +{\delta }_{ik}{p}_j{p}_l{p}_i-4p_i{p}_j{p}_k{p}_l)R+({\delta }_{ik}{\delta }_{jl}{p}_i{p}_j+{\delta }_{il}{\delta }_{jk}{p}_i{p}_j-{\delta }_{jl}{p}_i{p}_k{p}_j\\ & -{\delta }_{jk}{p}_i{p}_l{p}_j-{\delta }_{il}{p}_j{p}_k{p}_i-{\delta }_{ik}{p}_j{p}_l{p}_i+2p_i{p}_j{p}_k{p}_l)R_{AB}.\end{array}$$

$$\begin{array}{rl}& +p_k{p}_l({\delta }_{ij}{p}_i-p_i{p}_j)F_A+p_i{p}_j({\delta }_{kl}{p}_k-p_k{p}_l)F_B+({\delta }_{ij}{\delta }_{kl}{p}_i{p}_k-{\delta }_{kl}{p}_i{p}_j{p}_k\\ & -{\delta }_{ij}{p}_i{p}_k{p}_l+p_i{p}_j{p}_k{p}_l)F_{AB}+2({\delta }_{ijl}{p}_k{p}_{ijl}+{\delta }_{ijk}{p}_l{p}_{ijk}-p_i{p}_k{p}_{jl}-p_i{p}_l{p}_{jk}\\ & -p_j{p}_k{p}_{il}-p_j{p}_l{p}_{ik}-2p_k{p}_l{p}_{ij}+4p_i{p}_j{p}_k{p}_l)G_A+2({\delta }_{jkl}{p}_i{p}_j+{\delta }_{ikl}{p}_i{p}_k-p_i{p}_k{p}_{jl}\\ & -p_i{p}_l{p}_{jk}-p_j{p}_k{p}_{il}-p_j{p}_l{p}_{ik}-2p_i{p}_j{p}_{kl}+4p_i{p}_j{p}_k{p}_l)G_B\end{array}+\begin{array}{c}(p_{ijkl}-p_i{p}_{jkl}-p_j{p}_{ikl}-p_k{p}_{ijl}-p_l{p}_{ijk}+2p_i{p}_j{p}_{kl}+2p_i{p}_k{p}_{jl}+2p_i{p}_l{p}_{jk}\\ +2p_j{p}_k{p}_{il}+2p_j{p}_l{p}_{ik}+2p_k{p}_l{p}_{ij}-p_{ij}{p}_{kl}-p_{ik}{p}_{jl}-p_{il}{p}_{jk}-6p_i{p}_j{p}_k{p}_l)]\end{array}H,$$ (7)

where, for shorthand, $p_{ij}={\delta }_{ij}{p}_i$.

In this expression, there are eight relationship coefficients (R_AB, R_ABAB, F_A, F_B, F_AB, G_A, G_B, H), which in principle will specify eight different classes of marker genotypes. This probability of four alleles, $f_{ijkl}$, is then fitted to the observed frequencies in a sample.

For equations that solve for all eight parameters, the choice is somewhat arbitrary but a natural set of eight classes, in which identity-by-state mirrors the identity-by-descent, is: A_iA_iA_iA_i (all identical by state, or “ibs”), A_iA_kA_iA_k, A_iA_iA_kA_k, (two pairs ibs), A_iA_iA_iA_k, A_iA_kA_kA_k, (one triplet ibs) and A_iA_iA_jA_k, A_iA_jA_kA_k, and A_iA_jA_jA_k. (one pair ibs between A and B). Thus, we seek the expected frequencies in the vector (f_iiii, f_ijij, f_iijj, f_iiij, f_iijk, f_ijik).

The frequency of A_iA_iA_iA_i, is obtained from Equation (7), where all δ = 1 and all marker frequencies are p_i:

$$f_{iiii}=p_i\lceil p_i^3+p_i^2{q}_i(F_A+F_B+4R)+p_i{q}_i^2(F_{AB}+2R_{AB})+2p_i{q}_i(1-2p_i)(G_A+G_B)+q_i(1-6p_i{q}_i)H.$$

Likewise, the frequency that A and B are both heterozygous for A_i and A_j is, irrespective of order or phase,

$$f_{ijij}=4p_i{p}_j\lceil p_i{p}_j(1-F_A-F_B-F_{AB})+(p_i+p_j-4p_i{p}_j)R+(1-p_i-p_j+2p_i{p}_j)R_{AB}+(-p_i-p_j+4p_i{p}_j)(G_A+G_B)-(1-2p_i-2p_j+6p_i{p}_j)H$$

and homozygous for alternative alleles A_i and A_j is

$$f_{iijj}=2p_i{p}_j\lceil p_i{p}_j(1-4R+2R_{AB})+q_i{p}_j{F}_A+p_i{q}_j{F}_B+(1-p_i-p_j+p_i{p}_j)F_{AB}-2p_j(1-2p_i)G_A-2p_i(1-2p_j)G_B-(1-2p_i-2p_j+6p_i{p}_j)H$$

for triplets of identity-by-state $f_{iiij}$

$$f_{iiij}=2p_i{p}_j\lceil p_i^2+p_i{q}_i{F}_A-p_i^2{F}_B-p_i{q}_i{F}_{AB}+(p_i+p_j-4p_i{p}_j)R-(p_i+p_j-2p_i{p}_j)R_{AB}+(1-4p_i+4p_i{p}_j)G_A-2p_i(1-2p_j)G_B-(1-6p_i{q}_i)H$$

Finally, a single allele pair ii can be shared only within individual A,

$$f_{iijk}=2p_i{p}_j{p}_k\lceil p_i(1-4R+2R_{AB}-F_B+F_{AB})+q_i{F}_A-2(1-2p_i)G_A+4p_i{G}_B+2(1-3p_i)H$$

or shared only once between A and B:

$$f_{ijik}=4p_i{p}_j{p}_k\lceil p_i(1-F_A-F_B+F_{AB})+(1-4p_i)R-(1-2p_i)R_{AB}-(1-4p_i)(G_A+G_B)+2(1-3p_i)H.$$

The expressions for $f_{ijjj}\mathrm{and}{f}_{ijkk}$ are obtained by symmetry, and the expression for $f_{ijjk}$ is summed over all four pairings of j between A and B: $f_{ijjk},f_{ijkj},f_{jijk},\mathrm{and}{f}_{jikj}$.

The appendix gives the 8 × 8 matrix of probabilities of observing the marker gene frequencies f given the relatedness coefficients. Of course, this depends upon the particular array of f's used (there are others than the above). In this case, the determinant of the matrix is

$$\begin{array}{rl}& 512p_i^8{p}_j^7{p}_k^3(p_i-1)(96p_i^7-160p_i^6-4p_i^5(24p_j^2-16p_j-9)\\ & +3p_i^4(32p_j^2-80p_j+47)-p_i^3(132p_j^2-302p_j+147)\\ & +p_i^2(89p_j^2-154p_j+59)+\mathrm{p}(1-p_j)(23p_j-11)+(p_j-1)(2p_j-1))\end{array}$$

That it is nonzero indicates all 8 parameters are jointly estimable, but a linear approach which uses residuals to simplify things is needed at this point.

Joint values of H and identity disequilibrium

$$\left[\begin{array}{c}{f}_{AAAA}^{\mathrm{\prime }}\\ f_{AAaa}^{\mathrm{\prime }}\end{array}\right]=\left[\begin{array}{cc}p(1-p)(1-6p(1-p))& p^2{(1-p)}^2\\ -2pq(1-2p-2q+6pq)& 2p^2{q}^2\end{array}\right]\left[\begin{array}{c}H\\ F_{ab}\end{array}\right]$$

whose solution is

$$\hat{H}={\displaystyle \frac{q^2{f}_{AAAA}^{\mathrm{\prime }}-{(1-p)}^2{f}_{AAaa}^{\mathrm{\prime }}}{pq(1-p)(1-2p)(1-p-q)}}$$

$$\widehat{F_{ab}}={\displaystyle \frac{q(1-2p-2q+6pq)f_{AAAA}^{\mathrm{\prime }}-(1-p)(1-6p(1-p))f_{AAaa}^{\mathrm{\prime }}}{pq(1-p)(1-2p)(1-p-q)}.}$$

Joint estimates of H and joint identity disequilibrium

$$\left[\begin{array}{c}{f}_{AAAA}\\ f_{AaAa}\\ f_{AAaa}\end{array}\right]=\left[\begin{array}{ccc}p(1-p)(1-6p(1-p))& p^2{(1-p)}^2& 2p^2{(1-p)}^2\\ -4pq(1-2p-2q+6pq)& 4p^2{q}^2& 4pq((1-p)(1-q)+pq\\ -2pq(1-2p-2q+6pq)& 2pq(1-p-q+pq)& 4p^2{q}^2\end{array}\right]\left[\begin{array}{c}H\\ F\\ R\end{array}\right]$$

$$\hat{H}={\displaystyle \frac{2q(1-p-q+3pq){f^{\mathrm{\prime }}}_{AAAA}-p{(1-p)}^2(f_{AAaa}^{\mathrm{\prime }}+{f^{\mathrm{\prime }}}_{AaAa})}{2pq(1-p)(1-p-q)(1-3p)}}$$

$$\widehat{F_{ab}}={\displaystyle \frac{2q(1-2p-2q+6pq){f^{\mathrm{\prime }}}_{AAAA}+(1-p)(2p^2-4p+1)f_{AAaa}^{\mathrm{\prime }}+p(1-2p)f_{AaAa}^{\mathrm{\prime }}}{2pq(1-p)(1-p-q)(1-3p)}}$$

$$\widehat{R_{abab}}={\displaystyle \frac{4q(1-2p-2q+6pq){f^{\mathrm{\prime }}}_{AAAA}+(1-p)p(1-2p){f^{\mathrm{\prime }}}_{AAaa}-(4p^2-5p+1)f_{AaAa}^{\mathrm{\prime }}}{2pq(1-p)(1-p-q)(1-3p)}.}$$

The denominator shows that at least three alleles required in the population, and marker frequencies of $p_i$ = 1/3 are noninformative.

Identity disequilibrium has classically been characterized as the excess of homozygosity above that expected from the squared gene frequencies, as proposed by Hill (1975) and Ohta (1980). The identity excess is closely correlated to the expectation of the total squared linkage disequilibrium (Takahata 1982). Some of the problem is that haploid gametes are not directly assayed but rather imputed (Vitalis and Couvet 2001b).

The “classic” procedure for estimating identity disequilibrium involves comparing observed vs expected double homozygotes. From Vitalis and Couvet (2001a), one such estimator can be written in the form,

$$\widehat{R_{AB}}={\displaystyle \frac{a{f}_{AAAA}-b{f}_{AA,aa}}{c},}$$

where $a=6(6pq-2p-2q+1),b=(1-6pq),c=6pq(1-p-q)$  $(1-3p)$. The numerator is positive as a is always larger than b, but denominator can be either positive or negative and negative values occurs when b is larger an a. Such is life with four genes. It is also amazing the four-gene frequencies f_AAAA and f_AA,aa are directly used in an effective two gene estimator, as f_AAAA is the observed double homozygotes and f_AA,aa represents homozygotes expected with no zygotic association. Equally amazing is that Vitalis and Couvet (2001a) and others have given estimators for identity disequilibrium which omits this cumulant.

Discussion and Conclusion

A main feature of higher-order relatedness is the covariance of homozygosity between pairs of marker genes, this is effectively a covariance of second moments. Such a “covariance of covariance” arises when pedigrees occur in a mixture distribution (McLachlan et al. 2019). Such a distribution generates the genomic variation of homozygosity necessary for the existence of covariance of homozygosity between individuals at specific loci. The simplest mixture distribution is that of two populations with gene frequency p + a and p − a; in this case, covariance of heterozygosity, after mixing in equal proportions, equals a⁴ + 6a²p².

Another feature of higher-order relatedness is that the four-marker coefficient of gene identity must be described with cumulants and not moments. As an example of the necessity of cumulants, for population gene frequency p, the fourth central moment for four markers, denoted $(X_1,X_2,X_3\mathrm{and}{X}_4)$, is ${\sigma }_{12}^2{\sigma }_{34}^2+{\sigma }_{13}^2{\sigma }_{24}^2+{\sigma }_{23}^2{\sigma }_{13}^2+{\kappa }_4$, where ${\sigma }_{ij}^2$ is the covariance of $X_i$ and $X_j$ and ${\kappa }_4$ is a fourth-order cumulant which does not appear in the moment. Some type of term (not involving the product of variances) is needed for ${\kappa }_4$ and it could be any rational number. In summary, incorporating cumulants into four marker measures only requires some value X in the expansion of the fourth central moment ${\sigma }_{12}^2{\sigma }_{34}^2+{\sigma }_{13}^2{\sigma }_{24}^2+{\sigma }_{23}^2{\sigma }_{13}^2+X$ and this X is numerically estimated in the same way as the lower order cumulant terms.

Cumulants do have useful properties for models of quantitative traits, the most important is that the cumulant of the sum of two random variables X + Y is M(X + Y)=M(X) + M(Y); differential equations for models of selection on quantitative traits that involve cumulants are simpler than models involving moments (Burger 1991; Turelli and Barton 1994). This cumulant will also be key in deriving a marker-based estimator for Q_st (Ritland in prep) and for a portrayal of higher order population structure that separately accounts for both the correlation of relationship and the squared linkage disequilibrium (Ritland in prep).

We give probabilities of relationship for a homogenous population of just one generation. Such populations are most commonly assayed in genomics; however, it should be noted that the levels of nucleotide variation (for SNPs) is not high and loci with more than two alleles are uncommon; in fact, only about 5% of human SNPs are triallelic (Cao et al. 2015) although microsatellites and other types of repeat markers show greater variation. Reconstruction of pedigree relationship has traditionally involved cumbersome graph-tracing algorithms, and simpler recursive methods which require at least two generations of records (Karigl 1981; Thompson 1988; Whittemore and Halpern 1994). Also, current recursive methods (Zheng et al. 2018) assume a known pedigree (Kirkpatrick et al. 2019). This is somewhat like estimating Q_st with current methods, where aspects of the pedigree must be known.

Normalization constants

Relatedness coefficients are obtained by calculating the pairwise covariance of relatives and dividing it by a normalization constant that converts the covariance into a correlation. This constant is the maximum possible value that the covariance can take. For cases where pairwise comparisons involve the frequency of identical genotypes, it is simple to calculate as a binomial variance. For the two-marker relationship coefficient as described in Equation (1), the maximum covariance between two genotypes, conditioned upon observing allele i, is $\mathrm{E}[A_i{A}_i]-\mathrm{E}[A_i{]}^2=$  p_i(1 − p_i) when R = 1. Likewise, the three marker coefficient has a normalization constant of p_i(1 − p_i)(1 − 2p_i) for $A_i{A}_i{A}_i$, and the four-marker coefficients are normalized by p_i(1 − p_i)(1 − 6p_i(1 − p_i)) for $A_i{A}_i{A}_i{A}_i$. The normalization constants for combinations of alleles falls out of the analyses.

Ackerman et al. (2017) provided a different set of normalization constants for three and four marker measures than given here. In their Equation 6, they normalized the third central moment by the geometric mean gene frequency of the three central moments (rather than by p(1 − p)(1 − 2p) as done here). Their justification was a similarity of this “third moment correlation” to a bivariate correlation formula. Their normalization constant for the four-marker coefficient (Equation 8) involves a parameter α that mixes the unknown proportions of the two types of higher order identity (all identical vs 2 pairs identical), resulting in an inference that may be subject to biases.

Estimation

Table 2 gives example estimates of R, G, and H at a single locus. As expected, the variances decrease with numbers of alleles. While the variances for R are reasonable, those for H and the variances for the identity disequilibrium R_AB are quite large. Interestingly, and not remarkably, the variances for G are all over the place and it is not even estimate in one case (P = 0.5).

Table 2.

Allele states and denominators of cumulants.

Allele states	Expectation	Denominator
Two alleles
$i=j$	$p_i^2+p_i(1-p_i)F$	$p_i(1-p_i)$
$i\ne j$	$2p_i{p}_j(1-F)$	$-2p_i{p}_j$
Three alleles
$i=j=k$	$p_i^3+p_i^2(1-p_i)(F+2R)+p_{i}G$	$p_i(1-p_i)(1-2p_i)$
$i=j\ne k$	$p_i^2{p}_k(2R-2G)$	$-p_i{p}_k(1-2p_i)$
$i\ne j\ne k$	$p_i{p}_j{p}_k(1-F-2R+2G)$	$2p_i{p}_j{p}_k$
Four alleles
$i=j=k=l$	$p_i^2{q}_i^2(F_{AB}+2R_{AB})+q_i(1-6p_i{q}_i)H$	$p_i(1-p_i)(1-6p_i(1-p_i))$
$i=j=k\ne l$	$2p_i{p}_j(F_{AB}+(p_i+p_j-2p_i{p}_j)R_{AB}-(1-6p_i{q}_i)H)$	$p_i{p}_k(1-2p_i-2p_k+6p_i{p}_k)$
$i=j\ne k=l$	$2p_i{p}_j(2p_i{p}_j{R}_{AB}+(1-p_i-p_j+p_i{p}_j)F_{AB}-(1-2p_i-2p_j+6p_i{p}_j)H)$	$p_i{p}_k(1-2p_i-2p_k+6p_i{p}_k)$
$i=j\ne k\ne l$	$2p_i{p}_j{p}_k(p_i(2R_{AB}+F_{AB})+2(1-3p_i)H)$	$2p_i{p}_j{p}_k(1-3p_i)$
$i\ne j\ne k\ne l$	$p_i{p}_j{p}_k{p}_l(1-4R-2F+4G-8H)$	$-6p_i{p}_j{p}_k{p}_l$

Likelihood requires numerical solutions which introduces complications, as the numerical solution is normally iterated until convergence. We used the expectation–maximization method which has slow convergence. When using likelihood, I found the number of marker loci needed for adequate convergence was about 20 loci for three-gene coefficients and roughly double at 30–50 loci for four gene coefficients. Interestingly, it was found that loci with fewer alleles are more likely to give convergent estimates because the problem with nonconvergence arises when relatives do not share the same marker allele.

Calculating the probabilities of higher-order relationship poses an interesting set of obstacles. Equation solvers such as Derive or Mathematica help with the derivation and interpretation of complicated formulae. The coefficients can also undefined at certain intermediate frequencies (p = 1/2 or 1/3) and show high statistical variance about those frequencies (Ritland 1987).

Possible future approaches

The complications of correctly estimating population structure are discussed and treated by Weir and Goudet (2017). They developed two-marker moment estimators that can describe the “relativity” and this requires an explicit reference population. They develop their estimator in a multilevel approach (within individuals, between individual within populations, and between populations) which promoted a unified treatment of relatedness and population structure. Clearly, further progress will depend upon adequate definitions and applications of models.

“Relatedness mapping” (Albrechtsen et al. 2009) uses relatedness to identifying causative mutations, using the principle that affected individuals share higher relatedness about the mutation. A somewhat related activity is “IBD mapping”, in which segments of identity-by-descent (IBD) present in high-density genomic data are used to map casual variants (Browning and Browning 2012). However, the data by itself only reveal the presence of the variant.

Other fields have adopted the use of cumulants, which may show new approaches that population genetics can undertake. The central 4th cumulant has been used to detect early stages of termite infestation, as it can separate termite alarm signals from background noise (de la Rosa and Moreno Muñoz 2008). Advances in electrophysiological and imaging techniques are used to study the synchrony of neuron cell firings in the brain (Staude et al. 2010b ) and have highlighted the need for correlation measures that go beyond simple pairwise analyses, taking advantage of the “interaction property” of higher order cumulants as measures of correlation (Staude et al. 2010a ). In information systems, a “covariance of covariance” approach for individual pixels has been developed for image description and classification (Serra et al. 2009).

Data availability

All data necessary for confirming the conclusions of the article are present within the article's text, figures, and tables.

Funding

This work was supported by NSERC discovery grants to KR.