Skip to main content
. 2017 May 26;206(4):2085–2103. doi: 10.1534/genetics.116.198424

Table 1. Notation.

Quantity Notation
Allele Ajki for allele k{1,2}, individual j{1,2,.ni}, population i{1,2,r}
Allelic indicator xjkui for allele Ajki being of type u
Allele frequency πu expected value of xjkui for all i,j,k
piu actual frequency for allele type u in population i
piu observed frequency for allele type u in sample from population i
Theta θjk,jkii is probability of ibd between allele k in individual j from population i and allele k in individual j from population i
Inbreeding coefficient Fji is the ibd probability for the two alleles for individual j in population i: Fji=12k=12k=1,kk2θjk,jkii
Coancestry coefficient Coancestry θjji is the ibd probability for a pair of alleles drawn from individuals j,j in population i: θjji=14k=12k=12θjk,jkii. θSi is the average of θjji for all pairs j,j. θS is the average over populations of θSi. For any two distinct alleles drawn from population i, the ibd probability is θWi. The average over populations of θWi is θW. θB is the average of ibd probabilities for alleles from different populations
Relative inbreeding The relative inbreeding coefficient for individual j in population i is βji=(FjiθR)/(1θR): reference θR is θSi or θB
Relative coancestry The relative coancestry coefficient for individuals j,j in population i is βjji=(θjjiθR)/(1θR): reference θR is θSi or θB
Population-specific FST βWTi=(θWiθB)/(1θB) is probability two alleles drawn from population i are ibd, relative to the probability an allele drawn from one population is ibd to an allele drawn from another population. βSTi=(θSiθB)/(1θB) is for alleles drawn from two individuals in population i