Inbreeding and the Effective Number of Allele Copies

Why Inbreeding Changes Frequency Uncertainty

For a diploid sample, each individual contributes two allele copies at a locus. If those copies behaved like independent draws from the population, $n$ individuals would contribute $2n$ independent copies. Inbreeding makes the two copies within an individual more alike than independent draws would be. The expected allele frequency can remain the same, but the copies provide less independent information about it.

This tutorial derives that reduction in information for a biallelic locus. The key quantity is the inbreeding coefficient $F$, which describes the dependence between the two homologous alleles. We will treat $F$ and the population allele frequency as specified model quantities. The goal is to understand how they determine genotype proportions, sampling variance, and the effective number of allele copies.

Definition: Consider a diploid population and a biallelic locus. Let $p$ be the population frequency of the allele of interest and let $q=1-p$ be the frequency of the other allele. The inbreeding coefficient $F$ is the probability that the two alleles in an individual are identical by descent relative to a specified reference population. Under the model used here, with probability $F$ the two alleles share one ancestral draw, and with probability $1-F$ they are independent draws with allele frequency $p$. We take $0\leq F\leq 1$.

Identity by descent is a statement about the alleles’ ancestry, not merely whether they happen to have the same observed allele. Two alleles can both carry the allele of interest without being identical by descent. In the model above, $F=0$ gives independent allele draws within an individual; positive $F$ creates positive dependence.

Genotype Proportions Under Inbreeding

Write the genotype as the number of copies of the allele of interest: 0, 1, or 2. Under the shared-ancestral-draw part of the model, the two copies are both the allele of interest with probability $p$, and both are the other allele with probability $q$. Under the independent-draw part, the usual probabilities for 0, 1, and 2 copies are $q^2$, $2pq$, and $p^2$. Weighting these two possibilities by $F$ and $1-F$ gives the genotype proportions.

\begin{aligned} P(G=2) &= Fp+(1-F)p^2 = p^2+Fpq,\\ P(G=1) &= (1-F)2pq,\\ P(G=0) &= Fq+(1-F)q^2 = q^2+Fpq. \end{aligned}

These probabilities sum to one: $p^2+2pq+q^2+2Fpq-2Fpq=(p+q)^2=1$. The heterozygote proportion is reduced from $2pq$ to $2pq(1-F)$, while each homozygote proportion increases by $Fpq$. The marginal frequency of the allele of interest is still $p$, because the expected number of its copies per individual is

$$ 2P(G=2)+P(G=1) =2(p^2+Fpq)+2pq(1-F) =2p. $$

Worked Example: Genotype Proportions at a Biallelic Locus

Suppose the allele of interest has population frequency $p=0.30$, so $q=0.70$, and the inbreeding coefficient is $F=0.20$. Find the three genotype proportions and compare the heterozygote proportion with the value when $F=0$.

For two copies of the allele of interest,

$$ P(G=2)=p^2+Fpq =(0.30)^2+(0.20)(0.30)(0.70) =0.090+0.042=0.132. $$

For one copy,

$$ P(G=1)=2pq(1-F) =2(0.30)(0.70)(0.80) =0.336. $$

For zero copies,

$$ P(G=0)=q^2+Fpq =(0.70)^2+0.042=0.532. $$

The probabilities add to $0.132+0.336+0.532=1.000$. The expected allele frequency is $\{2(0.132)+0.336\}/2=0.300$, as required. With $F=0$, the heterozygote proportion would be $2pq=0.420$; at $F=0.20$, it is $0.336$, a reduction of $0.084=Fpq\cdot2$.

From Allele Dependence to Dosage Variance

Let $Y_1$ and $Y_2$ be indicators for whether the first and second allele copies are the allele of interest: each indicator is 1 for that allele and 0 otherwise. The allele dosage is $G=Y_1+Y_2$. Each indicator has mean $p$ and variance $pq$. Under the inbreeding model, the covariance between the two indicators is $Fpq$: in the shared-draw case the indicators are identical, and in the independent-draw case their covariance is zero. Therefore the variance of dosage is

\begin{aligned} \operatorname{Var}(G) &=\operatorname{Var}(Y_1)+\operatorname{Var}(Y_2) +2\operatorname{Cov}(Y_1,Y_2)\\ &=pq+pq+2Fpq\\ &=2pq(1+F). \end{aligned}

This result can also be checked directly from the genotype probabilities. Since $G$ takes values 0, 1, and 2, its mean is $2p$, and its second moment is

\begin{aligned} E(G^2) &=P(G=1)+4P(G=2)\\ &=2pq(1-F)+4(p^2+Fpq). \end{aligned}

Subtracting $\{E(G)\}^2=4p^2$ gives $E(G^2)-\{E(G)\}^2=2pq(1+F)$, the same variance. Inbreeding does not change the expected dosage $2p$, but it increases its variance by the factor $1+F$.

Result: If $n$ sampled individuals are independent of one another, all have the same allele frequency $p$ and inbreeding coefficient $F$, and their true diploid genotypes are observed, the allele-frequency estimator $\hat{p}=\sum_{i=1}^{n}G_i/(2n)$ has expectation $p$ and variance $$ \operatorname{Var}(\hat{p})=\frac{pq(1+F)}{2n}. $$ The factor $1+F$ accounts for dependence between the two allele copies within each individual. Independence between individuals is a separate assumption.

To see the variance result, the total allele count is $T=\sum_{i=1}^{n}G_i$. Independence between individuals gives $\operatorname{Var}(T)=n\operatorname{Var}(G)=2npq(1+F)$. Since $\hat{p}=T/(2n)$, dividing the variance of $T$ by $(2n)^2$ yields $pq(1+F)/(2n)$. Thus the standard error under this model is the square root of that variance.

Worked Example: Variance and Standard Error With Inbreeding

Suppose $n=500$ unrelated sampled individuals are diploid at the locus, with $p=0.20$, $q=0.80$, and $F=0.10$. Assume the genotypes are observed without error and the same $p$ and $F$ apply to every sampled individual. Calculate the variance and standard error of the allele-frequency estimator.

There are $2n=1{,}000$ sampled allele copies, but the two copies within each individual are correlated. The model variance is

\operatorname{Var}(\hat{p}) =\frac{pq(1+F)}{2n} =\frac{(0.20)(0.80)(1.10)}{1{,}000} =\frac{0.176}{1{,}000} =0.000176.

The standard error is

SE(\hat{p})=\sqrt{0.000176}=0.01327.

For comparison, treating all 1,000 copies as independent would give variance $(0.20)(0.80)/1{,}000=0.000160$ and standard error $\sqrt{0.000160}=0.01265$. That calculation understates uncertainty because it ignores the within-individual dependence. The assumptions about a common $p$, common $F$, independent individuals, diploidy, and accurate genotype calls are part of the calculation; a sample with related individuals or population structure may need a different variance model.

The Effective Number of Allele Copies

An effective number of allele copies is a variance-equivalent quantity. It is the number of independent allele draws that would give the same allele-frequency variance as the observed sample under the inbreeding model. If $m_{\mathrm{eff}}$ denotes this number, independent draws have variance $pq/m_{\mathrm{eff}}$. Equating that to $pq(1+F)/(2n)$ gives

m_{\mathrm{eff}}=\frac{2n}{1+F}.

Definition: Under the specified diploid inbreeding model, the effective number of allele copies is $m_{\mathrm{eff}}=2n/(1+F)$. It is the number of independent allele copies that has the same sampling variance for $\hat{p}$, not a literal count of observed copies. For $F=0$, $m_{\mathrm{eff}}=2n$; for positive $F$, $m_{\mathrm{eff}}<2n$.

The corresponding variance is $pq/m_{\mathrm{eff}}$. This equivalence is useful because it expresses the effect of within-individual dependence in the familiar form for independent allele draws. It does not mean that the sample contains fewer physical alleles, nor does it by itself account for dependence between different individuals, unequal sampling, uncertain genotypes, or other sources of variation.

Worked Example: Effective Copies and a Target Standard Error

A study plans to sample diploid individuals at a locus with $p=0.25$, $q=0.75$, and $F=0.25$. How many individuals are needed for the model-based standard error of $\hat{p}$ to be at most $0.0100$? Assume independent individuals, a common $F$, and accurately observed genotypes.

The variance must be no greater than $0.0100^2=0.000100$. Since $pq=(0.25)(0.75)=0.1875$, the effective number of copies must satisfy

\frac{pq}{m_{\mathrm{eff}}}\leq 0.000100 \quad\Longrightarrow\quad m_{\mathrm{eff}}\geq\frac{0.1875}{0.000100}=1{,}875.

With $n$ diploid individuals, $m_{\mathrm{eff}}=2n/(1.25)$. Therefore,

\frac{2n}{1.25}\geq1{,}875 \quad\Longrightarrow\quad n\geq\frac{(1{,}875)(1.25)}{2}=1{,}171.875.

The sample size must be a whole number, so the minimum is $n=1{,}172$ individuals. This gives $m_{\mathrm{eff}}=2(1{,}172)/1.25=1{,}875.2$, and $SE(\hat{p})=\sqrt{0.1875/1{,}875.2}=0.0099995$, which is just below the target. The answer depends on the specified allele frequency and inbreeding coefficient; if those inputs differ, the required sample size can differ too.

Interpretation and Assumptions

The central consequence is straightforward: positive inbreeding reduces heterozygosity and increases the variance of allele counts within individuals. For the same number $n$ of unrelated diploid individuals, the frequency estimator has $1+F$ times the variance it would have under independent allele copies. Because variance is proportional to the reciprocal of sample size, the effective number of copies is reduced by dividing $2n$ by $1+F$.

These equations rely on a specific sampling and genetic model. They assume a biallelic locus, diploid individuals, one common population frequency $p$, one common inbreeding coefficient $F$, independent sampled individuals, and correctly observed genotypes. If individuals are related to one another, their allele counts may also be dependent across individuals; the calculation above does not correct for that. Likewise, mixing populations with different allele frequencies can create additional variation that is not represented by this single-$p$, single-$F$ expression.

Common pitfall: “There are $2n$ physical allele copies, so the frequency variance must be $pq/(2n)$.” This treats the two copies within every individual as independent. Under the inbreeding model their covariance is $Fpq$, so the correct variance is $pq(1+F)/(2n)$, and the variance-equivalent number of independent copies is $2n/(1+F)$.

The effective-copy calculation is a way to express uncertainty, not a replacement for checking the study design. In particular, it should not be used as if it automatically corrected for population structure, related samples, nonrandom missing genotypes, or uncertainty in genotype calls. Those features require their own modeling assumptions and adjustments.

Check Your Understanding

Use the model and definitions in this tutorial to answer the following questions.

For $p=0.40$ and $F=0.15$, calculate the expected heterozygote proportion and compare it with the value when $F=0$.
Explain why a positive $F$ increases the variance of an individual’s allele dosage without changing its expected value.
For $n=200$ diploid individuals and $F=0.20$, calculate the effective number of allele copies.
State the assumptions needed to use $\operatorname{Var}(\hat{p})=pq(1+F)/(2n)$.
Why is the effective number of allele copies not the same as the number of physical allele copies in the sample?