Inbreeding Coefficient And Hardy–Weinberg Departures

Why Measure a Heterozygote Deficit?

At a biallelic locus, allele frequencies determine the Hardy–Weinberg expected genotype frequencies. In particular, if allele \(A\) has frequency \(p\) and allele \(a\) has frequency \(q=1-p\), the expected heterozygote frequency under Hardy–Weinberg equilibrium is \(2pq\). A population can have the same allele frequencies but a different proportion of heterozygotes. The inbreeding coefficient gives a compact way to describe that difference relative to the Hardy–Weinberg expectation.

The coefficient quantifies a departure; by itself, it does not identify the cause. A positive value describes fewer heterozygotes than expected at the specified allele frequencies. A negative value describes more. Population structure, mating patterns, or other processes may contribute to a departure, but the coefficient alone does not distinguish among explanations.

The word relative matters: \(F\) is not the absolute difference \(H_E-H_O\). For example, if the expected heterozygote frequency is \(0.40\) and the observed frequency is \(0.30\), the absolute deficit is \(0.10\), while the relative deficit is \(0.10/0.40=0.25\). The coefficient says the heterozygote frequency is 25% below its Hardy–Weinberg expectation.

This definition applies when both alleles are present, so that \(0<p<1\) and \(H_E=2pq>0\). If either allele has frequency zero, the expected heterozygote frequency is zero and the relative-deficit ratio is undefined. In that case, there is no positive Hardy–Weinberg heterozygote expectation against which to measure a relative deficit.

Genotype Frequencies in Terms of \(p\), \(q\), and \(F\)

Write \(f_{AA}\), \(f_{Aa}\), and \(f_{aa}\) for the population frequencies of the three genotypes. The heterozygote frequency follows directly from the definition of \(F\): it is the Hardy–Weinberg expectation multiplied by \(1-F\). Given that heterozygote frequency and the allele frequencies, the two homozygote frequencies are determined by the allele marginals.

Here is an algebraic justification of the formulas. By the definition of \(F\), the heterozygote frequency is \(H_O=2pq(1-F)\). Each \(Aa\) individual contributes one \(A\) allele and one \(a\) allele, so half the heterozygote frequency contributes to each allele's marginal frequency. Therefore \(p=f_{AA}+H_O/2\) and \(q=f_{aa}+H_O/2\). Substituting the expression for \(H_O\) gives

The final equalities use \(1-q=p\) and \(1-p=q\). These expressions preserve the allele frequencies: \(f_{AA}+f_{Aa}/2=p\) and \(f_{aa}+f_{Aa}/2=q\). They also sum to one, since

Thus, at fixed allele frequencies, increasing \(F\) shifts frequency away from heterozygotes and into both homozygote classes. The amount added to each homozygote frequency is \(Fpq\), while the heterozygote frequency falls by \(2Fpq\). This is a description of genotype frequencies, not a claim about the biological mechanism that produced them.

Worked Examples

Interpreting the Coefficient and Checking Validity

The formula \(F=1-H_O/(2pq)\) gives a useful sign and scale interpretation:

• \(F=0\): the heterozygote frequency equals its Hardy–Weinberg expectation at the specified allele frequencies.
• \(F>0\): there is a relative heterozygote deficit. For instance, \(F=0.20\) means \(H_O\) is 20% below \(2pq\).
• \(F<0\): there is a relative heterozygote excess. For instance, \(F=-0.20\) means \(H_O=1.20(2pq)\), or 20% above the expectation.

If the coefficient is used as a signed measure of a population's heterozygote departure, it need not be restricted to values from zero to one. However, not every real value gives valid genotype frequencies. For \(0<p,q<1\), nonnegative heterozygote frequency requires \(F\leq1\); nonnegative homozygote frequencies require \(F\geq-p/q\) and \(F\geq-q/p\). Together, these conditions are

These bounds express a basic requirement: each genotype frequency must be between zero and one. In particular, a proposed heterozygote excess cannot exceed the amount possible given the allele frequencies. If \(F\) is being used specifically to describe a deficit, it is nonnegative by that description; the signed formula also accommodates excess for a broader description of departure.

When \(p\) and \(q\) are estimated from genotype counts, \(F\) is calculated relative to the Hardy–Weinberg expectation at those estimated allele frequencies. It is therefore important to use allele frequencies that correspond to the same population or sample as the heterozygote frequency. Comparing \(H_O\) with \(2pq\) calculated from unrelated allele frequencies would not measure the intended relative deficit.

Common Mistake

A frequent error is to call \(H_E-H_O\) the inbreeding coefficient. That quantity is the absolute heterozygote-frequency difference. The coefficient is the difference divided by the expected heterozygote frequency: \(F=(H_E-H_O)/H_E\). For example, an absolute deficit of \(0.10\) corresponds to \(F=0.25\) when \(H_E=0.40\), but to \(F=0.50\) when \(H_E=0.20\). The same absolute shortfall can represent different relative deficits.

A second mistake is to interpret a positive coefficient as proof of a particular cause. It establishes the direction and relative size of the heterozygote departure under this definition; it does not establish a cause or demonstrate that the departure is statistically distinguishable from sampling fluctuation.