What Observed Heterozygosity Measures
A genotype call at a diploid locus is heterozygous when the two called alleles differ, such as \(A/G\), and homozygous when they are the same, such as \(A/A\). A sample’s observed heterozygosity summarizes how often its called genotypes are heterozygous across a specified set of variants. It is a descriptive fraction of observed calls, not a statement about why those genotypes occur.
The calculation depends on two choices that must be made explicit: which variants are eligible for inclusion, and which of those variants have a called genotype for the sample. The denominator is the number of eligible variants with a call for that sample. A missing genotype contributes neither a heterozygote nor a homozygote because its genotype status is unknown.
The notation \(j:C_{ij}=1\) means that the numerator sums only over variants where sample \(i\) has a call. This matters: \(H_{ij}\) is defined only for a called genotype. For a missing call, it is not appropriate to multiply an undefined \(H_{ij}\) by \(C_{ij}=0\). Instead, the formula explicitly restricts the numerator to called variants.
The result is a proportion between 0 and 1, often reported as a percentage. For example, \(0.25\) means that 25% of the sample’s called genotypes in the specified set are heterozygous. It does not mean that 25% of all possible sites in the genome are heterozygous; the value depends on the variant set and call status.
Choosing the Variant Set and Denominator
For a straightforward calculation, use a defined set of eligible, usually autosomal, diploid variants and classify each called genotype as heterozygous or homozygous. The same variant set should be used when comparing samples unless the purpose is specifically to describe each sample’s own callable set. Multiallelic diploid genotypes can also be classified as heterozygous when the two alleles differ, but the variant set and genotype representation must be consistent.
Do not mix loci with different ploidy into this calculation without an explicitly appropriate ploidy rule. For example, a haploid genotype has no pair of alleles to classify as heterozygous or homozygous in the same way as a diploid genotype. A practical analysis can restrict the calculation to comparable diploid loci or analyze different ploidy regions separately.
The denominator is not automatically the total number \(M\) of eligible variants. It equals \(M\) only if the sample has a called genotype at every eligible variant. If there are missing calls, using \(M\) would treat each missing genotype as though it were a non-heterozygote, which is not what observed heterozygosity measures. The earlier material on sample call rate covers the fraction of eligible variants that are called; that quantity is related to the denominator but is not the heterozygosity rate.
Worked Example: Calculating One Sample’s Rate
A hypothetical sample is evaluated at eight eligible diploid variants. Its calls are \(A/G\), \(C/C\), \(T/G\), missing, \(G/G\), \(C/T\), missing, and \(A/A\). Calculate its observed heterozygosity.
The called genotypes are \(A/G\), \(C/C\), \(T/G\), \(G/G\), \(C/T\), and \(A/A\): six calls in total. Of these, \(A/G\), \(T/G\), and \(C/T\) are heterozygous, so the numerator is three.
Substituting the counts into the definition gives $$ \widehat{H}_{\mathrm{obs}} = \frac{3\ \text{heterozygous calls}} {6\ \text{called genotypes}} = 0.5000. $$
The sample’s observed heterozygosity is 0.5000, or 50.00%, across these eight eligible variants’ called genotypes. The two missing genotypes are not in the denominator and are not counted as homozygous. Dividing by eight would give \(3/8=0.3750\), but that would be the heterozygote count divided by the eligible-variant count, not the observed heterozygosity as defined here.
Why the Formula Is a Mean
The definition can be justified by viewing each called genotype as an indicator: its value is 1 if heterozygous and 0 if homozygous. The mean of these indicators across a sample’s called loci is the number of ones divided by the number of indicators. If there are \(K_i\) called loci and \(Q_i\) heterozygous calls, then
This yields the ratio in the definition. Because each included \(H_{ij}\) is either 0 or 1, \(0\leq Q_i\leq K_i\), and therefore \(0\leq \widehat{H}_{i,\mathrm{obs}}\leq 1\). If \(K_i=0\), there are no observed genotype indicators to average, so the rate is undefined rather than zero.
Worked Example: Comparing Rates With Different Call Counts
At six eligible diploid variants, two hypothetical samples have the following calls:
| Variant | Sample A | Sample B |
|---|---|---|
| 1 | A/G | A/G |
| 2 | C/C | Missing |
| 3 | C/T | C/T |
| 4 | G/G | Missing |
| 5 | Missing | A/T |
| 6 | C/C | Missing |
Sample A has five called genotypes and two heterozygous calls, at variants 1 and 3. Thus $$ \widehat{H}_{A,\mathrm{obs}}=\frac{2}{5}=0.4000. $$ Sample B has three called genotypes and three heterozygous calls, at variants 1, 3, and 5. Thus $$ \widehat{H}_{B,\mathrm{obs}}=\frac{3}{3}=1.000. $$
On their respective called genotypes, Sample A’s rate is 40.00% and Sample B’s is 100.0%. However, Sample B has fewer calls, and its rate describes a different subset of loci from Sample A’s. The arithmetic is correct for each sample, but the comparison should be qualified: the rates are not based on identical observed loci. These figures alone do not establish why the samples differ.
Comparisons Require Comparable Calls
When the goal is to compare samples, using the same eligible variant list is a useful starting point, but it does not guarantee that their observed rates are based on the same loci. If missingness differs, a sample’s fraction may summarize a different subset of the eligible set. One diagnostic is to calculate rates on the loci called in every sample being compared, while also reporting how many loci remain. This makes the locus basis common but can substantially reduce the data available, especially with many samples.
A common set is not always the only useful summary. Reporting each sample’s rate over its own called genotypes preserves more observations, but comparisons should be read alongside call counts and missingness. A pooled fraction across all samples is also different from the average of sample-level rates: pooling gives more weight to samples with more calls, whereas an unweighted average of sample rates gives each sample equal weight. State which quantity is being reported.
Worked Example: Comparing on a Shared Called Set
Two hypothetical samples are called at different subsets of eight eligible variants. Their genotypes are:
| Variant | Sample A | Sample B |
|---|---|---|
| 1 | A/G | A/G |
| 2 | C/T | C/C |
| 3 | A/A | A/G |
| 4 | G/G | G/G |
| 5 | Missing | C/T |
| 6 | T/T | Missing |
| 7 | Missing | A/A |
| 8 | C/C | C/C |
Across each sample’s own calls, Sample A has six calls, two of which are heterozygous, so its rate is \(2/6=0.3333\). Sample B has seven calls, three of which are heterozygous, so its rate is \(3/7=0.4286\).
The shared called set consists of variants 1, 2, 3, 4, and 8. Sample A has heterozygous calls at variants 1 and 2, giving \(2/5=0.4000\). Sample B has heterozygous calls at variants 1 and 3, also giving \(2/5=0.4000\). On this common set the rates are equal, although the rates over each sample’s own calls differ. This example shows how the choice of denominator and the loci included can affect a comparison; it does not imply that either summary is universally preferable.
Diagnostics, Alternatives, and Limitations
Before interpreting or reporting a rate, check that genotype calls were classified consistently, that the eligible variant set is stated, and that the denominator equals the number of called eligible loci. Compare call counts or call rates alongside heterozygosity. If samples have different missingness patterns, inspect whether their rates rely on substantially different loci; a shared-call calculation may help assess sensitivity. A rate near 0 or 1 can be mathematically valid, but should prompt a check of the calls and denominator rather than an automatic conclusion about sample quality.
Several quantities can be confused with observed heterozygosity. The proportion of eligible loci called measures call completeness, not the fraction of called genotypes that are heterozygous. Genotype frequencies at one locus summarize samples at that locus, whereas a sample-level observed heterozygosity summarizes one sample across a specified set of loci. Expected heterozygosity is a different quantity based on allele frequencies; it is not calculated by counting the sample’s heterozygous calls. The earlier material on Hardy-Weinberg diagnostics addresses a different question and should not be substituted for this direct genotype-count summary.
Observed heterozygosity is also sensitive to which variants are included. A set enriched for particular allele frequencies can yield a different rate from another set, even if the same samples are analyzed. Thus, for meaningful comparisons, specify the variant inclusion rules and use consistent genotype processing. The rate is descriptive: by itself, it does not identify a biological or technical cause of a difference, nor does it provide a formal significance test.
Define the loci and ploidy convention before counting genotypes.
Include a variant in the denominator for a sample only when that genotype is called.
Among the called genotypes, count those with different alleles.
Divide the heterozygote count by the call count; state the rate, eligible set, and denominator.
Check Your Understanding
Answer each question, then compare with the worked solutions below.
- A sample has 12 eligible variants, 2 missing calls, and 4 heterozygous calls among the called genotypes. What is its observed heterozygosity?
- Why is an observed heterozygosity undefined when a sample has no called genotypes in the eligible set?
- Sample X has 8 heterozygous calls among 20 called loci; Sample Y has 6 among 12. Calculate and compare their rates.
- Two samples have different own-call rates but equal rates on the loci called in both. What does this tell you about the comparison?
Solutions. (1) There are \(12-2=10\) called genotypes, so \(\widehat{H}_{\mathrm{obs}}=4/10=0.4000\), or 40.00%. (2) The denominator is zero, so there are no observed genotype indicators to average; a zero rate would incorrectly assert that homozygous calls were observed. (3) Sample X has \(8/20=0.4000\), or 40.00%; Sample Y has \(6/12=0.5000\), or 50.00%. These rates describe their respective called genotypes; call counts and locus overlap matter when comparing them. (4) The own-call rates summarize different observed subsets, while the common-locus calculation uses the same loci for both samples. The difference in the own-call rates may therefore depend on which loci were callable, and the shared-set result gives a comparison on a common basis, with fewer loci potentially available.