Population Stratification as Allele-Frequency Variation

Why Allele-Frequency Differences Matter

A genotype data set records many variants for each person. Even when each variant is measured accurately, individuals need not come from one genetically homogeneous population. If the sampled groups differ in allele frequencies, their genotypes will differ systematically across the genome. Those patterns are not necessarily evidence of a technical error, nor do they require nearby variants to be linked: differences in group membership alone can create them.

The key idea is to distinguish variation within groups from variation between groups. At a particular variant, a group with a higher frequency of one allele will tend to have a higher average count of that allele per person. When several such differences occur across many variants, individuals from the same group may share a broad pattern of genotype values. This tutorial defines that structure, works through its arithmetic, and clarifies what it does—and does not—tell us.

Definition: For a biallelic variant, the allele frequency of a specified allele in a group is the proportion of observed chromosome copies in that group carrying that allele. With diploid genotype dosage \(G\) coded as the number of copies of that allele, \(G\) is 0, 1, or 2, and the sample allele frequency is \(\hat{p}=\sum_i G_i/(2n)\) when all \(n\) individuals have observed genotypes. Population stratification is the presence in a sample of subpopulations or ancestry groups whose allele frequencies differ at one or more variants. It can be discrete or continuous; the definition does not require groups to have sharp boundaries.

A Group Mixture Changes the Genotype Data

Let \(S\) denote an individual's group, and let \(p_s\) be the frequency of the counted allele in group \(s\). Under the dosage convention above, the expected genotype dosage in that group is \(E(G\mid S=s)=2p_s\): each of the two chromosome copies has probability \(p_s\) of carrying the allele. Thus, if two groups have different frequencies, their expected dosages differ as well.

The pooled frequency is a weighted average of the group frequencies, where the weights are the groups' shares of the sampled chromosomes. If all individuals contribute two observed chromosome copies, this is also a weighted average by sample size. A pooled frequency can therefore conceal substantial differences between groups.

Worked Example: Group-Specific and Pooled Allele Frequencies

Consider hypothetical genotype counts at one variant, with \(G\) counting copies of the allele of interest. Group A has 40 individuals: 18 with \(G=0\), 16 with \(G=1\), and 6 with \(G=2\). Group B has 60 individuals: 12 with \(G=0\), 30 with \(G=1\), and 18 with \(G=2\). Assume the genotypes are called correctly and observed for everyone.

In Group A, the number of counted allele copies is \(16+2(6)=28\), out of \(2(40)=80\) chromosome copies. Therefore, \(\hat{p}_A=28/80=0.350\), and the group's average dosage is \(28/40=0.700\).

In Group B, the allele-copy count is \(30+2(18)=66\), out of \(2(60)=120\), so \(\hat{p}_B=66/120=0.550\). Its average dosage is \(66/60=1.10\).

The pooled frequency is \((28+66)/(80+120)=94/200=0.470\). Equivalently, it is \((40/100)(0.350)+(60/100)(0.550)=0.470\). The pooled value accurately describes this particular mixture, but it does not describe either group: the group frequencies differ by 0.200, and their mean dosages differ by 0.400 allele copies per person.

The assumptions in that calculation are modest: the allele is consistently defined, the dosage represents its copy count, and the observed chromosomes are counted correctly. No Hardy–Weinberg assumption is needed to calculate allele frequency or mean dosage. Additional assumptions are needed if we want to predict the full genotype distribution from the allele frequency.

Model condition: Within a group, Hardy–Weinberg equilibrium at a biallelic variant means the two allele copies combine as if they were independently drawn with allele frequency \(p_s\). Under that condition, the probabilities for \(G=0,1,2\) are \((1-p_s)^2\), \(2p_s(1-p_s)\), and \(p_s^2\), respectively. This is a within-group model, not an automatic property of a pooled sample.

Worked Example: A Mixture Can Have Fewer Heterozygotes

Imagine a hypothetical sample made up equally of two groups. At one biallelic variant, their allele frequencies are \(p_1=0.200\) and \(p_2=0.800\). Assume Hardy–Weinberg equilibrium within each group and equal group sizes.

In Group 1, the expected genotype proportions for \(G=0,1,2\) are \(0.8^2=0.640\), \(2(0.2)(0.8)=0.320\), and \(0.2^2=0.040\). In Group 2, they are \(0.2^2=0.040\), \(2(0.8)(0.2)=0.320\), and \(0.8^2=0.640\). Averaging the proportions across the equally sized groups gives pooled proportions \(0.340\), \(0.320\), and \(0.340\).

The pooled allele frequency is \((0.200+0.800)/2=0.500\). If the pooled sample itself followed Hardy–Weinberg proportions at that frequency, its expected heterozygote proportion would be \(2(0.5)(0.5)=0.500\), not \(0.320\). The difference does not contradict the assumption of equilibrium within each group: pooling groups with different allele frequencies changes the combined genotype distribution. The calculation illustrates a possible pattern, not a universal diagnostic; other departures from the model can also affect genotype proportions.

How Group Membership Creates Patterns Across Loci

A genome-wide pattern can arise even if two variants are independent within each group. The reason is that group membership affects both genotypes. Once group membership is left unobserved, people from the same group tend to have group-specific expected dosages at both loci. This shared group label can create a marginal association between dosages.

The justification follows from the law of total covariance. For genotype dosages \(G_1\) and \(G_2\), condition first on group \(S\). If the dosages are independent within each group, then \(\operatorname{Cov}(G_1,G_2\mid S)=0\). The remaining covariance is the covariance of their group-specific means:

\operatorname{Cov}(G_1,G_2) = E\!\left[\operatorname{Cov}(G_1,G_2\mid S)\right] + \operatorname{Cov}\!\left(E[G_1\mid S],E[G_2\mid S]\right) = \operatorname{Cov}(2p_{1,S},2p_{2,S}).

The first term is within-group covariance; it is zero under the stated conditional-independence assumption. The second term is between-group covariance. It is positive if groups with higher mean dosage at one variant also tend to have higher mean dosage at the other, and negative if the group-specific means tend to move in opposite directions. Real genotype data need not satisfy conditional independence: linkage and other within-group relationships can also contribute covariance.

Worked Example: Between-Group Covariance at Two Independent Loci

Suppose a hypothetical sample has two equally sized groups and two variants. At Variant 1, the counted-allele frequencies are \(0.200\) and \(0.800\) in Groups 1 and 2. At Variant 2, they are \(0.800\) and \(0.200\), respectively. Assume dosage means are \(2p\), and assume the two dosages are independent within each group.

The group-specific expected dosages are \((0.400,1.600)\) at Variant 1 and \((1.600,0.400)\) at Variant 2. Each pooled mean is \(1.000\). The covariance of these group means, using equal weights, is

\frac{1}{2}(0.400-1.000)(1.600-1.000) + \frac{1}{2}(1.600-1.000)(0.400-1.000) = -0.360.

Because the within-group covariance is assumed to be zero, the total covariance is also \(-0.360\) dosage-squared. The negative value arises entirely from the groups' opposing frequency patterns; it does not imply that the variants are physically linked or negatively associated within either group. If both variants instead had frequencies \(0.200\) and \(0.800\) across the two groups, the group-mean covariance would be \(+0.360\): the deviations from each pooled mean would have the same sign in each group.

Across a genome, group-specific frequencies vary from variant to variant. Consequently, some variants can show similar group patterns and others opposite patterns; not every pair of loci will have the same covariance. The central point is that a sample's group composition can impose a systematic component on genotype data across many loci, over and above individual variation within groups.

Distinguishing Population Structure From Other Sources

An observed pattern across genotype columns has more than one possible explanation. Population structure is a biological feature of the sample: individuals come from groups with differing allele frequencies. Linkage disequilibrium, by contrast, is non-independence between alleles at different loci within a population. It can produce locus-to-locus association without group differences. The covariance example above isolates a different mechanism by assuming conditional independence within groups.

Technical batch effects are another alternative. If samples processed in different batches also differ in ancestry composition, a technical and a biological pattern may overlap. Earlier material on batch effects emphasizes that patterns tied to processing can be mistaken for biological differences, and vice versa. Genotype quality checks, sample processing records, and group-specific frequency summaries provide complementary evidence; no single broad pattern identifies its cause by itself.

Possible source	What varies	How it differs conceptually
Population stratification	Allele frequencies between groups of individuals	A shared group label shifts expected genotypes at many variants.
Linkage disequilibrium	Alleles at different loci within a population	Association can remain within a group; it does not require a mixture of groups.
Technical batch effect	Measured genotypes or call quality across processing batches	The source is measurement or processing, not necessarily biological frequency differences.
Sampling variation	Observed counts in a finite sample	Frequencies may differ by chance even when underlying group frequencies are equal.

Assumptions, Diagnostics, and Limits

Frequency comparisons are most interpretable when allele coding is consistent, genotype calls are sufficiently reliable, and the groups being compared are defined independently of the frequency pattern under investigation. With missing calls, the denominator should be the number of observed chromosome copies, rather than \(2n\) without adjustment. Frequency estimates from small groups are noisy, so an apparent difference may reflect sampling variation; estimates should be interpreted with their sample sizes and uncertainty in mind.

Useful checks include summarizing allele frequencies separately by known sampling groups, checking whether differences recur across many variants, and inspecting whether patterns align more closely with ancestry labels, collection sites, or processing batches. A few unusual variants may indicate locus-specific issues rather than broad structure. Conversely, weak differences at any one variant can still combine into a recognizable genome-wide pattern. These checks are clues, not proofs: group labels can be incomplete, and technical and biological sources can coexist.

Population groups are useful statistical descriptions, not claims that human genetic variation falls into a small set of perfectly separated categories. Frequencies can change gradually across geography, and an individual's ancestry can reflect contributions from more than one source. The simple group-mixture calculations are therefore teaching models. Their interpretation should not turn a frequency difference into a causal claim about a trait, nor should a broad genetic pattern be treated as a direct measure of any social category.

Common Pitfalls

Using only the pooled frequency. A pooled estimate can accurately summarize a mixture while hiding large group-specific differences. Compare frequencies by relevant group as well as overall.
Assuming pooled Hardy–Weinberg proportions from within-group equilibrium. Equilibrium within groups does not guarantee equilibrium after pooling groups with different frequencies.
Calling every cross-locus pattern linkage. Group membership can induce covariance between loci even when they are independent within each group. Conversely, linkage can exist without population stratification.
Treating a frequency difference as evidence of a technical error or a causal effect. It is a descriptive pattern that needs biological and technical context; it does not establish either conclusion by itself.

Key takeaway: Population stratification means that sampled groups differ in allele frequencies. Those differences shift group-specific mean dosages and, through group membership, can generate systematic genotype patterns across the genome. The pattern is real in the data, but its source and implications must be distinguished from linkage, sampling variation, and technical effects.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

A group has 25 individuals, of whom the total counted-allele dosage is 20. What is the sample allele frequency, assuming all genotypes are observed?
Two equally sized groups have allele frequencies 0.300 and 0.700. What is the pooled frequency? Does that pooled value show that the groups have the same frequency?
Why can a sample have fewer heterozygotes than expected from its pooled allele frequency even when each constituent group is in Hardy–Weinberg equilibrium?
Two loci are independent within each of two groups. What additional condition determines whether between-group mixing induces positive or negative dosage covariance?
Name one way population stratification differs from linkage disequilibrium, and one reason an observed genome-wide pattern might instead be technical.

Solutions. (1) There are \(2(25)=50\) chromosome copies, so \(\hat{p}=20/50=0.400\). (2) The pooled frequency is \((0.300+0.700)/2=0.500\). It does not show that the groups are alike; their frequencies differ by 0.400. (3) The group-specific genotype proportions are averaged, and that mixture need not equal the Hardy–Weinberg proportions calculated from the pooled frequency. (4) The directions of the groups' deviations in expected dosage at the two loci matter: aligned deviations give positive between-group covariance, while opposing deviations give negative covariance. (5) Stratification comes from allele-frequency differences between groups, whereas linkage disequilibrium is association between loci within a population. A processing batch correlated with sample grouping can create technical differences in measured genotypes or call quality.