Why Population Structure Matters for Pairwise Kinship
A pair of people can share alleles because they inherited them from a recent common ancestor, but also because they come from a population in which those alleles are common. A kinship estimator that does not separate these sources can report apparent relatedness between unrelated people from the same ancestry group. This is especially important in genomic data: a small per-marker bias can accumulate across many markers and look like a strong pairwise signal.
The earlier material on kinship coefficients and method-of-moments estimation establishes the target and the basic frequency-based strategy. Here the central issue is the estimator's sensitivity to the allele frequencies used for centering. A robust estimate is not a guarantee against every form of bias; it is an estimate designed to rely less on a potentially inappropriate pooled frequency model.
Let \(G_{ik}\in\{0,1,2\}\) be the count of a chosen allele for individual \(i\) at marker \(k\), and let \(p_k\) be its frequency in a reference population. Under a common-population model with random mating and no unmodelled structure, the expected dosage is \(2p_k\), and the dosage variance is \(2p_k(1-p_k)\). For a pair from that population, the covariance in dosages is approximately \(4\phi_{ij}p_k(1-p_k)\). This motivates the moment estimate
Here \(S\) is the set of markers used, typically after quality control and linkage-disequilibrium pruning or other steps to limit redundant information. The estimator assumes that the selected markers are sufficiently well measured, that the chosen frequencies represent the relevant population, and that the marker contributions provide a useful approximation to the moment model. Linkage disequilibrium can make the effective number of independent contributions much smaller than the marker count.
How Pooled Frequencies Create Bias
Suppose a sample contains ancestry groups with different frequencies at a marker. Even if two people are unrelated conditional on their ancestry, their dosages can be correlated across the sample: people from the same group tend to have similar expected dosages. Centering both people at a single pooled frequency leaves that shared group-level shift in both residuals. Their product is then positive on average.
Worked Example: Unrelated People Can Look Related
Consider two equally represented ancestry groups. At one hypothetical marker, the allele frequency is \(0.10\) in group A and \(0.70\) in group B. Assume the two people are unrelated but belong to the same group, and use the pooled frequency for centering.
The pooled frequency is \(p=(0.10+0.70)/2=0.40\), so \(4p(1-p)=4(0.40)(0.60)=0.96\). Conditional on group A, each person's expected dosage is \(2(0.10)=0.20\), giving a centered mean of \(0.20-0.80=-0.60\). The expected product of the two centered dosages is \((-0.60)(-0.60)=0.36\). The resulting one-marker moment ratio is \(0.36/0.96=0.375\).
The analogous calculation for two unrelated people in group B gives centered means \(1.40-0.80=0.60\), again a product of \(0.36\), and the same ratio, \(0.375\). These are not credible estimates of recent kinship: the positive numerator arises from shared ancestry-specific frequency shifts, not IBD. In real data the estimate averages across markers, but a consistent structure-driven shift can persist.
The calculation also shows why adding markers does not automatically solve the problem. If the same population structure affects many markers, averaging more markers can make the biased estimate more stable rather than correct it. The relevant question is whether the centering model is appropriate, not simply whether the marker count is large.
Two Ways to Reduce Sensitivity to Structure
One strategy is to improve the frequency model. If ancestry groups are known or adequately estimated, center each person's dosage relative to an appropriate ancestry-specific frequency. For a pair from the same well-defined group \(s\), one can use \(p_{ks}\) in place of a pooled \(p_k\). This removes the group-level dosage shift under the model. It remains a frequency-based strategy: its performance depends on estimating representative frequencies and assigning ancestry appropriately.
A second strategy is a genotype-based robust estimator, commonly referred to as KING-robust. Its construction uses pairwise genotype-sharing patterns rather than centering both people at an externally pooled allele frequency. In particular, genotype configurations involving heterozygotes and homozygotes provide information about whether their sharing is greater than expected from broad population differences. Because the estimate does not depend on a single pooled frequency reference in the same way as the moment estimator above, it is less sensitive to ancestry-related frequency differences.
“Robust” does not mean structure-proof, nor does it mean that every pair receives an unbiased estimate under every ancestry configuration. The pairwise genotype patterns still need adequate informative markers, valid genotype calls, and a setting in which the estimator's assumptions are reasonable. KING-robust estimates are particularly useful as a screen when a homogeneous reference frequency is doubtful; they should be considered alongside ancestry information and data-quality checks.
Worked Example: Re-centering Within Ancestry Groups
Return to the two groups in the previous example. Suppose both people are known to be in group A. Their expected dosages are \(0.20\), and the group-specific frequency is \(p_A=0.10\). After group-specific centering, each centered dosage has expected value \(0.20-2(0.10)=0\). For unrelated people whose genotypes are independent conditional on group A, the expected product is \(0\times0=0\). The expected numerator contribution is therefore zero, rather than \(0.36\).
For group B, the same reasoning gives expected dosage \(1.40\), centered value \(1.40-2(0.70)=0\), and expected product zero. The denominator for group A is \(4(0.10)(0.90)=0.36\), while for group B it is \(4(0.70)(0.30)=0.84\); both are positive. Thus the expected frequency-based kinship contribution is zero for unrelated pairs when the group-specific frequencies are correct and the model assumptions hold.
This correction depends on correctly specifying the relevant group and frequency. If an individual's ancestry is intermediate, uncertain, or different across genomic regions, a single group label may not give a suitable baseline. Local ancestry or finer structure can also remain unmodelled. Robust pairwise estimation is useful in such situations, but it does not eliminate the need to examine ancestry patterns.
Checking a Frequency-Based Estimate
A small numerical example makes the scaling explicit. Suppose three hypothetical, independent markers have frequencies \(0.20\), \(0.30\), and \(0.40\). For two individuals, the dosage pairs are respectively \((1,1)\), \((0,1)\), and \((2,2)\). The numerator contributions are:
The numerator is \(0.36-0.24+1.44=1.56\). The denominator is \(4(0.20)(0.80)+4(0.30)(0.70)+4(0.40)(0.60)=0.64+0.84+0.96=2.44\). The estimate is \(1.56/2.44=0.6393\). This value is only an illustration of the arithmetic: three markers are far too few for a stable estimate, and the stipulated independence and frequency assumptions would need to be credible. It should not be treated as a precise relationship classification.
Worked Example: A Larger Marker Set and Its Uncertainty
Suppose a hypothetical analysis of 1,000 quality-controlled markers gives a summed centered dosage product of \(122.0\) and a summed denominator of \(488.0\). Under the frequency-based moment formula, \(\widehat{\phi}_{ij}=122.0/488.0=0.2500\). The numerical value is a point estimate; it does not by itself establish that the pair has a particular pedigree relationship.
Now suppose ancestry diagnostics show that the pair comes from the same subpopulation, but the frequencies used were pooled across two subpopulations with appreciably different allele frequencies. The estimate may reflect both IBD and shared ancestry. A robust pairwise estimator can provide a useful comparison because its construction is less dependent on that pooled baseline. If the robust result is much smaller, the discrepancy is a warning to investigate structure and frequency choice, not proof that either estimate is correct.
The marker count alone does not justify a confidence interval or a claim of precision. Correlated markers, frequency estimation error, genotyping error, and ancestry misspecification all affect uncertainty. A sensible report identifies the estimator, the frequency or ancestry model, the marker set, and relevant diagnostics.
Assumptions, Diagnostics, and Alternatives
No estimator can recover a clean kinship signal from data that violate its basic requirements without consequence. Before interpreting pairwise estimates, check that genotype calls are reliable, that markers are aligned and coded consistently, and that variants with very low frequency or poor quality are handled appropriately. Inspect ancestry patterns using the principal-component methods already covered in the course, and compare estimates across plausible frequency choices or estimator types.
Useful warning signs include a broad upward shift in estimates among people from the same ancestry cluster, sharp changes when the frequency reference changes, and disagreement between a pooled-frequency estimate and a robust pairwise estimate. Estimates can also be unstable when few informative markers remain, when LD causes many markers to repeat essentially the same signal, or when missingness differs systematically between individuals or ancestry groups. These patterns call for investigation rather than automatic thresholding.
The main alternatives have different assumptions and practical roles:
| Approach | Strength | Important limitation |
|---|---|---|
| Pooled-frequency moment estimator | Simple, interpretable, and efficient when the frequency reference fits the sample. | Shared ancestry-specific frequency shifts can inflate pairwise covariance. |
| Ancestry-aware frequency estimator | Centers genotypes using frequencies closer to the pair's relevant population. | Requires suitable ancestry assignments and accurate group-specific frequencies. |
| Genotype-based robust estimator, such as KING-robust | Less dependent on a pooled allele-frequency baseline and useful as a structure-aware comparison. | Still relies on informative, reliable genotype patterns and can be affected by complex structure or data errors. |
These approaches are complementary. A frequency-based estimate can be efficient and meaningful when the reference population is appropriate. An ancestry-aware estimate can address known group differences explicitly. A robust estimate is valuable when pooled frequencies are questionable, but should not be interpreted as a universal replacement for ancestry modelling. The strongest conclusions are those that are stable under reasonable choices and consistent with ancestry and genotype-quality diagnostics.
A Short Justification for the Structure Bias
The source of the pooled-frequency bias follows from conditioning on ancestry. Let \(P_s\) denote the allele frequency in ancestry group \(s\), and let \(p\) be the pooled frequency. For an unrelated person in group \(s\), the conditional expected dosage residual is \(E(G-2p\mid s)=2(P_s-p)\). For two people who are unrelated conditional on the same group, their genotypes are conditionally independent, so
Averaging over groups gives \(4E[(P_s-p)^2]\), which is nonnegative and is positive whenever the group frequencies vary. Thus the pooled-centered numerator can be positive even when the pair has no recent IBD relationship. With group-specific centering at the correct \(P_s\), each conditional residual mean is zero, so the conditional expected product is zero for unrelated people under the stated independence model. This derivation explains both the attraction and the limitation of ancestry-aware centering: it removes this particular shift only when the group frequencies are adequately represented.
Common Pitfall
Incorrect reasoning: “The kinship estimate is positive across thousands of SNPs, so the pair must be related.” A large marker count can reduce random variation, but it does not remove systematic covariance caused by shared population structure. The estimate may become very stable around a biased value.
Correction: Check which allele frequencies were used and whether they represent the pair's ancestry. Compare pooled-frequency estimates with robust pairwise estimates, examine ancestry patterns, and assess sensitivity to marker selection and quality control. Interpret the result as evidence about pairwise genetic similarity under a specified model—not as proof of a pedigree relationship or of causation.
Check Your Understanding
Work through each question, then compare your reasoning with the solutions.
- Two unrelated individuals are in the same ancestry group, where a marker's allele frequency is \(0.65\). What is the expected centered dosage residual for each if the pooled frequency is \(0.45\), and what is the expected product under conditional independence?
- Why can increasing the number of markers make a structure-biased estimate more stable without making it more accurate?
- For a frequency-based estimator, compute the denominator contribution at a marker with \(p=0.25\).
- Name one advantage and one limitation of ancestry-aware frequency centering compared with a genotype-based robust estimator.
Solutions. (1) The expected dosage is \(2(0.65)=1.30\). The residual relative to the pooled frequency is \(1.30-2(0.45)=0.40\); the expected product is \(0.40^2=0.16\). (2) If many markers share the same systematic ancestry shift, averaging reduces random noise around that shift but does not remove it. (3) The denominator contribution is \(4(0.25)(0.75)=0.7500\). (4) Ancestry-aware centering can explicitly remove group-specific frequency shifts when the assignments and frequencies are accurate. It depends on those choices; a genotype-based robust estimator relies less on a pooled frequency baseline but still requires reliable, informative genotype patterns and can be affected by complex structure.