Why Prune Correlated Variants Before PCA?
Genotype PCA summarizes patterns across variants. If many nearby variants carry nearly the same information, including all of them can give that local genomic region more influence than regions represented by fewer variants. The resulting PCs may then reflect a mixture of broad ancestry differences and unusually dense local signals, rather than a balanced summary across the genome. Linkage disequilibrium (LD) pruning reduces this redundancy by retaining a subset of variants with limited pairwise correlation within genomic windows.
Pruning is a preprocessing choice, not a guarantee that the remaining variants are independent or that the PCs represent ancestry alone. Its purpose is narrower: to limit repeated contributions from locally correlated markers. The appropriate settings depend on the marker density, the ancestry composition of the sample, and the scientific goal.
Dosage correlation is a practical measure of how similarly two variants vary across the sampled individuals. It is not itself a direct estimate of every biological feature of LD: it can depend on the sample, and population structure can contribute to correlations. In particular, if allele frequencies differ between ancestry groups, pooling those groups can induce correlation even between variants whose within-group correlations are small. For local pruning, it is therefore important to consider whether the sample used to estimate correlations is reasonably homogeneous, or whether LD should be estimated within suitable groups.
PCA commonly begins with a genotype matrix whose rows are individuals and columns are variants, after the chosen centering and scaling. No particular scaling choice makes correlated columns cease to be redundant. The key issue here is that the same pattern repeated across columns is counted repeatedly when the matrix is used to form individual-level covariance.
How Redundancy Changes PCA Weight
A short calculation shows why a cluster of correlated variants can matter. Suppose two columns of a centered, standardized genotype matrix are identical, and each has variance 1. Their contribution to the covariance matrix across variants is
The vector \((1,1)\) is an eigenvector with eigenvalue 2, while \((1,-1)\) has eigenvalue 0. With only one of these columns, the corresponding variance contribution would be 1. Thus, duplicating the same standardized pattern doubles its contribution in this simple case. More generally, a group of \(k\) identical columns contributes an eigenvalue \(k\) in the shared direction. Highly correlated rather than identical variants have a similar, though less exact, effect.
This does not mean that a PC must be “caused by” one region whenever correlated variants are present. It means that repeated measurements of a local pattern can increase its weight relative to other genomic regions. Pruning reduces this repeated weighting; it does not remove all local structure or identify a uniquely correct PCA.
Worked Example: The Effect of Duplicate Variant Columns
Consider three hypothetical individuals and one centered genotype pattern \(z=(-1,0,1)\), scaled so its variance under a divisor of \(n\) is \(2/3\). For clarity, rescale it to \(z^*=(-\sqrt{3/2},0,\sqrt{3/2})\), which has variance 1. Suppose two variants have exactly this same standardized pattern.
The two-column matrix has columns \(z^*\) and \(z^*\). Using divisor \(n\), its covariance across variants is
Multiplying this matrix by \((1,1)^\mathsf{T}\) gives \((2,2)^\mathsf{T}=2(1,1)^\mathsf{T}\), so the shared direction has eigenvalue 2. If one duplicate column is removed, the remaining direction has variance 1. The two variants have not supplied two distinct patterns; they have counted one pattern twice. A pruning rule would retain one of them if their \(r^2\) exceeded its threshold.
This is an illustrative calculation, not a claim that real variants are exact duplicates. In real data, high but imperfect correlation produces partial redundancy, and the effect also depends on the rest of the genome.
Estimating Correlation and Applying a Pruning Rule
A typical procedure compares dosage correlations for variant pairs within a moving physical or genetic window. If a pair exceeds the \(r^2\) threshold, one member is removed, often by a greedy rule that prioritizes a specified ordering or a variant-quality criterion. The window then advances and comparisons are repeated. Because a variant may be compared with different neighbors as the window moves, the exact implementation and ordering can affect the retained set.
For a simple pair, the sample correlation can be calculated from centered dosages. If \(x_i\) and \(y_i\) are the dosages for individual \(i\), let \(\bar{x}\) and \(\bar{y}\) be their sample means. Then
The \(n-1\) divisors in the sample covariance and variances cancel in this ratio. The squared value measures the strength of the linear dosage association without retaining its sign. Complementary allele coding can change the sign of \(r\), but not \(r^2\).
Worked Example: Calculating Dosage \(r^2\)
Suppose six hypothetical individuals have dosages \(x=(0,0,1,1,2,2)\) at variant X and \(y=(0,1,0,1,2,2)\) at variant Y. Both sample means are 1. Their centered vectors are \((-1,-1,0,0,1,1)\) and \((-1,0,-1,0,1,1)\).
The sum of cross-products is \(1+0+0+0+1+1=3\). Each centered sum of squares is \(1+1+0+0+1+1=4\). Therefore,
If a pruning rule uses an \(r^2\) threshold of 0.50 and the variants are in the same window, this pair exceeds the threshold. The rule would remove one member according to its selection policy. It would not establish that one variant is biologically unimportant; it would limit redundant representation in this PCA input.
Choosing and Interpreting Pruning Settings
A window defines which pairs are candidates for comparison; the \(r^2\) threshold defines how much correlation is tolerated. A stricter threshold removes more highly correlated pairs, while a larger window allows comparisons across a wider genomic span. Neither setting has a universal best value. Physical distance is convenient, but equal physical windows need not contain equal numbers of variants or correspond to equal recombination patterns. Genetic-distance windows can better reflect recombination where reliable maps are available.
A practical choice should be tied to the intended PCA. For broad ancestry structure, analysts often seek a set with limited local redundancy while retaining genome-wide representation. The settings should be reported, including the window definition, window size and step (if applicable), \(r^2\) threshold, sample used to estimate LD, and the rule for selecting which variant to keep. It is also useful to check whether retained variants remain reasonably distributed across the genome.
Worked Example: A Greedy Window Pruning Pass
Consider five hypothetical variants A–E in one window, ordered by a predetermined quality ranking. Their relevant pairwise values are \(r^2_{AB}=0.82\), \(r^2_{AC}=0.12\), \(r^2_{BC}=0.18\), \(r^2_{CD}=0.67\), \(r^2_{CE}=0.08\), and \(r^2_{DE}=0.09\). Other pairwise values are below 0.20. Use threshold 0.20 and process variants in the stated order, retaining a variant unless it exceeds the threshold with an already retained variant.
Start with A and retain it. B has \(r^2_{AB}=0.82>0.20\), so remove B. C has \(r^2_{AC}=0.12<0.20\), so retain C. D has \(r^2_{CD}=0.67>0.20\), so remove D. E is below the threshold with both retained variants: \(r^2_{AE}<0.20\) and \(r^2_{CE}=0.08<0.20\), so retain E. The resulting set is A, C, E.
This example makes the ordering rule explicit. A different prioritization could produce a different retained subset, even with the same threshold. The goal is not to identify the one mathematically inevitable set, but to apply and document a reproducible rule that limits redundant pairs.
Diagnostics, Alternatives, and Limitations
Pruning should be checked as part of the PCA workflow. Useful diagnostics include the number and genomic distribution of retained variants, the distribution of pairwise \(r^2\) among retained variants in the relevant windows, and whether a small number of regions still dominate the input. Comparing PCA results before and after pruning can reveal substantial sensitivity: for example, whether broad group patterns remain similar or whether leading axes change sharply. Such comparisons are diagnostic, not proof that either result is the truth.
Unexpected results can arise when pooled ancestry groups inflate local dosage correlations, when a window or threshold is unsuitable for the marker density, or when the selection order repeatedly favors variants from a limited set of regions. A very strict threshold can discard useful information and reduce the number of retained variants; a lenient threshold can leave substantial redundancy. Large shifts in ancestry patterns after modest changes to settings warrant investigation rather than automatic acceptance of one setting.
| Approach | What it does | Main trade-off |
|---|---|---|
| LD pruning | Removes one variant from sufficiently correlated pairs in specified windows. | Reduces local redundancy, but results depend on window, threshold, sample, and selection rule. |
| Random thinning | Samples a subset of variants without using their correlations. | Simple, but can retain correlated pairs and may discard useful representatives by chance. |
| No pruning | Uses all available variants in PCA. | Preserves all measurements, but dense correlated regions can receive repeated weight. |
| LD-aware weighting or modeling | Attempts to account for dependence rather than simply removing variants. | Can retain more information, but requires additional modeling choices and is not equivalent to pruning. |
Pruning is a compromise, not a universal correction. It can discard variants that are informative for a particular locus or a specific ancestry contrast, and it does not remove long-range correlations or all structure-induced dependence. Nor does a low pairwise \(r^2\) guarantee global independence. Results should therefore be interpreted as PCA of a defined, pruned marker set, with sensitivity to reasonable settings considered when ancestry conclusions are important.
Common Mistake
Wrong reasoning: “The pruning threshold is 0.20, so all pairs in the retained data are independent, and the resulting PCs must represent ancestry alone.” The correction is that the threshold only governs pairwise dosage correlations considered by the specified window and algorithm. Correlations can remain outside the window or arise through population structure, and PCA can capture other sources of variation. Pruning reduces a particular form of local redundancy; it does not certify independence or assign a biological interpretation to each PC.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- Two standardized variant columns are identical. What are the eigenvalues of their two-by-two covariance matrix, and what do they imply about redundancy?
- If a pair has dosage correlation \(r=-0.60\), calculate \(r^2\). Would it exceed a pruning threshold of 0.30?
- Why can estimating \(r^2\) in a pooled sample of ancestry groups produce a different pruning decision than estimating it within groups?
- Name two settings or implementation details that should be reported for an LD-pruned PCA, and explain why.
Solutions. (1) The matrix is \(\begin{pmatrix}1&1\\1&1\end{pmatrix}\), with eigenvalues 2 and 0. The shared direction is counted twice and the contrast direction has no variance. (2) \(r^2=(-0.60)^2=0.36\), which exceeds 0.30, so the pair would trigger pruning if it is in the comparison window. (3) Differences in allele frequencies between groups can induce pooled dosage correlation, so pooled \(r^2\) may reflect both within-group association and between-group structure. (4) Examples include the window size and whether it is measured in physical or genetic distance, the \(r^2\) threshold, the LD-estimation sample, and the rule for selecting which variant to retain. These choices determine which pairs are compared and which markers enter PCA.