Worked Comparison of Clumping and Thresholding and Bayesian Scores

Why Compare These Score-Building Strategies?

A polygenic score combines genetic variants using estimated effects from a genome-wide association study (GWAS). Two broad strategies make different choices about noisy effects and linkage disequilibrium (LD). Clumping-and-thresholding (C+T) selects a relatively small set of variants, then weights them by their GWAS effect estimates. Bayesian shrinkage uses a prior to pull uncertain effects toward zero while accounting for correlation among variants. Neither strategy guarantees better prediction: performance depends on the discovery data, target population, LD reference, tuning procedure, and validation sample.

For a fair comparison, both scores must start from harmonized variant data and be evaluated on the same individuals using the same outcome and validation split. Alleles must be aligned so that each effect estimate multiplies the dosage of the same effect allele. A mismatch can reverse a contribution and distort either method.

Constructing a C+T Score

C+T has two main operations. Clumping uses an LD reference panel to retain a representative variant from a region of correlated variants, typically prioritizing the smallest GWAS p-value. Thresholding retains clumped variants whose GWAS p-values are at or below a chosen threshold \(t\). The score then sums their target-sample dosages multiplied by their GWAS effect estimates. A threshold is a tuning choice; different values can produce scores containing very different numbers of variants.

This basic recipe assumes compatible variant identifiers, genome build, effect alleles, and effect scales. It also assumes the LD reference is suitable for the target data and that the chosen clumping window and LD cutoff are defensible. Clumping is a practical way to reduce redundant correlated signals, not proof that retained variants are biologically independent. A GWAS p-value measures evidence against a null association, not the probability that an effect is true or the size of its predictive contribution.

Bayesian Shrinkage: A Small Summary-Data Derivation

Bayesian methods use a prior distribution to regularize effect estimates. A simple illustration makes the role of LD explicit. Let \(\boldsymbol{\beta}\) be the vector of joint variant effects and \(\mathbf{b}\) the vector of marginal GWAS effect estimates. Suppose the summary-data likelihood is

where \(R\) is the LD correlation matrix, \(v\) is a sampling-variance scale, and \(\tau^2\) controls the prior spread of effects. This is a simplified, specified model rather than a universal likelihood for every GWAS. It presumes the summary estimates and LD reference are compatible and that this covariance approximation is adequate. The posterior mean is also the posterior mode in this Gaussian model.

The likelihood contributes precision \(R/v\) and linear term \(\mathbf{b}/v\); the prior contributes precision \(I/\tau^2\). Adding them gives posterior precision \(R/v+I/\tau^2\). Multiplying its inverse by the linear term and multiplying numerator and denominator by \(v\) yields

This expression illustrates two features: the estimated effects are jointly adjusted through \(R\), and the prior adds a positive diagonal penalty that shrinks estimates. More elaborate Bayesian score methods can use different priors, LD models, or mixtures of effect-size distributions; they should not be treated as identical to this simple Gaussian example.

Tuning and Validation Without Leakage

C+T may require choices about the p-value threshold, clumping window, and LD cutoff. Bayesian methods require choices such as prior scale or other prior parameters, and may require computational settings. These choices should be made using training data or an inner validation procedure. A final held-out test set should be used only after choosing the method and settings; repeatedly selecting settings based on test performance makes the reported performance optimistic.

A suitable comparison fits or constructs each candidate using the same discovery GWAS and allele conventions, tunes both using comparable validation information, and evaluates both on the same untouched target test individuals. The outcome measure should match the task. For a continuous outcome, mean squared error or correlation may be useful; for a binary outcome, discrimination and calibration measures answer different questions. For example, a larger area under the ROC curve does not establish good calibration. The earlier material on discrimination and calibration distinguishes these prediction properties.

Computational Trade-Offs, Diagnostics, and Limits

C+T is often computationally light after GWAS summary statistics and an LD reference are available: clumping can be performed once, then several p-value thresholds can be scored by summation. Bayesian methods can require more computation because they account for LD while estimating many weights. The exact cost varies substantially by method, implementation, number of variants, and whether computation can be parallelized. Bayesian shrinkage is not automatically more accurate simply because it is more complex; C+T is not automatically safer simply because its score is sparse.

Useful checks include confirming allele alignment and variant matching, inspecting the number of retained C+T variants at each threshold, and checking whether clumping decisions are stable to reasonable LD-reference choices. For Bayesian scores, check that the LD matrix is compatible with the analyzed variants and is numerically usable; strong mismatch or poor conditioning can undermine estimates. For both methods, inspect score distributions and prediction performance across relevant target subgroups, while interpreting subgroup differences in light of sample size and population structure.

Unexpectedly extreme scores, a sharp performance drop between validation and test data, or large changes after minor tuning adjustments can indicate overfitting, allele errors, data leakage, or a discovery-to-target mismatch. Differences in allele frequency and LD can change how well discovery weights transfer. A score's apparent performance also depends on the target population and outcome definition; validation in one sample does not establish equal performance elsewhere.

Common Mistakes

• Choosing the winner on the test set. Trying many thresholds or priors against test outcomes and reporting the best result turns the test set into tuning data. Choose settings first, then evaluate once on held-out data.
• Treating clumping as complete LD correction. Clumping reduces redundancy under a specified reference and rule; it does not remove every correlation or guarantee that the reference matches the target.
• Assuming Bayesian weights are automatically unbiased or optimal. Posterior weights depend on the likelihood, prior, and LD inputs. A misspecified prior or mismatched LD reference can harm prediction.
• Comparing different samples or scales. Validation metrics are most interpretable when the candidate scores are compared on the same individuals, outcome, and scale, with the same evaluation procedure.
• Equating a better metric with better clinical decisions. Prediction metrics summarize particular aspects of performance. Their implications for decisions depend on the outcome, thresholds, and context.