The Bias-Variance Trade-Off in Weight Shrinkage

Why Shrink Noisy GWAS Weights?

A polygenic score combines genotype values with estimated genetic-effect weights. As established in the earlier material on deriving score weights from GWAS regression effects, the weights are estimates rather than known constants. Their sampling error matters: a weight that is too large can make the score respond too strongly to a variant whose estimated effect was partly noise. Shrinking weights toward zero can reduce that noise, but it also moves the weights away from their true effects.

This is a bias-variance trade-off. The calculation in this tutorial isolates that trade-off for a simple multiplicative rule. It does not account for linkage disequilibrium, selection of variants, or other sources of bias; those features can change the appropriate analysis. The assumptions below make clear what the calculation does and does not establish.

Definition: Suppose a GWAS estimates a true effect \(\beta\) by \(\hat{\beta}\), and the estimate is multiplied by a shrinkage factor \(c\). The shrunken weight is \(\tilde{\beta}=c\hat{\beta}\). Shrinkage toward zero uses \(0\leq c\leq 1\): \(c=1\) leaves the estimate unchanged, while \(c=0\) sets the weight to zero.

For the basic calculation, assume \(\hat{\beta}\) is unbiased for a fixed \(\beta\), with sampling variance \(\tau^2\): \(\mathbb{E}(\hat{\beta}\mid\beta)=\beta\) and \(\operatorname{Var}(\hat{\beta}\mid\beta)=\tau^2\). Here \(\mathbb{E}\) denotes expectation over repeated discovery studies. In practice, \(\tau\) may be estimated by the GWAS standard error. The effect and its standard error must be expressed on the same genotype and phenotype scales.

Bias, Variance, and Mean Squared Error

For a fixed true effect, the bias of the shrunken estimate is its expected value minus the true effect. Its variance measures how much it varies across repeated discovery studies. Squared bias plus variance gives the mean squared error (MSE), the expected squared distance between the estimated and true weight.

\operatorname{Bias}(\tilde{\beta}\mid\beta)=(c-1)\beta,\qquad \operatorname{Var}(\tilde{\beta}\mid\beta)=c^2\tau^2

The bias follows because \(\mathbb{E}(\tilde{\beta}\mid\beta)=c\beta\). The variance follows because multiplying a random variable by \(c\) multiplies its variance by \(c^2\). Therefore,

\operatorname{MSE}(c\mid\beta) =\mathbb{E}\big[(\tilde{\beta}-\beta)^2\mid\beta\big] =(1-c)^2\beta^2+c^2\tau^2.

The first term is squared bias: stronger shrinkage increases it unless \(\beta=0\). The second is sampling variance: reducing \(c\) lowers it. The unshrunk estimate has MSE \(\tau^2\), while setting the weight to zero has MSE \(\beta^2\). Which endpoint is better depends on the relative size of the true effect and the estimation noise.

Worked Example: Shrinking One Noisy Effect Estimate

Suppose a hypothetical GWAS estimates an effect on a standardized phenotype, per one-standard-deviation increase in a standardized genotype. For illustration, take the true effect to be \(\beta=0.08\) phenotype standard deviations and the estimate's standard error to be \(\tau=0.04\). These values are hypothetical; the true effect is treated as known here only to compare the methods' repeated-sampling error.

The noise variance is \(\tau^2=0.04^2=0.0016\). If the estimate is not shrunk, \(c=1\), its MSE is \(0.0016\). If it is set to zero, \(c=0\), its MSE is \(\beta^2=0.08^2=0.0064\). Zeroing this particular nonzero effect is worse than leaving it unshrunk.

Now use \(c=0.8\). The squared bias is \((1-0.8)^2(0.08)^2=0.000256\), and the variance is \(0.8^2(0.0016)=0.001024\). Thus its MSE is \(0.000256+0.001024=0.001280\), lower than the unshrunk MSE of \(0.0016\). For example, if the observed estimate were \(0.10\), the shrunken weight would be \(0.8(0.10)=0.08\) phenotype standard deviations per genotype standard deviation. The MSE comparison concerns repeated samples under the stated model, not certainty that this particular shrunken estimate equals the truth.

The Risk-Minimizing Shrinkage Factor

For fixed \(\beta\) and \(\tau^2\), minimize the MSE over \(c\). Differentiating the quadratic risk and setting its derivative to zero gives the unique minimum:

\frac{\partial\operatorname{MSE}(c\mid\beta)}{\partial c} =-2(1-c)\beta^2+2c\tau^2=0 \quad\Longrightarrow\quad c^*=\frac{\beta^2}{\beta^2+\tau^2}.

The second derivative is \(2(\beta^2+\tau^2)>0\) when the total is positive, so this stationary point is a minimum. If both \(\beta=0\) and \(\tau^2=0\), every value of \(c\) yields zero error; otherwise the displayed formula gives the minimizer. For a nonzero effect, \(c^*\) is closer to one when the effect is large relative to its standard error, and closer to zero when estimation noise is large relative to the effect.

This is an oracle result: it depends on the unknown true effect. It explains the trade-off, but does not directly provide an implementable factor for an individual variant. In particular, substituting the noisy estimate \(\hat{\beta}\) for \(\beta\) does not automatically produce an estimate with the same optimality property.

Result: Under an unbiased estimate with sampling variance \(\tau^2\), the fixed-effect MSE at shrinkage factor \(c\) is \((1-c)^2\beta^2+c^2\tau^2\). Its minimizer is \(c^*=\beta^2/(\beta^2+\tau^2)\). More precise estimates (smaller \(\tau^2\)) call for less shrinkage when the effect size is held fixed.

Worked Example: Discovery Precision Changes the Preferred Shrinkage

Consider two hypothetical discovery settings for the same true effect, \(\beta=0.05\) phenotype standard deviations per standardized genotype. In the first setting, suppose the standard error is \(0.02\); in the second, a smaller discovery sample gives a standard error of \(0.05\). Assume unbiased estimates in both settings.

In the first setting, \(\tau^2=0.0004\), so \(c^*=0.05^2/(0.05^2+0.02^2)=0.0025/0.0029=0.8621\). The estimate is relatively precise, and the minimizing factor is close to one. In the second setting, \(\tau^2=0.0025\), so \(c^*=0.0025/(0.0025+0.0025)=0.5000\). With greater sampling noise, the MSE-minimizing factor is smaller. The comparison assumes the same true effect and the stated variances; it does not imply that every variant in a real GWAS should receive the same factor.

Several Effects and Genetic Architecture

A score combines effects across variants, so it is useful to state the assumptions under which weight error translates directly into score error. Suppose there are \(m\) variants, with centered, standardized genotype values \(x_j\) that are uncorrelated in the target population: \(\mathbb{E}(x_j)=0\), \(\operatorname{Var}(x_j)=1\), and \(\operatorname{Cov}(x_j,x_k)=0\) for \(j\ne k\). Let the true genetic component be \(g=\sum_{j=1}^m x_j\beta_j\), and the score using shrunken estimates be \(\tilde{g}=\sum_{j=1}^m x_j c\hat{\beta}_j\). Assume each GWAS estimate is unbiased, has variance \(\tau_j^2\), and has sampling error independent of the target genotypes. These assumptions exclude LD among the variants in this calculation.

\mathbb{E}\big[(\tilde{g}-g)^2\big] =\sum_{j=1}^m\left\{(1-c)^2\beta_j^2+c^2\tau_j^2\right\}.

To see why, condition first on the estimated weight errors. The squared score error is the square of \(\sum_j x_j(c\hat{\beta}_j-\beta_j)\). Averaging over target genotypes removes cross-products because distinct genotypes have covariance zero, and each genotype has variance one. Averaging next over GWAS sampling gives the squared bias plus variance for each weight. Thus the expression is the sum of the per-variant MSEs. Differentiating this sum shows that the best single common factor under these assumptions is

c^*=\frac{\sum_{j=1}^m\beta_j^2} {\sum_{j=1}^m\beta_j^2+\sum_{j=1}^m\tau_j^2}.

This formula makes the role of genetic architecture explicit. The numerator measures the total squared signal across effects; the noise term measures total GWAS sampling variance. If many effects are near zero, total squared signal can be small relative to noise, favoring stronger overall shrinkage. If effects are larger in aggregate, less shrinkage can minimize this particular expected squared score error. The formula is still an oracle expression because the true effects are unknown.

Worked Example: The Effect Distribution Changes the Average Optimum

Suppose, hypothetically, each of \(m=100\) variants has an unbiased estimate with common variance \(\tau^2=0.0025\). Compare two possible architectures, summarized by the average squared true effect \(v\). The first has \(v=0.0004\), and the second has \(v=0.004\). These values describe hypothetical average squared effects in the same units as \(\tau^2\).

When \(\sum_j\beta_j^2=mv\) and \(\sum_j\tau_j^2=m\tau^2\), the factor is \(c^*=v/(v+\tau^2)\). For the first architecture, \(c^*=0.0004/(0.0004+0.0025)=0.1379\). For the second, \(c^*=0.004/(0.004+0.0025)=0.6154\). The architecture with smaller average squared effects has less aggregate signal relative to sampling noise and therefore favors stronger shrinkage in this risk calculation.

The minimized expected error per variant is \(v\tau^2/(v+\tau^2)\). It is \(0.0004(0.0025)/0.0029=0.0003448\) in the first case and \(0.004(0.0025)/0.0065=0.001538\) in the second. These different minimum errors reflect different underlying effect distributions; the smaller shrinkage factor in the first case does not mean its absolute prediction error must be larger.

Putting the Result Into a Score

The next example illustrates how one common factor changes several weights at once. It uses hypothetical known effects to calculate expected error, not to suggest that the true effects are available when constructing a real score. The score is evaluated in standardized, mutually uncorrelated genotypes, so the score-error formula above applies.

Worked Example: Comparing Score Error Across Shrinkage Choices

Suppose three hypothetical variants have true effects \((0.05,0.03,0)\) phenotype standard deviations per standardized genotype. Their GWAS estimates are \((0.06,0.01,0.02)\), and each estimate has standard error \(0.04\). The total squared signal is \(0.05^2+0.03^2+0^2=0.0034\), and total estimation variance is \(3(0.04^2)=0.0048\). Hence the oracle common factor for expected score squared error is \(c^*=0.0034/(0.0034+0.0048)=0.4146\).

The unshrunk weights have expected squared error \(0.0048\). Shrinking all weights by \(0.4146\) produces \((0.02488,0.004146,0.008293)\) and expected squared error \((1-0.4146)^2(0.0034)+(0.4146)^2(0.0048)\approx0.001990\). The zero score has error \(0.0034\), the total squared signal. Under these assumptions, the shrunken score has lower expected squared error than either endpoint. Its third weight is not zero because the observed third estimate is not zero; shrinkage reduces a weight rather than removing it.

This example also highlights the distinction between an oracle calculation and a practical choice. The factor used in the calculation depends on the true effects, which are not known in an applied analysis. The example establishes how a factor would perform under specified hypothetical truths; it does not identify a data-driven factor without further assumptions or evidence.

Assumptions and Common Pitfalls

The simple result depends on the estimation model and the target of optimization. If estimates are biased, for example after selecting variants based on unusually large observed associations, then \(\mathbb{E}(\hat{\beta}\mid\beta)=\beta\) need not hold. The bias and MSE formulas must then include that selection bias. If genotypes are correlated, cross-products in the score error generally do not vanish, so the sum of independent per-variant risks is no longer the full score risk. If standard errors differ, the aggregate formula includes their separate variances; treating them as equal without justification can misstate the trade-off.

There is also no guarantee that the factor minimizing squared weight error maximizes predictive correlation or another chosen performance measure. Those are distinct objectives. The derivation identifies a precise optimum for a stated loss and assumptions, not a universal rule for all score-construction goals.

Mistake: “Shrinkage always improves weights because it reduces variance.” The correction is that it also introduces bias. Compare both terms: for a large true effect and precise estimate, shrinkage can increase total MSE.
Mistake: “The same shrinkage factor must be optimal for every architecture.” The correction is that the common-factor optimum depends on total squared signal relative to total sampling variance.
Mistake: “The oracle factor can be calculated from the observed effect estimate by replacing the unknown truth.” The correction is that the formula depends on true effects; a noisy plug-in substitution does not inherit the oracle guarantee.
Mistake: “The independent-variant score formula applies unchanged under LD.” The correction is that correlated genotypes create cross-terms in prediction error, so the assumptions and risk expression must change.

Key takeaway: Shrinking a GWAS estimate toward zero trades squared bias against sampling variance. With unbiased estimates and a specified squared-error target, stronger shrinkage is favored when estimation noise is large relative to aggregate genetic signal. The optimum depends on discovery precision, genetic architecture, and the assumptions used to evaluate score error.

Check Your Understanding

Use the stated assumptions in each question, and distinguish an oracle result from a factor that can be calculated directly from observed data.

For one unbiased effect estimate with variance \(\tau^2\), write the squared-bias and variance terms for a weight shrunk by \(c\).
Why does the MSE-minimizing factor move closer to one when the estimate becomes more precise, with the true effect held fixed?
For several uncorrelated standardized genotypes, what two aggregate quantities determine the oracle common shrinkage factor?
Why can selecting variants for large observed GWAS effects make the unbiased-estimate calculation unreliable?
What feature of correlated genotypes prevents the score-error calculation from reducing to a sum of independent per-variant risks?