Calibrating Prior Variance and Polygenicity

Why Prior Calibration Matters

Shrinkage methods use assumptions about the sizes and prevalence of genetic effects to moderate noisy GWAS estimates. Two assumptions are especially consequential: the total genetic variance attributed to the variants being modeled, and the fraction of those variants expected to have nonzero effects. They work together. If the expected total variance is held fixed while fewer variants are assumed causal, the prior must allow larger effects among those variants.

This relationship is useful both for specifying a model and for understanding its behavior. It also explains why a score can change when either assumption changes, even when the other is held constant. A sensitivity analysis makes that dependence visible; it does not, by itself, establish that one setting is biologically correct or predicts better.

From Heritability to Prior Effect Variance

For a precise derivation, suppose the phenotype has been standardized to variance 1, the \(M\) genotype columns are centered and scaled to variance 1, and the additive genetic component is \(g=X\beta\). Write \(R\) for the genotype correlation matrix in the modeled population. Then the genetic variance for a fixed effect vector is \(\operatorname{Var}(g\mid\beta)=\beta^\mathsf{T}R\beta\). Assume the effects are independent across variants, each has mean zero, and each has variance \(\pi\tau^2\) across the full mixture, since it is nonzero with probability \(\pi\). Also assume the diagonal entries of \(R\) are 1.

The expected genetic variance across draws from this prior is

The cross-products have expectation zero because distinct effects are independent and centered. Equating this expected genetic variance to the target \(h^2\) gives \(\tau^2=h^2/(M\pi)\). This calibrates the prior’s expected genetic variance; it does not require every realized effect vector to have genetic variance exactly \(h^2\).

The assumptions matter. The formula uses a defined set of \(M\) modeled variants, standardized genotype columns, and an effect prior with the stated independence and zero-mean properties. Linkage disequilibrium appears in \(R\), but does not alter this expectation under those assumptions: the off-diagonal terms vanish in expectation. The formula is not a license to replace \(M\) with an informal “effective number” of variants. Such a substitution changes the prior and requires a clearly defined model. If genotype or phenotype scales differ from those assumed here, the effect variances must be interpreted or transformed on the corresponding scales.

For an infinitesimal normal prior, every variant is assigned a nonzero effect from a zero-mean normal distribution, so \(\pi=1\) and the variance is \(h^2/M\). A sparse mixture instead has \(\pi<1\), giving larger variance among the nonzero effects when \(h^2\) is held fixed. The overall effect variance across variants remains \(\pi\tau^2=h^2/M\) in either calibration.

How Prior Variance Changes Shrinkage

A one-variant normal model isolates the variance’s direct effect on shrinkage. Suppose an unbiased estimated effect \(b\) has sampling model \(b\mid\beta\sim N(\beta,s_b^2)\), where \(s_b\) is its standard error. Under a normal prior \(\beta\sim N(0,\tau^2)\), the posterior mean is a weighted version of the estimate:

The multiplier lies between 0 and 1. Increasing \(\tau^2\) increases the multiplier and reduces shrinkage toward zero; decreasing \(\tau^2\) does the opposite. In a multi-variant model with LD, the posterior calculations involve the joint information across variants, so this scalar expression is an illustration of the mechanism, not a replacement for the full model.

Polygenicity and the Mixture Prior

In a mixture model, \(\pi\) affects two parts of the prior. It sets the prior probability that an effect is nonzero, and, if \(h^2\) is held fixed, it also changes \(\tau^2=h^2/(M\pi)\). Lowering \(\pi\) therefore makes nonzero effects larger on average but makes any particular variant less likely to be included. These effects can pull a posterior estimate in different directions, so “less polygenic” does not automatically mean more or less shrinkage for every observed variant.

This can be seen with a single estimate \(b\) having sampling variance \(s_b^2\). Under a point mass at zero versus a normal slab \(N(0,\tau^2)\), the Bayes factor for the slab against the null is

The posterior inclusion probability is \(\pi BF_{10}/(1-\pi+\pi BF_{10})\). The posterior mean under this mixture equals that inclusion probability multiplied by the slab’s conditional posterior mean, \(\{\tau^2/(\tau^2+s_b^2)\}b\). Thus both the inclusion probability and the conditional effect size matter. This single-variant calculation illustrates the trade-off; LD-aware genome-wide methods use information across variants as well.

Using Sensitivity Analysis to Assess Score Stability

A sensitivity analysis evaluates a prespecified range of plausible prior settings and records what changes. A useful grid varies \(h^2\) and \(\pi\) separately, rather than varying only their ratio: the ratio determines the slab variance under this calibration, while \(\pi\) also determines the prior inclusion probability. For each setting, record the resulting effects and scores using the same variants, alleles, genotype scaling, and analysis sample. Those controls make differences more interpretable.

For comparison, an infinitesimal prior fixes \(\pi=1\) and spreads variance across all modeled variants. A spike-and-slab prior explicitly represents a nonzero fraction and separates inclusion from effect size. Continuous-shrinkage priors allow a range of effect scales without a single literal inclusion fraction, while penalization methods such as lassosum use a penalty parameter rather than this exact probability model. Their tuning parameters are not interchangeable with \(h^2\), \(\pi\), or \(\tau^2\); comparisons should focus on the assumptions and resulting scores, not on matching parameter labels.

A common mistake is to lower \(\pi\) at fixed \(h^2\) and conclude that all effects must be less shrunk because \(\tau^2\) increases. That reasoning considers only the slab: the prior probability of inclusion falls at the same time. Another mistake is to interpret similar scores across a narrow grid as proof that the chosen heritability is correct. Similarity only indicates limited sensitivity over the settings examined. Conversely, instability can signal that conclusions depend strongly on prior assumptions, but it does not identify which setting is right.

Assumptions, Diagnostics, and Limits

The calibration is most directly interpretable when the phenotype and genotype scales are explicit, the modeled variants are clearly defined, and the prior assumptions are reasonable approximations. Diagnostics include checking whether the implied nonzero-effect standard deviations are extreme, whether score rankings or effect signs change substantially across plausible settings, and whether results are driven by a small subset of variants. A large change in score scale with stable rank correlation should be reported as scale sensitivity, not as full stability.

Heritability and polygenicity are often uncertain, and their values may depend on the population, phenotype definition, and variant set. A prior can be internally consistent and still be misspecified. The calibration also describes a model expectation, not a guarantee about the realized genetic variance, and a stable score need not be well calibrated or predictive. Sensitivity analysis is therefore a disciplined way to expose dependence on assumptions; it cannot replace appropriate evaluation of the score.