Ridge Regression Weights Under Correlated Genotypes

Why Estimate Genotype Weights Jointly?

A polygenic score adds genotype values multiplied by their weights. When variants are correlated through linkage disequilibrium, the genotype columns in a regression contain overlapping information. A small change in the phenotype data can then produce large changes in separate unpenalized regression weights, even when the combined predictions change little. Ridge regression addresses this instability by penalizing the size of the weights while estimating them jointly.

The penalty does not make correlated variants independent, nor does it identify which variant is biologically responsible for an association. It changes the estimation problem: instead of minimizing prediction errors alone, ridge regression trades some fit to the observed data for smaller, more stable weights. This tutorial derives that trade-off using individual-level genotypes and phenotypes.

Model and Ridge Objective

Suppose \(n\) individuals have a continuous phenotype and genotype measurements at \(m\) variants. Let \(y_i\) denote the phenotype of individual \(i\), and let \(x_{ij}\) denote that individual’s genotype value at variant \(j\), often coded as an effect-allele count or dosage. The model represents the phenotype as an intercept, a weighted sum of the genotype values, and a residual:

In matrix notation, \(y\) is the \(n\)-vector of phenotypes, \(X\) is the \(n \times m\) genotype matrix, \(\boldsymbol{\beta}\) is the vector of \(m\) variant weights, and \(\boldsymbol{\varepsilon}\) is the residual vector. The intercept \(\alpha\) is not penalized. For the usual linear-model interpretation, assume the mean phenotype is described by this additive form and the errors have conditional mean zero; independent errors with common variance are a common additional working assumption. Genotype columns may be correlated. The ridge calculation itself does not require the errors to be normally distributed.

Center each genotype column and the phenotype by subtracting its sample mean. The intercept can then be fitted separately, and the centered-data ridge estimate is defined by minimizing the residual sum of squares plus a squared-weight penalty:

The objective uses the residual sum of squares (RSS), not an average RSS. Consequently, the numerical value of \(\lambda\) depends on this convention and on the scale of the data. If the columns are not centered, the intercept should still be fitted but excluded from the penalty. Centering makes that convention especially transparent.

Deriving the Joint Ridge Weights

Expanding the objective gives \(y^\mathsf{T}y-2\boldsymbol{\beta}^\mathsf{T}X^\mathsf{T}y+\boldsymbol{\beta}^\mathsf{T}X^\mathsf{T}X\boldsymbol{\beta}+\lambda\boldsymbol{\beta}^\mathsf{T}\boldsymbol{\beta}\). Differentiating with respect to \(\boldsymbol{\beta}\) and setting the gradient equal to zero yields the ridge normal equations:

For \(\lambda>0\), the matrix \(X^\mathsf{T}X+\lambda I\) is invertible, even if some genotype columns are exact linear combinations of others. Thus the unique ridge estimate is:

At \(\lambda=0\), this reduces to the ordinary least-squares normal equations; a unique ordinary least-squares coefficient vector requires \(X\) to have full column rank. The key stabilizing effect of a positive \(\lambda\) can be seen directly. For any nonzero vector \(v\), \(v^\mathsf{T}(X^\mathsf{T}X+\lambda I)v=\|Xv\|^2+\lambda\|v\|^2>0\). The matrix is therefore positive definite and invertible.

This derivation also clarifies what is meant by joint weights: each estimate depends on the full matrix \(X^\mathsf{T}X\), which contains both the genotype-column sums of squares and their cross-products, as well as on all the phenotype-genotype cross-products in \(X^\mathsf{T}y\). A marginal regression of phenotype on one variant at a time uses neither the same predictor set nor, in general, the same weights.

What Correlation Does to the Weights

When two genotype columns are strongly correlated, they can explain much of the same variation in the phenotype. The data may provide limited information about how to divide that shared signal between their separate coefficients. With no penalty, different combinations of large positive and negative weights can yield similar fitted values. This is why unstable individual coefficients can occur even when the overall fitted phenotype is comparatively stable.

Ridge regression discourages large weights by adding \(\lambda\) to the diagonal of the genotype cross-product matrix. It therefore improves the conditioning of the system being solved. It usually shrinks correlated weights toward one another in the sense that it reduces their overall magnitude, but it does not in every dataset make each coefficient smaller in absolute value than its unpenalized counterpart. Correlation and the pattern of phenotype-genotype cross-products determine how the weights change.

A useful way to see the effect is to express the problem in the directions of the genotype matrix’s singular vectors. If \(X=UDV^\mathsf{T}\), where the diagonal entries \(d_k\) of \(D\) are the singular values, then the ridge estimate can be written as:

A direction with a small singular value corresponds to a combination of genotype columns with little independent variation in the data. Its multiplier \(d_k/(d_k^2+\lambda)\) is reduced by the penalty. In a highly correlated region, such weakly identified directions are precisely where unpenalized estimates can be especially sensitive to small data changes. Ridge stabilizes these directions rather than trying to estimate them without constraint.

Scaling, Units, and the Choice of Penalty

The penalty acts on the numerical coefficients, so changing genotype scale changes the ridge fit unless the penalty is adjusted accordingly. If one genotype column is measured in dosage units and another is rescaled, the same value of \(\lambda\) does not penalize their effects in the same way. A common approach is to center genotype columns and scale them to comparable variance before fitting. The resulting coefficients then correspond to that scaled genotype representation, not directly to a one-allele dosage change.

The following small calculation shows why scaling matters. It uses one centered genotype column, so there is no correlation with another variant; the point is solely the dependence of ridge weights on scale.

Interpreting and Using Ridge Weights

Given a fitted vector of weights, the genetic component of the predicted phenotype for an individual is the sum of their genotype values multiplied by the corresponding weights. If the model includes an intercept, its prediction is the intercept plus that sum. The weights are joint prediction coefficients conditional on the other genotype columns included in the model. Their interpretation depends on the phenotype scale, genotype coding and scaling, variants included, and penalty value.

Ridge weights are generally not exactly zero when \(\lambda>0\). The penalty shrinks weights continuously rather than removing selected variants from the model. It can therefore be useful when many variants may contribute and correlated genotype columns make an unpenalized joint fit unstable. The resulting weights are not guaranteed to be unbiased estimates of biological effects: the penalty deliberately introduces shrinkage, and prediction performance depends on how well the fitted model transfers to the intended prediction population.

The parameter \(\lambda\) controls the strength of the penalty, but the best value is not determined by the normal equations alone. It must be chosen in a way that reflects the prediction goal and avoids using evaluation data to fit or tune the model. The algebra here explains what a chosen \(\lambda\) does to the weights; it does not establish that any particular value will generalize well.

Common Mistakes

• Treating joint weights as marginal GWAS effects. A marginal effect comes from fitting a variant separately; a ridge weight is estimated alongside the other included genotype columns. The two can differ, especially when variants are correlated.
• Assuming a positive penalty solves every modeling problem. Ridge makes the penalized coefficient system invertible when \(\lambda>0\), but it does not correct phenotype misspecification, genotype errors, ancestry mismatch, or inappropriate validation.
• Comparing weights without checking genotype scaling. A coefficient per raw dosage unit is not directly comparable to one per standardized genotype unit. Report the coding and scaling used.
• Interpreting a shrunken coefficient as a causal effect. The weight is a penalized prediction coefficient conditional on the included columns. Shrinkage alone does not establish causation.
• Penalizing the intercept by accident. The standard setup leaves the intercept unpenalized. Centering the outcome and genotype columns provides a convenient way to fit it separately.