Interpreting Shrunk Coefficients in Genomic Prediction Models

What a Shrunk Coefficient Does—and Does Not—Say

A genomic prediction model may assign a nonzero coefficient to a gene-expression feature or variant, but the coefficient is not automatically an estimate of that feature’s independent biological effect. In a penalized model, coefficient values are deliberately modified to improve prediction, particularly when the number of features is large relative to the number of individuals. The size and even the sign of a coefficient can depend on the penalty, feature scaling, and which other correlated features are included.

The earlier material on elastic-net prediction introduces the model-fitting goal. Here the focus is how to read its fitted coefficients. The central distinction is between a coefficient that helps form predictions under a specified penalized model and an unpenalized effect estimate with an inferential interpretation. In a high-dimensional setting, the latter may not even be uniquely estimable without additional assumptions.

The Elastic-Net Coefficient Scale

Consider \(n\) individuals and \(p\) genomic predictors. Suppose the response and predictor columns have been centered, each predictor has been scaled to have sample standard deviation one, and the intercept is left unpenalized. For clarity, take the centered response so the intercept is zero. A common elastic-net estimate minimizes

Here \(y\) is the response vector, \(X\) is the predictor matrix, and \(\beta\) is the vector of coefficients. The parameter \(\lambda\) controls the overall penalty strength; \(\alpha=1\) gives the lasso and \(\alpha=0\) gives ridge regression. The \(L_1\) part can set coefficients exactly to zero, while the \(L_2\) part continuously penalizes large values. This objective specifies the optimization procedure; it does not require normally distributed errors. Interpreting the model as a conditional mean, however, assumes that the mean response is adequately represented by the chosen linear predictor.

Standardization matters because the penalty acts on coefficient values. Without scaling, a predictor measured in large numerical units can have a small coefficient for purely unit-related reasons, while another predictor measured on a smaller scale can have a large coefficient. Standardization gives coefficients a common predictor scale, but it does not make them comparable as biological effects when correlations, coding, or outcome units differ.

Why Orthogonal Predictors Give a Clean Shrinkage Rule

A useful special case makes the mechanics explicit. Suppose the centered, standardized predictor columns are orthogonal, so \(X^\mathsf{T}X/n=I\). For feature \(j\), define \(z_j=x_j^\mathsf{T}y/n\), the unpenalized least-squares coefficient in this orthogonal setting. The objective separates into one term per coefficient:

For a nonzero coefficient, the derivative of the smooth parts is \((1+\lambda(1-\alpha))\beta_j-z_j\). The \(L_1\) term has derivative \(\lambda\alpha\) when \(\beta_j>0\) and \(-\lambda\alpha\) when \(\beta_j<0\). At zero, its subgradient ranges from \(-\lambda\alpha\) to \(\lambda\alpha\). Thus zero minimizes the term whenever \(|z_j|\leq\lambda\alpha\); otherwise solving the derivative equation gives the signed, thresholded value:

In this special case, the lasso component first thresholds small values to zero, and the ridge component further divides surviving values by a factor greater than or equal to one. The sign of every nonzero coefficient agrees with \(z_j\). With correlated genomic predictors, the objective no longer separates by feature, so this simple rule does not apply coordinate by coordinate.

Correlation Can Change Magnitudes and Signs

Genomic features are often correlated. When two features carry overlapping information, the model’s coefficients describe their contributions conditional on each other. A coefficient can therefore be small even when its feature is strongly associated with the response on its own. Conversely, an estimated coefficient can become larger in absolute value as the penalty changes, or can cross zero. “Shrinkage” refers to the penalty’s effect on the fitted vector as a whole; it does not guarantee that every individual coordinate moves monotonically toward zero.

This point can be seen directly for ridge regression, the \(\alpha=0\) elastic-net case. With standardized predictors, let \(G=X^\mathsf{T}X/n\) and \(c=X^\mathsf{T}y/n\). The ridge estimate satisfies \((G+\lambda I)\hat{\beta}=c\). If \(G\) is diagonal, each coordinate is simply reduced by a factor. When \(G\) has off-diagonal entries, each coefficient depends on the other features’ information as well.

Interpreting Units and Prediction Contributions

If both the response and predictor are standardized, a coefficient of \(0.24\) means that, within the fitted linear predictor and holding the other included predictors fixed, a one-standard-deviation increase in that predictor corresponds to a \(0.24\)-standard-deviation increase in the model’s predicted response. It is a conditional predictive contribution in the selected model. Correlation among predictors can make that conditional comparison quite different from a one-feature-at-a-time association.

Suppose the predictor was standardized from original units with standard deviation \(s_x\), and the response from original units with standard deviation \(s_y\). A standardized coefficient \(\hat{\beta}_{\mathrm{std}}\) corresponds to the original-scale coefficient \(\hat{\beta}_{\mathrm{raw}}=\hat{\beta}_{\mathrm{std}}s_y/s_x\), provided the transformations were ordinary centering and scaling. This conversion changes units; it does not undo penalization or turn a predictive coefficient into an unbiased effect estimate.

Predictive Coefficients Versus Effect Estimates

An unpenalized regression coefficient is defined by a different fitting criterion: minimizing residual sum of squares without a coefficient penalty. When \(p>n\), the ordinary least-squares coefficients are generally not unique, because many coefficient vectors can produce the same fitted values. A penalized solution selects a particular vector through the penalty, but its uniqueness does not establish that its coordinates are biological effects.

Even when an unpenalized model can be fit, a penalized coefficient is intentionally biased by the penalty. It may offer better prediction by trading some coefficient bias for reduced variability. The earlier material on post-lasso refitting discusses refitting without shrinkage after selection; that produces a different estimator and does not erase the fact that selection occurred. Neither a penalized coefficient nor a post-selection coefficient alone supplies valid uncertainty quantification or causal identification.

A predictive coefficient is conditional on the response definition, predictor coding, training data, model family, scaling, selected \(\lambda\), and \(\alpha\). If these change, coefficients can change even when predictions remain similar. For example, correlated genes can substitute for one another in forming predictions. The feature receiving more coefficient weight may be less important as an isolated biological target than the numerical coefficient suggests.

Diagnostics, Alternatives, and Limitations

Before interpreting coefficients, verify that preprocessing was learned from training data only and then applied unchanged to validation or test data. Check the scaling convention, allele or category coding where relevant, and whether the response itself was standardized. Examine correlations among features with large coefficients, and compare coefficient paths across plausible penalty values. Large changes in magnitude or sign are a warning that feature-level interpretation is unstable, even if prediction error changes little.

The main alternatives answer different questions. Ridge regression retains all predictors and distributes weight across correlated features; lasso can produce exact zeros but may select one of several correlated features somewhat arbitrarily; elastic net combines these behaviors. Unpenalized fitting targets residual fit rather than regularized prediction and may be undefined or highly variable when \(p\) is large relative to \(n\). None of these choices automatically makes coefficients causal or provides valid effect-size inference.

A common mistake is to say, “The gene has a positive biological effect because its fitted coefficient is positive.” The correction is: “In this specified penalized prediction model, the gene has a positive coefficient conditional on the other included predictors and the stated coding and scaling.” A stronger biological or causal claim requires a design and assumptions suited to that target, not simply reading the sign of a prediction coefficient.