Elastic-Net Cox Regression Across Correlated Features

Why Combine Sparsity With Shrinkage?

Omics measurements often contain sets of features that carry overlapping information. A group of highly correlated transcripts, for example, may each reflect a common biological signal. A lasso Cox model can select one member of such a set and exclude the rest, with the chosen member sensitive to small changes in the data. Elastic-net Cox regression adds a quadratic penalty to the lasso penalty. The absolute-value component can set coefficients exactly to zero; the quadratic component discourages one correlated feature from taking all the weight.

This balance is useful when a compact risk score is desired but selecting only one arbitrary representative from a correlated set would be unstable. It does not guarantee that every relevant feature is retained, nor does it identify a causal molecular mechanism. The fitted coefficients describe a penalized association with the modeled hazard, conditional on the included covariates and assumptions.

Model and Penalized Objective

Definition: In a Cox proportional-hazards model, subject $i$, with covariate vector $x_i$, has hazard $h_i(t)=h_0(t)\exp(x_i^\mathsf{T}\beta)$. Here $h_0(t)$ is an unspecified baseline hazard and $\beta$ is the vector of log hazard-ratio coefficients. For $p$ penalized features, elastic-net Cox regression estimates $\beta$ by minimizing $$ -\ell(\beta)+\lambda\left\{\eta\sum_{j=1}^{p}|\beta_j|+\frac{1-\eta}{2}\sum_{j=1}^{p}\beta_j^2\right\}, $$ where $\ell(\beta)$ is the Cox partial log-likelihood, $\lambda\geq 0$ controls overall penalty strength, and $0\leq\eta\leq1$ sets the lasso-to-ridge balance. At $\eta=1$ this is the lasso; at $\eta=0$ it is ridge penalization. Coefficients can be exactly zero when $\eta>0$.

With no tied event times, the partial likelihood is $$ L_{\mathrm{partial}}(\beta)=\prod_{i:\delta_i=1}\frac{\exp(x_i^\mathsf{T}\beta)}{\sum_{k\in R(T_i)}\exp(x_k^\mathsf{T}\beta)},\qquad \ell(\beta)=\log L_{\mathrm{partial}}(\beta), $$ where $T_i$ is the observed time, $\delta_i=1$ indicates an event, and $R(T_i)$ is the set at risk just before that event. A specified ties method is needed when event times are tied; the partial likelihood and its approximation should be used consistently throughout fitting and validation.

The usual working assumptions include proportional hazards for the modeled covariate effects, correctly defined event times and risk sets, and censoring that is noninformative conditional on the modeled information. Penalization addresses estimation when there are many or correlated predictors; it does not repair a wrong survival estimand, unmodeled time-varying effects, or informative censoring. The earlier material on proportional-hazards diagnostics is relevant when assessing the model form.

Features are generally standardized before applying the penalty, because an absolute-value or squared-coefficient penalty otherwise depends on measurement units. Clinical covariates may be left unpenalized when scientifically justified, while omics coefficients are penalized; that choice must be declared. Standardization, imputation, and any feature filtering must be estimated using training data alone within each validation split. This avoids information leakage from held-out subjects.

What the Quadratic Component Changes

The lasso can produce sparse solutions, but with strongly correlated features its likelihood may change little when one feature substitutes for another. The quadratic component penalizes large coefficients and rewards distributing a shared signal rather than assigning a large coefficient to a single arbitrary feature. This is often called a grouping effect. It is a tendency, not a promise that all correlated features will have nonzero coefficients.

A short derivation makes the distinction precise. Suppose two standardized features have positive, nonzero fitted coefficients. At a differentiable optimum, the derivative of the penalized objective with respect to either coefficient is zero. Writing $g_j=\partial[-\ell(\beta)]/\partial\beta_j$, the conditions are $$ g_j+\lambda(1-\eta)\beta_j+\lambda\eta=0,\qquad g_k+\lambda(1-\eta)\beta_k+\lambda\eta=0. $$ Subtracting the equations cancels the lasso terms, giving $$ \lambda(1-\eta)(\beta_j-\beta_k)=-(g_j-g_k). $$ Thus, for $\eta<1$, similar partial-likelihood gradients imply similar coefficients. Highly correlated, similarly scaled features often have similar gradients, so the quadratic component can encourage them to share the fitted signal. When $\eta=1$, this argument provides no control over the coefficient difference: the lasso may choose one feature or another, and exact duplicates can yield nonunique coefficient allocations.

Sparsity is still possible because of the absolute-value term. At a candidate optimum where coefficient $j$ is zero, the subgradient condition is $$ \left|\frac{\partial[-\ell(\beta)]}{\partial\beta_j}\right|\leq\lambda\eta. $$ The quadratic penalty contributes derivative $\lambda(1-\eta)\beta_j=0$ at zero, while the absolute-value penalty supplies the threshold. A feature whose loss gradient falls within this threshold can remain excluded. The condition is evaluated with the other coefficients at their fitted values, not in isolation from the rest of the model.

Worked Example: Two Identical Correlated Features

Consider a hypothetical risk set with two subjects at an event time: one experiences the event and has standardized features $(1,1)$; the other remains event-free at that time and has features $(0,0)$. There are no tied events. For coefficients $\beta_1,\beta_2$, the negative partial log-likelihood is $$ -\ell(\beta)=\log\{1+\exp[-(\beta_1+\beta_2)]\}. $$ The likelihood depends only on the sum $t=\beta_1+\beta_2$, so the features are indistinguishable to this likelihood.

For the lasso with $\lambda=0.2$, positive coefficients incur penalty $0.2t$. Minimizing $\log(1+\exp(-t))+0.2t$ gives derivative $-1/(1+\exp(t))+0.2=0$, hence $t=\log(4)=1.386$. Every nonnegative split of this sum has the same objective: $(1.386,0)$, $(0.693,0.693)$, and $(0,1.386)$ are equivalent. The lasso has no likelihood-based reason to prefer one feature.

Now use $\eta=0.5$ and $\lambda=0.2$. For fixed positive sum $t$, the quadratic penalty is smallest when $\beta_1=\beta_2=t/2$, since $\beta_1^2+\beta_2^2\geq t^2/2$. The objective along this equal split is $$ \log(1+\exp(-t))+0.2\left(\frac{t^2}{8}+\frac{t}{2}\right). $$ Its derivative is $-1/(1+\exp(t))+0.05t+0.1$. Setting this to zero gives approximately $t=1.538$, so $\beta_1=\beta_2\approx0.769$. The negative partial log-likelihood is approximately $\log(1+\exp(-1.538))=0.1946$; the penalty is approximately $0.2(0.2955+0.769)=0.2129$. The quadratic component has selected an equal allocation in this deliberately exact example. Real correlated features are not identical, so their likelihood gradients differ and their coefficients need not match.

Fitting and Tuning in Practice

Specify the survival analysis.
Define the event, time origin, censoring rule, and any delayed-entry risk sets. Choose a ties method if needed. These choices determine the likelihood being penalized.

Prepare predictors within training data.
Standardize penalized predictors and handle missingness using training-split estimates. Decide in advance which covariates, if any, are unpenalized.

Choose a penalty grid and balance.
Fit a range of $\lambda$ values for one or more prespecified $\eta$ values. Larger $\lambda$ generally increases shrinkage; changing $\eta$ changes the balance between exact sparsity and quadratic shrinkage.

Validate and refit.
Choose tuning values using validation data that preserve the intended prediction setting. Refit the selected model on the full training set, then apply its preprocessing and coefficients to an independent test set or target population.

Cross-validation can compare values of $\lambda$ and $\eta$ using held-out partial likelihood or another survival prediction criterion appropriate to the aim. Splits must respect the intended deployment setting, such as separating people from the same cohort when evaluating transport to a new cohort. If the same data are used to choose tuning parameters and to report final performance, that performance can be optimistic; use an independent test set or an appropriately nested validation design.

Worked Example: Applying the Zero-Coefficient Condition

Suppose a hypothetical standardized omics feature is being considered at a candidate fit with $\lambda=0.30$ and $\eta=0.60$. The threshold in the zero-coefficient condition is $\lambda\eta=0.30(0.60)=0.18$. Assume the loss gradient for this feature, evaluated with the other coefficients fixed at their candidate values, is $-0.14$.

Because $|-0.14|=0.14\leq0.18$, a zero coefficient satisfies the subgradient condition. The feature can remain excluded at this fit. If instead its gradient were $-0.24$, then $0.24>0.18$, so zero would not satisfy that condition at the same values of the other coefficients; the fitted coefficient would need to move away from zero to meet the optimum conditions. This comparison is a condition for an optimum, not a guarantee that the feature is biologically unimportant or that it will remain excluded under another sample or tuning choice.

Worked Example: Selecting a Penalty by Validation Loss

Suppose a hypothetical three-fold validation exercise compares three values of $\lambda$ at a fixed $\eta$. The table reports each fold's held-out negative partial log-likelihood per event; smaller values are preferred.

$\lambda$	Fold 1	Fold 2	Fold 3	Mean
0.10	0.62	0.70	0.66	0.660
0.20	0.59	0.63	0.61	0.610
0.40	0.64	0.60	0.62	0.620

For $\lambda=0.10$, the mean is $(0.62+0.70+0.66)/3=1.98/3=0.660$. For $\lambda=0.20$, it is $(0.59+0.63+0.61)/3=1.83/3=0.610$. For $\lambda=0.40$, it is $(0.64+0.60+0.62)/3=1.86/3=0.620$. The smallest mean is 0.610, so this criterion selects $\lambda=0.20$ among the candidates. This selects a tuning value for the stated validation criterion; it does not establish a universally optimal penalty or prove that the selected feature set is stable.

Diagnostics, Alternatives, and Limitations

Check whether the model is failing in ways that the penalty cannot fix. Useful checks include proportional-hazards diagnostics, the number and pattern of events, convergence across the penalty path, sensitivity to preprocessing, and coefficient or selection stability across resamples. A feature selected only under a narrow range of $\lambda$, or one whose selection changes across folds, should not be presented as a robust discovery. Strong correlations can make individual coefficients difficult to interpret even when the combined risk score is useful.

Elastic net differs from two nearby approaches:

Approach	Typical behavior with correlated predictors	Main trade-off
Lasso Cox model ($\eta=1$)	Can select one predictor and exclude close substitutes.	Produces sparse coefficients, but selection can be unstable across correlated features.
Elastic-net Cox model ($0<\eta<1$)	Can shrink correlated predictors together while still setting some coefficients to zero.	Balances sparsity and shared retention; requires choosing both penalty strength and balance.
Ridge Cox model ($\eta=0$)	Shrinks correlated predictors without setting coefficients exactly to zero.	Can stabilize prediction, but does not perform feature selection by zeroing coefficients.

If the scientific goal is to select or retain entire predefined omics blocks, ordinary elastic net is not a block-selection penalty; that is a different modeling objective. Likewise, an elastic-net coefficient is a conditional, penalized association, not an unpenalized effect estimate with ordinary standard errors. Standard inferential procedures do not automatically account for feature selection and tuning. External validation and careful reporting of the full preprocessing and tuning procedure remain important.

Common Pitfalls

Calling grouped retention a guarantee. Correlated predictors may be retained together, but differences in their information, scaling, or gradients can still lead to different coefficients or exclusions.
Interpreting zero as no association. A zero coefficient means the penalized optimization excluded the feature at the chosen fit. It is not proof of no biological or statistical association.
Comparing coefficients on incompatible scales. Penalties act on coefficient magnitudes. Standardize penalized features using training data and state the scale used for interpretation.
Tuning on the test set. Choosing $\lambda$, $\eta$, or preprocessing decisions using test outcomes compromises the independence of the final performance assessment.
Using the penalty to excuse a wrong model. Shrinkage does not resolve non-proportional hazards, a poorly defined estimand, or informative censoring.

Key takeaway: Elastic-net Cox regression minimizes the negative partial log-likelihood plus a penalty that combines an absolute-value term for sparsity and a squared-coefficient term for shrinkage. The quadratic part can stabilize the allocation of signal among correlated features, while validation and survival-model diagnostics remain essential.

Check Your Understanding

Answer each question, then compare with the solutions.

What does $\eta=1$ give, and what does $\eta=0$ give?
For $\lambda=0.25$ and $\eta=0.40$, what is the zero-coefficient threshold $\lambda\eta$? Would a loss gradient of $0.08$ satisfy the zero-coefficient condition?
Why can two identical predictors have an ambiguous allocation under the lasso but an equal allocation under elastic net in the worked example?
Why must standardization and feature filtering be repeated using only each training split during validation?

Solutions. (1) $\eta=1$ gives the lasso penalty; $\eta=0$ gives ridge penalization. (2) The threshold is $0.25(0.40)=0.10$; since $|0.08|\leq0.10$, zero satisfies the condition. (3) The likelihood depends only on the sum of the identical predictors' coefficients, so the lasso assigns the same penalty to any nonnegative split of that sum. The quadratic penalty is minimized by equal coefficients for a fixed sum. (4) Using held-out data to estimate preprocessing or select features allows information from validation subjects to influence the fitted model, making validation performance optimistic.

Approach	Typical behavior with correlated predictors	Main trade-off
Lasso Cox model (\(\eta=1\))	Can select one predictor and exclude close substitutes.	Produces sparse coefficients, but selection can be unstable across correlated features.
Elastic-net Cox model (\(0<\eta<1\))	Can shrink correlated predictors together while still setting some coefficients to zero.	Balances sparsity and shared retention; requires choosing both penalty strength and balance.
Ridge Cox model (\(\eta=0\))	Shrinks correlated predictors without setting coefficients exactly to zero.	Can stabilize prediction, but does not perform feature selection by zeroing coefficients.