Sparse-Group Cox Models Within Omics Blocks

Why Penalize at Two Levels?

An omics block may contain thousands of features, while the number of patients and observed events is much smaller. A group penalty can encourage the model to include or exclude an entire block, but when a block is included it does not necessarily reduce that block to a small feature set. A feature-level penalty can create sparsity within blocks, but does not by itself encourage block-level selection. The sparse-group Cox model combines both goals: it can exclude a block and, within a retained block, set some coefficients exactly to zero.

The model is useful when features have a meaningful block structure, such as separate assay layers, pathways, or prespecified feature sets. The resulting selection is conditional on the chosen block definitions, preprocessing, clinical covariates, penalty weights, and tuning procedure; it is not by itself evidence that a selected feature causes an outcome.

Model and Penalized Objective

For subject \(i\), let \(T_i\) be the event or censoring time, \(\delta_i\) indicate whether the event was observed, and \(x_i\) be the vector of omics features. Let \(z_i\) contain clinical covariates whose coefficients are not penalized. Partition the omics features into \(G\) nonoverlapping blocks, with coefficient vector \(\beta_g\) and \(p_g\) features in block \(g\). The Cox model is

h_i(t\mid x_i,z_i)=h_0(t)\exp(x_i^\mathsf{T}\beta+z_i^\mathsf{T}\gamma).

Here \(h_0(t)\) is an unspecified baseline hazard. With distinct event times, define the risk set \(R(T_i)=\{j:T_j\geq T_i\}\) for each observed event. The partial log-likelihood is

\ell(\beta,\gamma)= \sum_{i:\delta_i=1} \left[ x_i^\mathsf{T}\beta+z_i^\mathsf{T}\gamma -\log\left\{\sum_{j\in R(T_i)} \exp(x_j^\mathsf{T}\beta+z_j^\mathsf{T}\gamma)\right\} \right].

For tied event times, a specified tie method, such as the Breslow or Efron method, replaces the distinct-time expression. The method must be used consistently in fitting and evaluation. The sparse-group estimate minimizes the negative partial log-likelihood plus a combined penalty:

Q(\beta,\gamma)= -\ell(\beta,\gamma) +\lambda\left[ \eta\sum_{g=1}^{G}w_g\|\beta_g\|_2 +(1-\eta)\|\beta\|_1 \right].

The tuning parameter \(\lambda\geq 0\) controls the overall penalty, and \(0\leq\eta\leq1\) controls its balance. A common group weight is \(w_g=\sqrt{p_g}\), which adjusts for block size; other prespecified weights may be appropriate. When \(\eta=1\), the penalty is group lasso. When \(\eta=0\), it is lasso. For intermediate values, both block-level and feature-level sparsity are encouraged. The clinical coefficients \(\gamma\) are unpenalized in this formulation.

Definition: A sparse-group Cox model is a Cox proportional-hazards model estimated by minimizing the negative partial log-likelihood plus a weighted sum of group \(L_2\) penalties and an overall featurewise \(L_1\) penalty. A block is selected when at least one coefficient in it is nonzero; feature selection within a selected block is determined by which individual coefficients remain nonzero.

The usual assumptions include independent subjects, correctly defined event and censoring times, noninformative right censoring conditional on modeled covariates, and proportional hazards for the included effects. The omics feature values must be available at the prediction time intended by the analysis. If observations are dependent, risk sets or uncertainty calculations must reflect that design. The partial likelihood handles the unspecified baseline hazard, but it does not repair violations of proportional hazards or informative censoring. The earlier material on high-dimensional Cox regression and Group-Lasso Cox Models For Omics Blocks provides context for the likelihood and the group-only penalty.

Worked Examples

Worked Example: Computing a Cox Loss and Sparse-Group Penalty

Consider a hypothetical three-subject study with one omics feature in a single block and no clinical covariates. Subject A has an event at time 1, subject B has an event at time 2, and subject C is censored after time 2. Their feature values are \(x_A=1\), \(x_B=0\), and \(x_C=-1\). There are no tied event times, so the distinct-time partial likelihood applies. Evaluate the objective at \(\beta=\log(2)\), with \(\lambda=0.2\), \(\eta=0.5\), and \(w_1=1\).

At time 1, all three subjects are at risk. The event subject is A, so the partial-likelihood contribution is

\frac{\exp(\beta x_A)} {\exp(\beta x_A)+\exp(\beta x_B)+\exp(\beta x_C)} = \frac{2}{2+1+1/2} =\frac{4}{7}.

At time 2, B is the only subject with an observed event and the risk set is \(\{B,C\}\). Its contribution is

\frac{\exp(\beta x_B)} {\exp(\beta x_B)+\exp(\beta x_C)} = \frac{1}{1+1/2} =\frac{2}{3}.

Thus \(L(\beta)=(4/7)(2/3)=8/21\), and the negative partial log-likelihood is \(-\log(8/21)=\log(21/8)\approx0.9651\). Since the block has one feature, its group norm and the featurewise absolute value are both \(|\beta|=\log(2)\). The penalty is

0.2\left[0.5\log(2)+0.5\log(2)\right] =0.2\log(2)\approx0.1386.

Therefore \(Q(\beta)\approx0.9651+0.1386=1.104\). This calculation checks the risk sets and penalty at a specified coefficient; it does not establish that this coefficient minimizes the objective.

Worked Example: A Proximal Step Can Keep Only Some Features

Suppose a fitting algorithm takes a proximal-gradient step for one four-feature block. Let the vector before applying the penalty be \(u=(0.8,-0.3,0.1,0)\), with step size absorbed into \(u\). Use \(\lambda=0.5\), \(\eta=0.4\), and \(w_g=\sqrt{4}=2\). First apply featurewise soft thresholding at \(\lambda(1-\eta)=0.3\):

S_{0.3}(u) = \bigl(\operatorname{sign}(u_k)(|u_k|-0.3)_+\bigr)_{k=1}^4 =(0.5,0,0,0).

The group threshold is \(\lambda\eta w_g=0.5(0.4)(2)=0.4\). The norm of the thresholded vector is \(0.5\), so group shrinkage multiplies it by \(1-0.4/0.5=0.2\). The updated block is \((0.1,0,0,0)\): the block remains selected, but only its first feature is nonzero.

For comparison, if a second block has pre-threshold vector \(u=(0.25,-0.2,0.1,0)\), each absolute entry is at most \(0.3\). Featurewise thresholding therefore gives the zero vector, and group shrinkage leaves it zero. The two stages explain how a block can be discarded altogether or retained with a sparse subset of features.

Worked Example: Checking Whether a Block Can Stay Zero

For a differentiable loss \(f=-\ell\), a zero block satisfies the optimality condition when the block gradient can be balanced by both penalty subgradients. In a proximal step this is equivalent to checking whether the feature-thresholded negative gradient has norm no greater than the group threshold. Consider two hypothetical blocks with \(\lambda=0.4\), \(\eta=0.5\), and four features per block, so the feature threshold is \(0.2\) and the group threshold is \(0.4\).

For the first block, suppose its loss gradient is \(g=(0.55,-0.35,0.12,-0.08)\). Soft-threshold \(-g\) at \(0.2\):

S_{0.2}(-g)=(-0.35,0.15,0,0), \qquad \|S_{0.2}(-g)\|_2 =\sqrt{0.35^2+0.15^2} =\sqrt{0.145} \approx0.3808.

Since \(0.3808\leq0.4\), the zero-block condition holds at this gradient, so the block can remain zero at an optimum. For a second block, let \(g=(0.8,-0.6,0.3,0)\). Then

S_{0.2}(-g)=(-0.6,0.4,-0.1,0), \qquad \|S_{0.2}(-g)\|_2 =\sqrt{0.36+0.16+0.01} =\sqrt{0.53} \approx0.7280.

Here \(0.7280>0.4\), so the zero-block condition fails: the penalty is not sufficient to keep this block at zero at the stated gradient. This is an optimality check, not a guarantee that the block’s final coefficients or selected features will remain unchanged as the other model coefficients are updated.

Why the Proximal Update Works

The negative Cox partial log-likelihood is convex in the linear predictors, and the sparse-group penalty is convex. A common fitting approach uses a gradient step for the smooth loss followed by a proximal step for the penalty. For a block vector \(u\), the proximal operation solves

\underset{b}{\operatorname{argmin}}\; \frac{1}{2}\|b-u\|_2^2 +\lambda(1-\eta)\|b\|_1 +\lambda\eta w_g\|b\|_2.

The featurewise \(L_1\) term first gives \(v=S_{\lambda(1-\eta)}(u)\). The remaining group penalty shrinks this vector radially toward zero. The result is

\operatorname{prox}(u)= \begin{cases} 0, & \|v\|_2\leq\lambda\eta w_g,\\[4pt] \left(1-\dfrac{\lambda\eta w_g}{\|v\|_2}\right)v, & \|v\|_2>\lambda\eta w_g. \end{cases}

To see why the group threshold is \(\lambda\eta w_g\), consider the group-only minimization after feature thresholding. Away from zero, the derivative of the quadratic term is \(b-v\), while the derivative of the group norm is \(\lambda\eta w_g b/\|b\|_2\). Setting their sum to zero makes \(b\) point in the direction of \(v\), with its length reduced by \(\lambda\eta w_g\). At zero, the group-norm subgradient is the unit \(L_2\) ball, so zero minimizes the expression precisely when \(\|v\|_2\leq\lambda\eta w_g\).

This argument also gives the block-level optimality condition: for a block at zero, soft-threshold the negative loss gradient coordinatewise at \(\lambda(1-\eta)\); its \(L_2\) norm must be no greater than \(\lambda\eta w_g\). For an active block, zero coefficients within it must satisfy the corresponding featurewise subgradient condition, accounting for the group-norm derivative from the other active coefficients.

Practical Fitting, Diagnostics, and Alternatives

Feature scales matter because the featurewise penalty acts on coefficient magnitudes. Omics features are commonly centered and scaled using training data only, with the same transformations applied to validation data. The group weights should reflect the block definition and size; \(w_g=\sqrt{p_g}\) is common, but it does not make different blocks biologically equivalent. Clinical covariates that are not penalized should be included consistently in every fit. Tuning \(\lambda\) and \(\eta\) should use validation that respects the intended prediction setting and avoids information leakage. If performance is compared across candidates, the same folds and a prespecified survival prediction criterion should be used.

Useful diagnostics include convergence of the objective, small optimality or KKT residuals, and agreement of the fitted solution across reasonable solver tolerances. Along a penalty path, larger \(\lambda\) should generally produce greater shrinkage; abruptly changing selections can indicate weakly identified or highly correlated predictors. Refit stability across resamples can reveal whether selection depends heavily on a small number of subjects. Proportional-hazards diagnostics remain relevant after fitting: sparsity does not make the proportional-hazards assumption true.

Approach	What it encourages	Important distinction
Lasso Cox	Featurewise sparsity	Does not explicitly favor selecting whole blocks
Group-lasso Cox	Blockwise inclusion or exclusion	Typically retains all features in a selected block
Sparse-group Cox	Blockwise and within-block sparsity	Requires tuning both overall strength and penalty balance
Elastic-net Cox	Feature sparsity plus coefficient shrinkage for correlated predictors	Does not use block-level selection unless a group penalty is added

Sparse-group selection has limits. Correlated features may substitute for one another, so a small change in data or tuning can change which feature is selected even when their joint signal is similar. Overlapping groups require a formulation that explicitly handles overlapping membership; simply applying the displayed penalty to overlapping lists can count a coefficient multiple times. The penalty also does not provide ordinary unpenalized standard errors or account automatically for selection uncertainty. Selected coefficients are shrunken, and their values should not be interpreted as unbiased effect estimates. The subsequent material on Bayesian shrinkage addresses a different regularization framework; it does not remove the need to examine model assumptions and validation design.

Common Pitfalls

Equating a selected block with an all-included block. The group penalty can retain a block while the feature penalty sets many of its coefficients to zero.
Reading a zero coefficient as proof of no association. It means the fitted penalized objective set that coefficient to zero under this feature set, tuning choice, and preprocessing.
Comparing unscaled coefficients across features without checking units. Scaling changes the penalty’s effect, so preprocessing must be reported and applied without using held-out data.
Treating tuning as inference. Cross-validation selects a predictive tuning choice under a chosen validation design; it does not turn selected features into confirmed discoveries.

Key takeaway: Sparse-group Cox regression applies a group norm to encourage block selection and an \(L_1\) norm to encourage feature selection within blocks. Both the resulting support and its scientific interpretation depend on assumptions, scaling, tuning, and stability.

Check Your Understanding

Answer each question, then compare with the solutions below.

In the displayed penalty, what happens when \(\eta=1\), and what happens when \(\eta=0\)?
A four-feature block has \(w_g=2\), \(\lambda=0.3\), and \(\eta=0.5\). What are its feature threshold and group threshold?
For a proximal step, the feature-thresholded vector has norm \(0.25\), and the group threshold is \(0.3\). Is the updated block zero or nonzero?
Why can two highly correlated features in the same block have unstable individual selection?
Name one model assumption that sparse-group penalization does not enforce.

Solutions. (1) At \(\eta=1\), the featurewise \(L_1\) component vanishes and the penalty is group lasso. At \(\eta=0\), the group component vanishes and the penalty is lasso. (2) The feature threshold is \(\lambda(1-\eta)=0.3(0.5)=0.15\); the group threshold is \(\lambda\eta w_g=0.3(0.5)(2)=0.3\). (3) The block is zero because \(0.25\leq0.3\). (4) Correlated features can carry similar information, allowing small data or tuning changes to shift the fitted coefficient from one feature to the other. (5) Examples include proportional hazards and noninformative censoring; neither follows from penalization.