Penalty Geometry And Sparse Solutions

Why Penalty Geometry Matters

Regularized regression changes coefficient estimates by adding a penalty to the least-squares loss. The geometry of that penalty helps explain two distinct effects: estimates are pulled toward zero, and some estimates may become exactly zero. For the elastic net, both effects are present because its penalty combines an L1 term, which has corners, and an L2 term, which is smooth.

The distinction matters when interpreting a fitted model. A small coefficient is not the same as a zero coefficient: an exact zero excludes that predictor from the fitted linear combination, while a nonzero coefficient, however small, remains in it. Geometry provides a useful visual explanation, and the optimality conditions make the explanation precise.

Definition: For a response vector $y$, a design matrix $X$, coefficient vector $\beta$, penalty strength $\lambda \geq 0$, and mixture parameter $0 \leq \alpha \leq 1$, the elastic net estimate minimizes $$ \frac{1}{2n}\|y-X\beta\|_2^2 +\lambda\left\{\alpha\sum_{j=1}^{p}|\beta_j|+ \frac{1-\alpha}{2}\sum_{j=1}^{p}\beta_j^2\right\}. $$ The intercept, when included, is ordinarily not part of the penalized coefficient vector. The L1 component is $\sum_j|\beta_j|$; the squared L2 component is $\sum_j\beta_j^2$.

This tutorial uses the objective above, including its factor of $1/2$ on the squared L2 penalty. Changing that convention changes some algebraic factors, so the objective must be identified before applying a coefficient formula. Unless stated otherwise, examples use centered response and predictors scaled so that each predictor has squared length $n$. This scaling makes the loss curvature for an individual coefficient equal to one in an orthogonal design.

Contours and the First-Contact Picture

A contour of the least-squares loss consists of coefficient vectors with the same loss. With two predictors, these contours are typically ellipses; their orientation and width reflect the predictors’ relationship and the scale of the loss. A penalty contour consists of coefficient vectors with the same penalty value. Minimizing the penalized objective can be visualized as expanding a loss contour outward from the least-squares solution until it first reaches an allowed penalty contour. Equivalently, for an appropriate constraint level, one can minimize loss subject to the penalty being no greater than that level.

For two coefficients, an L1 penalty contour is a diamond. Its sharp points lie on the coordinate axes, where one coefficient is zero. A least-squares contour can first touch the diamond at one of these points; this is the geometric reason that lasso-type penalties can produce exact zeros. By contrast, an L2 penalty contour is a circle under equal scaling, with no corners. Its smooth boundary generally pulls coefficients inward without making a coefficient exactly zero.

An elastic net contour combines the two shapes. When $\alpha>0$, the absolute-value term retains a cusp wherever a coefficient is zero, while the squared term adds smooth curvature. The contour is therefore not simply a diamond or a circle: its geometry depends on both components. The cusp still makes exact zeros possible; the L2 component changes the cost of moving away from the origin and the resulting amount of shrinkage.

Penalty component	Two-coefficient contour	Geometric implication
L1: $\sum_j\|\beta_j\|$	Diamond with axis corners	A first contact at a corner can set one coefficient to zero
L2: $\sum_j\beta_j^2$	Smooth circle under equal scaling	Pulls coefficients toward zero, but has no corner that encourages an exact zero
Elastic net: both components	Rounded contour retaining cusps when $\alpha>0$	Combines smooth shrinkage with the possibility of exact zeros

Why the L1 Component Can Create Exact Zeros

The corner argument has an exact counterpart in the optimality conditions. For a differentiable loss $L(\beta)=\|y-X\beta\|_2^2/(2n)$, the derivative with respect to $\beta_j$ is $-X_j^{\mathsf T}(y-X\beta)/n$. The absolute-value function is not differentiable at zero. Its subgradient is the interval $[-1,1]$ at zero, but is the single value $\operatorname{sign}(\beta_j)$ when $\beta_j\ne 0$. This interval allows the loss derivative to be balanced at a zero coefficient.

Theorem: A coefficient vector $\hat{\beta}$ minimizes the elastic net objective if, for each coordinate $j$, $$ 0\in-\frac{X_j^{\mathsf T}(y-X\hat{\beta})}{n} +\lambda\alpha\,\partial|\hat{\beta}_j| +\lambda(1-\alpha)\hat{\beta}_j. $$ In particular, if $\hat{\beta}_j=0$, the necessary coordinate condition is $$ \left|\frac{X_j^{\mathsf T}(y-X\hat{\beta})}{n}\right|\leq \lambda\alpha. $$ The quantity on the left is the absolute residual correlation for predictor $j$, evaluated with the other fitted coefficients in place. The bound applies to a proposed solution with that coefficient zero; it does not by itself determine the other coefficients.

The condition follows by setting zero within the subgradient of the convex objective. At a zero coefficient, the L2 derivative contributes $\lambda(1-\alpha)\hat{\beta}_j=0$. The L1 subgradient can take any value between $-1$ and $1$, so it can offset the loss derivative exactly when the absolute residual correlation is at most $\lambda\alpha$. Away from zero, the subgradient is fixed by the coefficient’s sign, and the L2 derivative contributes additional shrinkage.

Evaluate the loss gradient.
At a candidate fit, calculate the residual correlation $X_j^{\mathsf T}(y-X\hat{\beta})/n$ for the coordinate being checked.

Check a zero candidate.
If $\hat{\beta}_j=0$, compare the absolute residual correlation with $\lambda\alpha$. A value no greater than this threshold is consistent with a zero coefficient.

Check a nonzero candidate.
If $\hat{\beta}_j\ne0$, use the subgradient $\operatorname{sign}(\hat{\beta}_j)$ and include the L2 term $\lambda(1-\alpha)\hat{\beta}_j$ in the stationarity equation.

Orthogonal Predictors: Thresholding and Shrinkage

The geometry becomes especially transparent when predictors are orthogonal: $X_j^{\mathsf T}X_k/n=0$ for $j\ne k$, and $X_j^{\mathsf T}X_j/n=1$. Define $z_j=X_j^{\mathsf T}y/n$. Apart from a term that does not depend on the coefficients, the objective separates into one problem per coordinate:

\frac{1}{2}(\beta_j-z_j)^2 +\lambda\alpha|\beta_j| +\frac{\lambda(1-\alpha)}{2}\beta_j^2.

For $z_j>0$, a positive solution must satisfy the derivative equation $\beta_j-z_j+\lambda\alpha+\lambda(1-\alpha)\beta_j=0$. If $z_j\leq\lambda\alpha$, the zero subgradient condition is satisfied instead. Applying the same argument for negative $z_j$ gives the signed threshold formula below. The numerator performs soft thresholding; the denominator accounts for the extra L2 shrinkage.

\hat{\beta}_j= \frac{\operatorname{sign}(z_j)(|z_j|-\lambda\alpha)_+} {1+\lambda(1-\alpha)}, \qquad (a)_+=\max(a,0).

Worked Example: Separate Thresholding from L2 Shrinkage

Method. Use the coordinate formula for an orthogonal, centered and scaled design. Suppose a hypothetical risk study has two predictors with $z=(1.2,0.3)$, and uses $\lambda=0.4$ and $\alpha=0.5$. Find both elastic net estimates. The intercept is unpenalized and is not part of this calculation.

Step 1: Calculate the L1 threshold. The threshold is $\lambda\alpha=0.4(0.5)=0.2$. The denominator for each coordinate is $1+\lambda(1-\alpha)=1+0.4(0.5)=1.2$.

Step 2: Calculate the first coefficient. Since $z_1=1.2>0.2$, its soft-thresholded numerator is $1.2-0.2=1.0$. Thus $\hat{\beta}_1=1.0/1.2=5/6$, approximately $0.8333$.

Step 3: Calculate the second coefficient. Since $z_2=0.3>0.2$, its numerator is $0.3-0.2=0.1$. Thus $\hat{\beta}_2=0.1/1.2=1/12$, approximately $0.0833$.

The estimates are $(5/6,1/12)$, or approximately $(0.8333,0.0833)$. Check: both $z_j$ values exceed the $0.2$ threshold, so both estimates are nonzero; the positive L2 denominator makes each smaller than its thresholded numerator. A common mistake is to use a threshold of $\lambda=0.4$, or to omit the denominator. The threshold is $\lambda\alpha=0.2$, and the denominator is $1.2$. This calculation avoids treating thresholding and L2 shrinkage as the same operation.

Worked Example: A Coefficient at and Above the Threshold

Method. Apply the orthogonal-design formula and compare each absolute $z_j$ with the L1 threshold. For a hypothetical centered, scaled data set, let $z_1=0.20$ and $z_2=0.25$, with $\lambda=0.5$ and $\alpha=0.4$.

Step 1: Find the threshold and denominator. The threshold is $\lambda\alpha=0.5(0.4)=0.2$. The denominator is $1+\lambda(1-\alpha)=1+0.5(0.6)=1.3$.

Step 2: Evaluate the first coordinate. Since $|z_1|=0.20$ is exactly equal to the threshold, $(|z_1|-\lambda\alpha)_+=0$. Therefore $\hat{\beta}_1=0/1.3=0$.

Step 3: Evaluate the second coordinate. Its numerator is $0.25-0.20=0.05$. Therefore $\hat{\beta}_2=0.05/1.3=1/26$, approximately $0.0385$.

The estimates are $(0,1/26)$, or approximately $(0,0.0385)$. Check: the first coefficient is zero at the boundary, while the second is nonzero because $0.25>0.20$. A common mistake is to use the full $\lambda=0.5$ as the threshold and conclude that both coefficients are zero. The zero threshold here is $\lambda\alpha=0.2$, not $\lambda$. This example also shows why the equality case belongs in the zero region.

Correlated Predictors and the Residual-Correlation Check

With correlated predictors, the objective does not separate into independent coordinate problems. A coefficient cannot generally be computed from its own $z_j$ alone, because the other fitted coefficients affect its residual correlation. The subgradient condition remains valid: to assess a proposed zero, calculate the residual using the full candidate coefficient vector, then compare the absolute residual correlation with $\lambda\alpha$.

Worked Example: Checking a Zero with Correlated Predictors

Method. Use the KKT conditions, including the residual correlation created by the other coefficient. For a hypothetical two-predictor study, suppose the centered, scaled design has Gram matrix $X^{\mathsf T}X/n=\begin{pmatrix}1&0.5\\0.5&1\end{pmatrix}$, and $X^{\mathsf T}y/n=(1,0.2)$. Let $\lambda=0.4$ and $\alpha=0.5$. Find and verify a candidate with the second coefficient zero.

Step 1: Set the second coefficient to zero and solve the first-coordinate equation. The threshold is $\lambda\alpha=0.4(0.5)=0.2$. If $\beta_2=0$ and $\beta_1>0$, stationarity for coordinate 1 gives $1-\beta_1=0.2+0.2\beta_1$. Rearranging, $0.8=1.2\beta_1$, so $\beta_1=2/3$.

Step 2: Check the zero-coordinate condition. The residual correlation for coordinate 2 is $z_2-0.5\beta_1=0.2-0.5(2/3)=0.2-1/3=-2/15$. Its absolute value is $2/15$, approximately $0.1333$, which is no greater than the threshold $0.2$. Thus the second coefficient can indeed be zero.

Step 3: Verify stationarity for coordinate 1. Its residual correlation is $1-\beta_1-0.5\beta_2=1-2/3=1/3$. The required positive-coefficient penalty contribution is $\lambda\alpha+\lambda(1-\alpha)\beta_1=0.2+0.2(2/3)=1/3$, so the equation is satisfied.

The candidate is $(2/3,0)$. The Gram matrix has eigenvalues $0.5$ and $1.5$, so the squared-error loss is strictly convex in the coefficients; adding the convex penalty preserves strict convexity, and this verified candidate is the unique minimizer. Check: both coordinates satisfy their stationarity conditions, including the zero-coordinate inequality. A common mistake is to compare $z_2=0.2$ directly with the threshold and stop. With correlated predictors, the relevant quantity is the residual correlation $z_2-0.5\beta_1$, not $z_2$ alone.

Common Interpretation Traps

Calling every shrunken coefficient sparse. Shrinkage means movement toward zero; sparsity requires exact zero coefficients. A positive L2 component can shrink a coefficient without making it zero.
Using $\lambda$ as the zero threshold. Under the stated objective, the zero-coordinate KKT threshold is $\lambda\alpha$. The L2 component changes the nonzero solution through its derivative, not the zero-coordinate threshold.
Applying the orthogonal formula to correlated predictors. The formula involving $z_j$ alone requires the orthogonal, normalized design described above. With correlation, check residual correlations or solve the coupled optimality conditions.
Ignoring scaling and objective conventions. Predictor scaling affects the geometry, and a different normalization of the loss or penalty changes the numerical threshold. State the objective and scaling before interpreting a coefficient calculation.

Key takeaway: The L1 component creates a nonsmooth cusp at zero, allowing its subgradient to balance a range of residual correlations and produce exact zeros. The L2 component adds smooth curvature and further shrinkage. For an orthogonal normalized design, threshold by $\lambda\alpha$ and then divide by $1+\lambda(1-\alpha)$; for correlated predictors, check the residual correlation at the proposed fit.

Check Your Understanding

For each problem, use the stated elastic net objective and show the threshold, any denominator, and the relevant optimality check before selecting an answer.

A hypothetical orthogonal, centered and scaled design has $z=-0.9$, $\lambda=0.5$, and $\alpha=0.6$. Find the coefficient estimate, showing the threshold and denominator. (Answer: $-0.5$)
For one coordinate in an orthogonal design, $z=0.24$, $\lambda=0.4$, and $\alpha=0.6$. Determine whether the estimate is zero, including the equality or inequality used. (Answer: $0$, since $|z|=\lambda\alpha=0.24$)
A two-predictor orthogonal design has $z=(0.8,-0.1)$, $\lambda=0.3$, and $\alpha=0.5$. Compute both coefficients to four decimal places. (Answer: $(0.5652,0)$)
A hypothetical standardized two-predictor design has Gram matrix $\begin{pmatrix}1&0.25\\0.25&1\end{pmatrix}$, crossproducts $X^{\mathsf T}y/n=(0.9,0.3)$, $\lambda=0.4$, and $\alpha=0.5$. Test the candidate $\beta_2=0$, solve for $\beta_1>0$, and check the residual correlation for coordinate 2. (Answer: $(7/12,0)$; the absolute residual correlation is $0.1542$, below the $0.2$ threshold)
For an orthogonal design with $z=1.5$, $\lambda=0.5$, and $\alpha=0.6$, calculate the threshold, denominator, and coefficient. (Answer: threshold $0.3$, denominator $1.2$, coefficient $1$)

Penalty component	Two-coefficient contour	Geometric implication
L1: \(\sum_j\|\beta_j\|\)	Diamond with axis corners	A first contact at a corner can set one coefficient to zero
L2: \(\sum_j\beta_j^2\)	Smooth circle under equal scaling	Pulls coefficients toward zero, but has no corner that encourages an exact zero
Elastic net: both components	Rounded contour retaining cusps when \(\alpha>0\)	Combines smooth shrinkage with the possibility of exact zeros

What You'll Learn