How Alpha Changes Elastic-Net Shrinkage

Why Alpha Matters

Elastic net combines two kinds of coefficient penalty. The lasso component can set coefficients exactly to zero, while the ridge component shrinks coefficients continuously and can stabilize estimates when predictors are correlated. The parameter $ \alpha $ controls the mixture. Understanding that mixture is essential when interpreting an elastic-net fit: changing $ \alpha $ changes not just the number of selected predictors, but also how the penalty treats the coefficients that remain.

The discussion below uses the usual squared-error objective for a linear regression fit. Examples are hypothetical actuarial prediction settings with fixed, standardized design information; no claim is made about a real insurer or portfolio. The focus is the penalty’s behavior, not the choice of a tuning procedure.

Definition: For a response vector $y$, predictor matrix $X$, coefficient vector $\beta$, nonnegative tuning parameter $ \lambda $, and mixture parameter $ \alpha\in[0,1] $, the elastic-net estimate minimizes $$ \frac{1}{2n}\|y-X\beta\|_2^2 +\lambda\left\{\alpha\|\beta\|_1+\frac{1-\alpha}{2}\|\beta\|_2^2\right\}. $$ Here $n$ is the number of observations, $ \|\beta\|_1=\sum_j|\beta_j| $, and $ \|\beta\|_2^2=\sum_j\beta_j^2 $. The intercept, if included, is conventionally not penalized. At $ \alpha=1 $, the penalty is lasso; at $ \alpha=0 $, it is ridge; values strictly between 0 and 1 combine both.

This objective assumes a squared-error fitting criterion. It does not describe a penalized logistic regression objective, which uses a different loss function. The coefficient-penalty interpretation of $ \alpha $ is still useful there, but the squared-error calculations below should not be transferred unchanged to a different loss.

What Changes as Alpha Moves

The ridge part is proportional to $1-\alpha$, and the lasso part is proportional to $ \alpha $. Consequently, increasing $ \alpha $ puts more weight on the lasso component and less on the ridge component, with $ \lambda $ held fixed. This does not mean every nonzero coefficient must steadily get smaller as $ \alpha $ increases. The components have different shapes, and the resulting optimum can move in either direction.

For orthogonal standardized predictors, the effect can be seen directly. Suppose each predictor has mean zero and squared length $x_j^\mathsf{T}x_j/n=1$, and distinct predictors have zero cross-product. Let $z_j=x_j^\mathsf{T}y/n$ be the unpenalized score for predictor $j$, with the other terms in the response handled consistently with the intercept. The separate coefficient update is

\hat{\beta}_j = \frac{\operatorname{sign}(z_j)(|z_j|-\lambda\alpha)_+} {1+\lambda(1-\alpha)}, \qquad (a)_+=\max(a,0).

Thus the lasso component creates a threshold: the coefficient is exactly zero whenever $ |z_j|\leq\lambda\alpha $. Above the threshold, the coefficient is reduced by the ridge denominator as well. When $ \alpha=0 $, the threshold is zero and the update is ridge shrinkage. When $ \alpha=1 $, the denominator is one and the update is soft thresholding, the lasso solution for this orthogonal case.

Why the Orthogonal Update Holds

Under the stated orthogonality and scaling assumptions, the part of the objective involving one coefficient $ \beta_j $, after terms constant in that coefficient are removed, is

\frac{1}{2}(\beta_j-z_j)^2 +\lambda\alpha|\beta_j| +\frac{\lambda(1-\alpha)}{2}\beta_j^2.

For a positive candidate coefficient, differentiating gives $ (\beta_j-z_j)+\lambda\alpha+\lambda(1-\alpha)\beta_j=0 $. Solving yields $ \beta_j=(z_j-\lambda\alpha)/[1+\lambda(1-\alpha)] $, valid when $z_j>\lambda\alpha$. For a negative candidate, the derivative of $ |\beta_j| $ is $-1$, giving $ \beta_j=(z_j+\lambda\alpha)/[1+\lambda(1-\alpha)] $, valid when $z_j<-\lambda\alpha$. At zero, the absolute-value term has subgradient between $-1$ and $1$; zero minimizes the expression precisely when $ |z_j|\leq\lambda\alpha $. Combining these three cases gives the displayed update. The formula depends on orthogonal, unit-scaled predictors and squared-error loss; for correlated predictors the coefficient updates are coupled and this separate formula does not generally apply.

Worked Example: Same Lambda, Different Alpha

A hypothetical pricing model has one standardized predictor orthogonal to the other predictors. Its score is $z=1.2$, and the fixed penalty parameter is $ \lambda=0.4 $. Calculate its coefficient at $ \alpha=0 $, $0.5$, and $1$. Use the orthogonal update and retain at least four decimals in intermediate values.

At $ \alpha=0 $ (ridge): the threshold is $0.4(0)=0$, and the denominator is $1+0.4(1)=1.4$. Therefore $ \hat{\beta}=1.2/1.4=0.857142\ldots $.

At $ \alpha=0.5 $: the threshold is $0.4(0.5)=0.2$, so the numerator is $1.2-0.2=1.0$. The denominator is $1+0.4(0.5)=1.2$. Thus $ \hat{\beta}=1.0/1.2=0.833333\ldots $.

At $ \alpha=1 $ (lasso): the threshold is $0.4$, the numerator is $1.2-0.4=0.8$, and the denominator is $1$. Thus $ \hat{\beta}=0.8 $. Rounded to four decimals, the estimates are 0.8571, 0.8333, and 0.8000. Each is positive and exceeds its applicable threshold before division, so none is set to zero. The values decrease in this particular example; that is not a general monotonicity rule.

Alpha, Sparsity, and Shrinkage Are Not Synonyms

Sparsity refers to having many coefficients exactly equal to zero. In the orthogonal update, a positive $ \alpha $ creates a nonzero cutoff $ \lambda\alpha $; increasing $ \alpha $ at fixed $ \lambda $ raises that cutoff. Ridge alone has no such cutoff, so with nonzero scores it typically leaves all coefficients nonzero. By contrast, shrinkage describes reducing coefficient magnitudes toward zero. Both ridge and lasso shrink, but only the lasso component produces exact zeros in the standard elastic-net penalty.

A larger $ \alpha $ therefore tends to make exact zeros more available, but it does not guarantee that a particular fitted model will have more zeros in every dataset. For correlated predictors, coefficients are jointly optimized, not updated independently. Further, the coefficient sizes among those that remain depend on both the threshold and the ridge denominator. It is safer to describe the change in penalty geometry than to promise a fixed change in every coefficient.

Worked Example: Which Predictors Become Zero?

A hypothetical claim-frequency model is being illustrated using the squared-error elastic-net objective and three orthogonal, standardized predictors. Their scores are $z_1=1.1$, $z_2=0.7$, and $z_3=0.25$. At $ \lambda=0.5 $, calculate the coefficients at $ \alpha=0.8 $, then compare them with $ \alpha=0.2 $. This is a fixed-data illustration, not a statement that a frequency outcome should generally be modeled with squared error.

For $ \alpha=0.8 $: the threshold is $0.5(0.8)=0.4$ and the denominator is $1+0.5(0.2)=1.1$. The first coefficient is $(1.1-0.4)/1.1=0.7/1.1=0.6364$. The second is $(0.7-0.4)/1.1=0.3/1.1=0.2727$. Since $0.25\leq0.4$, the third coefficient is zero. The result is approximately $(0.6364,0.2727,0)$.

For $ \alpha=0.2 $: the threshold is $0.5(0.2)=0.1$ and the denominator is $1+0.5(0.8)=1.4$. The coefficients are $(1.1-0.1)/1.4=1.0/1.4=0.7143$, $(0.7-0.1)/1.4=0.6/1.4=0.4286$, and $(0.25-0.1)/1.4=0.15/1.4=0.1071$. All three are nonzero. The smaller alpha has reduced the lasso threshold, so the weakest score survives. These calculations check against the rule that a coefficient is zero exactly when its absolute score does not exceed the applicable threshold.

Correlated Predictors and the Grouping Tendency

When predictors are strongly correlated, many combinations of their coefficients can produce similar fitted values. Lasso may choose one predictor and set another to zero, particularly when the data provide little information to distinguish their contributions. Adding a ridge component discourages large, uneven coefficient patterns because it penalizes the sum of squared coefficients. This produces a grouping tendency: highly correlated predictors can be treated more similarly than under lasso alone.

The grouping tendency is not a guarantee that correlated predictors receive equal coefficients or are always selected together. Exact equality follows in a symmetric special case, not as a universal property. The following calculation shows why ridge can favor sharing a combined effect.

Worked Example: Two Identical Predictors Share an Effect

Suppose two standardized predictors are identical, so their fitted contribution depends only on $t=\beta_1+\beta_2$. Their common score is $z=1.2$. Use $ \lambda=0.5 $ and $ \alpha=0.4 $, and restrict attention to the optimal nonnegative coefficients. Find the total coefficient and the individual coefficients.

For a fixed positive total $t$, the lasso term is $ \lambda\alpha t=0.2t $, regardless of how the total is divided. The ridge term is minimized by an equal split: for any $ \beta_1+\beta_2=t $, $ \beta_1^2+\beta_2^2=t^2/2+(\beta_1-\beta_2)^2/2\geq t^2/2 $, with equality at $ \beta_1=\beta_2=t/2 $. The objective, omitting constants, is therefore $ \frac12(t-1.2)^2+0.2t+0.075t^2 $, because $0.5(1-0.4)/2=0.15$ multiplies $ \beta_1^2+\beta_2^2=t^2/2 $.

Differentiate with respect to $t$: $ (t-1.2)+0.2+0.15t=0 $. Hence $1.15t=1.0$, so $t=1/1.15=0.869565\ldots$. The equal coefficients are $t/2=0.434782\ldots$ each. To check the role of the ridge component, with lasso alone the objective depends on the positive coefficients through their sum, so any nonnegative split of the optimal total $t=1.2-0.5=0.7$ has the same penalty and fit. The ridge term makes the equal split strictly preferable in this identical-predictor example. Real predictors are rarely identical, so this calculation illustrates a tendency rather than a universal equality rule.

Common Exam Trap: Assuming Alpha Only Changes Sparsity

A common mistake is to say that increasing $ \alpha $ must make every nonzero coefficient smaller because the lasso component becomes stronger. The calculation in the first worked example is a useful correction: at the same $ \lambda=0.4 $, the coefficient at $ \alpha=0.5 $ is $0.8333$, not a value obtained by simply subtracting half the lasso threshold from the ridge coefficient. The correct calculation changes both the threshold and the denominator.

For example, a tempting but wrong calculation at $ \alpha=0.5 $ is to use the lasso threshold $0.2$ but leave the ridge denominator at $1.4$, giving $ (1.2-0.2)/1.4=0.7143 $. This combines pieces from different alpha values. The correct denominator is $1+\lambda(1-\alpha)=1.2$, giving $1.0/1.2=0.8333$. Another error is to assume a higher alpha means more shrinkage in every nonzero coefficient; the formula shows that thresholding and the denominator both change, and correlated predictors add further coupling.

Alpha setting	Penalty emphasis	Typical interpretation
$ \alpha=0 $	Ridge only	Continuous shrinkage; ordinarily no exact-zero selection
$0<\alpha<1$	Ridge and lasso	Shrinkage plus the possibility of exact zeros; ridge can encourage grouping
$ \alpha=1 $	Lasso only	Threshold-based shrinkage and exact-zero selection; correlated predictors may compete

In R interfaces such as glmnet, the argument alpha represents this mixture parameter and lambda controls the overall penalty scale. A comparison at the same lambda is a comparison of the stated objectives, not necessarily a comparison at equal predictive performance or equal effective shrinkage. Alpha should be interpreted alongside the fitted coefficients and the chosen lambda; tuning both parameters together is a separate decision.

Key takeaway: Increasing alpha shifts the penalty from ridge toward lasso. This generally makes exact zeros more attainable and reduces the ridge-based grouping influence, but it does not imply that every nonzero coefficient decreases monotonically. Interpret the mixture and the fitted result, not alpha in isolation.

Check Your Understanding

Use the orthogonal standardized squared-error update where specified. Each problem is hypothetical; answers are rounded to four decimal places when needed.

A standardized predictor has score $z=0.9$. With $ \lambda=0.3 $ and $ \alpha=0.5 $, calculate its elastic-net coefficient. (Answer: $0.6522$)
A standardized predictor has score $z=0.4$, with $ \lambda=0.6 $ and $ \alpha=0.75 $. Determine whether its coefficient is zero, and give the threshold used. (Answer: Zero; the threshold is $0.4500$, which exceeds $|z|=0.4000$.)
A standardized predictor has score $z=1.5$, with $ \lambda=0.5 $ and $ \alpha=0.2 $. Calculate the coefficient, showing the threshold and denominator. (Answer: Threshold $0.1000$, denominator $1.4000$, coefficient $1.0000$.)
Two identical standardized predictors have common score $z=1.0$. With $ \lambda=0.4 $ and $ \alpha=0.5 $, assume optimal coefficients are nonnegative. Find the total coefficient and each individual coefficient. (Answer: Total $0.7273$; each coefficient $0.3636$.)
Two orthogonal standardized predictors have scores $0.8$ and $0.35$. At $ \lambda=0.4 $, calculate their coefficients for $ \alpha=0.5 $ and then for $ \alpha=1 $. What happens to the weaker predictor? (Answer: At $ \alpha=0.5 $, coefficients are $0.5000$ and $0.1250$; at $ \alpha=1 $, they are $0.4000$ and $0$. The weaker predictor is thresholded to zero at $ \alpha=1 $.)

Alpha setting	Penalty emphasis	Typical interpretation
\( \alpha=0 \)	Ridge only	Continuous shrinkage; ordinarily no exact-zero selection
\(0<\alpha<1\)	Ridge and lasso	Shrinkage plus the possibility of exact zeros; ridge can encourage grouping
\( \alpha=1 \)	Lasso only	Threshold-based shrinkage and exact-zero selection; correlated predictors may compete

What You'll Learn