The Alpha-Lambda Parameterization

Why Two Parameters Are Needed

Elastic net uses two penalty components: an L1 penalty based on the absolute values of coefficients and an L2 penalty based on their squares. Two different questions therefore arise. What is the mix of the two components? And how large is the combined penalty? The parameter $\alpha$ answers the first question; $\lambda$ answers the second.

This distinction matters when reading a fitted model, comparing software output, or transferring tuning values between formulations. A value called $\lambda$ in one objective need not have the same numerical meaning in another: the squared-error term may be normalized differently, or the L2 term may include a factor of one-half. The safest approach is to match the coefficients of each term in the actual objective.

Definition: For a linear regression with response values $y_i$, predictors $x_{ij}$, intercept $\beta_0$, and coefficient vector $\boldsymbol{\beta}=(\beta_1,\ldots,\beta_p)$, a common elastic net objective is $$ \frac{1}{2n}\sum_{i=1}^{n}\left(y_i-\beta_0-\sum_{j=1}^{p}x_{ij}\beta_j\right)^2 +\lambda\left[\alpha\sum_{j=1}^{p}|\beta_j|+\frac{1-\alpha}{2}\sum_{j=1}^{p}\beta_j^2\right]. $$ Here $\lambda\geq 0$ is the overall penalty strength and $0\leq\alpha\leq1$ is the mixing parameter. The intercept is unpenalized in this formulation. The formula assumes the same predictor scaling and loss normalization are used throughout the comparison.

Write $L_1(\boldsymbol{\beta})=\sum_{j=1}^{p}|\beta_j|$ and $L_2^2(\boldsymbol{\beta})=\sum_{j=1}^{p}\beta_j^2$. The penalty in the displayed objective is $\lambda\alpha L_1+\lambda(1-\alpha)L_2^2/2$. Thus, $\lambda\alpha$ is the coefficient multiplying $L_1$, while $\lambda(1-\alpha)/2$ is the coefficient multiplying the squared norm itself.

The factor $1/2$ is part of this convention, not a change to the definition of the squared norm. Some authors instead report the coefficient multiplying $L_2^2$ directly. That coefficient is $\lambda(1-\alpha)/2$, not $\lambda(1-\alpha)$. This distinction is a frequent source of conversion errors.

Separating the Mix from the Scale

A useful alternative writes the objective with separate component weights:

\frac{1}{2n}\sum_{i=1}^{n}\left(y_i-\beta_0-\sum_{j=1}^{p}x_{ij}\beta_j\right)^2 +\lambda_1 L_1(\boldsymbol{\beta})+\frac{\lambda_2}{2}L_2^2(\boldsymbol{\beta}).

For the same objective and the same loss normalization, coefficient matching gives $\lambda_1=\lambda\alpha$ and $\lambda_2=\lambda(1-\alpha)$. In this convention, $\lambda_2$ is the weight before the one-half factor. When $\lambda_1+\lambda_2>0$, the conversion back is $\lambda=\lambda_1+\lambda_2$ and $\alpha=\lambda_1/(\lambda_1+\lambda_2)$.

Identify the component weights.
Rewrite the penalty as $c_1L_1+c_2L_2^2/2$, keeping the one-half factor explicit.

Recover overall strength.
Set $\lambda=c_1+c_2$. This follows because the two weights in the alpha-lambda form sum to $\lambda$.

Recover the mix.
Set $\alpha=c_1/(c_1+c_2)$, provided the sum is positive. The remaining fraction $1-\alpha$ is $c_2/(c_1+c_2)$.

Why the Conversion Works

The conversion follows by equating the coefficients of the two penalty functions, not by comparing fitted coefficients or predictions. Suppose a penalty is written as $c_1L_1+c_2L_2^2/2$, with $c_1,c_2\geq0$. To express it as $\lambda[\alpha L_1+(1-\alpha)L_2^2/2]$, the coefficient of $L_1$ must satisfy $c_1=\lambda\alpha$, and the coefficient of $L_2^2/2$ must satisfy $c_2=\lambda(1-\alpha)$. Adding these equalities yields $\lambda=c_1+c_2$. Dividing the first equality by this sum gives $\alpha=c_1/(c_1+c_2)$. Substitution then reproduces both original weights exactly.

If $c_1=c_2=0$, then $\lambda=0$ and the penalty vanishes. In that case no particular value of $\alpha$ can be recovered: every $\alpha$ gives the same zero penalty. If the component weights are nonnegative and at least one is positive, the conversion produces $\lambda>0$ and $0\leq\alpha\leq1$. Negative component weights do not represent the usual elastic net penalty.

Worked Example: Reading Component Weights from Alpha and Lambda

A model uses the normalized objective above with $\lambda=4$ and $\alpha=0.30$. Write the penalty using separate weights, then evaluate the penalty at $\boldsymbol{\beta}=(2,-1)$. The intercept is excluded.

Method. Match the coefficients in the bracketed penalty. The L1 weight is $\lambda_1=\lambda\alpha=4(0.30)=1.20$. The weight before $L_2^2/2$ is $\lambda_2=\lambda(1-\alpha)=4(0.70)=2.80$. Thus the penalty is $1.20L_1+2.80L_2^2/2$.

For the given coefficients, $L_1=|2|+|-1|=3$, and $L_2^2=2^2+(-1)^2=4+1=5$. The L1 contribution is $1.20(3)=3.60$. The L2 contribution is $2.80(5/2)=2.80(2.5)=7.00$. The total penalty is $3.60+7.00=10.60$.

Check. Directly, the bracketed expression is $0.30(3)+0.70(5/2)=0.90+1.75=2.65$. Multiplying by $\lambda=4$ gives $4(2.65)=10.60$, as before. Common mistake avoided: the L2 contribution is not $2.80(5)$; the component weight $2.80$ multiplies $L_2^2/2$.

When the Squared-Norm Coefficient Is Reported Directly

Another convention writes the penalty as $\lambda_1L_1+c_2L_2^2$, with no one-half attached to the squared norm. In that notation, matching coefficients gives $\lambda_1=\lambda\alpha$ and $c_2=\lambda(1-\alpha)/2$. Consequently, the inverse conversion is $\lambda=\lambda_1+2c_2$ and $\alpha=\lambda_1/(\lambda_1+2c_2)$, when the denominator is positive.

Notice that the denominator is not $\lambda_1+c_2$. The L2 coefficient $c_2$ is half of the weight that would appear before $L_2^2/2$. Before converting, determine whether the stated coefficient multiplies $L_2^2$ or $L_2^2/2$.

Worked Example: Converting Direct L1 and Squared-Norm Weights

A penalized regression objective contains the penalty $0.90L_1+1.80L_2^2$. Express it in alpha-lambda form with the convention $\lambda[\alpha L_1+(1-\alpha)L_2^2/2]$.

Method. The coefficient of $L_1$ is $\lambda\alpha=0.90$. The coefficient of $L_2^2$ is $\lambda(1-\alpha)/2=1.80$, so the corresponding weight before $L_2^2/2$ is $2(1.80)=3.60$. Therefore, $\lambda=0.90+3.60=4.50$, and $\alpha=0.90/4.50=0.20$.

Check. Substitution gives $4.50[0.20L_1+0.80L_2^2/2]$. Its L1 coefficient is $4.50(0.20)=0.90$, and its $L_2^2$ coefficient is $4.50(0.80)/2=1.80$. Both match the stated penalty. Common mistake avoided: using $0.90+1.80$ as $\lambda$ would ignore the one-half convention and would give the wrong mix.

Changing the Scale of the Loss

Penalty weights depend on the scale of the loss term. Multiplying the entire objective by a positive constant does not change its minimizer, but it multiplies every term—including both penalty weights—by that constant. Therefore, values of $\lambda$, $\lambda_1$, or $\lambda_2$ cannot be compared across loss normalizations without adjusting for the multiplier.

For example, multiplying the normalized objective by $2n$ changes the squared-error term from $(1/(2n))\sum_i r_i^2$ to $\sum_i r_i^2$, where $r_i=y_i-\beta_0-\sum_jx_{ij}\beta_j$. The penalty weights become $2n\lambda\alpha$ before $L_1$, and $n\lambda(1-\alpha)$ before $L_2^2$. The mixing parameter remains $\alpha$, since both component weights have been multiplied by the same positive factor. The numerical overall penalty parameter changes with the objective scale.

Worked Example: Translating to an Unnormalized Residual Sum of Squares

Suppose $n=40$, $\lambda=0.05$, and $\alpha=0.60$ in the normalized objective. Rewrite the objective after multiplying every term by $2n$, and verify the component weights.

Method. Here $2n=80$. The original penalty has weight $\lambda\alpha=0.05(0.60)=0.03$ before $L_1$, and weight $\lambda(1-\alpha)=0.05(0.40)=0.02$ before $L_2^2/2$. After multiplying by 80, these become $80(0.03)=2.40$ before $L_1$, and $80(0.02)=1.60$ before $L_2^2/2$.

The transformed objective is therefore $\sum_{i=1}^{40}r_i^2+2.40L_1+1.60L_2^2/2$, or equivalently $\sum_{i=1}^{40}r_i^2+2.40L_1+0.80L_2^2$.

Check. The direct-squared-norm version has L1 weight $2.40$ and squared-norm coefficient $0.80$. Converting those weights back gives $\lambda=2.40+2(0.80)=4.00$ on the unnormalized scale, and $\alpha=2.40/4.00=0.60$. This agrees with multiplying the original $\lambda$ by 80: $80(0.05)=4.00$. Common mistake avoided: carrying over $0.05$ unchanged after rescaling the residual sum of squares would not preserve the objective.

Practical Interpretation and Exam Traps

With $\lambda>0$, increasing $\alpha$ increases the L1 share of the penalty and decreases the L2 share, while holding the total coefficient weight $\lambda$ fixed. Increasing $\lambda$ multiplies both component weights by the same factor when $\alpha$ is fixed. Thus, changing $\alpha$ and changing $\lambda$ are different operations. Neither parameter alone describes both the mixture and the scale.

The ratio of the component weights also offers a quick check. If the weights are $\lambda_1$ before $L_1$ and $\lambda_2$ before $L_2^2/2$, their sum is the overall $\lambda$, and their ratio determines the mix. If the squared-norm coefficient is instead reported directly as $c_2$ before $L_2^2$, its equivalent component weight is $2c_2$. These checks apply only when the loss normalization and coefficient scaling have been identified.

Do not confuse strength and mix. $\lambda$ scales the combined penalty; $\alpha$ allocates that scale between the two components.
Do not drop the half. A coefficient before $L_2^2$ is half the corresponding coefficient before $L_2^2/2$.
Do not compare lambda values blindly. First check whether the loss is a sum, a mean, or a sum divided by $2n$.
Do not infer alpha when the total penalty is zero. If both component weights are zero, the penalty contains no information about the mixing parameter.

Key takeaway: In the convention $\lambda[\alpha L_1+(1-\alpha)L_2^2/2]$, the component weights are $\lambda_1=\lambda\alpha$ and $\lambda_2=\lambda(1-\alpha)$. Add these weights to recover $\lambda$, and divide the L1 weight by their sum to recover $\alpha$. Adjust weights whenever the loss scale or the one-half convention changes.

Check Your Understanding

For each problem, identify the penalty convention before converting. Treat all stated weights as nonnegative.

An objective uses $\lambda=6$ and $\alpha=0.25$, with penalty $\lambda[\alpha L_1+(1-\alpha)L_2^2/2]$. What are the weights before $L_1$ and $L_2^2/2$? (Answer: $1.5$ and $4.5$, respectively.)
A penalty is $2L_1+6L_2^2/2$. Find its alpha-lambda parameters in the same convention. (Answer: $\lambda=8,\ \alpha=0.25$.)
A penalty is $3L_1+2L_2^2$, with no one-half in the displayed L2 term. Convert it to the alpha-lambda convention with $L_2^2/2$. (Answer: $\lambda=7,\ \alpha=3/7$.)
A normalized objective has $n=25$, $\lambda=0.08$, and $\alpha=0.50$. After multiplying the whole objective by $2n$, what are the weights before $L_1$ and $L_2^2/2$? (Answer: $2$ and $2$, respectively.)
A penalty is $0L_1+0L_2^2/2$. Can a unique alpha be recovered from these weights? (Answer: No. The overall penalty strength is zero, so every value of alpha gives the same zero penalty.)

What You'll Learn