Why Factor-Level Selection Needs Care
A categorical predictor with several levels is usually represented in a regression model by several indicator columns. Penalized regression applies its penalty to model coefficients, so the indicators for one factor are ordinarily treated as separate coefficients. A factor can therefore have some retained indicators and others set to zero. This distinction matters: retaining one indicator is not the same as selecting the entire factor, and setting one indicator to zero does not necessarily remove the factor’s remaining information.
This tutorial focuses on how the ordinary elastic net treats those separate indicators and how that behavior compares with lasso. The topic is aligned with the SOA Exam PA (Predictive Analytics) syllabus.
The Elastic Net Penalty on Indicator Coefficients
For a response vector \(y\), a design matrix \(X\), and coefficient vector \(\beta\), consider the Gaussian elastic net objective
Here \(n\) is the number of observations, \(\beta_0\) is an unpenalized intercept, \(\lambda\geq 0\) controls the overall penalty, and \(0\leq\alpha\leq1\) sets the balance between the absolute-value and squared-coefficient components. The loss is the squared-error loss scaled by \(1/(2n)\). When \(\alpha=1\), the penalty is lasso; when \(\alpha=0\), it is ridge. For values strictly between zero and one, the objective contains both components.
If a factor has indicator coefficients \(\beta_1,\ldots,\beta_{K-1}\), its contribution to the penalty is
There is no term in this ordinary penalty that says “keep every coefficient for this factor” or “set every coefficient for this factor to zero.” The lasso component can set individual coefficients exactly to zero. The squared-coefficient component shrinks coefficients toward zero but, by itself, does not usually set them exactly to zero. Combining the components does not change the fact that the standard penalty acts coefficient by coefficient.
Worked Example: Calculating the Penalty for One Factor
A four-level categorical predictor is represented by three indicators. In a fitted candidate model, their coefficients are \(0.6\), \(-0.2\), and \(0\). Calculate this factor’s elastic net penalty contribution for \(\lambda=0.4\) and \(\alpha=0.5\), and compare it with its lasso penalty contribution at the same \(\lambda\).
Method. Apply the penalty to the three indicator coefficients separately. The sum of absolute coefficients is \(0.6+0.2+0=0.8\). The sum of squared coefficients is \(0.6^2+(-0.2)^2+0^2=0.36+0.04=0.40\).
The elastic net contribution is \(0.4\{0.5(0.8)+(1-0.5)(0.40)/2\} =0.4(0.40+0.10)=0.20\).
For lasso, \(\alpha=1\), so the contribution is \(0.4(0.8)=0.32\). The third indicator has coefficient zero, but the first two are nonzero: this candidate has partial, not all-or-none, factor retention. The calculation checks because the elastic net contribution includes both an absolute-value part and a squared part, whereas the lasso contribution includes only the absolute-value part. The common mistake is to treat the three indicators as one coefficient or to count a zero indicator as proof that the entire factor was removed.
Why the Elastic Net Coordinate Update Still Acts Individually
The coordinate update makes the separate-indicator behavior explicit. Assume the Gaussian squared-error objective above, with the intercept handled separately. For one standardized column \(x_j\), assume \(x_j^\mathsf{T}x_j/n=1\). Hold all other coefficients fixed and let \(r_j\) be the residual after removing the current contribution of \(x_j\). Define the conditional score \(z_j=x_j^\mathsf{T}r_j/n\). The part of the objective that depends on the candidate value \(b\) for this coefficient, up to a constant, is
The first two terms follow by expanding the squared residual loss. The penalty adds an absolute-value term and another quadratic term. For a positive solution, differentiation gives \((1+\lambda(1-\alpha))b-z_j+\lambda\alpha=0\), so \(b=(z_j-\lambda\alpha)/(1+\lambda(1-\alpha))\), provided \(z_j>\lambda\alpha\). For a negative solution the corresponding condition is \(z_j<-\lambda\alpha\). If the score lies between these thresholds, the minimum is at zero. Together, these cases give the update below.
This derivation shows why an elastic net coordinate update can make one indicator zero while leaving another nonzero: its threshold is applied to each coefficient’s own conditional score. The squared-coefficient component changes the denominator and shrinks a nonzero update; the absolute-value component supplies the zero threshold. The update is conditional on the other coefficients being held fixed. In correlated designs, successive updates interact, so this formula alone does not determine the final fitted model in one step.
Worked Example: Comparing Indicator Updates Under Lasso and Elastic Net
During a coordinate step for two standardized indicators of a factor, suppose the conditional scores are \(z_1=0.90\) and \(z_2=0.08\). All other coefficients are held fixed. Use \(\lambda=0.5\) to calculate the updates for lasso and for elastic net with \(\alpha=0.2\). State what the comparison does and does not establish.
Method. Apply the coordinate update to each score. For lasso, \(\alpha=1\), so the denominator is \(1\) and the threshold is \(\lambda\alpha=0.5\). The first update is \(0.90-0.50=0.40\); the second score is below the threshold, so its update is \(0\).
For elastic net with \(\alpha=0.2\), the threshold is \(0.5(0.2)=0.10\), and the denominator is \(1+0.5(1-0.2)=1.40\). The first update is \((0.90-0.10)/1.40=0.80/1.40=0.5714\) to four decimal places. Since \(0.08<0.10\), the second update is \(0\).
In both coordinate steps, one indicator is updated to a nonzero value and the other to zero. This illustrates that elastic net does not force all indicators of a factor to share one selection decision. It is not a claim that these two numbers are the final coefficients from fitting an entire correlated design: the calculation is conditional on the other coefficients being held fixed. The common mistake is to interpret the smaller elastic net threshold as a guarantee that all factor indicators will be retained.
Lasso and Elastic Net: Similar Selection Unit, Different Shrinkage
For factor indicators, the key similarity is that both ordinary lasso and ordinary elastic net operate on individual coefficients. The lasso’s absolute-value penalty can produce exact zeros. Elastic net retains that feature when \(\alpha>0\), because its penalty still includes the absolute-value component. The added squared-coefficient component changes the amount of shrinkage and can affect which indicators survive, but it does not convert the factor into a single selection unit.
| Penalty | Effect on one factor’s indicators | What it does not guarantee |
|---|---|---|
| Lasso | Penalizes each indicator coefficient by its absolute value; some coefficients can be exactly zero. | It does not require every indicator to be retained or every indicator to be zero. |
| Elastic net with \(\alpha>0\) | Penalizes each coefficient with an absolute-value and a squared-coefficient component. | It does not group the indicators into one all-or-none selection decision. |
| Ridge, the \(\alpha=0\) limit | Shrinks each coefficient through a squared-coefficient penalty. | It ordinarily does not set coefficients exactly to zero or select a factor as a unit. |
Elastic net can behave differently from lasso among correlated predictors, but that does not mean its ordinary penalty selects factor levels as a block. Whether one or several indicators survive depends on the data, the design, the coding, and the penalty strength. A factor with one retained indicator and two zero indicators is a possible coefficient pattern under either lasso or elastic net with an absolute-value component.
Worked Example: Reference Coding Changes the Coefficient Penalty
A model has a three-level factor with fitted cell predictions of 10 for level A, 12 for level B, and 12 for level C. First use A as the reference; then use C as the reference. For each coding, calculate the sum of absolute indicator coefficients and the elastic net penalty contribution for \(\lambda=0.25\), \(\alpha=0.5\). Explain why the two penalties can differ even though the fitted cell predictions match.
Method. With A as reference, the intercept is 10 and the indicator coefficients are \(\beta_B=2\) and \(\beta_C=2\). Their absolute-value sum is \(2+2=4\), and their squared-coefficient sum is \(2^2+2^2=8\). The penalty contribution is \(0.25\{0.5(4)+0.5(8)/2\} =0.25(2+2)=1.00\).
With C as reference, the intercept is 12. The coefficient for A is \(10-12=-2\), and the coefficient for B is \(12-12=0\). The absolute-value sum is \(2+0=2\), and the squared-coefficient sum is \(4+0=4\). The penalty contribution is \(0.25\{0.5(2)+0.5(4)/2\} =0.25(1+1)=0.50\).
Both codings reproduce the same three cell predictions, but the coefficient vectors—and therefore their penalties—differ. A penalized fit can consequently depend on the factor’s coding convention. This calculation checks directly: each coding gives predictions 10, 12, and 12 for A, B, and C. The common mistake is to assume that identical unpenalized fitted values imply identical penalties for every coded coefficient representation.
When the Goal Is to Select a Whole Factor
If the modeling goal requires all indicators for a factor to enter or leave together, the ordinary lasso and elastic net penalties do not impose that requirement. A different penalty, such as a group lasso penalty, is designed to act on a group of coefficients together. In a simplified notation, a group penalty can include a term proportional to \(\|\beta_g\|_2\), where \(\beta_g\) contains all coefficients assigned to group \(g\). That group-level norm can set the group to zero as a whole. This is a different modeling choice from applying an ordinary elastic net penalty separately to each indicator.
Even with ordinary penalties, the phrase “factor selected” can be used informally to mean that at least one indicator coefficient is nonzero. If that shorthand is used, it should be stated clearly. It should not be confused with evidence that every level differs from the reference, nor with a formal all-or-none selection rule.
Common Traps and Practical Interpretation
- Equating a factor with one coefficient. A factor with \(K\) levels commonly contributes \(K-1\) indicator coefficients under treatment coding. Check how many indicators the model matrix actually contains.
- Calling partial retention whole-factor selection. If one indicator is nonzero and another is zero, the model retains a contrast for the factor but has not retained every indicator.
- Assuming elastic net always keeps correlated indicators together. Its squared-coefficient component can affect shrinkage among correlated predictors, but it does not guarantee that every indicator in a factor is selected.
- Assuming coding is irrelevant to a coefficient penalty. Changing the reference can change the coefficient vector and its penalty, even when the unpenalized cell predictions are unchanged.
- Confusing shrinkage with exact selection. The squared-coefficient component shrinks; exact zeros in elastic net with \(\alpha>0\) arise through the absolute-value component.
A sound interpretation names the coding and reports the indicator-level result. For example, “the model retained the contrast for level B relative to the stated reference, but set the level C contrast to zero” is more precise than “the factor was selected.” This distinction is useful when comparing penalized models, explaining results to stakeholders, and deciding whether the scientific question concerns individual contrasts or an entire categorical predictor.
Check Your Understanding
For each problem, track the individual indicator coefficients rather than treating a factor as a single coefficient.
- A five-level factor has four indicator coefficients \(0.7, 0, -0.3, 0\). For \(\lambda=0.2\) and \(\alpha=0.5\), calculate the elastic net penalty contribution. (Answer: \(0.129\).)
- In a coordinate update, a standardized indicator has conditional score \(z=0.62\). Calculate its elastic net update for \(\lambda=0.4\), \(\alpha=0.5\), with all other coefficients held fixed. (Answer: \(0.35\).)
- A factor has three coefficients \(1.2,-0.4,0\). For \(\lambda=0.3\), \(\alpha=0.8\), calculate its elastic net penalty contribution. (Answer: \(0.432\).)
- A three-level factor has cell predictions A = 8, B = 11, C = 8. With A as reference, state the two indicator coefficients and whether this representation has partial or complete indicator retention. (Answer: \(\beta_B=3,\ \beta_C=0\); partial indicator retention.)
- In one fitted elastic net model, two indicators for a factor are nonzero and a third is zero. Does this show that elastic net selected the whole factor as one unit? State the precise interpretation. (Answer: No; it shows that two indicator coefficients were retained and one was set to zero, so the factor was only partially retained at the coefficient level.)