Penalizing Dummy Variables and Interactions

Why Coding Matters for Penalized Regression

A categorical predictor can be represented by indicator variables, while an interaction adds a product of predictor columns to the regression model. With ordinary least squares, different full-rank codings can represent the same fitted values. Ridge and lasso, however, penalize the sizes of particular coefficients. If recoding changes those coefficient values, it can change the penalty and therefore the fitted model.

The practical consequence is that coding is not merely a way to label categories when fitting a penalized model. Centering, scaling, the choice of reference category, and the construction of interaction columns all affect which coefficient values are shrunk. The calculations below focus on the usual least-squares objectives with an unpenalized intercept.

Definition: For responses $y_1,\ldots,y_n$, an intercept $\beta_0$, and predictor coefficients $\boldsymbol{\beta}=(\beta_1,\ldots,\beta_p)$, ridge regression minimizes $$ \frac{1}{2n}\sum_{i=1}^n (y_i-\hat{y}_i)^2+\lambda\sum_{j=1}^p \beta_j^2, $$ and lasso regression minimizes $$ \frac{1}{2n}\sum_{i=1}^n (y_i-\hat{y}_i)^2+\lambda\sum_{j=1}^p |\beta_j|. $$ Here $\hat{y}_i=\beta_0+\sum_{j=1}^p x_{ij}\beta_j$, $\lambda\geq 0$, and the intercept is excluded from the penalty. These are least-squares objectives; the penalty sums include whichever encoded predictor and interaction coefficients are being fitted.

The factor $1/(2n)$ is a common objective convention; other conventions rescale the numerical value of $\lambda$. The key issue here is that the penalty acts on the coefficients in the chosen parameterization. It does not directly penalize a category, a fitted value, or an interaction effect independently of its coding.

Factor Coding Changes the Penalty

For a factor with three levels, reference coding uses an intercept for the reference level and one indicator coefficient for each other level. An alternative full-rank coding can use contrasts whose coefficients describe deviations relative to an overall level. Either coding can represent three distinct fitted group means, but the coefficient values need not be the same.

Worked Example: Three-Level Factor Under Two Codings

A hypothetical risk study has a categorical rating factor with levels A, B, and C. Suppose the fitted mean responses are 10, 12, and 16, respectively. Compare the ridge and lasso penalty components for reference coding with A as the reference and for the following effect coding: A has row $(1,0)$, B has row $(0,1)$, and C has row $(-1,-1)$. The intercept is not penalized. This example compares the penalty costs of representations of the same fitted group means; it does not claim that the penalized optimum is unchanged.

Reference coding. The intercept is the fitted mean for A, so $\beta_0=10$. The B indicator coefficient is $12-10=2$, and the C indicator coefficient is $16-10=6$. The ridge penalty component, before multiplication by $\lambda$, is

$$ 2^2+6^2=4+36=40. $$

The lasso penalty component is $|2|+|6|=8$.

Effect coding. Write the intercept as $\alpha$ and the two contrast coefficients as $\theta_A,\theta_B$. The fitted means are $\alpha+\theta_A$, $\alpha+\theta_B$, and $\alpha-\theta_A-\theta_B$. Adding the three fitted means gives $3\alpha=10+12+16=38$, so $\alpha=38/3$. Thus

\theta_A=10-\frac{38}{3}=-\frac{8}{3}, \qquad \theta_B=12-\frac{38}{3}=-\frac{2}{3}.

The C fitted mean checks: $38/3-(-8/3)-(-2/3)=48/3=16$. The ridge penalty component is

\left(-\frac{8}{3}\right)^2+\left(-\frac{2}{3}\right)^2 =\frac{64}{9}+\frac{4}{9} =\frac{68}{9}.

The lasso penalty component is $8/3+2/3=10/3$. Both codings reproduce the same three means, but their ridge costs are 40 and $68/9$, and their lasso costs are 8 and $10/3$. Therefore the penalty can favor different fitted means under different codings. The common mistake is to assume that equal fitted values imply equal penalty costs; the correction is to calculate the penalty from the actual coefficient vector in each coding.

The particular contrast definitions matter too. Changing which category is the reference changes the indicator coefficients. Changing the contrast columns changes contrast coefficients. A penalty on individual coefficients is not generally invariant to these reparameterizations. This does not mean one coding is universally preferable; it means the coding must be chosen deliberately and used consistently when comparing or interpreting penalized fits.

Centering and Scaling Indicator Variables

A binary indicator $D$ equals 0 for one category and 1 for the other. If its sample proportion of ones is $p$, centering gives $D-p$. Standardizing it gives $Z=(D-p)/s_D$, where the standard deviation using divisor $n$ is $s_D=\sqrt{p(1-p)}$. The intercept changes under these transformations, while the slope is rescaled. Since the intercept is not penalized but the slope is, the penalty changes with the scale.

Coefficient transformation: If $Z=(D-p)/s_D$, then $D=p+s_DZ$. Consequently, $$ \beta_0+\beta_DD=(\beta_0+p\beta_D)+(s_D\beta_D)Z. $$ The coefficient on $Z$ is $s_D\beta_D$. Its ridge penalty is $s_D^2\beta_D^2$, and its lasso penalty is $s_D|\beta_D|$. The fitted values are unchanged by the transformation, but the penalty generally is not.

Worked Example: Standardizing an Unbalanced Indicator

In a hypothetical portfolio study, a binary indicator $D$ identifies policies in a particular underwriting category. Exactly 20% of the observations have $D=1$. Suppose the fitted mean is 7 for $D=0$ and 12 for $D=1$. Compare the slope penalties before and after standardizing $D$, using the standard deviation with divisor $n$.

Method and original coding. In the model $\hat{y}=\beta_0+\beta_DD$, the fitted mean at $D=0$ gives $\beta_0=7$. The fitted mean at $D=1$ gives $7+\beta_D=12$, so $\beta_D=5$. The ridge penalty component is $5^2=25$, and the lasso penalty component is $|5|=5$.

Standardize. Here $p=0.20$, so

s_D=\sqrt{0.20(1-0.20)} =\sqrt{0.16} =0.40, \qquad Z=\frac{D-0.20}{0.40}.

Since $D=0.20+0.40Z$, substitution gives

$$ 7+5D=7+5(0.20+0.40Z)=8+2Z. $$

Thus the transformed intercept is 8 and the transformed slope is 2. For $D=0$, $Z=-0.20/0.40=-0.5$, giving $8+2(-0.5)=7$. For $D=1$, $Z=0.80/0.40=2$, giving $8+2(2)=12$. The predictions check. The transformed ridge penalty component is $2^2=4$; the transformed lasso component is 2. The common mistake is to standardize a column and retain its old slope. The correction is to transform the coefficient as well: here the slope changes from 5 to $0.40(5)=2$.

Standardization gives binary indicators equal variance only when their proportions are equal; rare indicators have smaller $s_D$. The coefficient on the standardized column is consequently smaller for a fixed original-scale group difference, and its unweighted penalty is smaller as well. If different indicator columns are standardized differently, the same numerical $\lambda$ does not impose the same penalty on each original-scale contrast.

Centering Predictors in an Interaction Model

An interaction coefficient is also affected by the coding of its component variables. Consider a binary indicator $D$, a quantitative predictor $x$, and the model $\hat{y}=\beta_0+\beta_xx+\beta_DD+\beta_{xD}xD$. Center $x$ at $c$ and $D$ at $p$, defining $x_c=x-c$ and $D_c=D-p$. The centered interaction column is $x_cD_c$.

To translate coefficients, substitute $x=x_c+c$ and $D=D_c+p$ into the original model. The product expands as $xD=x_cD_c+p x_c+cD_c+cp$. Collecting like terms gives the transformation below.

\begin{aligned} \beta_0'&=\beta_0+c\beta_x+p\beta_D+cp\beta_{xD},\\ \beta_x'&=\beta_x+p\beta_{xD},\\ \beta_D'&=\beta_D+c\beta_{xD},\\ \beta_{xD}'&=\beta_{xD}. \end{aligned}

The interaction coefficient stays the same when both variables are shifted but not rescaled. The main-effect coefficients generally change, because the effect of one variable depends on the value of the other. Since ridge and lasso penalize those main-effect coefficients as well as the interaction coefficient, centering can change the total penalty even when it leaves every prediction unchanged.

Worked Example: Centering Both Terms of an Interaction

A hypothetical severity model uses a quantitative measure $x$ and a binary indicator $D$. Its fitted equation is $\hat{y}=10+2x+3D+4xD$. Re-express the model using $x_c=x-2$ and $D_c=D-0.25$, then compare the ridge and lasso penalty components on the three non-intercept coefficients.

Method. Here $\beta_0=10$, $\beta_x=2$, $\beta_D=3$, $\beta_{xD}=4$, $c=2$, and $p=0.25$. The transformed intercept is

\beta_0'=10+2(2)+0.25(3)+(2)(0.25)(4) =10+4+0.75+2 =16.75.

The transformed quantitative main effect is $\beta_x'=2+0.25(4)=3$. The transformed indicator main effect is $\beta_D'=3+2(4)=11$. The interaction coefficient remains $\beta_{xD}'=4$. Thus the centered equation is $\hat{y}=16.75+3x_c+11D_c+4x_cD_c$.

Check the prediction at $x=2,D=1$. The original equation gives $10+2(2)+3(1)+4(2)(1)=25$. In centered coordinates, $x_c=0,D_c=0.75$, so the transformed equation gives $16.75+3(0)+11(0.75)+4(0)(0.75)=25$. At $x=0,D=0$, the original prediction is 10; the centered equation gives $16.75+3(-2)+11(-0.25)+4(-2)(-0.25)=10$. Both checks agree.

Before centering, the ridge penalty component is $2^2+3^2+4^2=29$, and the lasso component is $2+3+4=9$. After centering, the ridge component is $3^2+11^2+4^2=9+121+16=146$, and the lasso component is $3+11+4=18$. The intercept is excluded in both calculations. The common mistake is to change the interaction column but leave its main-effect coefficients untouched. Expanding the product shows why that is wrong: centering the components moves part of the interaction contribution into both main effects.

Practical Checks and Exam Trap

A reliable calculation keeps the fitted equation and the penalty calculation separate. First translate the entire equation into the new columns. Then apply the penalty to the transformed non-intercept coefficients. A coefficient transformation that preserves predictions does not, by itself, preserve the ridge or lasso objective.

Write the columns.
State whether indicators are reference-coded, effect-coded, centered, or standardized, and define each interaction as the product of the columns actually used.

Translate the full equation.
Substitute the original variables in terms of the new columns and collect the intercept, main effects, and interaction terms.

Compute the penalty from those coefficients.
Square and sum for ridge, or take absolute values and sum for lasso; exclude the intercept when using the conventional objective stated here.

Exam trap: assuming a centering transformation leaves the penalty unchanged. In the interaction example, one might incorrectly use the original slopes and report a ridge penalty of $2^2+3^2+4^2=29$ after centering. That calculation combines coefficients from the original equation with columns from the centered equation. The transformed coefficients are $3,11,4$, so the correct ridge penalty component is $3^2+11^2+4^2=146$. Predictions can match exactly while these coefficient penalties differ.

Key takeaway: Ridge and lasso penalties are attached to encoded coefficients. To compare penalized models across factor codings, scaling choices, or centered interaction columns, transform the coefficients first and then calculate the penalty in the new parameterization.

Check Your Understanding

For each problem, track the fitted equation and the penalty separately. Unless stated otherwise, omit the intercept from the penalty.

A three-level factor has fitted means 5, 9, and 11. With the first level as reference, find the two indicator coefficients and their ridge and lasso penalty components. Then use effect coding with rows $(1,0),(0,1),(-1,-1)$ and find the two contrast coefficients and their penalty components. (Answer: Reference coefficients $4,6$, ridge component $52$, lasso component $10$; effect-coded coefficients $-10/3,2/3$, ridge component $104/9$, lasso component $4$.)
A binary indicator has prevalence $0.36$ and an original-scale fitted difference of 10 between its two groups. Using standard deviation $\sqrt{p(1-p)}$, find the standardized slope and the ridge and lasso penalty components before and after standardization. (Answer: Standard deviation $0.48$, standardized slope $4.8$; ridge components $100$ and $23.04$; lasso components $10$ and $4.8$.)
A model is $\hat{y}=6+x+2D+3xD$. Set $x_c=x-1.5$ and $D_c=D-0.4$. Find all transformed coefficients and the ridge penalty component before and after centering. (Answer: Transformed coefficients $\beta_0'=10.1,\ \beta_x'=2.2,\ \beta_D'=6.5,\ \beta_{xD}'=3$; ridge components $14$ and $56.09$.)
A binary variable $D$ is replaced by $W=2D$ in a model containing only an intercept and that predictor. If the original slope is 7, find the slope on $W$ that preserves the same fitted values, then compare the ridge and lasso penalty components. (Answer: The slope on $W$ is $3.5$; ridge components $49$ and $12.25$; lasso components $7$ and $3.5$.)

What You'll Learn