Factor Encoding and Ridge Penalties

Why Factor Encoding Matters for Ridge

A categorical predictor such as plan type or repair category does not have numerical distances that can be used directly in a regression. It must first be represented by numerical columns. In ordinary least squares, different full-rank encodings of the same factor can represent the same fitted values. Ridge regression adds a penalty on coefficients, however, and that penalty is calculated from the coefficients created by the chosen encoding. Consequently, two encodings that describe the same category effects can impose different amounts of shrinkage.

The practical point is not that one encoding is always preferable. It is that a ridge penalty applies to the specified coefficient vector, not to an abstract categorical effect that is automatically invariant to coding. A candidate interpreting a ridge fit should know the reference level, the indicator columns, and which coefficients are penalized.

Reference Coding and the Ridge Objective

For a Gaussian response, a common ridge objective is residual sum of squares plus a penalty on the slopes. With \(p\) predictors and an unpenalized intercept, write

Here \(y_i\) is the response for observation \(i\), \(x_{ij}\) is its value on predictor \(j\), \(\beta_0\) is the intercept, and \(\lambda\) controls the penalty. The intercept is excluded from the penalty. For a factor represented by reference indicators, each indicator coefficient is included in the sum of squared slopes. The exact convention used for the objective can vary, but the encoding issue remains: the penalty acts on the coefficients in the objective.

For a factor with levels A, B, and C and A as the reference, the model’s category contribution is \(\beta_0\) for A, \(\beta_0+\beta_B\) for B, and \(\beta_0+\beta_C\) for C. Thus \(\beta_B\) and \(\beta_C\) describe differences from A. Ridge shrinks these differences toward zero, which pulls the fitted category effects toward the reference level’s effect, conditional on the rest of the model.

Why Changing the Reference Level Changes the Penalty

Suppose the fitted category values are \(m_A,m_B,m_C\), with no other predictors for the moment. Under A-reference coding, the intercept is \(m_A\), and the factor coefficients are \(m_B-m_A\) and \(m_C-m_A\). The ridge penalty for these coefficients is therefore

If B is instead the reference, the coefficients become \(m_A-m_B\) and \(m_C-m_B\), giving penalty

These expressions generally differ. The reason is that changing the reference level changes the intercept and the set of differences being penalized. The change is not just a relabeling of the same slope vector with an unchanged sum of squares.

Other Encodings: Sum-to-Zero and Rescaled Contrasts

Reference coding is not the only option. In sum-to-zero coding, category effects are constrained to add to zero, while the intercept represents the average category contribution under the chosen convention. For three category values, the effects are found by subtracting their average from each value. This can make the intercept and effects easier to describe symmetrically, but the squared effects need not equal the squared reference-coded coefficients.

A different issue arises when a contrast column is rescaled. If a predictor column is multiplied by a constant, its coefficient must be divided by that constant to preserve fitted values. The sum of squared coefficients will then generally change. Thus coding choices include not only which level is the reference, but also the scale of contrast columns.

A Short Justification: Why Rescaling Changes Shrinkage

Suppose a factor is represented by a single indicator \(x\), and its coefficient is \(\beta\). Its contribution to the linear predictor is \(x\beta\), and the penalty is \(\lambda\beta^2\). Now define a rescaled column \(z=cx\), for nonzero \(c\). To keep the same fitted contribution, its coefficient \(\theta\) must satisfy \(z\theta=x\beta\). Since \(z=cx\), this requires \(\theta=\beta/c\). The new penalty is therefore

Unless \(|c|=1\), this is not the original penalty. For example, if \(\beta=4\) and \(c=2\), the equivalent coefficient is \(\theta=2\). The squared coefficient changes from 16 to 4. Predictions are unchanged for that coefficient conversion, but the ridge cost is not. Optimization consequently has a different incentive to shrink the factor effect.

Practical Interpretation and Common Traps

With reference coding, ridge shrinkage pulls each non-reference difference toward zero. That does not mean every category value is pulled toward zero: the intercept is unpenalized, and other predictors and the data also affect the fit. In a model with several factors and numerical predictors, the fitted effect of one factor can depend on the other columns and on their scales.

• Do not interpret an indicator coefficient without naming the reference. A coefficient of 3 means a difference of 3 from the chosen reference level, not an absolute category value of 3.
• Do not confuse a raw difference with a ridge estimate. The worked two-category example had a raw difference of 12 but a ridge coefficient of 4.
• Do not assume reference-level invariance. A different reference changes which contrasts appear in the penalty.
• Do not overlook scaling. A rescaled contrast can represent the same fitted category effects with a different coefficient norm and thus a different penalty.
• State which columns are penalized. An intercept is commonly unpenalized, but software and modeling conventions should be checked rather than presumed.

A full set of \(K\) indicators together with an intercept is linearly redundant because the indicators sum to one. Some modeling setups avoid this by omitting a reference indicator; others use a constraint or a penalty convention that handles the redundancy. These choices can affect the coefficient representation and its penalty. The key is to identify the actual design columns and penalty, rather than assuming that the phrase “factor effect” uniquely determines them.

For an actuarial analysis, coding affects both the numerical fit and how a model’s category effects should be communicated. A penalty may favor one reference level over another in the sense that it assigns different costs to the same fixed set of category values. Therefore, document the factor levels, coding, scaling, and penalty convention when comparing ridge models. A numerical difference between two fits may reflect the representation and not a change in the underlying data.