Why Two Reasonable Models Can Disagree
An unpenalized generalized linear model (GLM) and an elastic-net model can give different coefficients, fitted values, or predictions even when they are trained on the same observations. The difference is not automatically evidence that one model is wrong. It may reflect the elastic-net penalty pulling coefficients toward zero, correlated predictors sharing predictive information, or a difference in the predictors or terms supplied to the two models.
A useful comparison begins by checking that both models target the same response, use comparable observations and exposure treatment, and make predictions on the same scale. Only then should a difference in fitted effects be attributed to penalization. The examples below use hypothetical insurance and measurement settings; their numbers are illustrative, not industry data.
What the Elastic-Net Penalty Changes
Suppose a GLM has coefficient vector \(\boldsymbol{\beta}\), including an intercept, and a specified response distribution, link function, and design matrix. The unpenalized fit estimates coefficients by maximizing the likelihood, or equivalently minimizing the negative log-likelihood. Its interpretation depends on the chosen model and assumptions: for example, a Poisson log-link model specifies a conditional mean \(\mu_i\) with \(\log(\mu_i)=\eta_i\), where \(\eta_i\) is the linear predictor.
An elastic net adds a penalty to the fitting objective. A common form, with the intercept excluded from the penalty, is:
Here \(L(\boldsymbol{\beta})\) is the model's negative log-likelihood (or a stated loss), \(\lambda\geq 0\) controls penalty strength, and \(0\leq\alpha\leq1\) sets the mix of penalties. At \(\alpha=1\) the penalty is lasso; at \(\alpha=0\) it is ridge. Intermediate values combine them. The precise objective scaling can differ between software implementations, so \(\lambda\) values are not automatically comparable across packages or differently scaled objectives. Predictors also need consistent preprocessing: a penalty on raw coefficients can treat variables differently merely because their units differ.
In a GLM, the penalty acts on coefficients, not directly on a desired amount of change in the response. Consequently, a coefficient pulled toward zero may produce a small or large change in predicted counts, probabilities, or costs depending on the link function, the other terms, and the case being predicted. The intercept is commonly left unpenalized, but the fitting specification should be checked rather than assumed.
Why Shrinkage Occurs: A Short Derivation
The shrinkage effect can be seen exactly in a restricted setting: Gaussian squared-error regression with one standardized predictor, no intercept in the calculation, and objective \(\frac{1}{2n}\sum_i(y_i-x_i\beta)^2+\lambda[\alpha|\beta|+(1-\alpha)\beta^2/2]\). Assume \(\sum_i x_i^2/n=1\), and define \(z=\sum_i x_i y_i/n\). Expanding the squared-error term gives a constant independent of \(\beta\), plus \(\beta^2/2-z\beta\). Thus the part of the objective depending on \(\beta\) is:
For \(\beta>0\), differentiating and setting the derivative to zero gives \((1+\lambda(1-\alpha))\beta-z+\lambda\alpha=0\). This solution is positive only if \(z>\lambda\alpha\). For \(\beta<0\), the derivative of \(|\beta|\) is \(-1\); the corresponding solution is negative only if \(z<-\lambda\alpha\). If \(|z|\leq\lambda\alpha\), the minimum is at zero. Combining these cases yields:
This derivation explains both the lasso threshold, which can set a coefficient to zero, and the additional ridge-like denominator, which shrinks nonzero values. It relies on the stated one-predictor, standardized Gaussian setup. It is not a formula for the exact coefficient of a general correlated-predictor GLM; there the fit is obtained by optimizing the full penalized objective.
Worked Example: A Shrunk Coefficient and Its Fitted Value
A hypothetical measurement study uses one standardized predictor in Gaussian regression. For the fitting calculation, \(n^{-1}\sum_i x_i^2=1\), \(z=n^{-1}\sum_i x_i y_i=0.90\), \(\lambda=0.20\), and \(\alpha=0.50\). Find the elastic-net coefficient and compare its fitted contribution at \(x=1\) with the unpenalized coefficient.
Method. Use the one-predictor solution derived above. Since \(z=0.90>\lambda\alpha=0.20(0.50)=0.10\), the coefficient is positive and is not thresholded to zero.
The numerator after the lasso threshold is \(0.90-0.10=0.80\). The denominator is \(1+\lambda(1-\alpha)=1+0.20(0.50)=1.10\). Therefore, \(\widehat{\beta}_{\mathrm{EN}}=0.80/1.10=0.727272\ldots\), or \(0.7273\) to four decimals. In this standardized, one-predictor setup, the unpenalized coefficient is \(z=0.9000\). At \(x=1\), the elastic-net predictor contribution is \(0.7273\), compared with \(0.9000\) without penalization.
The difference in contribution is \(0.9000-0.727272\ldots=0.172727\ldots\), or \(0.1727\) to four decimals. Check: the penalized coefficient is closer to zero, as expected. This example avoids treating the difference as a response-scale difference: the response-scale change would also depend on the model's link and intercept.
Correlated Predictors and Shared Signal
When predictors are correlated, each may contain overlapping information about the response. A penalized fit can distribute or shrink their coefficients differently from an unpenalized fit. The fitted linear predictor may nevertheless remain similar over the observed data, because it is the combined contribution of predictors that determines the fitted value. Conversely, predictions for cases with unusual combinations of correlated predictors can differ more.
This is a diagnostic point, not a license to interpret every coefficient difference as a penalty effect. Check the design matrix and preprocessing, and compare the combined fitted contributions as well as individual coefficients. If two columns are identical, for example, their separate effects cannot be distinguished by the data alone. An elastic-net penalty may give a well-defined penalized solution in such a setup, but that does not create new information about which input is responsible for the outcome.
Worked Example: Two Identical Predictors Share a Contribution
In a hypothetical Gaussian regression, two standardized predictors are identical: \(x_1=x_2=z\), with \(n^{-1}\sum_i z_i^2=1\). The response is \(y_i=2z_i\). Use the elastic-net objective with \(\lambda=0.40\), \(\alpha=0.50\), and no intercept in this calculation. Find the symmetric penalized coefficients and compare the fitted value at \(z=1\) with the unpenalized fitted value.
Method. By symmetry, set \(\beta_1=\beta_2=b\). The fitted value is \((\beta_1+\beta_2)z=2bz\), so the squared-error part of the objective, per observation under the stated normalization, is \(\frac{1}{2}(2-2b)^2\). For positive \(b\), the penalty is \(\lambda[\alpha(2b)+(1-\alpha)(2b^2)/2]\).
Substituting \(\lambda=0.40\) and \(\alpha=0.50\), the objective as a function of positive \(b\) is \(\frac{1}{2}(2-2b)^2+0.40[0.50(2b)+0.50(2b^2)/2]\). Its derivative is \(-2(2-2b)+0.40(1+ b)\), or \(-4+4b+0.40+0.40b\). Setting it to zero gives \(4.40b=3.60\), hence \(b=3.60/4.40=9/11=0.818181\ldots\). This is positive, so the assumed derivative of the absolute-value terms is valid.
The combined coefficient is \(2b=18/11=1.636363\ldots\), or \(1.6364\) to four decimals. At \(z=1\), the penalized fitted value is \(1.6364\). Without a penalty, the fitted value is \(2\); the data identify the combined coefficient as \(2\), not a unique allocation between the identical columns. The penalized fit's fitted value is lower by \(2-18/11=4/11=0.3636\) to four decimals. Check: both elastic-net coefficients are equal, and their sum—not either coefficient alone—determines the prediction here. This avoids interpreting the allocation as evidence that one of the identical predictors has a stronger effect.
Separate Shrinkage from Model Specification
A comparison can only isolate the effect of the penalty when the models use aligned responses, predictors, transformations, factor encodings, offsets, weights, and observations. If the GLM includes an interaction and the elastic-net design matrix does not, the comparison mixes a penalty difference with a model-specification difference. Elastic net penalizes the supplied terms; it does not automatically invent a missing interaction or nonlinear transformation.
For a log-link model, compare the linear predictors first, then apply the inverse link to obtain response-scale predictions. For a binomial model with a logit link, for example, a difference in log-odds is not the same numerical difference as a probability difference. The same principle applies to counts: exponentiating a difference in log means gives a ratio of means, not a difference in means.
Worked Example: A Missing Interaction Can Mimic Strong Shrinkage
A hypothetical count analysis compares two supplied fitted models. The unpenalized Poisson log-link GLM is \(\log(\mu)=\log(4)+0.40x+0.30z+0.50xz\). The elastic-net Poisson model is \(\log(\mu)=\log(4)+0.35x+0.25z\). For \(x,z\in\{0,1\}\), calculate both predictions at \((1,1)\), and determine whether the gap can be attributed to shrinkage alone.
Method. Compute each model's linear predictor using the terms it actually contains, then exponentiate. At \((1,1)\), the GLM has \(\eta_{\mathrm{GLM}}=\log(4)+0.40+0.30+0.50=\log(4)+1.20\). Thus \(\widehat{\mu}_{\mathrm{GLM}}=4e^{1.20}=4(3.3201169\ldots)=13.2805\) expected counts, to four decimals.
The elastic-net model has \(\eta_{\mathrm{EN}}=\log(4)+0.35+0.25=\log(4)+0.60\). Therefore, \(\widehat{\mu}_{\mathrm{EN}}=4e^{0.60}=4(1.8221188\ldots)=7.2885\) expected counts, to four decimals. Calculating the difference before rounding gives \(4(e^{1.20}-e^{0.60})=5.9919925\ldots\), or \(5.9920\) expected counts to four decimals.
The models agree at \((0,0)\), where both predict \(4\). At \((1,1)\), however, the GLM has an interaction term \(0.50xz\), and the elastic-net model does not. The gap therefore cannot be attributed to coefficient shrinkage alone: the design matrices differ. Check: aligning the terms is necessary before treating a comparison as a diagnosis of the penalty's effect. This avoids wrongly labeling an omitted interaction as shrinkage.
Compare Predictions on the Response Scale
The following example uses aligned predictors and the same exposure offset. It illustrates why a difference in linear predictors should not be reported as a difference in expected counts.
Worked Example: Convert a Log-Mean Gap to Expected Counts
A hypothetical frequency comparison uses Poisson log-link fits with the same exposure of two units and the same predictor values. Their fitted equations for this case reduce to \(\log(\mu)=\log(2)-0.20\) for the GLM and \(\log(\mu)=\log(2)-0.15\) for the elastic net. Calculate both expected counts and their difference.
Method. Exponentiate each linear predictor. The offset is identical in both models, so \(\mu_{\mathrm{GLM}}=2e^{-0.20}=2(0.818730753\ldots)=1.637461506\ldots\), or \(1.6375\) to four decimals. Similarly, \(\mu_{\mathrm{EN}}=2e^{-0.15}=2(0.860707976\ldots)=1.721415952\ldots\), or \(1.7214\) to four decimals.
Calculate the difference from the unrounded values: \(2e^{-0.15}-2e^{-0.20}=0.083954446\ldots\), or \(0.0840\) expected counts to four decimals. The mean ratio is \(e^{0.05}=1.051271\ldots\), so the elastic-net expected count is about \(5.13\%\) higher in this case. Check: the higher log mean produces the higher expected count. This avoids confusing the \(0.05\) difference in log means with the response-scale difference.
A Practical Diagnostic Sequence
When a comparison raises questions about model effects or predictions, use a consistent order. It prevents a difference caused by mismatched inputs from being explained as a penalty effect.
Verify the response definition, prediction target, analysis observations, weights, and offsets. Confirm that both models predict the same quantity.
Compare variables, transformations, factor contrasts, interactions, and any preprocessing. Check whether scaling was performed consistently and whether predictions use the same transformations.
Record the GLM and elastic-net coefficients, the penalty settings, and the scale on which coefficients are reported. For correlated inputs, inspect combined contributions rather than treating each coefficient as an isolated prediction.
For the same case, calculate each linear predictor and apply the inverse link. Report differences on the scale relevant to the decision, such as expected counts or probabilities.
A coefficient change alone does not establish a predictive improvement or loss. Compare the models using aligned validation data and a metric appropriate to the prediction task.
Common Exam Trap
A frequent error is to subtract linear predictors and report the result as the difference in expected counts. In the preceding example, the linear-predictor gap is \((-0.15)-(-0.20)=0.05\). The incorrect calculation is therefore to report a count difference of \(0.05\). That is on the log-mean scale, not the expected-count scale.
The correction is to apply the inverse link to each model and then subtract: \(2e^{-0.15}-2e^{-0.20}=0.083954446\ldots\), which is \(0.0840\) expected counts to four decimals. A related check is to calculate the difference before rounding the two predictions. Subtracting \(1.7214-1.6375\) gives \(0.0839\) because each value has already been rounded; the correctly rounded difference from the unrounded calculations is \(0.0840\).
Check Your Understanding
Work through each comparison, keeping coefficient scale, linear-predictor scale, and response scale distinct.
- For a standardized one-predictor Gaussian fit, let \(z=0.90\), \(\lambda=0.20\), and \(\alpha=0.50\). Find the elastic-net coefficient. (Answer: \(0.7273\) to four decimals.)
- A Poisson log-link model has \(\log(\mu)=\log(e)+\log(3)+0.20x\), where \(e=2\) exposure units and \(x=1\). Find the expected count. (Answer: \(6e^{0.20}=7.3284\) to four decimals.)
- For the same case, two aligned Poisson models have \(\log(\mu_{\mathrm{GLM}})=\log(5)+0.40\) and \(\log(\mu_{\mathrm{EN}})=\log(5)+0.25\). Find the expected-count difference, GLM minus elastic net. (Answer: \(5(e^{0.40}-e^{0.25})=1.0390\) to four decimals.)
- A GLM includes \(0.50xz\) but an elastic-net fit contains only \(x\) and \(z\). Both fits predict at \(x=z=1\). Can their prediction gap be attributed to shrinkage alone? (Answer: No; the models have different specifications because only the GLM includes the interaction.)