Recovering Coefficients on the Original Scale

Why Recover the Original-Scale Coefficients?

Ridge and lasso are often fitted after centering and scaling predictors so their penalty does not depend arbitrarily on the predictors’ measurement units. The fitted coefficients then describe changes in a standardized response per standardized predictor unit, not changes in the original response per dollar, year, or other original unit. To interpret the model or calculate a prediction from raw measurements, convert the slopes and intercept back together.

The conversion is an algebraic re-expression of the same fitted function. It does not undo shrinkage, change which lasso coefficients are zero, or produce a new fit. It does require keeping track of exactly how the response was handled: left unchanged, centered only, or both centered and scaled.

Definition: For predictor \(j\), let \(\bar{x}_j\) be its training-sample mean and \(s_j>0\) its training-sample scale, commonly its sample standard deviation. Its standardized value is \(z_j=(x_j-\bar{x}_j)/s_j\). Write the fitted model on the transformed variables as \(w=\theta_0+\sum_{j=1}^p\theta_j z_j\). If the response is transformed as \(w=(y-c_y)/s_y\), where \(s_y>0\), then \(\theta_j\) is the fitted coefficient for standardized predictor \(j\), and \(\theta_0\) is the transformed-scale intercept.

Here \(c_y\) is the response centering constant and \(s_y\) is the response scale. Keeping both symbols explicit avoids confusing a centered response with an unchanged one. If the response is fully standardized, usually \(c_y=\bar{y}\) and \(s_y\) is its scale. If it is centered but not scaled, \(c_y=\bar{y}\) and \(s_y=1\). If it is left completely unchanged, \(c_y=0\) and \(s_y=1\).

Conversion Formulas

For each predictor, the original-scale slope is the transformed-scale slope multiplied by the response scale and divided by the predictor scale. The original-scale intercept must also account for every predictor mean and for the response centering constant.

\beta_j=\frac{s_y\theta_j}{s_j},\qquad j=1,\ldots,p,

\beta_0=c_y+s_y\theta_0-\sum_{j=1}^p \beta_j\bar{x}_j =c_y+s_y\theta_0-\sum_{j=1}^p\frac{s_y\theta_j}{s_j}\bar{x}_j.

Thus the same prediction can be written either as \(c_y+s_y(\theta_0+\sum_j\theta_j z_j)\) using transformed variables or as \(\beta_0+\sum_j\beta_jx_j\) using original variables. Apply the training means and scales to new observations; do not recompute them for each prediction.

Record each transformation.
For every predictor, write its training mean \(\bar{x}_j\) and scale \(s_j\). For the response, identify \(c_y\) and \(s_y\).

Convert the slopes.
Calculate \(\beta_j=s_y\theta_j/s_j\) separately for each predictor.

Convert the intercept.
Calculate \(c_y+s_y\theta_0\), then subtract \(\beta_j\bar{x}_j\) for every predictor.

Check a fitted value.
Evaluate one observation on both scales. The two predictions must agree, apart from any rounding.

Why the Formula Holds

The conversion follows by substituting the predictor standardization into the fitted equation and then collecting the constant and predictor terms. No assumption that the fit is ordinary least squares is needed for this algebra.

\begin{aligned} y &=c_y+s_yw\\ &=c_y+s_y\left(\theta_0+\sum_{j=1}^p\theta_j\frac{x_j-\bar{x}_j}{s_j}\right)\\ &=c_y+s_y\theta_0 +\sum_{j=1}^p\frac{s_y\theta_j}{s_j}x_j -\sum_{j=1}^p\frac{s_y\theta_j}{s_j}\bar{x}_j. \end{aligned}

The coefficient multiplying \(x_j\) is therefore \(\beta_j=s_y\theta_j/s_j\). All terms not multiplying an \(x_j\) form the intercept, giving the formula above. This identity also shows why adjusting slopes but leaving the intercept unchanged generally changes every prediction.

Worked Conversions

Worked Example: Fully Standardized Response and Two Predictors

A hypothetical insurer models a standardized annual claim-cost index using two standardized predictors. In the training data, the predictors have means \(\bar{x}_1=10\), \(\bar{x}_2=100\) and scales \(s_1=2\), \(s_2=20\). The response has mean \(\bar{y}=50\) and scale \(s_y=10\), and was transformed as \(w=(y-\bar{y})/s_y\). A ridge fit gives \(\theta_0=0.5\), \(\theta_1=0.8\), and \(\theta_2=-0.3\). Convert the coefficients and check the prediction at \(x_1=12,\ x_2=80\).

Method. Use the general conversion formulas. The ridge penalty determines the fitted \(\theta\) values, but the conversion itself is the same algebra used for any fitted linear predictor.

Slopes. The first original-scale slope is \(\beta_1=10(0.8)/2=4\). The second is \(\beta_2=10(-0.3)/20=-0.15\).

Intercept. First calculate \(\bar{y}+s_y\theta_0=50+10(0.5)=55\). Then subtract the predictor-mean contributions: \(\beta_1\bar{x}_1=4(10)=40\) and \(\beta_2\bar{x}_2=(-0.15)(100)=-15\). Hence \(\beta_0=55-40-(-15)=30\). The original-scale fitted equation is \(\hat{y}=30+4x_1-0.15x_2\).

Prediction check. At \(x_1=12,\ x_2=80\), the standardized predictors are \(z_1=(12-10)/2=1\) and \(z_2=(80-100)/20=-1\). The transformed-scale prediction is \(\hat{w}=0.5+0.8(1)+(-0.3)(-1)=1.6\), so \(\hat{y}=50+10(1.6)=66\). On the original scale, \(\hat{y}=30+4(12)-0.15(80)=30+48-12=66\). The predictions agree. The common mistake avoided here is adding the second mean contribution, \(-15\), rather than subtracting it as a signed quantity.

Worked Example: Centered but Unscaled Response

A hypothetical portfolio model uses one predictor with training mean \(\bar{x}=20\) and scale \(s=4\). The response was centered but not scaled, so \(w=y-\bar{y}\), with \(\bar{y}=200\) and \(s_y=1\). A lasso fit gives \(\theta_0=0.4\) and \(\theta_1=2\). Recover the original-scale equation and predict at \(x=24\).

Method. Use \(c_y=\bar{y}=200\) and \(s_y=1\), because centering does not mean the response is unchanged. The lasso estimate is already given; no refitting is part of this conversion.

Slope. \(\beta_1=1(2)/4=0.5\).

Intercept. The transformed intercept contribution is \(c_y+s_y\theta_0=200+0.4=200.4\). Subtract \(\beta_1\bar{x}=0.5(20)=10\), giving \(\beta_0=190.4\). Thus \(\hat{y}=190.4+0.5x\).

Prediction check. At \(x=24\), \(z=(24-20)/4=1\). The transformed-scale prediction is \(\hat{w}=0.4+2(1)=2.4\), so the response prediction is \(200+2.4=202.4\). The original-scale equation gives \(190.4+0.5(24)=190.4+12=202.4\). The results agree. The key mistake to avoid is treating the centered response as fully unchanged and dropping the response mean.

Worked Example: Response Left Completely Unchanged

A hypothetical claim-frequency model leaves its response in its original units. Its single predictor has training mean \(\bar{x}=5\) and scale \(s=3\). The standardized-predictor ridge fit is \(\hat{y}=\theta_0+\theta_1z\), with \(\theta_0=100\) and \(\theta_1=6\). Convert the equation and check its prediction at \(x=8\).

Method. Since the response was left completely unchanged, set \(c_y=0\) and \(s_y=1\). This is different from centering the response, for which \(c_y\) would be its mean.

Slope. \(\beta_1=1(6)/3=2\).

Intercept. \(\beta_0=0+1(100)-2(5)=90\), so the original-scale equation is \(\hat{y}=90+2x\).

Prediction check. At \(x=8\), \(z=(8-5)/3=1\), and the standardized-predictor equation gives \(100+6(1)=106\). The original-scale equation gives \(90+2(8)=106\). The predictions agree. The common mistake avoided is restoring a response mean that was never removed.

What Changes and What Does Not

The transformation changes the numerical values and units of the coefficients, not the underlying fitted predictions. If a standardized lasso estimate is exactly zero, its original-scale slope remains zero because \(s_y/s_j\) is positive. Nonzero signs are also preserved when the scales are positive. Ridge coefficients likewise convert by the same formula.

Standardization matters to the penalty as well as to interpretation. For example, if the standardized-scale objective adds \(\lambda\sum_j\theta_j^2\), then substituting \(\theta_j=s_j\beta_j/s_y\) expresses that penalty in original-scale slopes as \(\lambda\sum_j s_j^2\beta_j^2/s_y^2\). For a lasso penalty \(\lambda\sum_j|\theta_j|\), the corresponding expression is \(\lambda\sum_j s_j|\beta_j|/s_y\). Thus a standardized fit is not generally the same optimization problem as applying an unweighted penalty directly to the original-unit slopes. The coefficient conversion preserves the fitted model; it does not imply that the penalty is unchanged under a different parameterization.

These formulas assume a linear predictor with a free intercept and the stated affine transformations. In the usual ridge and lasso setup, the intercept is not penalized. If an implementation omits the intercept, constrains it, or uses a different transformation, verify the fitting convention before applying the formula. A predictor with zero scale cannot be standardized by this formula; it is constant in the training data and must be handled separately. At prediction time, use the training-sample means and scales, not new means and scales computed from the prediction cases.

Common Trap: Restoring the Wrong Response Mean

Suppose a response was centered but not scaled: \(w=y-\bar{y}\). Its correct conversion uses \(c_y=\bar{y}\) and \(s_y=1\). Setting \(c_y=0\) just because \(s_y=1\) would incorrectly treat the response as unchanged. For instance, in the centered-response example, that error would give the intercept \(0+0.4-10=-9.6\), instead of \(200+0.4-10=190.4\). At \(x=24\), the erroneous equation predicts \(-9.6+12=2.4\), which is the centered prediction, not the original response prediction \(202.4\).

A reliable check is to write the response transformation explicitly before calculating: unchanged means \(w=y\), centered only means \(w=y-\bar{y}\), and fully standardized means \(w=(y-\bar{y})/s_y\). Only after identifying that transformation should you set \(c_y\) and \(s_y\).

Key takeaway: Convert each slope by multiplying by the response scale and dividing by the predictor scale. Convert the intercept by restoring the response centering, adding the transformed intercept contribution, and subtracting every converted slope times its predictor mean. Check at least one prediction on both scales.

Check Your Understanding

For each problem, identify the response transformation before converting. Answers are given at the end of each question.

A response is fully standardized with \(\bar{y}=30\) and \(s_y=4\). A predictor has \(\bar{x}=8\), \(s=2\), and the fitted standardized-scale coefficients are \(\theta_0=0.25\) and \(\theta_1=1.5\). Find the original-scale equation and its prediction at \(x=10\). (Answer: \(\beta_1=3,\ \beta_0=7;\ \hat{y}(10)=37\).)
A response is centered but not scaled, with \(\bar{y}=80\). A predictor has \(\bar{x}=12,\ s=3\), and a lasso fit has \(\theta_0=-0.5,\ \theta_1=2.4\). Find the original-scale equation and predict at \(x=15\). (Answer: \(\beta_1=0.8,\ \beta_0=69.9;\ \hat{y}(15)=81.9\).)
A response is left unchanged. A predictor has \(\bar{x}=4,\ s=2\), and a ridge fit has \(\theta_0=25,\ \theta_1=-6\). Find the original-scale equation and predict at \(x=5\). (Answer: \(\beta_1=-3,\ \beta_0=37;\ \hat{y}(5)=22\).)
A response is fully standardized with \(\bar{y}=100,\ s_y=5\). Two predictors have means \(2,10\), scales \(1,4\), and fitted coefficients \(\theta_0=-0.2,\ \theta_1=0.6,\ \theta_2=-0.8\). Find the original-scale equation and predict at \((x_1,x_2)=(3,6)\). (Answer: \(\beta_1=3,\ \beta_2=-1,\ \beta_0=103;\ \hat{y}=106\).)
A response is centered but unscaled with \(\bar{y}=45\). A predictor has \(\bar{x}=6,\ s=2\), and a fitted coefficient \(\theta_0=1,\ \theta_1=0\). Give the original-scale equation and explain why the zero slope stays zero. (Answer: \(\beta_1=0,\ \beta_0=46;\ \hat{y}=46\) for every \(x\), because multiplying the zero transformed slope by \(s_y/s\) leaves it zero.)

What You'll Learn