Centering Predictors and the Intercept

Why Center Predictors?

In multiple regression, the intercept describes the fitted response when every predictor equals zero. That can be mathematically valid but practically unhelpful if zero is far from the values observed. Centering gives zero a more useful reference: subtract a chosen value from each numeric predictor, such as its sample mean or a meaningful benchmark. The resulting intercept describes the fitted response at those reference values.

Centering changes the coordinate system used to describe the same additive regression surface. Provided the model includes an intercept and the design matrix has full column rank, it changes the intercept but not the slopes or any fitted value. The algebra is important: centering is not the same as changing predictor units, and it does not, by itself, alter the model's fit.

Definition: In a multiple linear regression with \(p\) numeric predictors \(x_1,\ldots,x_p\), choose fixed constants \(c_1,\ldots,c_p\) and define the centered predictors \(z_j=x_j-c_j\). The constants \(c_j\) are the centering values. If \(c_j=\bar{x}_j\), the sample mean of predictor \(j\), the predictor is mean-centered.

How the Coefficients Change

Assume the model includes an intercept and has the additive form

Y_i=\beta_0+\beta_1x_{i1}+\cdots+\beta_px_{ip}+\varepsilon_i,\qquad i=1,\ldots,n.

Here \(Y_i\) is the response for observation \(i\), \(\beta_0\) is the intercept, \(\beta_j\) is the slope for predictor \(j\), and \(\varepsilon_i\) is the error. For the usual OLS interpretation of the model, the errors have conditional mean zero given the predictors; constant conditional variance is additionally assumed when using the usual homoskedastic standard-error formulas. The centering identities below are algebraic and do not require normal errors.

Substitute \(x_{ij}=z_{ij}+c_j\) into the model and collect the constant and predictor terms:

\begin{aligned} Y_i &=\beta_0+\sum_{j=1}^{p}\beta_j(z_{ij}+c_j)+\varepsilon_i\\ &=\left(\beta_0+\sum_{j=1}^{p}\beta_jc_j\right) +\sum_{j=1}^{p}\beta_jz_{ij}+\varepsilon_i. \end{aligned}

Thus the centered-model intercept is \(\gamma_0=\beta_0+\sum_{j=1}^{p}\beta_jc_j\), while each centered-model slope is \(\gamma_j=\beta_j\). The intercept moves because the origin has moved; the slopes stay the same because a one-unit change in \(z_j\) is still a one-unit change in \(x_j\), with the other predictors held fixed.

Key result: With an intercept, replacing each predictor \(x_j\) by \(z_j=x_j-c_j\) gives the same fitted response at every observation. The centered intercept is the original model's fitted response at \(x_1=c_1,\ldots,x_p=c_p\). This result assumes the same additive terms are included and the OLS coefficient representation is unique (full column rank).

Worked Example: Re-expressing a Two-Predictor Fit

A hypothetical model estimates annual claim cost \(Y\), measured in thousands of dollars, using two numeric predictors \(x_1\) and \(x_2\). Its fitted equation is \(\widehat{Y}=12+3x_1-2x_2\). Re-express the fit after centering \(x_1\) at 4 and \(x_2\) at 5, then verify the fitted value at \(x_1=6,\ x_2=2\).

Method. Use \(z_1=x_1-4\) and \(z_2=x_2-5\), so \(x_1=z_1+4\) and \(x_2=z_2+5\). Substitute these into the original equation:

\begin{aligned} \widehat{Y} &=12+3(z_1+4)-2(z_2+5)\\ &=12+3z_1+12-2z_2-10\\ &=14+3z_1-2z_2. \end{aligned}

The centered intercept is \(14=12+(3)(4)+(-2)(5)\). Both slopes are unchanged. At the chosen center, \(z_1=z_2=0\), the fitted cost is 14 thousand dollars.

For \(x_1=6,\ x_2=2\), the centered values are \(z_1=6-4=2\) and \(z_2=2-5=-3\). The centered equation gives

\widehat{Y}=14+3(2)-2(-3)=14+6+6=26.

The original equation gives \(12+3(6)-2(2)=12+18-4=26\), so the two representations agree. The common mistake to avoid is changing a slope when only the origin has changed.

Why OLS Fitted Values Are Unchanged

The coefficient transformation above does more than produce matching formulas for one observation: it gives a one-to-one correspondence between the original and centered coefficient choices. For any original coefficients \((\beta_0,\beta_1,\ldots,\beta_p)\), define \(\gamma_0=\beta_0+\sum_j\beta_jc_j\) and \(\gamma_j=\beta_j\). The substitution shows that the predicted response for every observation is identical under the two coefficient vectors. Conversely, \(\beta_0=\gamma_0-\sum_j\gamma_jc_j\) and \(\beta_j=\gamma_j\), so every centered coefficient vector also corresponds to an original one.

OLS chooses coefficients to minimize the residual sum of squares (RSS), the sum of squared differences between observed and fitted responses. Because corresponding coefficient vectors give the same fitted responses, they give the same residuals and the same RSS. They therefore have the same minimizing fitted values. If the design matrix has full column rank, the minimizing coefficient vector is unique in each representation, so the slopes correspond exactly as stated. This is the justification for the invariance result; it is not a claim that the intercept itself stays fixed.

Choose reference values.
Set \(z_{ij}=x_{ij}-c_j\) for each predictor and observation.

Translate the coefficients.
Keep each slope and set the new intercept to \(\beta_0+\sum_j\beta_jc_j\).

Check a prediction.
Evaluate both equations at the same original predictor values; the predictions must agree.

Worked Example: Mean-Centered Predictors and the Response Mean

In a hypothetical full-rank OLS model for annual cost in thousands of dollars, \(x_1\) is a numeric exposure measure and \(x_2\) is a numeric risk score. For 80 policies, the sample means are \(\bar{x}_1=10\), \(\bar{x}_2=3\), and the sample response mean is \(\bar{Y}=18.6\). The fitted slopes are \(b_1=0.4\) and \(b_2=-0.2\). Find the original and mean-centered intercepts, then predict for \(x_1=12,\ x_2=4\).

Method. In OLS with an intercept, the residuals sum to zero. Hence the sample response mean equals the sample mean of the fitted responses. The fitted equation is linear, so averaging its fitted values gives

\overline{\widehat{Y}} =b_0+b_1\bar{x}_1+b_2\bar{x}_2.

Since \(\overline{\widehat{Y}}=\bar{Y}\), solve for the original intercept:

\begin{aligned} b_0 &=\bar{Y}-b_1\bar{x}_1-b_2\bar{x}_2\\ &=18.6-(0.4)(10)-(-0.2)(3)\\ &=18.6-4+0.6=15.2. \end{aligned}

Now center at the predictor means: \(z_1=x_1-10\) and \(z_2=x_2-3\). The centered intercept is

\begin{aligned} b_0+b_1\bar{x}_1+b_2\bar{x}_2 &=15.2+(0.4)(10)+(-0.2)(3)\\ &=15.2+4-0.6=18.6. \end{aligned}

Thus the mean-centered fitted equation is \(\widehat{Y}=18.6+0.4z_1-0.2z_2\). At \(x_1=12,\ x_2=4\), the centered values are \(z_1=2,\ z_2=1\), giving \(\widehat{Y}=18.6+(0.4)(2)+(-0.2)(1)=18.6+0.8-0.2=19.2\). Checking in the original equation gives \(15.2+(0.4)(12)+(-0.2)(4)=15.2+4.8-0.8=19.2\). Both representations agree. The trap is to treat the centered intercept as the original intercept; here it is the response mean, while the original intercept is 15.2.

Interpreting the Centered Intercept

The centered intercept is the model's fitted response when all centered predictors equal zero. Since \(z_j=0\) means \(x_j=c_j\), it is the fitted response at the chosen combination of reference values. The combination need not occur as an actual observation: the intercept remains a model prediction at that point. Its practical usefulness depends on whether those reference values are meaningful and whether interpreting the model there is reasonable.

When every predictor is centered at its sample mean, the intercept equals \(\bar{Y}\) in an OLS fit with an intercept. This follows from the normal equation for the intercept, which makes the residuals sum to zero, together with the fact that the mean of each centered predictor is zero. Specifically, the mean fitted response is the centered intercept plus the slopes multiplied by the centered predictor means; those latter terms vanish. Mean-centering does not make the intercept an estimate of a new slope or change the response units.

Worked Example: Translating a Centered Equation Back to Original Units

A hypothetical fitted equation for a service-cost response is reported as \(\widehat{Y}=7+1.5(x_1-20)-0.4(x_2-5)\). Give the equivalent equation in the original predictors and predict at \(x_1=24,\ x_2=2\). State what the centered intercept represents.

Method. The centering values are \(c_1=20\) and \(c_2=5\). Expand each term:

\begin{aligned} \widehat{Y} &=7+1.5x_1-1.5(20)-0.4x_2+0.4(5)\\ &=7+1.5x_1-30-0.4x_2+2\\ &=-21+1.5x_1-0.4x_2. \end{aligned}

The original intercept is \(-21\); the slopes remain \(1.5\) and \(-0.4\). The centered intercept, 7, is the fitted response at \(x_1=20,\ x_2=5\), not at \(x_1=x_2=0\).

At \(x_1=24,\ x_2=2\), the centered predictor values are \(24-20=4\) and \(2-5=-3\). Therefore,

\widehat{Y}=7+1.5(4)-0.4(-3)=7+6+1.2=14.2.

The original equation confirms the result: \(-21+1.5(24)-0.4(2)=-21+36-0.8=14.2\). The common error is to report 7 as the original-coordinate intercept; it is the intercept only in the centered coordinates.

Conditions and Common Traps

The invariance result depends on the model structure. It applies to an additive linear model with an intercept when the predictors are shifted by constants and the same terms are retained. It does not say that any alteration of predictor columns preserves fitted values. In particular, if a model is fitted without an intercept, its predictor columns generally do not include the constant column needed to absorb the shift. Centering can then change the set of possible fitted values and the OLS fit.

Centering also leaves the predictor's unit and scale unchanged: a difference of one unit in \(z_j\) is exactly a difference of one unit in \(x_j\). Consequently, the slope has the same units and interpretation in both equations. Choosing a different center changes the intercept again, but not the slopes or predictions. A reliable calculation is to substitute \(x_j=z_j+c_j\) and collect the constant terms, rather than trying to guess the direction of the intercept adjustment.

Key takeaway: With an intercept, centering numeric predictors translates the intercept by \(\sum_j\beta_jc_j\) and leaves slopes, fitted values, residuals, and RSS unchanged. The new intercept is the fitted response at the centering values; when those values are the sample means, it equals the sample response mean in an OLS fit.

Check Your Understanding

For each problem, distinguish the change in coordinates from a change in the fitted regression model.

A fitted equation is \(\widehat{Y}=9+2x_1+3x_2\). Center \(x_1\) at 4 and \(x_2\) at 1. Give the centered equation and its prediction at \(x_1=5,\ x_2=3\). (Answer: \(\widehat{Y}=20+2z_1+3z_2\); prediction \(28\).)
An OLS model with an intercept uses two mean-centered predictors and has slopes \(-0.5\) and \(1.2\). The sample response mean is 32. What is the centered intercept, and what condition on the fit makes this result hold? (Answer: Intercept \(32\); the model must be an OLS fit with an intercept, so the residuals sum to zero.)
A centered equation is \(\widehat{Y}=11+0.8(x_1-15)+2(x_2-4)\). Give the original-coordinate intercept and predict at \(x_1=20,\ x_2=3\). (Answer: Original intercept \(-9\); prediction \(13\).)
A regression is fitted through the origin, with no intercept, and then a predictor is centered at a nonzero constant. Is it guaranteed that the refitted centered model has the same fitted values? Explain the key reason. (Answer: No; without an intercept, the shifted predictor generally cannot be represented by the same model columns, so the available fitted-value set can change.)
A centered fit is \(\widehat{Y}=5+2z_1-z_2\), where \(z_1=x_1-3\) and \(z_2=x_2-8\). Translate it to the original predictors and predict at \(x_1=4,\ x_2=10\). (Answer: \(\widehat{Y}=7+2x_1-x_2\); prediction \(5\).)

What You'll Learn