Why Centering Leaves Additive Slopes Unchanged

Why Centering Can Change the Intercept Without Changing the Slopes

In multiple regression, a predictor’s numerical zero may not be a useful reference point. Subtracting a chosen constant from each predictor moves that reference point, making the intercept describe the fitted outcome at the chosen values instead. But does changing the predictors this way also change the slopes or the model’s predictions? For an additive ordinary least squares model that includes an intercept, the answer follows directly from rewriting the same equation.

Definition: For $n$ observations and $p$ predictors, an additive linear regression model with an intercept is $$ y_i=\beta_0+\sum_{j=1}^{p}\beta_j x_{ij}+\varepsilon_i,\qquad i=1,\ldots,n, $$ where $y_i$ is the outcome for observation $i$, $x_{ij}$ is its value on predictor $j$, $\beta_0$ is the intercept, $\beta_j$ is the slope for predictor $j$, and $\varepsilon_i$ is the error. Center predictor $j$ by subtracting a constant $c_j$: $x_{ij}^{*}=x_{ij}-c_j$. The constants may differ across predictors.

The OLS fit chooses coefficients to minimize the sum of squared residuals, $\sum_{i=1}^{n}(y_i-\widehat{y}_i)^2$. For the algebraic result here, use the same observations in both fits and include an intercept in both models. Assume the model is additive: its predictor terms are the individual $x_{ij}$, with no additional terms. If the design matrix has full column rank, the OLS coefficients are unique. The fitted values are unique even in some rank-deficient cases, but individual coefficients need not be; the main result below is stated for the usual full-rank case.

The Coefficient Transformation

Because $x_{ij}=x_{ij}^{*}+c_j$, substitute this expression into the original fitted equation:

\widehat{y}_i =\widehat{\beta}_0+\sum_{j=1}^{p}\widehat{\beta}_j(x_{ij}^{*}+c_j) =\left(\widehat{\beta}_0+\sum_{j=1}^{p}\widehat{\beta}_j c_j\right) +\sum_{j=1}^{p}\widehat{\beta}_j x_{ij}^{*}.

This is the same fitted equation expressed using the centered predictors. Its centered intercept is the original intercept plus the sum of each original slope multiplied by its centering constant. Each slope remains attached to the same predictor and has the same value.

Theorem: In an additive OLS model with an intercept, centering predictors at constants $c_1,\ldots,c_p$ leaves each slope unchanged. The centered intercept is $$ \widehat{\gamma}_0=\widehat{\beta}_0+\sum_{j=1}^{p}\widehat{\beta}_j c_j, \qquad \widehat{\gamma}_j=\widehat{\beta}_j\quad (j=1,\ldots,p). $$ The two parameterizations have identical fitted values and residuals for every observation.

The equation establishes how to rewrite a particular set of coefficients. To show that the OLS coefficients themselves transform in this way, we also need to show that OLS is optimizing over the same possible fits in both parameterizations.

Map original coefficients to centered coefficients.
For any original coefficient vector, set $\gamma_j=\beta_j$ for each slope and $\gamma_0=\beta_0+\sum_{j=1}^{p}\beta_jc_j$. The equation above shows that this mapping preserves every fitted value.

Reverse the mapping.
Given any centered coefficients, set $\beta_j=\gamma_j$ and $\beta_0=\gamma_0-\sum_{j=1}^{p}\gamma_jc_j$. This recovers an original coefficient vector with the same fitted values.

Compare the OLS objectives.
For each corresponding pair of coefficient vectors, the fitted values match observation by observation. Therefore the residuals match, and so do their sums of squares. Since the mapping works in both directions, minimizing the sum of squared residuals in either parameterization produces the same fitted values; with full column rank, the coefficient mapping is unique.

This argument does not depend on the centering constants being sample means. They can be any fixed values. Choosing different constants changes the numerical value and reference point of the intercept, but the algebraic relationship between the two models remains the same.

Worked Example: Centering Two Predictors in an Exact Fit

Suppose a hypothetical data set has three observations with these predictor and outcome values. An additive OLS model with two predictors and an intercept is fit.

Observation	$x_1$	$x_2$	$y$
1	1	0	12
2	0	1	7
3	1	1	9

The three design rows are linearly independent, so the three coefficients are uniquely determined. Since there are three observations and three coefficients, solve the equations for an exact fit:

\begin{aligned} \widehat{\beta}_0+\widehat{\beta}_1&=12,\\ \widehat{\beta}_0+\widehat{\beta}_2&=7,\\ \widehat{\beta}_0+\widehat{\beta}_1+\widehat{\beta}_2&=9. \end{aligned}

Subtracting the first equation from the third gives $\widehat{\beta}_2=-3$. Subtracting the second from the third gives $\widehat{\beta}_1=2$. Substitution into the first gives $\widehat{\beta}_0=12-2=10$. Thus the original fitted equation is $\widehat{y}=10+2x_1-3x_2$.

Now center the predictors at $c_1=1$ and $c_2=1$, so $x_1^{*}=x_1-1$ and $x_2^{*}=x_2-1$. The centered intercept is

\widehat{\gamma}_0 =10+(2)(1)+(-3)(1)=9.

The slopes remain $2$ and $-3$, giving $\widehat{y}=9+2x_1^{*}-3x_2^{*}$. For observation 1, the centered values are $x_1^{*}=0$ and $x_2^{*}=-1$, so its centered prediction is $9+2(0)-3(-1)=12$, the same as its original prediction $10+2(1)-3(0)=12$. The same identity holds for every observation by the coefficient transformation.

A Noisy Example: The OLS Fit Is Preserved

The argument is not limited to data that fall exactly on a fitted plane. In a typical OLS fit, residuals are nonzero. The next example checks their role explicitly: a residual vector orthogonal to the intercept and predictor columns satisfies the OLS normal equations, and centering preserves the resulting fitted values.

Worked Example: Checking a Centered Fit With Nonzero Residuals

Consider the following hypothetical data, fit with an intercept and two additive predictors.

Observation	$x_1$	$x_2$	$y$
1	1	10	5
2	3	10	7
3	1	14	1
4	3	14	7

Center at $c_1=2$ and $c_2=12$. The centered predictor columns, in observation order, are $u=(-1,1,-1,1)$ and $v=(-2,-2,2,2)$. Consider the equation $\widehat{y}=5+2u-0.5v$. It gives fitted values $(4,8,2,6)$, and therefore residuals $e=(1,-1,-1,1)$.

To verify this is the OLS fit, check that the residuals have zero dot product with each model column. Their sum is $1-1-1+1=0$; their dot product with $u$ is $-1-1+1+1=0$; and their dot product with $v$ is $-2+2-2+2=0$. These are the normal equations for the intercept and the two predictors. The predictor columns are linearly independent, so the coefficients are unique. The residual sum of squares is $1^2+(-1)^2+(-1)^2+1^2=4$.

In terms of the original predictors, the same equation is

$$ 5+2(x_1-2)-0.5(x_2-12) =7+2x_1-0.5x_2. $$

Thus the original intercept is $7$, and the original slopes are $2$ and $-0.5$. Centering changes the intercept to $5$, while leaving both slopes unchanged. The residuals and their sum of squares also remain $e=(1,-1,-1,1)$ and $4$, respectively.

Reading the Intercept and Checking Predictions

In the original model, the intercept is the fitted outcome when all original predictors equal zero. In the centered model, it is the fitted outcome when $x_j=c_j$ for every predictor, because those values make every centered predictor equal to zero. This shift in reference point explains why the intercept changes. It does not imply that the model’s predictions at any particular set of original predictor values change.

Worked Example: Confirming a Prediction at New Centering Values

Suppose a hypothetical additive OLS fit for an outcome measured in points is $\widehat{y}=40+1.5x_1-0.8x_2$. Here $x_1$ is measured in years and $x_2$ in kilometres. Center $x_1$ at 50 years and $x_2$ at 20 kilometres.

The new intercept is $40+(1.5)(50)+(-0.8)(20)=40+75-16=99$. With $x_1^{*}=x_1-50$ and $x_2^{*}=x_2-20$, the centered equation is $\widehat{y}=99+1.5x_1^{*}-0.8x_2^{*}$. The slope on $x_1$ remains 1.5 points per year, and the slope on $x_2$ remains $-0.8$ points per kilometre.

For a hypothetical record with $x_1=54$ years and $x_2=18$ kilometres, the original equation gives $40+1.5(54)-0.8(18)=40+81-14.4=106.6$ points. The centered equation gives $99+1.5(4)-0.8(-2)=99+6+1.6=106.6$ points. Both equations predict the same outcome, with the same units.

Common Mistakes and Scope

Thinking that a different intercept means different predictions. The intercept changes because zero in the centered coordinates corresponds to the selected centering values in the original coordinates. The compensating intercept change keeps the complete fitted equation unchanged.
Changing the intercept but not checking its formula. Add $\widehat{\beta}_j c_j$ for every predictor, including negative slopes or negative centering constants. The signs are part of the calculation.
Assuming that centering changes a slope’s unit or per-unit interpretation. Subtracting a constant does not rescale a predictor. Each slope still describes the fitted change in outcome for a one-unit increase in its predictor, holding the other predictors fixed.
Applying the result to a different model without checking its terms. The result shown here is for an additive model with an intercept. It does not automatically apply if the intercept is omitted or if the model’s terms are changed.

Centering can make the intercept’s reference point more informative while preserving the predictions and additive slopes. Algebraically, it is a change of coordinates: each centered predictor is the original predictor minus a constant, and the intercept absorbs the corresponding constant contributions from the slopes.

Check Your Understanding

Use the coefficient transformation to answer each question.

A model is $\widehat{y}=6+3x_1-2x_2$. If $x_1$ is centered at 4 and $x_2$ at 5, what are the centered intercept and slopes?
In an additive model with an intercept, why do corresponding original and centered coefficient vectors produce the same residual sum of squares?
A predictor is centered at 10. What original predictor value corresponds to a centered value of $-3$?
For $\widehat{y}=12+0.5x_1$, center $x_1$ at 8. Verify that the original and centered equations give the same prediction when $x_1=10$.
Why is including an intercept an important condition for the result as stated?

Observation	\(x_1\)	\(x_2\)	\(y\)
1	1	0	12
2	0	1	7
3	1	1	9