Centering the Predictor in Least-Squares Calculations

Why Center the Predictor?

A least-squares line can be written using the original predictor values or using deviations from their mean. The second form can simplify calculations and make the intercept easier to identify. The key is to distinguish the two intercepts: the centered intercept is the fitted response at the mean predictor value, while the ordinary intercept is the fitted response at predictor value zero.

Centering is an algebraic change of coordinates, not a new fitting method. Provided the predictor values are not all identical, the fitted slope and fitted responses remain the same. This tutorial develops that result and shows how to use it in calculations.

Definition: For observed pairs $(x_i,y_i)$, $i=1,\ldots,n$, define the sample means $\bar{x}=\frac{1}{n}\sum_{i=1}^n x_i$ and $\bar{y}=\frac{1}{n}\sum_{i=1}^n y_i$. The centered predictor is $z_i=x_i-\bar{x}$. The ordinary least-squares line in centered coordinates is written $\hat{y}_i=c_0+c_1z_i$, where $c_0$ and $c_1$ minimize $\sum_{i=1}^n(y_i-c_0-c_1z_i)^2$. The predictor must vary across the observations for the slope to be uniquely determined.

For comparison, write the ordinary-coordinate line as $\hat{y}_i=b_0+b_1x_i$. The symbols $b_0,b_1$ refer to the ordinary intercept and slope; $c_0,c_1$ refer to the centered intercept and slope. The two lines describe the same fitted values when their coefficients are converted appropriately.

Why the Centered Intercept Is the Response Mean

Centering makes the sum of the transformed predictor values zero: $\sum_{i=1}^n z_i=\sum_{i=1}^n(x_i-\bar{x})=0$. That identity is what makes the centered intercept especially simple. The following argument obtains both centered least-squares coefficients directly from the squared-error criterion.

Write the centered criterion.
For a proposed line, its sum of squared errors is $Q(c_0,c_1)=\sum_{i=1}^n(y_i-c_0-c_1z_i)^2$.

Differentiate with respect to the intercept.
At a least-squares minimum, $\frac{\partial Q}{\partial c_0}=-2\sum_{i=1}^n(y_i-c_0-c_1z_i)=0$. Thus $\sum_i y_i-nc_0-c_1\sum_i z_i=0$. Since $\sum_i z_i=0$, this gives $c_0=\bar{y}$.

Differentiate with respect to the slope.
The slope equation is $\sum_i z_i(y_i-c_0-c_1z_i)=0$. Substituting $c_0=\bar{y}$ gives $c_1=\frac{\sum_i z_i(y_i-\bar{y})}{\sum_i z_i^2}$, as long as $\sum_i z_i^2>0$.

Theorem: If the observed predictor values are not all equal, the least-squares coefficients in centered coordinates are $$ c_0=\bar{y},\qquad c_1=\frac{\sum_{i=1}^n(x_i-\bar{x})(y_i-\bar{y})}{\sum_{i=1}^n(x_i-\bar{x})^2}. $$ The centered slope $c_1$ equals the ordinary least-squares slope $b_1$. The ordinary intercept is $b_0=\bar{y}-b_1\bar{x}$, so $c_0$ is generally not $b_0$.

The slope formula follows because $z_i=x_i-\bar{x}$ and $\sum_i z_i=0$, so $\sum_i z_i(y_i-\bar{y})=\sum_i z_iy_i$. This is the same numerator and denominator used for the ordinary least-squares slope. Equivalently, substituting $x_i=z_i+\bar{x}$ into $b_0+b_1x_i$ gives $(b_0+b_1\bar{x})+b_1z_i$. Therefore $c_1=b_1$ and $c_0=b_0+b_1\bar{x}=\bar{y}$.

These are algebraic facts about least-squares fitting; they do not require a probability model. If the fitted line is also used as part of a statistical model for inference, that model has additional assumptions. Centering by itself neither adds nor removes those assumptions.

Worked Example: Centering a Small Data Set

A hypothetical study records a predictor $x$ and response $y$ for three observations: $(2,5)$, $(4,8)$, and $(6,10)$. Find the least-squares line in centered coordinates and in ordinary coordinates. Use the least-squares centered formulas, and avoid treating the centered intercept as the ordinary intercept.

Step 1: Calculate the means. $$ \bar{x}=\frac{2+4+6}{3}=4,\qquad \bar{y}=\frac{5+8+10}{3}=\frac{23}{3}. $$

Step 2: Calculate the centered slope. The pairs of deviations are $(-2,-8/3)$, $(0,1/3)$, and $(2,7/3)$. Therefore, $$ \sum_i(x_i-\bar{x})(y_i-\bar{y}) =(-2)(-8/3)+(0)(1/3)+(2)(7/3)=\frac{16}{3}+\frac{14}{3}=10, $$ $$ \sum_i(x_i-\bar{x})^2=(-2)^2+0^2+2^2=8,\qquad c_1=\frac{10}{8}=\frac{5}{4}. $$

Step 3: Write the centered fit. The centered intercept is the response mean, so $c_0=23/3$. Hence $$ \hat{y}=\frac{23}{3}+\frac{5}{4}(x-4). $$

Step 4: Convert to ordinary coordinates. Expanding the centered expression gives $$ \hat{y}=\frac{23}{3}+\frac{5}{4}x-5 =\frac{5}{4}x+\frac{8}{3}. $$ Thus $b_1=5/4$ and $b_0=8/3$.

Check. At $x=4=\bar{x}$, the fitted value is $23/3$, the response mean. At $x=2$, both forms give $23/3+(5/4)(-2)=31/6$. The ordinary form gives $(5/4)(2)+8/3=31/6$ as well. The common mistake would be to put $23/3$ in front of $x$ in the ordinary form; that would confuse $c_0$ with $b_0$.

Using Centering When Predictor Values Repeat

The centered calculation works with repeated predictor values just as it does with distinct ones: each observation contributes to the sums. A predictor value appearing three times contributes three terms, not one. The example below illustrates how centering also makes the fitted value at the mean predictor immediately visible.

Worked Example: Replicated Predictor Values

For a hypothetical process study, the observed pairs are $(10,6)$, $(10,8)$, $(12,9)$, $(14,10)$, and $(14,12)$. Find the centered least-squares line and the ordinary intercept. Use every observation in the centered sums; do not replace repeated observations with unweighted predictor levels.

Step 1: Find the means. $$ \bar{x}=\frac{10+10+12+14+14}{5}=12,\qquad \bar{y}=\frac{6+8+9+10+12}{5}=\frac{45}{5}=9. $$

Step 2: Find the centered sums. The predictor deviations are $-2,-2,0,2,2$; the response deviations are $-3,-1,0,1,3$. Thus $$ \sum_i(x_i-\bar{x})(y_i-\bar{y}) =(-2)(-3)+(-2)(-1)+0(0)+2(1)+2(3) =6+2+0+2+6=16, $$ $$ \sum_i(x_i-\bar{x})^2=4+4+0+4+4=16. $$ Therefore $c_1=16/16=1$.

Step 3: Write both forms. Since $c_0=\bar{y}=9$, the centered fit is $\hat{y}=9+(x-12)$. Expanding gives $\hat{y}=x-3$, so the ordinary intercept is $-3$.

Check. At $x=12$, the fitted value is $9$. The predictions at $x=10,12,14$ are $7,9,11$, respectively. Using the five original observations, the squared deviations from those predictions are $1,1,0,1,1$, summing to $4$. The check also shows why the two observations at each of $x=10$ and $x=14$ must both be included in the calculation.

Centering When Predictor Values Are Large

Centering can be especially convenient when predictor values are large relative to their spread. The centered form expresses changes from the sample mean directly, and it avoids repeatedly combining large predictor values with the slope. The fitted line itself is unchanged; only its coordinate representation differs.

Worked Example: Large Predictor Values and Two Intercepts

A hypothetical instrument records four predictor values $1001,1002,1003,1004$, with responses $20,22,21,25$. Calculate the centered fit, convert it to ordinary coordinates, and check a fitted value. Use centered deviations to calculate the slope rather than subtracting large raw sums.

Step 1: Calculate the means. $$ \bar{x}=\frac{1001+1002+1003+1004}{4}=1002.5,\qquad \bar{y}=\frac{20+22+21+25}{4}=22. $$

Step 2: Calculate the slope. The predictor deviations are $-1.5,-0.5,0.5,1.5$; the response deviations are $-2,0,-1,3$. Therefore, $$ \sum_i(x_i-\bar{x})(y_i-\bar{y}) =(-1.5)(-2)+(-0.5)(0)+(0.5)(-1)+(1.5)(3) =3+0-0.5+4.5=7, $$ $$ \sum_i(x_i-\bar{x})^2=2.25+0.25+0.25+2.25=5,\qquad c_1=\frac{7}{5}=1.4. $$

Step 3: Write the centered line. The centered intercept is $c_0=\bar{y}=22$, so $\hat{y}=22+1.4(x-1002.5)$.

Step 4: Convert to the ordinary intercept. $$ b_0=22-(1.4)(1002.5)=22-1403.5=-1381.5. $$ The ordinary form is $\hat{y}=-1381.5+1.4x$.

Check. At $x=1001$, the centered form gives $22+1.4(-1.5)=19.9$. The ordinary form gives $-1381.5+(1.4)(1001)=-1381.5+1401.4=19.9$. The common mistake is to report $22$ as the ordinary intercept. It is the centered intercept, which is the fitted response at $x=1002.5$, not at $x=0$.

Common Mistakes and Checks

Centering is simple once the coordinate system is clear, but several errors can produce a plausible-looking line with incorrect coefficients. Use these checks when completing a calculation:

Using the wrong intercept. In centered coordinates, $c_0=\bar{y}$. In ordinary coordinates, $b_0=\bar{y}-b_1\bar{x}$. They are equal only in special cases, such as $\bar{x}=0$ or $b_1=0$.
Changing the slope while centering. Subtracting $\bar{x}$ from every predictor value does not change the least-squares slope. If the computed slopes differ, check the centered sums and whether every observation was included.
Counting predictor levels instead of observations. If predictor values repeat, each observation still has its own response and contributes separately to the sums.
Assuming a unique slope when the predictor is constant. If every $x_i=\bar{x}$, then $\sum_i(x_i-\bar{x})^2=0$, so the slope formula is undefined. A line's slope cannot be identified from those predictor values.
Checking only the coefficient labels. Substitute a predictor value, preferably $\bar{x}$, into both forms. They should give the same fitted response, and the centered form should give $\bar{y}$ at $x=\bar{x}$.

Key takeaway: Centering replaces $x_i$ by $x_i-\bar{x}$. For a least-squares fit with varying predictor values, the slope stays the same, the centered intercept is $\bar{y}$, and the ordinary intercept is $\bar{y}-b_1\bar{x}$.

Check Your Understanding

For each problem, identify the centered intercept and slope. Convert to ordinary coordinates when requested.

For observations $(1,4)$, $(3,7)$, and $(5,8)$, calculate $\bar{x}$, $\bar{y}$, and the centered least-squares line. (Answer: $\bar{x}=3$, $\bar{y}=19/3$, and $\hat{y}=19/3+(x-3)$.)
For observations $(2,3)$, $(2,5)$, $(5,8)$, and $(5,10)$, calculate the centered line and the ordinary intercept, accounting for all four observations. (Answer: $\bar{x}=3.5$, $\bar{y}=6.5$, $c_1=5/3$, centered line $\hat{y}=6.5+(5/3)(x-3.5)$, and $b_0=2/3$.)
A data set has $\bar{x}=40$, $\bar{y}=18$, $\sum_i(x_i-\bar{x})^2=50$, and $\sum_i(x_i-\bar{x})(y_i-\bar{y})=-35$. Give both forms of its least-squares line. (Answer: $c_1=-0.7$, centered line $\hat{y}=18-0.7(x-40)$, and ordinary line $\hat{y}=46-0.7x$.)
For observations $(5,12)$, $(7,13)$, $(9,17)$, and $(11,18)$, calculate the centered line, convert it to ordinary coordinates, and verify the fitted value at the mean predictor. (Answer: $\bar{x}=8$, $\bar{y}=15$, $c_1=1.1$, centered line $\hat{y}=15+1.1(x-8)$, ordinary line $\hat{y}=6.2+1.1x$, and the fitted value at $x=8$ is $15$.)
A least-squares line in ordinary coordinates is $\hat{y}=-26+2.5x$, and the data have $\bar{x}=12$. Find the centered intercept and write the centered line. (Answer: $c_0=-26+(2.5)(12)=4$, so $\hat{y}=4+2.5(x-12)$; for a least-squares fit, $\bar{y}=4$.)

What You'll Learn