Why Centering Preserves Quadratic Fitted Values

Why Centering Does Not Change the Quadratic Fit

Centering a predictor means subtracting a fixed reference value from it. This can make the predictor easier to work with, but it also raises a natural question: if the regression uses both the predictor and its square, does centering change the curve that can be fitted? With an intercept and both polynomial terms included, the answer is no. Centering changes the way the curve is written, not the set of curves available to least squares.

This is an algebraic statement about the model space. It does not say that the numerical coefficients stay the same, or that centering improves a model's predictions. The coefficients generally change; the fitted values do not, provided the original and centered models contain an intercept, a linear term, and a quadratic term.

Definition: For predictor values \(x_i\), choose a fixed center \(c\) and define the centered predictor \(z_i=x_i-c\). The original quadratic model has mean function \(\beta_0+\beta_1x_i+\beta_2x_i^2\). The centered quadratic model has mean function \(\gamma_0+\gamma_1z_i+\gamma_2z_i^2\). Both include an intercept and both powers, first and second, of their predictor.

For the regression setting, write the original model as \(Y_i=\beta_0+\beta_1x_i+\beta_2x_i^2+\varepsilon_i\). The usual mean-model assumption is \(E(Y_i\mid x_i)=\beta_0+\beta_1x_i+\beta_2x_i^2\), equivalently \(E(\varepsilon_i\mid x_i)=0\). Least squares chooses coefficients to minimize the sum of squared residuals. The equivalence proved below needs no normality or constant-variance assumption: it follows from the design columns being different bases for the same set of fitted vectors. For a unique coefficient solution, assume the design matrix has full column rank; the fitted values remain unique even if coefficients are not.

Rewriting the Polynomial

Since \(z=x-c\), the original predictor is \(x=z+c\). Substitute this expression into the original quadratic mean function and expand:

\begin{aligned} \beta_0+\beta_1x+\beta_2x^2 &=\beta_0+\beta_1(z+c)+\beta_2(z+c)^2\\ &=(\beta_0+c\beta_1+c^2\beta_2) +(\beta_1+2c\beta_2)z+\beta_2z^2. \end{aligned}

Thus the centered coefficients that represent the same curve are

\gamma_0=\beta_0+c\beta_1+c^2\beta_2,\qquad \gamma_1=\beta_1+2c\beta_2,\qquad \gamma_2=\beta_2.

These equations show directly that every original quadratic can be expressed in centered form. The reverse is also true: substitute \(z=x-c\) into the centered polynomial. The resulting coefficients are

\beta_0=\gamma_0-c\gamma_1+c^2\gamma_2,\qquad \beta_1=\gamma_1-2c\gamma_2,\qquad \beta_2=\gamma_2.

The two transformations are inverses. Therefore, neither parameterization excludes any quadratic curve that the other can represent. This conclusion depends on retaining all three terms. For example, if a model includes an intercept and a square but omits the linear term, centering generally introduces a linear term, so the restricted model spaces need not agree.

Worked Example: Re-expressing a Quadratic Curve

Suppose a hypothetical fitted curve, with \(x\) measured in units, is \(\hat{y}=12+2x-0.5x^2\). Re-express it using the centered predictor \(z=x-4\), so the chosen center is \(c=4\).

Here \(\beta_0=12\), \(\beta_1=2\), and \(\beta_2=-0.5\). Apply the coefficient conversion:

\(\gamma_0=12+4(2)+4^2(-0.5)=12+8-8=12\).

\(\gamma_1=2+2(4)(-0.5)=2-4=-2\), and \(\gamma_2=-0.5\).

The centered equation is therefore \(\hat{y}=12-2z-0.5z^2\). Check it at three predictor values. When \(x=2\), \(z=-2\): the original gives \(12+2(2)-0.5(2^2)=14\), and the centered equation gives \(12-2(-2)-0.5(-2)^2=14\). When \(x=4\), \(z=0\), and both give \(12\). When \(x=6\), \(z=2\): both give \(6\). The equations describe the same fitted curve, although their linear coefficients differ.

Why Least-Squares Fitted Values Are Preserved

The algebra above establishes equality of the possible curves. The least-squares consequence follows because a data set's fitted vector must be chosen from that set of possible vectors. To make this explicit, let \(X\) be the design matrix whose columns are the intercept, \(x\), and \(x^2\), and let \(Z\) be the design matrix whose columns are the intercept, \(z\), and \(z^2\). Each column is a vector of values across the observations.

Because \(z=x-c\) and \(z^2=x^2-2cx+c^2\), every column of \(Z\) is a linear combination of the columns of \(X\). Conversely, \(x=z+c\) and \(x^2=z^2+2cz+c^2\), so every column of \(X\) is a linear combination of the columns of \(Z\). Their column spaces are equal: they contain exactly the same vectors of possible fitted values.

Theorem: If the original and centered quadratic regression models both include an intercept, a linear term, and a quadratic term, then for the same response data and fixed center \(c\), their least-squares fitted values are identical for every observation. Their residuals and residual sum of squares are consequently identical as well.

Here is the least-squares justification. Let \(\mathbf{y}\) be the vector of observed responses. Least squares selects a vector in the model's column space that minimizes the squared distance to \(\mathbf{y}\). Since the two column spaces are equal, they offer the same candidate fitted vectors and the same objective values. Thus their minimizing fitted vector is the same. Equivalently, the fitted vector is the orthogonal projection of \(\mathbf{y}\) onto that common column space. If the design has full column rank, the coefficient vector representing that fitted vector is unique in either basis; if not, coefficients might be nonunique, but the projection and fitted vector are still unique.

This explains why the result is stronger than saying that two equations happen to agree at a few selected points. The equality holds for all observations and for any predictor value at which either polynomial is evaluated, because the two polynomial expressions are algebraically identical after the coefficient conversion.

Worked Example: Checking Predictions After Centering

Use the same hypothetical curve \(\hat{y}=12+2x-0.5x^2\), now centered at \(c=4\). Its centered form is \(\hat{y}=12-2z-0.5z^2\), where \(z=x-4\). Compare predictions at \(x=3\) and \(x=7\).

At \(x=3\), \(z=3-4=-1\). In the original form, \(\hat{y}=12+2(3)-0.5(3^2)=12+6-4.5=13.5\). In the centered form, \(\hat{y}=12-2(-1)-0.5(-1)^2=12+2-0.5=13.5\).

At \(x=7\), \(z=7-4=3\). The original form gives \(\hat{y}=12+2(7)-0.5(7^2)=12+14-24.5=1.5\). The centered form gives \(\hat{y}=12-2(3)-0.5(3^2)=12-6-4.5=1.5\). These equal calculations illustrate the same prediction at different locations; the column-space argument establishes the equality for the entire model, not just these two checks.

Centering a Least-Squares Fit in a Small Data Set

A small hypothetical data set makes the fitted-value result concrete. Suppose the observed predictor-response pairs are \((-1,2)\), \((0,1)\), and \((2,7)\), with \(x\) measured in units and \(Y\) in response units. Fit a quadratic with an intercept. There are three observations and three coefficients, and the three distinct predictor values give a full-rank quadratic design.

Worked Example: Matching Least-Squares Fits in Two Bases

First solve for the original coefficients. The three points give the equations \(\beta_0-\beta_1+\beta_2=2\), \(\beta_0=1\), and \(\beta_0+2\beta_1+4\beta_2=7\). From the middle equation, \(\beta_0=1\). The first then gives \(-\beta_1+\beta_2=1\), and the third gives \(2\beta_1+4\beta_2=6\), or \(\beta_1+2\beta_2=3\). Solving yields \(\beta_2=4/3\) and \(\beta_1=1/3\).

The original fitted equation is \(\hat{y}=1+\frac{1}{3}x+\frac{4}{3}x^2\). At \(x=-1\), it gives \(1-\frac{1}{3}+\frac{4}{3}=2\); at \(x=0\), it gives \(1\); and at \(x=2\), it gives \(1+\frac{2}{3}+\frac{16}{3}=7\). It interpolates all three points, so its residual sum of squares is \(0\).

Now center at \(c=1\), giving \(z=x-1\). The coefficient conversion gives \(\gamma_0=1+1(1/3)+1^2(4/3)=8/3\), \(\gamma_1=1/3+2(1)(4/3)=3\), and \(\gamma_2=4/3\). The centered equation is \(\hat{y}=\frac{8}{3}+3z+\frac{4}{3}z^2\). The observed centered values are \(z=-2,-1,1\). Their predictions are respectively \(\frac{8}{3}-6+\frac{16}{3}=2\), \(\frac{8}{3}-3+\frac{4}{3}=1\), and \(\frac{8}{3}+3+\frac{4}{3}=7\). Thus the centered fit has the same fitted values and the same zero residual sum of squares.

This example is saturated: three coefficients match three observations exactly. In a larger data set the fit might not pass through every observation, but the column-space argument still guarantees that centering leaves its least-squares fitted values unchanged.

What Changes and What Does Not

Centering changes the coordinate used for the predictor. In general, the intercept and linear coefficient in the polynomial's written form change according to the conversion formulas. The quadratic coefficient is unchanged for a simple shift, as \(\gamma_2=\beta_2\). The fitted curve, fitted values, residuals, and residual sum of squares do not change. Those facts concern the equivalence of the parameterizations; they do not by themselves settle how any coefficient should be interpreted in a particular application.

The center \(c\) can be any fixed reference value for the algebraic equivalence. Choosing a convenient or substantively meaningful reference can make the model's representation more useful, but it is not necessary for preserving the fit. If the center is the sample mean, the same algebra applies to that chosen value. No matter how \(c\) is selected, the essential requirement for this result is that the intercept and both polynomial terms are retained.

Common Mistake

Wrong reasoning: “The linear coefficient changed after centering, so the model must give different fitted values.” A coefficient's numerical value depends on how the predictor terms are written, so this comparison alone does not determine whether the curve changed.
Correction: Convert all coefficients using the shift formulas, or compare the design column spaces. With the intercept and both powers included, the resulting polynomial is identical at every \(x\), and the least-squares fitted values are identical.
Another limitation: The result does not automatically apply if a term is dropped. For example, an intercept-plus-square model in \(x\) generally becomes an intercept, linear, and square expression in \(z\). Omitting the linear term in the centered model can therefore change the set of available fits.

Key takeaway: Centering a predictor in a complete quadratic regression is an invertible change of basis. It changes the coefficient representation but preserves the model space, so least squares produces the same fitted values.

Check Your Understanding

Use the algebra and model-space argument to answer these questions.

For a model with intercept, \(x\), and \(x^2\), what centered terms appear when \(z=x-c\) is substituted into the quadratic term?
Write the formulas that convert \(\beta_0,\beta_1,\beta_2\) into \(\gamma_0,\gamma_1,\gamma_2\).
Why do equal column spaces imply equal least-squares fitted values, even if coefficient solutions are not unique?
Which part of the quadratic coefficient conversion is unchanged by a shift, and what condition on the model terms is needed for the fitted-value result?
Would an intercept-plus-square model necessarily preserve its model space if centered and the linear term were still omitted? Explain briefly.