Scaling Predictors for Stable High-Degree Fits

Why Scale a Predictor Before Raising It to High Powers?

A polynomial model may include $x$, $x^2$, $x^3$, and still higher powers. If the predictor is measured in large units, these columns can have very different numerical magnitudes. For example, when $x$ is around 800, $x^5$ is around $10^{14}$, while the intercept column is always 1. Such differences can make the computer's numerical calculations less reliable, especially when polynomial columns also carry overlapping information.

Centering and scaling replace the original predictor with a more convenient coordinate. The purpose is computational, not to change the polynomial relationship being fit. The coefficients will generally change because they refer to the new coordinate, but, under the conditions described below, the fitted predictions need not change.

Definition: For a predictor $x$, a centered and scaled predictor is $$ z=\frac{x-c}{s}, $$ where $c$ is a chosen center and $s$ is a chosen positive scale. A polynomial model of degree $d$ in $z$ includes $1,z,z^2,\ldots,z^d$. Choosing $c$ near the middle of the observed $x$-values and $s$ so that the resulting $z$-values are moderate is often useful.

The center and scale have the same units as $x$, so $z$ is dimensionless. If observed values of $x$ run from 0 to 1000, for instance, $c=500$ and $s=500$ map that range to $-1$ through 1. Other choices can be appropriate; the important point is to keep the transformed predictor and its powers at manageable magnitudes.

What Scaling Changes—and What It Preserves

Write the original-coordinate model of degree $d$ as $$ \hat{y}=\hat{\beta}_0+\hat{\beta}_1x+\hat{\beta}_2x^2+\cdots+\hat{\beta}_dx^d. $$ In scaled coordinates, write the fitted polynomial as $$ \hat{y}=\hat{\gamma}_0+\hat{\gamma}_1z+\hat{\gamma}_2z^2+\cdots+\hat{\gamma}_dz^d. $$ The $\hat{\beta}_j$ and $\hat{\gamma}_j$ are usually different numbers. They describe the same fitted curve using different coordinates only when they are related by the change of variable $x=c+sz$. Comparing a coefficient in one coordinate system directly with the coefficient having the same index in the other is generally not meaningful.

Here is the key mathematical reason fitted values can be preserved. Substituting $x=c+sz$ into each original power gives $$ x^j=(c+sz)^j=\sum_{k=0}^{j}{j\choose k}c^{j-k}s^kz^k. $$ Thus every original-coordinate polynomial of degree at most $d$ can be written as a polynomial of degree at most $d$ in $z$. Conversely, $z=(x-c)/s$, so every polynomial in $z$ can be written as one in $x$. This reverse substitution is possible because $s\ne 0$.

Consequently, the two sets of possible fitted curves are the same. Least squares chooses the curve in that set that minimizes the sum of squared residuals, so an exact least-squares fit has the same fitted values in either coordinate system. This statement concerns the fitted curve and predictions, not equality of coefficient values or numerical results from imperfect computation.

Theorem: If $s\ne 0$, fitting a degree-$d$ polynomial in $x$ and fitting a degree-$d$ polynomial in $z=(x-c)/s$, with an intercept and every power from 1 through $d$, use the same set of possible fitted values at the observed predictor values. Their least-squares fitted values are therefore the same in exact arithmetic. The coefficients generally differ.

Worked Examples

Worked Example: Rescaling a Predictor That Is Near 500

Suppose a hypothetical cubic curve is written in terms of $z=(x-500)/10$ as $$ \hat{y}=20+3z-2z^2+0.5z^3. $$ Find the equivalent expression in $x$, and verify the prediction at $x=510$.

Since $z=x/10-50$, the powers expand as follows: $$ z^2=\frac{x^2}{100}-10x+2500, \qquad z^3=\frac{x^3}{1000}-1.5x^2+750x-125000. $$ For the cubic, the $x$-term comes from $3(x/10)(50^2)=750x$, and the constant is $(-50)^3=-125000$.

Substitute these expressions into the model and collect like terms: $$ \begin{aligned} \hat{y} &=20+3\left(\frac{x}{10}-50\right) -2\left(\frac{x^2}{100}-10x+2500\right)\\ &\quad+0.5\left(\frac{x^3}{1000}-1.5x^2+750x-125000\right)\\ &=0.0005x^3-0.77x^2+395.3x-67630. \end{aligned} $$ In particular, the linear coefficient is $0.3+20+375=395.3$, and the constant is $20-150-5000-62500=-67630$.

At $x=510$, the scaled value is $z=(510-500)/10=1$, so the prediction is $20+3-2+0.5=21.5$. Substitution in the expanded expression gives the same result: $$ 0.0005(510^3)-0.77(510^2)+395.3(510)-67630 =66325.5-200277+201603-67630=21.5. $$ The large terms in the expanded calculation nearly cancel. The scaled expression is much simpler to evaluate near $x=500$.

Worked Example: Comparing the Magnitudes of Fifth-Power Columns

A hypothetical predictor is recorded between 0 and 1000, and a fifth-degree polynomial is being considered. Compare the fifth-power column at $x=800$ before and after using $z=(x-500)/500$.

Without rescaling, the fifth-power value is $$ x^5=800^5=327{,}680{,}000{,}000{,}000\approx3.28\times10^{14}. $$ The intercept column has value 1, and the linear column has value 800. The columns therefore have very different numerical scales.

With the stated transformation, $z=(800-500)/500=0.6$, so $$ z^5=0.6^5=0.07776. $$ Across the original range $0\le x\le1000$, the transformed range is $-1\le z\le1$, and every power $z^j$ also lies between $-1$ and 1. This does not prove that fitting will be perfectly stable: columns can still be strongly related, and the data may not support a complicated curve. It does, however, avoid unnecessarily huge power values caused solely by the units and location of $x$.

Worked Example: Same Fitted Curve, Different Coefficients

Consider the following hypothetical observations, with $z=(x-100)/2$. Fit a quadratic by least squares and compare its scaled and original-coordinate forms.

$x$	$z$	$y$
98	-1	11
99	-0.5	9.75
100	0	10
101	0.5	11.75
102	1	15

The model in scaled coordinates is $\hat{y}=a+bz+gz^2$. The five distinct $z$-values give a full-rank quadratic design, so the least-squares coefficients are uniquely determined. The sums needed for the normal equations are $$ \sum z=0,\quad \sum z^2=2.5,\quad \sum z^3=0,\quad \sum z^4=2.125, $$ $$ \sum y=57.5,\qquad \sum zy=5,\qquad \sum z^2y=31.375. $$ The normal equations separate into $$ 5a+2.5g=57.5,\qquad 2.5a+2.125g=31.375,\qquad 2.5b=5. $$ The last equation gives $b=2$. Solving the first two gives $a=10$ and $g=3$: these values satisfy $5(10)+2.5(3)=57.5$ and $2.5(10)+2.125(3)=31.375$. Thus the fitted model is $\hat{y}=10+2z+3z^2$.

To express the same curve in $x$, substitute $z=(x-100)/2$: $$ \hat{y}=10+(x-100)+0.75(x-100)^2 =0.75x^2-149x+7410. $$ For example, at $x=101$, the scaled form gives $10+2(0.5)+3(0.5^2)=11.75$. The original-coordinate form gives $0.75(101^2)-149(101)+7410=11.75$. The coefficients changed substantially, but the prediction did not.

Practical Choices and Limits

A useful scale is often chosen to make the transformed observed values roughly between $-1$ and 1. If $x_{\min}$ and $x_{\max}$ are the observed minimum and maximum, one possible choice is $$ c=\frac{x_{\min}+x_{\max}}{2}, \qquad s=\frac{x_{\max}-x_{\min}}{2}. $$ This maps the endpoints to $-1$ and 1, provided the predictor varies. Another reasonable choice is a meaningful unit such as 10 years or 100 milligrams. The choice should be recorded clearly so that predictions and coefficients can be translated back correctly.

Scaling helps address avoidable numerical problems; it does not change the information in the data. In particular, it does not make a high-degree model appropriate, remove the effect of influential observations, or guarantee that the polynomial terms are not strongly related. It also cannot repair a rank-deficient design, such as trying to estimate more polynomial coefficients than the data can identify. A polynomial that behaves poorly outside the observed range remains the same polynomial after a change of scale.

When software is used, supply the transformed predictor consistently for both fitting and prediction. To predict at a new original value $x_{\text{new}}$, first calculate $z_{\text{new}}=(x_{\text{new}}-c)/s$, then use the fitted coefficients in $z$. Alternatively, convert the full polynomial back to $x$ with correct algebra. Mixing coefficients from one coordinate system with values from the other does not evaluate the intended curve.

Common Mistake

A common mistake is to see different coefficients after rescaling and conclude that the fitted relationship must have changed. That conclusion confuses a coordinate-dependent description with the curve itself. For example, the coefficient of $x^2$ in an original-coordinate cubic need not equal the coefficient of $z^2$ in its scaled-coordinate version; the lower-order coefficients are mixed during expansion. Compare fitted predictions at the same $x$-values, or convert the entire polynomial consistently, rather than comparing matching coefficient positions.

Key takeaway: Centering and scaling can make high-degree polynomial calculations more manageable. With an intercept and all powers through the same degree, a nonzero affine rescaling preserves the polynomial model space and its exact least-squares fitted values. Coefficients change because the predictor's coordinate system changed.

Check Your Understanding

Answer each question using the definitions and results in this tutorial.

If $x$ ranges from 200 to 600, give a center and scale that map the endpoints to $-1$ and 1.
For $z=(x-40)/5$, what original $x$-value corresponds to $z=2$?
Why can fitting a full quadratic in $x$ or in $z=(x-c)/s$, with $s\ne0$, produce the same exact least-squares fitted values?
Does scaling guarantee that polynomial terms are uncorrelated or that a high-degree model is well supported by the data? Explain briefly.
A fitted polynomial is expressed in $z$, but a prediction is requested at $x=75$. What must be done before evaluating the fitted expression?