Why Standardize Predictors?
In a multiple regression, a slope’s numerical size depends on the units used to measure its predictor. A coefficient for annual premium measured in dollars will generally have a different numerical value from the coefficient for the same premium measured in thousands of dollars. Standardizing a predictor replaces its original units with units of standard deviation, which can make coefficients easier to compare and calculations easier to interpret.
Standardization changes the coefficient scale, not the underlying fitted relationship. The essential distinction is whether only the predictors are standardized or whether the response is standardized as well. The first produces slopes in response units per predictor standard deviation; the second produces fully standardized slopes in response standard deviations per predictor standard deviation.
Coefficient Conversion in a Multiple Regression
Start with a multiple linear regression containing an intercept and \(p\) numeric predictors. The fitted equation, expressed in the predictors’ original units, is
Here \(b_j\) is the fitted change in the response for a one-unit increase in \(x_j\), holding the other predictors fixed. Replacing each \(x_{ij}\) by \(\bar{x}_j+s_{x_j}z_{ij}\) gives the equation in standardized predictor units:
Thus, if predictors alone are standardized, the coefficient for \(z_j\) is \(b_js_{x_j}\). It measures the fitted response change for a one-standard-deviation increase in \(x_j\), holding the other predictors fixed. The intercept is the fitted response when every standardized predictor is zero—that is, when every original predictor is at its sample mean.
The reverse conversion is equally important. If the fitted equation is already written using standardized predictors, divide each standardized-predictor slope by that predictor’s standard deviation to recover its original-unit slope. Then adjust the intercept to preserve the predictions.
Worked Example: Converting Two Slopes to Per-Standard-Deviation Units
A hypothetical insurer models annual claim cost \(y\), in thousands of dollars, using two numeric predictors: vehicle age \(x_1\), in years, and annual mileage \(x_2\), in thousands of miles. Its fitted equation is \(\hat{y}=12+0.8x_1-1.5x_2\). In the estimation sample, \(\bar{x}_1=10\), \(s_{x_1}=2\), \(\bar{x}_2=50\), and \(s_{x_2}=10\). Write the equation using standardized predictors, then predict for \(z_1=1\), \(z_2=-1\).
Method. Multiply each original-unit slope by the corresponding predictor standard deviation. Adjust the intercept by evaluating the original fitted equation at the predictor means.
Standardized-predictor slopes. For vehicle age, \(0.8(2)=1.6\). For mileage, \((-1.5)(10)=-15\). The slopes are in thousands of dollars per one standard deviation of the corresponding predictor.
Intercept. At the predictor means, the fitted value is \(12+0.8(10)-1.5(50)=12+8-75=-55\). Therefore the standardized-predictor equation is \(\hat{y}=-55+1.6z_1-15z_2\).
Prediction. At \(z_1=1\) and \(z_2=-1\), \(\hat{y}=-55+1.6(1)-15(-1)=-55+1.6+15=-38.4\) thousand dollars.
Check. These standardized values correspond to \(x_1=10+2(1)=12\) years and \(x_2=50+10(-1)=40\) thousand miles. The original equation gives \(12+0.8(12)-1.5(40)=12+9.6-60=-38.4\), the same prediction. The calculation avoids the common mistake of changing the slopes but leaving the intercept unchanged.
Why the Fitted Values Do Not Change
Standardizing predictors is an affine change of coordinates: each original value can be recovered from its standardized value using \(x_{ij}=\bar{x}_j+s_{x_j}z_{ij}\). Since \(s_{x_j}>0\), this conversion is reversible. With an intercept in the model, the original and standardized predictor columns span the same set of possible fitted vectors. Consequently, least squares can express the same fitted values in either coordinate system; only the coefficient representation changes.
Replace every \(x_{ij}\) by \(\bar{x}_j+s_{x_j}z_{ij}\) in the original fitted equation.
The constant becomes \(b_0+\sum_j b_j\bar{x}_j\), and the coefficient multiplying \(z_{ij}\) becomes \(b_js_{x_j}\).
Because the substituted equation is algebraically identical for each observation, it gives the same fitted response and residual for that observation.
This justification is algebraic; it does not require normally distributed errors. For the usual unique coefficient representation, assume the model includes an intercept, each standardized predictor has positive sample standard deviation, and the design matrix has full column rank. Standardization cannot make a predictor with zero variation usable: its standard deviation is zero, so its standardized value is undefined.
Standardizing the Response as Well
A different scale results when the response is also standardized. Let \(\bar{y}\) and \(s_y>0\) be the sample mean and sample standard deviation of the response, and define \(z_{y,i}=(y_i-\bar{y})/s_y\). If the original-unit slope for \(x_j\) is \(b_j\), the coefficient in a regression of standardized response on standardized predictors is
This fully standardized slope measures the fitted change in response standard deviations for a one-standard-deviation increase in predictor \(j\), holding the other predictors fixed. With an intercept and ordinary least squares, the standardized intercept is zero: the fitted response at the predictor means equals \(\bar{y}\). This follows because the OLS residuals sum to zero, so the mean fitted response equals the mean observed response.
Worked Example: Interpreting Fully Standardized Slopes
A hypothetical model of monthly service cost \(y\), measured in dollars, has original-unit equation \(\hat{y}=100+3x_1+0.4x_2\). The numeric predictors are service calls \(x_1\) and equipment age \(x_2\), measured in years. Their sample means are 20 and 100, and their sample standard deviations are 5 and 20. The response sample mean is 200 and its sample standard deviation is 40. Find the fully standardized slopes and predict at \(z_1=2\), \(z_2=-1\).
Check the response mean. At the predictor means, the original equation gives \(100+3(20)+0.4(100)=100+60+40=200\), matching the stated response mean.
Convert the slopes. For service calls, \(\beta_1=3(5)/40=15/40=0.375\). For equipment age, \(\beta_2=0.4(20)/40=8/40=0.2\). The fully standardized equation is \(\widehat{z}_y=0.375z_1+0.2z_2\).
Predict in standardized units. At \(z_1=2\), \(z_2=-1\), the prediction is \(0.375(2)+0.2(-1)=0.75-0.2=0.55\) response standard deviations above the mean.
Convert to dollars. The predicted response is \(200+40(0.55)=200+22=222\) dollars.
Check in the original equation. The predictor values are \(x_1=20+5(2)=30\) calls and \(x_2=100+20(-1)=80\) years. The original equation gives \(100+3(30)+0.4(80)=100+90+32=222\) dollars. The prediction agrees. The common mistake avoided here is treating a fully standardized coefficient as dollars per standard deviation; its units are response standard deviations per predictor standard deviation.
Recovering the Original Equation
A standardized equation may be supplied even when a prediction or interpretation is required in the original predictor units. To reverse the conversion, first divide each standardized-predictor slope by its own standard deviation. Then recover the original intercept by subtracting the slope-weighted predictor means. The following example checks both steps against a prediction.
Worked Example: Returning to Original Predictor Units
A hypothetical fitted model for quarterly reserve change, in thousands of dollars, is \(\hat{y}=600+24z_1-8z_2\), where \(z_1\) and \(z_2\) are standardized numeric predictors. Predictor \(x_1\) has mean 40 and standard deviation 8; predictor \(x_2\) has mean 12 and standard deviation 3. Recover the original-unit equation and predict when \(x_1=48\) and \(x_2=9\).
Recover the slopes. The original-unit slope for \(x_1\) is \(24/8=3\). The original-unit slope for \(x_2\) is \(-8/3\).
Recover the intercept. The standardized intercept is the fitted response at both predictor means, so \(600=b_0+3(40)+(-8/3)(12)\). Since \(3(40)=120\) and \((-8/3)(12)=-32\), \(600=b_0+88\), giving \(b_0=512\). The original-unit equation is \(\hat{y}=512+3x_1-(8/3)x_2\).
Predict using standardized values. For \(x_1=48\), \(z_1=(48-40)/8=1\). For \(x_2=9\), \(z_2=(9-12)/3=-1\). Thus \(\hat{y}=600+24(1)-8(-1)=600+24+8=632\) thousand dollars.
Check in original units. The original equation gives \(512+3(48)-(8/3)(9)=512+144-24=632\) thousand dollars. The two calculations agree.
Interpretation and Common Traps
A standardized-predictor slope is still a multiple-regression slope: it describes a change in the fitted response while the other predictors are held fixed. Standardizing does not remove that conditional interpretation. It changes the size of the predictor increment from one original measurement unit to one sample standard deviation.
Fully standardized slopes put response and predictor changes on standard-deviation scales, but they are not a universal measure of predictor importance. Their values depend on the predictors’ scales in the estimation sample and on the other predictors included in the model. A large standardized slope does not, by itself, establish causation or show that a predictor is useful for every decision.
A frequent exam error is to divide an original-unit slope by the predictor standard deviation when converting to a predictor-standardized slope. The direction is the opposite: since one standard deviation represents \(s_{x_j}\) original units, the change over that interval is \(b_js_{x_j}\). For example, if \(b_j=2.4\) response units per original predictor unit and \(s_{x_j}=5\), the slope per predictor standard deviation is \(2.4(5)=12\), not \(2.4/5=0.48\). To convert the standardized-predictor slope of 12 back to the original scale, divide by 5 and recover 2.4.
Check Your Understanding
For each problem, use the stated sample means and standard deviations. Answers are given for checking.
- A fitted model has slope 2.5 response units per unit of numeric predictor \(x\), and \(s_x=4\). What is the slope when only \(x\) is standardized? (Answer: 10 response units per standard deviation of \(x\).)
- A fitted model has original-unit slope \(-1.2\), predictor standard deviation 3, and response standard deviation 6. Find the fully standardized slope. (Answer: \(-0.6\).)
- A predictor-standardized equation is \(\hat{y}=70+5z\). The predictor mean is 8 and its standard deviation is 2. Find the original-unit equation. (Answer: \(\hat{y}=50+2.5x\).)
- A two-predictor model is \(\hat{y}=40+2x_1-0.5x_2\). The predictor means are 3 and 20, and their standard deviations are 2 and 4. Write the predictor-standardized equation and predict at \(z_1=1.5\), \(z_2=-0.5\). (Answer: \(\hat{y}=36+4z_1-2z_2\); prediction \(43\).)
- A fully standardized model has coefficient \(\beta=0.3\) for predictor \(x\). The sample standard deviations are \(s_x=8\) and \(s_y=12\). Recover the original-unit slope, then state the fitted response change for a four-unit increase in \(x\), holding other predictors fixed. (Answer: original slope \(0.3(12)/8=0.45\) response units per unit of \(x\); fitted change \(0.45(4)=1.8\) response units.)