Why Standardize Both Sides of a Regression
Regression slopes are expressed in the units of the outcome per unit of a predictor. For example, a slope might describe points of exam score per hour of practice. Those units are useful when making predictions in the original setting, but they can make a coefficient’s numerical size hard to interpret on its own. Standardizing the outcome as well as the predictors expresses fitted changes in standard-deviation units.
This tutorial derives that conversion for a fitted multiple linear regression with an intercept. The central idea is a change of units: the fitted predictions do not change when translated back to the original scale. Standardization does not, by itself, make a model causal or guarantee that its linear form is appropriate.
Use the same observations to calculate all means and standard deviations in this tutorial. Let \(y\) be the outcome and \(x_1,\ldots,x_k\) the predictors. Suppose an ordinary least-squares fit with an intercept gives \(\hat{y}=b_0+b_1x_1+\cdots+b_kx_k\). Here \(b_j\) is the fitted slope for predictor \(x_j\), holding the other predictors fixed. Standardizing each variable gives \(z_y=(y-\bar{y})/s_y\) and \(z_j=(x_j-\bar{x}_j)/s_{x_j}\).
Why the Conversion Formula Works
To express a fitted outcome in standard-deviation units, subtract its sample mean and divide by the outcome’s sample standard deviation: \(\hat{z}_y=(\hat{y}-\bar{y})/s_y\). For an ordinary least-squares fit with an intercept, the mean of the fitted values is the sample mean of the outcome, so \(\overline{\hat{y}}=\bar{y}\). The same fit also has \(b_0=\bar{y}-\sum_{j=1}^{k}b_j\bar{x}_j\). Substituting these facts into the fitted equation gives the conversion directly.
The second line follows because \(b_0+\sum_j b_j\bar{x}_j=\bar{y}\). In the third line, each centered predictor is written as \(x_j-\bar{x}_j=s_{x_j}z_j\). There is no constant term left because all variables have been centered at their sample means. More precisely, this zero intercept is a property of this transformed fitted equation; the original-scale intercept \(b_0\) is generally not zero.
Worked Conversions and Predictions
Worked Example: Converting Two Slopes
Suppose a hypothetical sample of 40 students has exam score \(y\), practice hours \(x_1\), and absences \(x_2\). The sample means are \(\bar{y}=70\) points, \(\bar{x}_1=5\) hours, and \(\bar{x}_2=4\) absences. The sample standard deviations are \(s_y=10\) points, \(s_{x_1}=2\) hours, and \(s_{x_2}=3\) absences. Suppose a fitted additive regression gives slopes \(b_1=3\) points per hour and \(b_2=-2\) points per absence. Find the intercept and standardized slopes.
Check the model’s intercept. For an ordinary least-squares fit with an intercept, the fitted regression passes through the sample means. Thus, \(b_0=\bar{y}-b_1\bar{x}_1-b_2\bar{x}_2 =70-(3)(5)-(-2)(4)=63\) points. The original fitted equation is therefore \(\hat{y}=63+3x_1-2x_2\).
Convert the practice-hours slope. Use the outcome SD in the denominator and the practice-hours SD in the numerator: \(b_1^*=b_1(s_{x_1}/s_y)=3(2/10)=0.600\).
Convert the absences slope. \(b_2^*=b_2(s_{x_2}/s_y)=-2(3/10)=-0.600\). The standardized fitted equation is \(\hat{z}_y=0.600z_1-0.600z_2\).
Holding absences fixed, a one-SD increase in practice hours corresponds to an increase of 0.600 outcome SDs in the fitted exam score. Holding practice hours fixed, a one-SD increase in absences corresponds to a decrease of 0.600 outcome SDs in the fitted score. The calculations use nonzero SDs and coefficients from one additive model; they do not establish that either predictor causes a score change.
Worked Example: Predicting in Standardized and Original Units
In a hypothetical sample, a fitted model predicts exam score from practice hours \(x_1\) and attendance percentage \(x_2\). Suppose \(\bar{y}=80\) points, \(s_y=8\) points, \(\bar{x}_1=4\) hours, \(s_{x_1}=1.5\) hours, \(\bar{x}_2=90\%\), and \(s_{x_2}=5\) percentage points. The fitted slopes are \(b_1=2\) points per hour and \(b_2=0.4\) points per percentage point. Find the prediction for a student with 5.5 practice hours and 85% attendance, using both scales.
First calculate the original intercept from the fitted mean relationship: \(b_0=80-(2)(4)-(0.4)(90)=36\) points. The original fitted model is \(\hat{y}=36+2x_1+0.4x_2\).
The predictor values standardize to \(z_1=(5.5-4)/1.5=1\) and \(z_2=(85-90)/5=-1\). The standardized slopes are \(b_1^*=2(1.5/8)=0.375\) and \(b_2^*=0.4(5/8)=0.250\). Thus the standardized fitted value is \(\hat{z}_y=(0.375)(1)+(0.250)(-1)=0.125\).
Return to the original outcome scale by multiplying by \(s_y\) and adding \(\bar{y}\): \(\hat{y}=80+(0.125)(8)=81\) points. Check directly in the original model: \(\hat{y}=36+(2)(5.5)+(0.4)(85)=36+11+34=81\) points. Both routes give the same prediction. This agreement is expected because standardization changes the scale of the equation, not the fitted predictions expressed in original units.
Worked Example: Recovering an Original-Unit Model
Suppose a hypothetical standardized regression predicts a measured outcome \(y\) from \(x_1\) and \(x_2\), with fitted equation \(\hat{z}_y=0.250z_1-0.400z_2\). For the same sample, let \(\bar{y}=50\), \(s_y=6\), \(\bar{x}_1=20\), \(s_{x_1}=4\), \(\bar{x}_2=10\), and \(s_{x_2}=2\). Recover the original-unit slopes and intercept.
Rearrange the conversion formula \(b_j^*=b_j(s_{x_j}/s_y)\) to obtain \(b_j=b_j^*(s_y/s_{x_j})\). Therefore, \(b_1=0.250(6/4)=0.375\) outcome units per unit of \(x_1\), and \(b_2=-0.400(6/2)=-1.20\) outcome units per unit of \(x_2\).
The original intercept is determined by the sample means: \(b_0=\bar{y}-b_1\bar{x}_1-b_2\bar{x}_2 =50-(0.375)(20)-(-1.20)(10)=54.5\). So the original fitted model is \(\hat{y}=54.5+0.375x_1-1.20x_2\).
In this fitted model, a one-unit increase in \(x_1\), holding \(x_2\) fixed, corresponds to a fitted increase of 0.375 outcome units. A one-unit increase in \(x_2\), holding \(x_1\) fixed, corresponds to a fitted decrease of 1.20 outcome units. The standardized equation and this original-unit equation express the same fit on different scales.
Conditions, Scope, and a Common Mistake
The conversion is a change of scale, not a separate fitting procedure: given the same observations and an ordinary least-squares fit with an intercept, transforming the fitted coefficients by \(s_{x_j}/s_y\) gives the standardized slopes. The formula is meaningful only when the outcome and each standardized predictor vary in the sample. If a predictor has \(s_{x_j}=0\), its standardized values cannot be calculated; if \(s_y=0\), outcome standardization is not defined.
The slope interpretation also depends on the model being additive in the predictors as written. It describes a fitted change when one predictor changes and the other included predictors are held fixed. It is not automatically the result of changing a predictor in the real world, and it does not account for variables that were not included in the model.
- Mistake: “The standardized slope is the original slope divided by the outcome SD.” Correction: The predictor must also be converted to SD units, so \(b_j^*=b_j(s_{x_j}/s_y)\).
- Mistake: “The standardized intercept is the original intercept divided by the outcome SD.” Correction: When both outcome and predictors are centered at their sample means, the standardized fitted intercept is zero. The original intercept is recovered from the sample means.
- Mistake: “A standardized slope is a percent change.” Correction: It is a change measured in standard deviations, not a percentage.
- Mistake: “A standardized slope proves a causal effect.” Correction: It describes a conditional fitted association; standardizing variables does not establish causation.
Standardized units can make a model’s fitted relationships easier to describe without carrying the original measurement units into every sentence. But the transformed coefficients remain conditional slopes from the same regression model. Their interpretation requires attention to the outcome, predictors, sample, and units that the original model represents.
Check Your Understanding
Use the conversion formula and the fitted-model interpretation to answer these questions.
- A fitted slope is 4 outcome units per predictor unit, with \(s_x=3\) and \(s_y=12\). What is the standardized slope?
- In a standardized fitted model, what does a slope of \(-0.5\) mean when the other predictors are held fixed?
- Why is the standardized intercept zero for an ordinary least-squares regression with an intercept when all variables are centered at their sample means?
- If \(b_j^*=0.30\), \(s_y=9\), and \(s_{x_j}=3\), what is the original-unit slope \(b_j\)?
- What happens to the standardization formula if a predictor has sample standard deviation zero?