Deriving the Adjusted Slope by Residualizing

Why Residualize Both Variables?

In a two-predictor regression, the coefficient of a focal predictor describes its linear association with the outcome after accounting for a control predictor. One way to understand this adjustment is to first remove from both the outcome and the focal predictor the parts that are linearly associated with the control. The remaining residuals represent deviations from those linear patterns. Their relationship gives the adjusted slope.

The central result is exact: the focal-predictor coefficient from a regression of the outcome on the focal predictor and the control is the least-squares slope obtained by regressing the outcome residuals on the focal-predictor residuals. This is an algebraic property of least squares, not an approximation. The derivation also explains why residualizing only one of the two variables generally does not give the adjusted coefficient.

Model and Definitions

For \(n\) observations, let \(y_i\) be an outcome, \(x_i\) a focal predictor, and \(z_i\) a control predictor. The two-predictor additive model is

The model has a common linear slope for \(x\) across values of \(z\); it contains no interaction term. The least-squares fitted equation is \(\hat y_i=\hat\beta_0+\hat\beta_1x_i+\hat\beta_2z_i\). The adjusted slope of interest is \(\hat\beta_1\): the fitted change in \(y\) for a one-unit increase in \(x\), comparing observations at the same value of \(z\) under this additive model.

To residualize a variable against \(z\), regress that variable on an intercept and \(z\), then subtract its fitted value from its observed value. Write the two auxiliary regressions as

Here \(r_{x,i}=x_i-(\hat a_x+\hat c_xz_i)\) and \(r_{y,i}=y_i-(\hat a_y+\hat c_yz_i)\). Each residual measures how far that observation is above or below its own fitted line using \(z\). Because both auxiliary regressions include an intercept, each set of residuals sums to zero. Their residuals are also uncorrelated, in the sample, with \(z\).

The Residualization Result

The denominator must be positive. If it were zero, all \(r_{x,i}\) would be zero, meaning \(x\) could be predicted perfectly from an intercept and \(z\). The data would then provide no separate information with which to estimate the coefficients of \(x\) and \(z\).

The result is a least-squares identity. It does not require normally distributed errors or constant error variance. To interpret the model as describing a conditional mean, a common assumption is \(E(\varepsilon_i\mid x_i,z_i)=0\); this assumption is distinct from the algebraic identity. Nor does residualizing by itself establish that changing \(x\) causes a change in \(y\).

Why the Formula Holds

Let \(e_i=y_i-\hat y_i\) be the residual from the full two-predictor regression. The least-squares normal equations say that these full-model residuals have zero sample cross-product with each included predictor and with the intercept:

The auxiliary regression of \(y\) on an intercept and \(z\) gives its fitted values using those two terms. In the full model, the contribution of \(x\) is \(\hat\beta_1x_i\). Removing the intercept-and-\(z\) part of \(y_i\) therefore leaves \(\hat\beta_1\) times the part of \(x_i\) not explained by an intercept and \(z\), plus the full-model residual. In symbols,

To see why the full-model residual remains \(e_i\) after removing its linear association with \(z\), note that it is already orthogonal to both the intercept and \(z\), by the normal equations. Its least-squares fitted values from a regression on those terms are therefore zero.

Multiply the displayed relationship by \(r_{x,i}\) and sum over observations:

The last cross-product is zero. Indeed, \(r_{x,i}=x_i-(\hat a_x+\hat c_xz_i)\), so

Thus \(\sum_i r_{x,i}r_{y,i}=\hat\beta_1\sum_i r_{x,i}^{\,2}\). Dividing by the positive denominator gives the theorem. This derivation shows why both variables must be residualized: the numerator describes how their remaining deviations move together, while the denominator scales that cross-product by the remaining variation in \(x\).

Connection to Centered Cross-Products

The formula can also be written using centered cross-products. This makes explicit how the control predictor changes the part of the focal predictor and outcome that contributes to the slope. Define \(S_{ab}=\sum_i(a_i-\bar a)(b_i-\bar b)\) for variables \(a\) and \(b\). Regressing on an intercept and \(z\) gives residual cross-products

when \(S_{zz}>0\). For example, the first expression subtracts from the total centered variation in \(x\) the amount accounted for by its linear association with \(z\). The second subtracts the part of the \(x\)-and-\(y\) cross-product attributable to their shared linear association with \(z\). These expressions follow by writing the fitted slope of each auxiliary regression as its centered cross-product with \(z\), divided by \(S_{zz}\), and substituting the fitted values into the residual cross-products.

What the Adjustment Does—and Does Not Do

Residualization removes only the linear association with the specified control predictor. It does not remove every possible relationship with \(z\), correct measurement errors, or ensure that the observations are comparable in all other ways. The adjusted slope depends on the model’s chosen control and its additive form. Its units are the units of \(y\) divided by the units of \(x\), just as for an ordinary regression slope.

A common mistake is to residualize \(x\) against \(z\) but leave \(y\) unchanged, then regress the raw \(y\) on the residualized \(x\) and call that slope the adjusted coefficient. This generally fails: the remaining linear association between \(y\) and \(z\) has not been removed. The theorem uses \(r_x\) and \(r_y\), both residualized against the same control, and their cross-product. Another mistake is to treat the residuals as observations at a common actual value of \(z\). They are deviations from fitted linear patterns, not matched measurements or causal effects.