Deriving Coefficients Under Affine Rescaling

Why Derive the Coefficient Changes?

A regression coefficient is expressed in the units used for its variables. Converting minutes to ten-minute units, or changing the zero point of a predictor, changes the numerical coefficients. But that does not necessarily change the fitted outcomes. A precise derivation helps distinguish a change in the model’s description from a change in its predictions.

Here we derive the changes for an additive multiple regression model when predictors and the outcome undergo affine transformations: each variable may be shifted and multiplied by a nonzero constant. The model includes an intercept. The derivation applies to OLS coefficients and makes clear why transforming the predictors alone preserves fitted outcomes, whereas transforming the outcome transforms the fitted outcomes too.

Model and Affine Transformations

Definition: For $n$ observations and $p$ predictors, an additive linear regression model with an intercept is $$ y_i=\beta_0+\sum_{j=1}^{p}\beta_j x_{ij}+\varepsilon_i,\qquad i=1,\ldots,n. $$ The OLS estimates $\hat{\beta}_0,\ldots,\hat{\beta}_p$ minimize the sum of squared residuals over all coefficient values. We assume the design matrix, whose columns are an intercept and the $p$ predictors, has full column rank, so the OLS coefficient estimates are unique. The algebra below does not require normally distributed errors.

For each predictor $j$, define a transformed version $x_{ij}^{*}=a_j+b_jx_{ij}$, where $a_j$ is a shift and $b_j\ne 0$ is a multiplier. Also allow the outcome to be transformed as $y_i^{*}=c+dy_i$, where $d\ne 0$. Nonzero multipliers ensure that each transformation can be reversed: $x_{ij}=(x_{ij}^{*}-a_j)/b_j$ and $y_i=(y_i^{*}-c)/d$.

Key result: If the original OLS model includes an intercept, the coefficients after these transformations are $$ \hat{\beta}_j^{*}=\frac{d\hat{\beta}_j}{b_j},\qquad j=1,\ldots,p, $$ and $$ \hat{\beta}_0^{*}=c+d\hat{\beta}_0-\sum_{j=1}^{p}\frac{d\hat{\beta}_j a_j}{b_j}. $$ The transformed fitted values satisfy $$ \hat{y}_i^{*}=c+d\hat{y}_i. $$

These formulas include shifts and changes of scale in the same calculation. The multiplier $b_j$ affects the slope for predictor $j$; the shift $a_j$ affects the intercept. An outcome multiplier $d$ scales all slopes, and an outcome shift $c$ contributes directly to the intercept.

Deriving the Coefficient and Fitted-Value Formulas

First consider how to express the original fitted regression in terms of the transformed variables. Substitute $x_{ij}=(x_{ij}^{*}-a_j)/b_j$ into the original fitted equation, then transform its outcome:

\begin{aligned} c+d\left(\hat{\beta}_0+\sum_{j=1}^{p}\hat{\beta}_j x_{ij}\right) &=c+d\hat{\beta}_0+\sum_{j=1}^{p}\frac{d\hat{\beta}_j}{b_j}(x_{ij}^{*}-a_j)\\ &=c+d\hat{\beta}_0-\sum_{j=1}^{p}\frac{d\hat{\beta}_j a_j}{b_j} +\sum_{j=1}^{p}\frac{d\hat{\beta}_j}{b_j}x_{ij}^{*}. \end{aligned}

This identifies the transformed intercept and slopes in the key result. It also suggests what happens to fitted values: the transformed regression gives $c+d\hat{y}_i$ for observation $i$.

To verify that these are the OLS estimates, not merely coefficients that produce a plausible equation, use the least-squares criterion. For any original coefficient vector, the corresponding transformed equation has residual $y_i^{*}-\hat{y}_i^{*}=d(y_i-\hat{y}_i)$. Its sum of squared residuals is therefore multiplied by $d^2$:

\sum_{i=1}^{n}(y_i^{*}-\hat{y}_i^{*})^2 =d^2\sum_{i=1}^{n}(y_i-\hat{y}_i)^2.

Because $d^2>0$, multiplying the original sum of squared residuals by $d^2$ does not change which coefficient choice minimizes it. The transformations are reversible, so every candidate transformed equation with an intercept corresponds to a candidate original equation with an intercept. Thus the transformed OLS minimizer corresponds exactly to the original OLS minimizer, establishing the formulas.

The full-rank condition is also retained: shifting or multiplying a predictor by a nonzero constant does not create or remove a linear dependence among the intercept and predictor columns. Therefore the transformed OLS coefficients are unique under the stated condition.

Worked Example: Shifting and Rescaling Predictors and Outcome

Suppose a hypothetical fitted model, with outcome measured in points, is $$ \hat{y}=5+3x, $$ where $x$ is measured in its original units. Define $x^{*}=(x-1)/2$ and $y^{*}=10+2y$. Find the transformed fitted equation and check it for $x=0,1,2$.

Write $x^{*}=a+bx$, so $a=-0.5$ and $b=0.5$. For the outcome, $c=10$ and $d=2$. The transformed slope is $$ \hat{\beta}_1^{*}=\frac{d\hat{\beta}_1}{b}=\frac{2(3)}{0.5}=12. $$ The transformed intercept is $$ \hat{\beta}_0^{*}=c+d\hat{\beta}_0-\frac{d\hat{\beta}_1a}{b} =10+2(5)-\frac{2(3)(-0.5)}{0.5}=26. $$ Thus $\hat{y}^{*}=26+12x^{*}$.

For $x=0,1,2$, the original fitted values are $5,8,11$. The transformed predictor values are $-0.5,0,0.5$, and the transformed outcomes corresponding to those fitted values are $10+2(5)=20$, $10+2(8)=26$, and $10+2(11)=32$. The new equation gives $26+12(-0.5)=20$, $26+12(0)=26$, and $26+12(0.5)=32$, as required.

Predictor Changes Alter Coefficients, Not Fitted Outcomes

If only the predictors are transformed, then $c=0$ and $d=1$, so the fitted-value formula reduces to $\hat{y}_i^{*}=\hat{y}_i$. This equality is about predictions for the same observations, with the transformed predictor values inserted into the transformed equation. The coefficient numbers may change because the predictor units or zero points have changed, even though the fitted outcome values have not.

Worked Example: Two Predictor Conversions

Suppose a hypothetical additive model is $$ \hat{y}=40+2x_1-0.5x_2. $$ Here $x_1$ is measured in minutes and $x_2$ is a score. Define $x_1^{*}=x_1/10$, so $a_1=0$, $b_1=0.1$, and define $x_2^{*}=x_2-4$, so $a_2=-4$, $b_2=1$. The outcome is not transformed.

The new slopes are $$ \hat{\beta}_1^{*}=\frac{2}{0.1}=20,\qquad \hat{\beta}_2^{*}=\frac{-0.5}{1}=-0.5. $$ The new intercept is $$ \hat{\beta}_0^{*}=40-\frac{2(0)}{0.1}-\frac{(-0.5)(-4)}{1}=38. $$ The transformed equation is $\hat{y}=38+20x_1^{*}-0.5x_2^{*}$.

For the original profile $x_1=10$, $x_2=6$, the original prediction is $40+2(10)-0.5(6)=57$. The transformed profile is $x_1^{*}=1$, $x_2^{*}=2$, which gives $38+20(1)-0.5(2)=57$. Both equations predict the same outcome for this profile.

Outcome Changes Transform Fitted Values

If predictors stay as they are and only the outcome is transformed, set $a_j=0$, $b_j=1$ for every predictor. The coefficient formulas then become $\hat{\beta}_0^{*}=c+d\hat{\beta}_0$ and $\hat{\beta}_j^{*}=d\hat{\beta}_j$. The transformed fitted value is $c+d\hat{y}_i$, just as the transformation prescribes for the observed outcome.

Worked Example: Rescaling an Outcome After OLS

Consider three hypothetical observations $(x,y)=(0,1),(1,2),(2,4)$, with $x$ measured in hours and $y$ in points. With an intercept, $\bar{x}=1$ and $\bar{y}=7/3$. The slope is the sum of products of deviations divided by the sum of squared predictor deviations: $$ \hat{\beta}_1= \frac{(-1)(-4/3)+(0)(-1/3)+(1)(5/3)} {(-1)^2+0^2+1^2} =\frac{3}{2}=1.5. $$ The intercept is $\hat{\beta}_0=\bar{y}-\hat{\beta}_1\bar{x}=7/3-3/2=5/6$. The fitted equation is $\hat{y}=5/6+1.5x$.

Now express the outcome on a new scale using $y^{*}=100+10y$. The transformed intercept and slope are $$ \hat{\beta}_0^{*}=100+10(5/6)=325/3\approx108.3,\qquad \hat{\beta}_1^{*}=10(1.5)=15. $$ So $\hat{y}^{*}=325/3+15x$. The original fitted values at $x=0,1,2$ are $5/6,7/3,23/6$. Applying $100+10\hat{y}$ gives $325/3,370/3,415/3$; substituting $x=0,1,2$ in the transformed equation gives those same values.

The original residuals are $1/6,-1/3,1/6$, so the original residual sum of squares is $(1/6)^2+(-1/3)^2+(1/6)^2=1/6$. After the transformation, residuals are multiplied by 10 and the residual sum of squares is $100(1/6)=50/3$. Its numerical value changes, but multiplying every candidate sum of squares by the same positive factor leaves the minimizing fit unchanged in the corresponding transformed coordinates.

Common Mistake and Practical Checks

A common mistake is to divide a slope by the predictor’s conversion multiplier without adjusting the intercept when the predictor has also been shifted. For example, replacing $x$ by $x^{*}=x-4$ leaves the slope unchanged, but it does not leave the intercept unchanged. Since $x=x^{*}+4$, the constant term gains $4\hat{\beta}_1$. The correct general formula includes the shift contribution $-d\hat{\beta}_j a_j/b_j$ for each predictor.

A reliable check is to choose a predictor profile, convert its values, and compute the fitted outcome both before and after the transformation. The predictions should agree if only predictors changed, and should be related by $c+d\hat{y}$ if the outcome changed too. Keep units visible as well: a slope has units of outcome per unit of predictor, so changing either variable’s scale changes the numerical slope accordingly.

These formulas are for a model with an intercept. The intercept makes predictor shifts expressible as changes to the constant term while preserving the same set of fitted values. Without an intercept, a shifted predictor may not preserve that set of fits, so the same formulas should not be applied automatically.

Key takeaway: With an intercept, invertible affine transformations preserve the fitted regression structure. Predictor shifts and scale changes alter coefficient values but leave fitted outcomes unchanged when the outcome is unchanged. An outcome transformation changes fitted values to $c+d\hat{y}_i$, and the slopes and intercept change according to the same affine conversion.

Check Your Understanding

Use the formulas and reasoning from this tutorial to answer the following.

If $x^{*}=3+2x$ and the outcome is unchanged, how does the slope transform? What additional coefficient must be adjusted because of the shift?
In a two-predictor model, suppose only the outcome is multiplied by $-2$, with no shift. How do the intercept, slopes, and fitted values change?
Why does multiplying the outcome by a nonzero constant leave the OLS minimizer in correspondence with the original one?
A predictor is converted by $x^{*}=x/60$, with no shift. If its original slope is 4 outcome units per original predictor unit, what is its transformed slope?
Why should the affine transformation formulas not be applied automatically to a regression model without an intercept?