Why Transform a Regression?
A linear regression is useful when the conditional mean of the response is adequately represented by a linear function of the predictors and the residual behavior is appropriate for the intended analysis. Sometimes the relationship is curved, or the residual spread changes systematically across fitted values. A transformation can help make the mean relationship more nearly linear or the residual spread more nearly constant. It does not automatically fix every model problem: the transformed model must still be checked with diagnostics.
A transformation changes what the model describes. If the response is transformed, the fitted regression concerns the conditional mean of that transformed response, not automatically the conditional mean of the original response. If a predictor is transformed, its coefficient describes a change in the response associated with a unit change in the transformed predictor, not necessarily a constant change per original predictor unit.
For ordinary least squares on this transformed scale, the model assumes that \(E[g(Y_i)\mid X_i]=\beta_0+\beta_1h(X_i)\), and that the errors have conditional mean zero, constant conditional variance, and are independent across observations. Normal errors are additionally assumed when using exact small-sample \(t\)- or \(F\)-inference. These conditions concern the transformed response and transformed predictors. A log response requires positive observed responses; a square-root response requires nonnegative responses. A log predictor likewise requires positive predictor values.
Use Diagnostics to Motivate a Transformation
Residual plots suggest possibilities; they do not prescribe a transformation mechanically. A curved residual pattern against fitted values can indicate that the mean relationship is not adequately linear. A fan-shaped pattern, in which residual spread grows or shrinks as fitted values increase, can indicate nonconstant variance. For a single predictor, a curved residual pattern against that predictor can motivate transforming the predictor. The response transformation can affect both the mean pattern and the variance pattern, so a candidate should be refit and checked rather than chosen from a rule alone.
A log response is often considered when responses are positive and variation appears to grow with the response level. One justification is a multiplicative-error model. If \(Y=m(X)\exp(\varepsilon)\), with \(m(X)>0\), then taking logarithms gives \(\log(Y)=\log(m(X))+\varepsilon\). Thus, multiplicative deviations on the original scale become additive deviations on the log scale. This algebra motivates the transformation; it does not guarantee that the transformed errors have constant variance or satisfy the other regression assumptions.
A square-root response can be a useful candidate for nonnegative responses when a less-than-proportional scale change is plausible. A log predictor can represent a relationship that changes rapidly at small predictor values and more slowly at large values. A square-root predictor can represent a different kind of diminishing change. In each case, the substantive meaning of the transformed scale and the diagnostic evidence matter.
Examine residuals against fitted values and relevant predictors. Distinguish a curved mean pattern from changing residual spread.
Choose a response or predictor transformation that is defined for the data and has a defensible interpretation.
Fit the model on the proposed scale, then recheck residual patterns and the assumptions required for the planned inference.
State what a coefficient means in terms of \(g(Y)\) and \(h(X)\). Do not silently treat it as an effect in the original units.
Worked Examples
Worked Example: Fitting a Square-Root Response Model
In a hypothetical study of service requests, analysts suspect that the response spread grows with the fitted response. They consider a square-root response. For four observations, the predictor values are \(X=0,1,2,3\), and the observed responses are \(Y=4,9,9,16\). Fit the least-squares line for \(Z=\sqrt{Y}\), calculate its residual variance estimate, and predict on the transformed scale at \(X=2.5\).
Method and transformation. The responses are nonnegative, so the square root is defined. Transforming gives \(Z=2,3,3,4\). Fit \(Z_i=\beta_0+\beta_1X_i+\varepsilon_i\) by least squares.
Calculate the coefficients. The means are \(\bar X=(0+1+2+3)/4=1.5\) and \(\bar Z=(2+3+3+4)/4=3\). The centered predictor sum of squares is \(S_{XX}=(-1.5)^2+(-0.5)^2+(0.5)^2+(1.5)^2=5\). The centered cross-product is \(S_{XZ}=(-1.5)(-1)+(-0.5)(0)+(0.5)(0)+(1.5)(1)=3\). Therefore \(\hat\beta_1=S_{XZ}/S_{XX}=3/5=0.6\), and \(\hat\beta_0=\bar Z-\hat\beta_1\bar X=3-(0.6)(1.5)=2.1\). The fitted equation is \(\hat Z=2.1+0.6X\).
Check the fit and calculate residual variance. At \(X=0,1,2,3\), the fitted \(Z\) values are \(2.1,2.7,3.3,3.9\). Observed minus fitted residuals are \(-0.1,0.3,-0.3,0.1\). Thus the residual sum of squares is \(0.01+0.09+0.09+0.01=0.20\). With \(n=4\) observations and two estimated coefficients, the residual variance estimate is \(s^2=0.20/(4-2)=0.10\).
Predict on the modeled scale. At \(X=2.5\), \(\hat Z=2.1+(0.6)(2.5)=3.6\). The requested fitted value is 3.6 square-root units. Squaring 3.6 gives 12.96, but that quantity is not thereby established as the conditional mean of \(Y\). The fitted regression directly estimates a conditional mean on the \(Z=\sqrt{Y}\) scale.
Common mistake avoided: treating the slope 0.6 as an increase of 0.6 original response units per unit of \(X\). It is an increase of 0.6 fitted square-root response units per unit of \(X\).
Worked Example: Interpreting a Log-Response Regression
A hypothetical analyst studies positive claim-processing amounts \(Y\) and suspects that their variability increases with their level. The analyst fits a model of \(\log(Y)\) against \(Q=\log(X)\), where \(X>0\). For four observations, the transformed predictor values are \(Q=0,1,2,3\), and the transformed responses are \(Z=\log(Y)=1,1.5,2,2.5\). Calculate the regression coefficients and interpret the slope on the modeled scale.
Method. Fit \(Z_i=\beta_0+\beta_1Q_i+\varepsilon_i\). The transformed means are \(\bar Q=(0+1+2+3)/4=1.5\) and \(\bar Z=(1+1.5+2+2.5)/4=1.75\).
The centered predictor sum of squares is \(S_{QQ}=(-1.5)^2+(-0.5)^2+(0.5)^2+(1.5)^2=5\). The centered cross-product is \(S_{QZ}=(-1.5)(-0.75)+(-0.5)(-0.25)+(0.5)(0.25)+(1.5)(0.75)=2.5\). Therefore \(\hat\beta_1=2.5/5=0.5\), and \(\hat\beta_0=1.75-(0.5)(1.5)=1.00\). The fitted equation is \(\widehat{\log(Y)}=1+0.5\log(X)\). The fitted transformed responses at \(Q=0,1,2,3\) are \(1,1.5,2,2.5\), exactly the four observed transformed values, so the residual sum of squares is zero for these supplied data.
Interpret the coefficient. A one-unit increase in \(\log(X)\) is associated with a 0.5-unit increase in fitted \(\log(Y)\). Since increasing \(\log(X)\) by one corresponds to multiplying \(X\) by \(e\), the model's fitted log response increases by 0.5 for that predictor change. This is an interpretation of the fitted log response, not a statement that the original response increases by 0.5 units.
Common mistake avoided: calling 0.5 the change in \(Y\) for a one-unit increase in \(X\). The fitted equation uses both \(\log(X)\) and \(\log(Y)\); its slope is measured in log-response units per log-predictor unit.
Worked Example: Transforming a Curved Predictor
A hypothetical analyst models a nonnegative exposure index \(X\) and a response \(Y\). A residual plot from a straight-line fit against \(X\) shows systematic curvature, so the analyst considers \(Q=\sqrt{X}\). The observed values are \(X=0,1,4,9\) and \(Y=5,7,9,11\). Fit the model using \(Q\), predict at \(X=16\), and compare the slope interpretation with a constant-slope model in \(X\).
Transform and fit. The transformed predictors are \(Q=0,1,2,3\). Their mean is \(\bar Q=1.5\), and the response mean is \(\bar Y=(5+7+9+11)/4=8\). The centered predictor sum of squares is \(S_{QQ}=5\). The centered cross-product is \(S_{QY}=(-1.5)(-3)+(-0.5)(-1)+(0.5)(1)+(1.5)(3)=10\). Thus \(\hat\beta_1=10/5=2\) and \(\hat\beta_0=8-(2)(1.5)=5\). The fitted equation is \(\hat Y=5+2\sqrt{X}\).
Check and predict. At the four observed predictor values, the fitted responses are \(5,7,9,11\), so all four residuals are zero. At \(X=16\), \(\sqrt{X}=4\), giving \(\hat Y=5+(2)(4)=13\).
Interpret the slope. The coefficient 2 describes a two-unit increase in fitted \(Y\) for a one-unit increase in \(\sqrt{X}\). For \(X>0\), the rate of change with respect to the original predictor is \(d\hat Y/dX=2/(2\sqrt X)=1/\sqrt X\). For example, at \(X=4\) this rate is \(1/2=0.5\), whereas at \(X=9\) it is \(1/3\). The effect per original \(X\)-unit is therefore not constant.
Common mistake avoided: interpreting 2 as the change in fitted \(Y\) for every one-unit increase in \(X\). It is the slope with respect to \(\sqrt{X}\); the slope with respect to \(X\) varies with \(X\).
Choosing and Checking a Transformation
A transformation should address a diagnosed feature while keeping the resulting model meaningful. For example, a log transformation cannot be applied directly to zero or negative observations. Adding a constant to make the logarithm defined changes the transformation and must be justified rather than treated as a harmless computational step. A square-root transformation is defined at zero, but not for negative values. If a predictor is transformed, the same transformation must be used consistently when calculating fitted values.
The least-squares criterion is also scale-specific. A regression of \(Y\) on \(X\) minimizes squared residuals in the units of \(Y\); a regression of \(\log(Y)\) on \(X\) minimizes squared residuals in log units. Consequently, residual sums of squares and \(R^2\) values from models with different response transformations do not directly measure error on a common response scale. Model choice should consider the diagnostic improvement, the intended use, and the scale on which errors matter.
One useful algebraic fact explains why a predictor transformation changes slope meaning. If the fitted model is \(\hat Y=\hat\beta_0+\hat\beta_1h(X)\), then a small change in \(X\) changes the fitted response at rate \(\hat\beta_1h'(X)\), where the derivative exists. For \(h(X)=\sqrt X\), this rate varies with \(X\); for \(h(X)=\log X\), it is \(\hat\beta_1/X\). The coefficient remains a constant slope in transformed-predictor units, but generally not in original-predictor units.
Check Your Understanding
For each case, identify the transformation or interpretation required and check the arithmetic before comparing with the answer.
- A model fitted to \(Z=\sqrt{Y}\) is \(\hat Z=1.5+0.4X\). What is the fitted value of \(Z\) at \(X=5\), and what does the slope describe? (Answer: \(\hat Z=3.5\); the fitted square-root response increases by 0.4 per one-unit increase in \(X\).)
- A fitted model is \(\widehat{\log(Y)}=0.8+0.3X\). Calculate the fitted log response at \(X=4\) and interpret the slope, without treating it as an original-unit effect. (Answer: 2.0; fitted \(\log(Y)\) increases by 0.3 per one-unit increase in \(X\).)
- A model is \(\hat Y=4+3\log(X)\), with \(X>0\). Calculate the fitted value at \(X=e^2\), and state the change in fitted response when \(X\) is multiplied by \(e\). (Answer: 10; multiplying \(X\) by \(e\) increases \(\log(X)\) by 1 and fitted \(Y\) by 3.)
- For four observations, \(Q=\sqrt X\) takes values \(0,1,2,3\), and \(Y\) takes values \(2,4,6,8\). Find the least-squares line for \(Y\) on \(Q\), then predict at \(X=25\). (Answer: \(\hat Y=2+2Q=2+2\sqrt X\); at \(X=25\), \(\hat Y=12\).)
- A residual plot from a model using positive \(Y\) shows residual spread increasing with fitted response. An analyst fits \(\log(Y)\) on \(X\), but the residual plot on the transformed scale still has a clear fan shape. Is the transformation alone sufficient evidence that the model assumptions now hold? (Answer: No; the transformed model must be diagnosed again, and the remaining pattern indicates the constant-variance assumption may still be inadequate.)