Transforming Predictors Versus Transforming Outcomes

Two Different Reasons to Transform

A transformation can enter a predictive model in two fundamentally different places. Transforming a predictor changes the form of the relationship between an input and the outcome. Transforming the outcome changes the scale on which the model is fitted, and therefore can change the quantity the fitted model directly estimates. These choices may both involve logarithms, but they do not have the same interpretation.

For instance, an analyst might model annual cost as a linear function of the logarithm of exposure. The predicted cost is still in dollars. Alternatively, the analyst might model the logarithm of annual cost. The fitted value is then on the log-cost scale; converting it back to dollars requires care, particularly when the business target is average cost rather than a typical or median cost.

Definition: A predictor transformation replaces an input $X$ with a specified function $g(X)$ in the model while leaving the response $Y$ on its original scale. An outcome transformation replaces $Y$ with a specified function $h(Y)$ as the response being modelled. The predictor transformation changes the model’s functional form in $X$; the outcome transformation changes the scale of the model target.

The transformation must be defined for the observed and future values to which the model will be applied. For example, $\log(X)$ requires $X>0$, and $\log(Y)$ requires $Y>0$. Other transformations have their own domains. A model that cannot transform a future input, or an outcome that includes values outside the transformation’s domain, is not usable as specified.

Transforming a Predictor: The Response Stays in Its Original Units

Suppose a regression model uses $g(X)$ as a predictor and assumes $Y=\beta_0+\beta_1g(X)+\varepsilon$, with $E[\varepsilon\mid X]=0$. Under that condition, its conditional mean prediction is $E[Y\mid X]=\beta_0+\beta_1g(X)$. In particular, if $g(X)=\log(X)$, the fitted outcome remains in the original units of $Y$. If $Y$ is measured in dollars, the prediction is in dollars.

The slope $\beta_1$ describes the change in predicted $Y$ for a one-unit increase in $\log(X)$, not for a one-unit increase in $X$. For a change from $x_1$ to $x_2$, the change in the model prediction is $\beta_1\log(x_2/x_1)$. Thus a log predictor often describes a relationship in which equal proportional changes in $X$ correspond to equal changes in predicted $Y$. This is an additive change on the response scale, not a percentage change in $Y$.

Worked Example: Logarithm of Exposure as a Predictor

A hypothetical service plan has annual cost $Y$, in dollars, modelled by $E[Y\mid X=x]=100+25\log(x/100)$, where $X>0$ is the number of eligible service units. Compare the predicted annual costs at $X=100$ and $X=200$, and interpret the difference. Assume the model’s conditional mean specification is appropriate.

Method. Substitute each exposure into the predictor transformation, then calculate the model prediction. The ratio inside the logarithm is dimensionless, and both exposures are positive, so the transformation is defined.

At $x=100$, $x/100=1$, so $\log(1)=0$. The prediction is $100+25(0)=\$100.00$.

At $x=200$, $x/100=2$, and $\log(2)\approx0.693147$. The prediction is $100+25(0.693147)=100+17.328675=\$117.33$, rounded to the nearest cent.

The predicted difference is $\$117.33-\$100.00=\$17.33$. Equivalently, doubling exposure changes the prediction by $25\log(200/100)=25\log(2)\approx\$17.33$. This is a dollar change because the response was not transformed. The common mistake to avoid is calling it a 17.33% increase: the model predicts a dollar difference, not a percentage difference.

Transforming an Outcome: The Model Targets a Different Scale

Suppose instead that the model is fitted to $h(Y)$, such as $\log(Y)$. Least squares on the transformed response estimates a conditional mean on that transformed scale, subject to the usual regression assumptions. It does not, in general, directly estimate the original-scale conditional mean $E[Y\mid X]$. Applying the inverse transformation to a fitted transformed-scale value produces a prediction, but its statistical meaning depends on the error distribution and the target of interest.

For a log-outcome model, write $\log(Y)=\eta(X)+\varepsilon$, where $\eta(X)$ is the model’s conditional location on the log scale. If $E[\varepsilon\mid X]=0$, then $\eta(X)=E[\log(Y)\mid X]$. Exponentiating $\eta(X)$ gives the geometric mean of $Y$ conditional on $X$, when that quantity is defined. It is not automatically the arithmetic mean $E[Y\mid X]$.

A frequently used special case makes the distinction explicit. If, conditional on $X=x$, $\varepsilon\sim N(0,\sigma^2)$ with constant variance $\sigma^2$, then $Y$ is conditionally lognormal. In this case, $\exp(\eta(x))$ is the conditional median, while the conditional arithmetic mean is $\exp(\eta(x)+\sigma^2/2)$. These formulae require the stated normal-error model and, for the displayed mean adjustment, the stated variance. Without those assumptions, the adjustment $\sigma^2/2$ is not generally valid.

\begin{aligned} \log(Y)\mid X=x &\sim N(\eta(x),\sigma^2),\\ \operatorname{Median}(Y\mid X=x)&=\exp(\eta(x)),\\ E[Y\mid X=x]&=\exp\!\left(\eta(x)+\frac{\sigma^2}{2}\right). \end{aligned}

Worked Example: Log-Outcome Model With Normal Errors

A hypothetical claim-cost model assumes $\log(Y)=\log(100)+0.2X+\varepsilon$, where, conditional on $X$, $\varepsilon\sim N(0,0.16)$. Calculate the conditional median and conditional arithmetic mean of $Y$ at $X=0$. State why the two predictions differ.

Method. First calculate the log-scale location. Then use the lognormal median and mean formulae; the variance $0.16$ is the second parameter of the normal distribution and is not its standard deviation.

At $X=0$, $\eta(0)=\log(100)+0.2(0)=\log(100)$. The conditional median is $\exp(\log(100))=100$.

The conditional mean is $\exp(\log(100)+0.16/2)=100\exp(0.08)$. Since $\exp(0.08)\approx1.083287$, the mean is approximately $100(1.083287)=\$108.33$, rounded to the nearest cent.

The model therefore gives a median of $\$100$ and a mean of $\$108.33$. The difference arises because the lognormal distribution is right-skewed: larger values in the upper tail raise the arithmetic mean above the median. Simply exponentiating the fitted log value would give $\$100$, which is not the conditional mean. That shortcut is correct for the median under these assumptions, but not for the mean.

Why Retransformation Needs an Adjustment

The general reason for the difference is that exponentiation is nonlinear. In general, the exponential of an average is not the average of exponentials. For the log-outcome model, $Y=\exp(\eta(X))\exp(\varepsilon)$. Taking a conditional expectation therefore gives $E[Y\mid X]=\exp(\eta(X))E[\exp(\varepsilon)\mid X]$. The second factor cannot usually be replaced by 1 just because $E[\varepsilon\mid X]=0$.

Under normal errors, the factor is $\exp(\sigma^2/2)$. More generally, if a suitable estimate of the conditional error distribution is available, an analyst can estimate the factor $E[\exp(\varepsilon)\mid X=x]$ directly. One practical approach is a smearing estimate: use the average of exponentiated log-scale residuals, preferably within appropriate groups or conditional on predictors if the error distribution varies with $X$. This is a correction for predicting an original-scale mean, not a universal constant independent of model assumptions.

\begin{aligned} Y&=\exp(\eta(X))\exp(\varepsilon),\\ E[Y\mid X=x]&=\exp(\eta(x))E[\exp(\varepsilon)\mid X=x]. \end{aligned}

The identity above also explains why the zero-mean condition on log-scale errors alone does not justify using $\exp(\eta(x))$ as the original-scale mean. A transformation may be useful for modelling, but the inverse transformation does not erase its implications for the prediction target.

Worked Example: A Smearing Adjustment From Two Residuals

At a fixed predictor value, a fitted log-cost model gives $\eta=\log(80)$. Suppose the conditional log-scale error takes the value $-\log(2)$ with probability $0.5$ and $+\log(2)$ with probability $0.5$. Estimate the original-scale conditional mean using the specified error distribution, and check the result from the implied costs.

Method. Calculate the mean of $\exp(\varepsilon)$, then multiply it by $\exp(\eta)$. The error has conditional mean zero because its two values are opposites with equal probabilities, but its exponentials do not average to 1.

The smearing factor is $0.5\exp(-\log(2))+0.5\exp(\log(2))$. Since $\exp(-\log(2))=1/2$ and $\exp(\log(2))=2$, the factor is $0.5(0.5)+0.5(2)=0.25+1=1.25$.

The original-scale mean estimate is $\exp(\log(80))(1.25)=80(1.25)=\$100.00$.

As a check, the two possible costs are $80\exp(-\log(2))=80(1/2)=\$40$ and $80\exp(\log(2))=80(2)=\$160$. Their equally weighted mean is $(40+160)/2=\$100$, agreeing with the smearing calculation. The common error is to set the smearing factor to 1 because the log errors average to zero. The correct calculation averages their exponentials.

Choosing the Scale That Matches the Question

The appropriate model depends partly on what the prediction is for. If decisions depend on expected total dollars, the target is ordinarily an original-scale conditional mean. If the aim is to describe a typical outcome in a strongly right-skewed population, a conditional median may be more relevant. The model and any retransformation adjustment should match that target. A predictor transformation, by contrast, can change the shape of the fitted relationship while leaving the response target and its units intact.

Outcome transformations can also change which errors are treated as large. Least squares on $Y$ minimizes the sum of squared residuals measured in the units of $Y$. Least squares on $\log(Y)$ minimizes squared differences between log outcomes and fitted log outcomes. For positive outcomes, log differences correspond to multiplicative discrepancies: a given ratio between observed and predicted values has the same log difference regardless of the dollar scale. These are different fitting objectives, and neither is automatically best in every application.

When evaluating predictions, identify the scale of the metric. RMSE calculated on the log scale measures error in log units, while RMSE calculated after retransformation measures error in the original outcome units. They answer different questions and should not be compared as if they were the same quantity. A useful assessment reports or evaluates performance on the scale relevant to the decision, while also making clear how the fitted model was trained.

An outcome transformation also imposes domain and interpretation constraints. A logarithm cannot be applied to zero or negative outcomes without changing the method; adding a constant changes the transformation and its meaning. For an outcome that can be zero, negative, or otherwise outside the chosen transformation’s domain, a different modelling approach may be required. Do not silently alter the data or use an inverse transformation outside its valid range.

Derivation: The Lognormal Mean Adjustment

The adjustment for a normal log-scale error follows from the moment-generating function of a normal random variable. If $\varepsilon\sim N(0,\sigma^2)$, then its moment-generating function is $E[\exp(t\varepsilon)]=\exp(t^2\sigma^2/2)$. Setting $t=1$ gives $E[\exp(\varepsilon)]=\exp(\sigma^2/2)$. Since $Y=\exp(\eta)\exp(\varepsilon)$ at fixed $X=x$, taking expectations yields the original-scale mean formula.

\begin{aligned} E[Y\mid X=x] &=E[\exp(\eta(x)+\varepsilon)\mid X=x]\\ &=\exp(\eta(x))E[\exp(\varepsilon)\mid X=x]\\ &=\exp(\eta(x))\exp(\sigma^2/2)\\ &=\exp\!\left(\eta(x)+\frac{\sigma^2}{2}\right). \end{aligned}

The derivation depends on the conditional normal distribution with mean zero and variance $\sigma^2$. If the variance changes with $x$, then the adjustment also changes with $x$. If the conditional errors are not normal, the general expectation identity remains true, but the factor $\exp(\sigma^2/2)$ need not be correct.

Common Exam Traps

Confusing the location of the transformation. A model for $Y$ using $\log(X)$ predicts $Y$ on its original scale. A model for $\log(Y)$ predicts on the transformed outcome scale before retransformation.
Interpreting a transformed-predictor slope as a percent effect on $Y$. In a model linear in $\log(X)$ but not $\log(Y)$, a proportional change in $X$ produces an additive change in predicted $Y$.
Exponentiating a log prediction and calling it the mean. Under a lognormal model, this is the median; the mean includes a variance adjustment. Under other error distributions, use the appropriate conditional retransformation factor.
Assuming a zero-mean log error has an exponential with mean 1. In general, $E[\varepsilon\mid X]=0$ does not imply $E[\exp(\varepsilon)\mid X]=1$.
Ignoring the scale of an error metric. A log-scale error and a dollar-scale error measure different discrepancies and are not directly interchangeable.

Key takeaway: Transforming a predictor changes how the model relates inputs to the response; transforming the outcome changes the scale of the fitted target. Always state what the model estimates, which scale a prediction is on, and what assumptions are needed to translate it to the decision-relevant scale.

Check Your Understanding

For each problem, identify the model’s scale and show the key calculation before checking the answer.

A model for annual service cost is $E[Y\mid X=x]=50+8\sqrt{x}$. Compare predicted costs at $x=9$ and $x=16$, and state whether the difference is on the dollar or percentage scale. (Answer: $\$74$ and $\$82$, a dollar increase of $\$8$.)
A log-cost model has $\log(Y)=\log(25)+0.2X+\varepsilon$, where $\varepsilon\mid X\sim N(0,0.09)$. At $X=0$, give the conditional median and mean to the nearest cent. (Answer: Median $=\$25.00$; mean $=25\exp(0.045)\approx\$26.15$.)
At a fixed predictor value, a log-outcome model has $\eta=\log(60)$. Its errors are equally likely to be $-\log(3)$ and $+\log(3)$. Find the smearing factor and original-scale mean. (Answer: Factor $=(1/3+3)/2=5/3$; mean $=60(5/3)=\$100$.)
An intercept-only least-squares model is fitted to two positive outcomes, $10$ and $40$, after taking natural logarithms. What is the fitted value on the log scale, and what is its exponential? Compare that exponential with the arithmetic mean of the original outcomes. (Answer: Fitted log value $=(\log(10)+\log(40))/2=\log(20)$; exponential $=20$, while the original-scale arithmetic mean is $25$.)
A model is specified as $E[Y\mid X=x]=12+5\log(x)$, with $Y$ in dollars and $x>0$. What is the predicted change in $Y$ when $x$ increases from 20 to 40, rounded to the nearest cent? Is that change an original-scale dollar difference or a percent change? (Answer: $5\log(40/20)=5\log(2)\approx\$3.47$, an original-scale dollar difference.)

What You'll Learn