Comparing Transformed Models on the Outcome Scale

Why Put Competing Models on the Outcome Scale?

A transformed-outcome model may fit a relationship well while producing coefficients that are difficult to interpret in the units people care about. A slope of 0.12 on a log scale, for example, is not a change of 0.12 outcome units. Nor can that number be directly compared with a slope of 3.0 from a model fitted to the untransformed outcome. The coefficients describe changes on different scales.

To compare what models imply, choose the same predictor values and the same prediction target, then translate each model’s predictions to the original outcome scale. This makes the comparisons interpretable in context: predicted points, minutes, dollars, or other original units. The translation must respect what each model estimates. In particular, exponentiating a fitted log outcome does not generally give the arithmetic mean outcome.

What Each Model Predicts

For a direct linear model, write \(Y_i=\beta_0+\beta_1x_i+\varepsilon_i\). The model assumes that the conditional mean of \(Y_i\) is \(\beta_0+\beta_1x_i\), so the fitted prediction \(\hat{y}(x)=\hat{\beta}_0+\hat{\beta}_1x\) is in the original units of \(Y\). The error has conditional mean zero, \(E(\varepsilon_i\mid x_i)=0\). For the usual least-squares calculations, observations are treated as independent; constant conditional error variance is needed for the usual standard-error formulas, though not for calculating the fitted line itself.

For a log-outcome model, the response must be positive, and the model is \(\log(Y_i)=\gamma_0+\gamma_1x_i+U_i\). It assumes a linear conditional mean on the log scale and, for the usual interpretation of the fitted log value, \(E(U_i\mid x_i)=0\). Exponentiating the fitted log value gives \(\exp(\hat{\gamma}_0+\hat{\gamma}_1x)\), which is a prediction on the original scale but is not generally the arithmetic conditional mean. Indeed,

The second factor accounts for residual variation. A common estimate is the smearing factor \(S=\frac{1}{n}\sum_{i=1}^{n}\exp(\hat{u}_i)\), where \(\hat{u}_i\) are fitted residuals on the log scale. The arithmetic-mean prediction is then estimated by \(S\exp(\hat{\gamma}_0+\hat{\gamma}_1x)\) when the residual distribution, and therefore the smearing factor, is reasonably stable across the predictor values being considered. If residual variation changes with \(x\), one common factor may not be suitable.

A square-root outcome model requires a different translation. Suppose \(Z_i=\sqrt{Y_i}=\delta_0+\delta_1x_i+e_i\), with \(Y_i\geq 0\), \(E(e_i\mid x_i)=0\), and conditional variance \(\operatorname{Var}(e_i\mid x_i)=\sigma_e^2(x_i)\). Since \(Y_i=Z_i^2\), its arithmetic conditional mean is

This follows by expanding the square: if \(m(x)=\delta_0+\delta_1x\), then \(E\{(m(x)+e)^2\mid x\}=m(x)^2+2m(x)E(e\mid x)+E(e^2\mid x)=m(x)^2+\sigma_e^2(x)\). Thus, simply squaring the fitted square-root value omits the contribution of residual variation when the target is the arithmetic mean.

Worked Predictions and Contrasts

Why Raw Coefficients Are Not Comparable

A coefficient’s numerical size depends on the scale on which the model was fitted. In the direct model above, the slope 3 means that the fitted conditional mean changes by 3 outcome points per additional hour. In the log model, the slope 0.12 is a change in fitted log outcome per hour. For a one-hour increase, exponentiation corresponds to a multiplicative factor of \(\exp(0.12)\approx1.127\) in the model’s outcome-scale predictions when the same smearing factor applies. This is about a 12.7% increase, not an increase of 0.12 points. In the square-root model, the slope 0.30 is measured in square-root points per hour and does not represent a constant change in the original outcome units.

Even coefficients fitted to the same kind of transformed outcome can be hard to compare if the predictor units differ or the predictor change differs. A useful comparison states the predictor values explicitly, translates the relevant predictions to the original outcome scale, and reports an outcome-scale difference or ratio in context. A difference answers “how many outcome units?” A ratio answers “how many times as large?” These are distinct summaries and may tell different parts of the story.

The comparison also depends on the prediction target. A back-transformed log location and an arithmetic conditional mean are not interchangeable. Before comparing models, state whether the target is, for example, a conditional mean or a median, and use a suitable translation for each model. If the smearing factor varies across \(x\), use an appropriate \(x\)-specific estimate or avoid presenting a single-factor correction as an arithmetic-mean estimate.

How to Make a Fair Comparison

Common Mistakes

• Comparing slope magnitudes across scales. Saying a slope of 3 is larger than a slope of 0.12 ignores that the coefficients have different units and interpretations. Compare outcome-scale predictions or contrasts instead.
• Exponentiating and calling the result an arithmetic mean. Exponentiating a fitted log value does not generally account for residual variation. Use a suitable smearing adjustment when the target is the arithmetic mean.
• Comparing different targets. A median-like prediction from one model and an arithmetic-mean prediction from another do not form a like-for-like comparison. Identify the target before calculating predictions.
• Leaving out units or predictor values. “The effect is 12” is incomplete. State, for example, that the fitted mean increases by 12.0 points as the predictor rises from 2 to 6 hours under the stated model.
• Interpreting fitted contrasts causally. A regression contrast describes model-based association unless the study design and assumptions support a causal interpretation.