Selecting Transformations In GLMs

Why Transformation Choice Matters

A continuous predictor does not have to enter a generalized linear model in its original units. For example, a model may use age, log age, or age squared. These choices represent different assumptions about how the predictor relates to the linear predictor, and therefore can produce different fitted means and predictions. The task here is to compare a prespecified set of transformations, not to search without limits for a curve that happens to fit the observed data.

A transformation applies to a predictor, not to the response distribution. If a Poisson GLM is used in every candidate, the response remains Poisson; what changes is the form of the predictor’s contribution to the linear predictor. A sound comparison also keeps the response observations, other terms, offsets, and likelihood convention fixed.

Definition: Let $x_i$ be a continuous predictor for observation $i$, and let $g_1,\ldots,g_M$ be prespecified transformations defined at all observed $x_i$. A candidate GLM uses one transformation at a time, for example $$ \eta_i=\beta_0+\beta_1 g_m(x_i)+\boldsymbol{z}_i^\mathsf{T}\boldsymbol{\gamma}+o_i,\qquad m\in\{1,\ldots,M\}, $$ where $\boldsymbol{z}_i$ denotes other predictors, $o_i$ is a fixed offset, and $\eta_i$ is the linear predictor. The candidate set consists of these specified models, together with their stated response distribution and link.

Common choices include $g(x)=x$, $g(x)=\log(x)$ when $x>0$, $g(x)=\sqrt{x}$ when $x\geq 0$, and $g(x)=x^2$. A power transformation must be defined on the observed values; a logarithm cannot be applied to zero or negative values without an explicitly chosen alternative definition. A transformation should also have a scientific rationale when possible. A logarithm might represent diminishing changes across a positive, highly skewed measurement, while a square might represent curvature.

Scale, Units, and Interpretation

The coefficient attached to a transformed predictor describes a change on the model’s linear-predictor scale. With a log link, for example, changing $g(x)$ by one unit changes the log of the mean by $\beta_1$, holding the other predictors fixed. The size of that “one unit” depends on the transformation and its units. A coefficient for age in years is not numerically the same as a coefficient for age in decades, even when those specifications describe exactly the same model.

Theorem: Suppose a GLM includes an intercept and an unrestricted coefficient for $g(x_i)$, and otherwise has fixed terms, response distribution, link, offsets, and observations. Replacing $g(x_i)$ by $a+b\,g(x_i)$, for fixed $a$ and $b\neq 0$, leaves the set of possible linear predictors unchanged. Consequently, it leaves the maximized likelihood and fitted means unchanged, provided the corresponding parameterization is mapped one-to-one.

Proof. Write the part of the linear predictor involving the intercept and transformation as $\beta_0+\beta_1g(x_i)$. Under the replacement $h(x_i)=a+b\,g(x_i)$, set $\gamma_1=\beta_1/b$ and $\gamma_0=\beta_0-a\beta_1/b$. Then $\gamma_0+\gamma_1h(x_i)=\beta_0+\beta_1g(x_i)$ for every $i$. Conversely, any $(\gamma_0,\gamma_1)$ gives the same form with $\beta_1=b\gamma_1$ and $\beta_0=\gamma_0+a\gamma_1$. Thus the two parameterizations generate exactly the same linear predictors. The response distribution and link are unchanged, so the same fitted means and likelihood values are available in both models. The parameter mapping is one-to-one because $b\neq0$. $\square$

Worked Example: Age in Years or Decades

Suppose a logistic GLM has linear predictor $\eta_i=\alpha+\beta x_i$, where $x_i$ is age in years. Re-express age in decades as $u_i=x_i/10$. Since $x_i=10u_i$,

\alpha+\beta x_i=\alpha+(10\beta)u_i.

Thus the same fitted probabilities are obtained with decade-scale coefficient $\gamma=10\beta$ and unchanged intercept $\alpha$. If the fitted coefficient per year is $0.04$, the coefficient per decade is $0.40$: a ten-year increase changes the log odds by $0.40$, and the corresponding odds ratio is $\exp(0.40)\approx1.49$. This is a change of units, not evidence that one model fits better.

A shifted transformation can work the same way. For positive age, $\log(x/10)=\log(x)-\log(10)$. With an intercept, the constant shift is absorbed into the intercept, so models using $\log(x)$ and $\log(x/10)$ have the same fitted values. Their intercepts, however, refer to different reference points.

This invariance does not make all transformations equivalent. In general, $x$ and $\log(x)$ do not differ by a shift and nonzero rescaling, so the sets of possible linear predictors differ. Their fitted means can differ, and a comparison criterion may favor one over the other. Interpretation should be examined alongside fit: for a differentiable transformation, the predictor’s local effect on the linear predictor is $\beta_1 g'(x)$, which can vary across the range of $x$.

Comparing Fit Without Rewarding Complexity for Free

A maximized likelihood measures fit to the observed data, but adding flexibility can improve it even when the extra flexibility does not predict well. Consider candidates fitted by maximum likelihood to the same observations, with a comparable likelihood. For a model with $k$ estimated parameters, AIC is $-2\ell(\hat{\theta})+2k$, while BIC is $-2\ell(\hat{\theta})+k\log(n)$. The parameter count must include all estimated parameters relevant to the likelihood, including a dispersion parameter when it is estimated. These criteria are comparable only when the underlying likelihoods refer to the same response data and are defined on the same scale.

A raw transformation and a log transformation are usually nonnested candidates: neither model need be a special case of the other. Their likelihoods can still be compared when the likelihoods are comparable, but a higher maximized likelihood alone is not a sufficient reason to choose the more flexible model. AIC and BIC apply different penalties; neither makes scientific interpretability irrelevant.

Theorem: Suppose a smaller candidate model is nested in a larger one: every parameter setting in the smaller model can be represented in the larger model. If both models use the same likelihood for the same observations, the larger model’s maximized log-likelihood cannot be smaller. If the larger model adds $q$ estimated parameters and the maxima exist, its AIC is lower only when its log-likelihood gain exceeds $q$.

Proof. Let $\Theta_S$ and $\Theta_L$ be the parameter spaces of the smaller and larger models, and suppose nesting means that the smaller model’s likelihood values are available in the larger model. Then $\sup_{\theta\in\Theta_L}\ell(\theta)\geq\sup_{\theta\in\Theta_S}\ell(\theta)$, because the supremum over a set containing all the smaller model’s possibilities cannot be lower. Write the log-likelihood gain as $\Delta\ell=\hat{\ell}_L-\hat{\ell}_S\geq0$. The AIC difference is \[ \mathrm{AIC}_L-\mathrm{AIC}_S=-2\Delta\ell+2q. \] It is negative exactly when $\Delta\ell>q$. Therefore an improvement in training likelihood alone does not guarantee an AIC improvement. $\square$

Worked Example: Comparing Two Gaussian Predictor Forms

Consider a Gaussian GLM with identity link and known variance $1$, using three observations: $x=(-1,0,1)$ and $y=(0,0,1)$. Compare an intercept plus $x$ with an intercept plus $x^2$. Both candidates have two estimated mean parameters. For known variance $1$, the log-likelihood at fitted means is $-\tfrac{3}{2}\log(2\pi)-\tfrac{1}{2}\mathrm{RSS}$, where RSS is the residual sum of squares.

For the model with $x$, least squares gives intercept $\bar y=1/3$ and slope $\sum_i x_i y_i/\sum_i x_i^2=1/2$. Its fitted values are $(-1/6,1/3,5/6)$, so $\mathrm{RSS}=1/36+1/9+1/36=1/6$.

For the model with $x^2$, the predictor values are $(1,0,1)$. The fitted value at $x^2=0$ is $0$; at $x^2=1$, it is the average of $0$ and $1$, which is $1/2$. Thus $\mathrm{RSS}=1/4+0+1/4=1/2$.

The log-likelihood for the $x$ model exceeds that for the $x^2$ model by $\tfrac{1}{2}(1/2-1/6)=1/6$. Since both models have the same parameter count, their AIC difference is $\mathrm{AIC}_{x}-\mathrm{AIC}_{x^2}=-2(1/6)=-1/3$, favoring $x$ by this criterion. This example compares two particular models; it does not establish that the relationship is linear outside these observations.

Validation Scores and the Search Scope

A validation score asks how well a fitted candidate predicts observations not used to fit it. For a GLM, one option is the negative log predictive density: score each held-out response using the candidate’s predictive distribution, then sum or average those losses. Lower scores are better. When using cross-validation, all data-dependent preprocessing—including estimated centering or scaling—must be learned from each training fold and then applied to its validation fold. A fixed, prespecified transformation such as $\log(x)$ does not need to be estimated, but its domain must still be checked.

Worked Example: Poisson Validation Scores

Suppose two Poisson GLMs, one using a linear predictor transformation and one using a log transformation, are fitted on training data. On a held-out set their predictive means for two observations are respectively $(1,1)$ and $(0.5,3)$. The observed counts are $(0,2)$. For a Poisson prediction with mean $\mu$ and count $y$, the negative log predictive density is $\mu-y\log(\mu)+\log(y!)$.

For the first model, the two losses are $1$ and $1-2\log(1)+\log(2)=1+\log(2)$. Its total score is $2+\log(2)\approx2.693$.

For the second model, the losses are $0.5$ and $3-2\log(3)+\log(2)$. Its total is $3.5-2\log(3)+\log(2)\approx1.996$. Dividing each total by two gives average scores of about $1.347$ and $0.998$, respectively. The second model predicts these held-out counts better under this score. The calculation supports a predictive comparison on this validation set; it does not establish that the second transformation will perform better in every sample.

Worked Example: Choosing a Defensible Candidate Set

Suppose a clinical study measures a positive biomarker $x$, and the proposed GLM uses the same outcome, link, and adjustment variables under three alternatives: $x$, $\log(x)$, and $\sqrt{x}$. All three are defined for every observed biomarker value. The analyst chooses this set before examining model-comparison results and evaluates it using AIC and a prespecified cross-validation score.

The comparison can answer which of these three specified models has the preferred score. It cannot establish that the best transformation among the three is the true response relationship, nor can it rule out other useful forms. Adding dozens of powers or cutpoints after seeing initial results creates a larger search than the stated set: the selected score can benefit from chance. A more defensible analysis records the candidate set and its rationale, reports the selection criterion, and treats the winning transformation as a modeling choice rather than a confirmatory test.

Practical Checks and Common Mistakes

Comparing unlike likelihoods. Check that candidates use the same response observations and comparable likelihood definitions. A numerical likelihood difference is not meaningful if one model uses a different response scale or data subset.
Treating units as model improvements. Replacing $x$ by $x/10$ changes a coefficient’s numerical value, but with an intercept it does not change the fitted model.
Choosing by training likelihood alone. A more flexible nested model cannot fit worse at its maximum likelihood. Use a complexity-aware criterion or an appropriate validation score rather than interpreting this mechanical improvement as evidence of better prediction.
Ignoring the transformation’s domain. A logarithm requires positive inputs, and a square-root transformation requires nonnegative inputs. Any alternative handling of values outside the domain must be explicitly specified.
Forgetting that a search has occurred. A score used to choose among many transformations is not an untouched assessment of the final choice. Keep the search limited and prespecified, and use validation data appropriately.

Transformation selection matters because it can change both the shape of the fitted relationship and the predictions made at clinically relevant values. A useful report names the candidate transformations, states the comparison criterion, describes any data-dependent scaling, and presents the chosen model’s effects on an interpretable scale. These steps make clear what the comparison supports—and what it does not.

Check Your Understanding

Use the definitions and results above to answer the following questions.

Why do models using $x$ and $x/10$ have the same fitted values when both include an intercept and an unrestricted coefficient?
What must be held fixed for likelihood-based comparison of transformations to be meaningful?
A nested model adds two parameters and improves the maximized log-likelihood by $1.5$. Does its AIC improve? Show the calculation.
For which observed values is $\log(x)$ defined, and what should be checked before fitting that candidate?
Why is a validation score for the selected transformation not an untouched assessment if the same validation results were used to select it?