Why the Response Scale Matters
A model’s likelihood is calculated from probability densities for the observed response. A density is defined relative to a particular scale: a model for \(Y\) assigns densities to values of \(Y\), while a model for \(Z=\log Y\) assigns densities to values of \(Z\). Even when these variables contain equivalent information, their density values—and therefore their log likelihoods—are not automatically comparable.
This matters when using Akaike’s information criterion (AIC) to compare, for example, a Gaussian model fitted directly to a positive response with a model that assumes the logarithm of that response is Gaussian. A valid comparison must express both likelihoods as densities for the same observed response on the same scale. The change-of-variables formula supplies the required adjustment.
The Likelihood Adjustment for a Transformation
Suppose each observed response \(y_i\) is transformed to \(z_i=g(y_i)\), where \(g\) is differentiable and one-to-one. Write \(f_Z\) for the density of the transformed response. The change-of-variables formula gives the density of the original response:
The absolute derivative is the Jacobian factor. For independent observations \(y_1,\ldots,y_n\), multiply their densities to obtain the likelihood, or equivalently add their log densities:
Here \(\ell_Z\) is the log likelihood calculated using the transformed observations, under the corresponding transformed-response model. If the transformation is fixed and does not add estimated parameters, the parameter count \(k\) is unchanged. Thus the AIC for the model expressed as a density for \(Y\) is
The adjusted value, not the unadjusted \(\mathrm{AIC}_Z\), is the one to compare with AIC values for other models of \(Y\). The Jacobian contribution is common to candidate models using the same fixed transformation and data, but it is not generally common to a model fitted on a different response scale.
Specify \(z_i=g(y_i)\) and verify that it is one-to-one over the observed response range.
Find \(\left|g'(y_i)\right|\) for each observation, then add the log Jacobian terms.
Add the Jacobian sum to the transformed-scale log likelihood, then calculate AIC using the correct parameter count.
Compare AIC values only after they represent densities for the same observed response on the same scale.
The Logarithmic Transformation
For a positive response, let \(Z=\log Y\). Then \(g(y)=\log y\), so \(g'(y)=1/y\). Since \(y>0\), the absolute Jacobian is \(1/y\). The original-scale log likelihood for a model that is Gaussian on the log scale is therefore
The minus sign is important: \(\log|g'(y_i)|=\log(1/y_i)=-\log y_i\). If the transformed-scale AIC is \(\mathrm{AIC}_Z=-2\ell_Z+2k\), its original-response-scale equivalent is
A Gaussian model fitted directly to \(Y\) already has a likelihood for \(Y\). Its AIC can be compared with the adjusted lognormal AIC, provided both models use the same observations and the likelihood assumptions are appropriate. In these comparisons, a lognormal model means that \(\log Y\) is normally distributed; it does not mean that its unadjusted log-scale likelihood is a density for \(Y\).
Worked Example: Finding the Jacobian Adjustment for One Observation
Suppose \(Y>0\), \(Z=\log Y\), and a specified model gives \(Z\) a standard normal density. For one hypothetical observation \(y=2\), calculate the density for \(Y\) using the density for \(Z\). The model here is specified, not fitted: \(Z\sim N(0,1)\).
The transformed observation is \(z=\log 2\approx0.6931\), and the standard normal density is
The logarithm transformation has Jacobian \(1/y\). Therefore the density for the original response value is
Equivalently, the original-scale log density is \(\log f_Y(2)=\log f_Z(\log 2)-\log 2\approx-1.852\). The Jacobian changes the density because the same probability is being described per unit of \(Y\), rather than per unit of \(Z\).
Comparing a Gaussian and a Lognormal Model
Consider two models for the same positive observations. Model G assumes that the responses are independent and normally distributed with a common mean and variance. Model L assumes that their logarithms are independent and normally distributed with a common mean and variance. Both are fitted by maximum likelihood. Each estimates two parameters: a mean and a variance, so \(k=2\). The normal maximum-likelihood variance uses divisor \(n\), not \(n-1\).
To compare these models by AIC, Model G’s likelihood is already on the \(Y\) scale. For Model L, calculate the normal log likelihood on the log scale, then subtract \(\sum_i\log y_i\) to obtain its log likelihood as a density for \(Y\). This is a comparison of fitted likelihoods, not a claim that either model describes the data well; the distributional assumptions still require scrutiny.
Worked Example: AIC for Gaussian and Lognormal Models
Use the hypothetical positive observations \(y=(1,e,e^2)\), where \(e\) is the base of the natural logarithm. For convenient exact calculations, their logarithms are \(z=(0,1,2)\). Assume independence and the two model forms described above.
Model G, Gaussian on the original response scale. The sample mean is \(\bar{y}=(1+e+e^2)/3\approx3.702\). The maximum-likelihood variance is
For independent normal observations with an estimated common mean and variance, the maximized log likelihood is \(\ell_G=-\frac{n}{2}\left[\log(2\pi)+1+\log(\hat{\sigma}^2_Y)\right]\). Substituting \(n=3\) and \(\hat{\sigma}^2_Y\approx7.288\) gives
With \(k=2\), \(\mathrm{AIC}_G=-2\ell_G+2k\approx2(7.236)+4=18.472\).
Model L, Gaussian on the logarithmic response scale. The mean of \(z=(0,1,2)\) is \(\bar{z}=1\); its maximum-likelihood variance is \(\hat{\sigma}^2_Z=[(0-1)^2+(1-1)^2+(2-1)^2]/3=2/3\). The log likelihood calculated for the transformed observations is
For these observations, \(\sum_i\log y_i=0+1+2=3\). The logarithmic transformation’s Jacobian therefore gives the log likelihood for \(Y\): \(\ell_L=\ell_Z-3\approx-6.649\). Its AIC on the \(Y\) scale is
The difference is \(18.472-17.297\approx1.175\), so Model L has the lower AIC by about 1.175 units for this hypothetical data set. Omitting the Jacobian would instead give an unadjusted transformed-scale AIC of \(-2\ell_Z+4\approx11.297\). That value is not comparable with Model G’s AIC, because it describes a density for \(Z\), not a density for the observed \(Y\).
Units and Response Transformations
A change in measurement units is itself a transformation of the response. For example, converting a measurement from kilograms to grams multiplies each value by 1000. The density’s numerical value changes under this conversion, so the AIC’s numerical value changes too. However, when all candidate models are expressed on the same converted scale, each receives the same AIC shift; their ranking is unchanged.
Worked Example: Converting the Response from Kilograms to Grams
Suppose a set of \(n=3\) responses is recorded in kilograms, and a model has log likelihood \(\ell_{\mathrm{kg}}\) and AIC \(\mathrm{AIC}_{\mathrm{kg}}=-2\ell_{\mathrm{kg}}+2k\). Convert the response to grams using \(t=1000y\). The inverse transformation is \(y=t/1000\), with derivative \(dy/dt=1/1000\). Thus the density per gram is \(f_{\mathrm{g}}(t)=f_{\mathrm{kg}}(t/1000)/1000\).
For three observations the transformed log likelihood is \(\ell_{\mathrm{g}}=\ell_{\mathrm{kg}}-3\log(1000)\). Consequently,
Every candidate model converted from kilograms to grams receives this same shift for the same three observations. For example, if two models had kilogram-scale AIC values of 20.000 and 21.000, their gram-scale values would be approximately 61.447 and 62.447. The difference remains 1.000, and the ranking is unchanged. This shared shift within a common scale is different from comparing an unadjusted log-scale AIC against a response-scale AIC.
Assumptions and a Common Pitfall
AIC comparisons require maximized likelihoods for the same observations under clearly specified probability models. For a lognormal model, all observed responses must be positive so that their logarithms exist. The independence and distributional assumptions used to construct each likelihood must be appropriate; in a regression setting, the likelihood must also correspond to the stated regression model. A fixed, one-to-one transformation permits the Jacobian adjustment above. If the transformation itself has estimated parameters, those parameters must be included in the parameter count, and the likelihood must account for their estimation.
Common mistake: A learner fits a normal model to \(Y\), fits another normal model to \(\log Y\), and compares the two software-reported AIC values without adjustment. The reasoning is that both values use the same formula, \(-2\ell+2k\), so they must be directly comparable. The correction is that the second likelihood is a density for \(\log Y\), not \(Y\). For a logarithmic transformation, add \(2\sum_i\log y_i\) to the transformed-scale AIC before comparing it with an AIC for \(Y\). Matching the formula is not enough; the likelihoods must refer to the same response scale.
Check Your Understanding
Use the definitions and formulas above to answer these questions.
- For \(Z=\log Y\), derive the Jacobian factor and state the adjustment that converts a log-scale log likelihood into a likelihood for \(Y\).
- A fixed logarithmic transformation is applied to three positive observations whose log values sum to 4. By how much does the transformed-scale AIC change when it is expressed on the original response scale?
- Why is a normal-model AIC for \(Y\) not directly comparable with the unadjusted AIC from a normal model fitted to \(\log Y\)?
- When responses are converted from kilograms to grams, why can AIC values change without changing the ranking of candidate models fitted on that common scale?
- For a fixed one-to-one response transformation, does the transformation alone increase \(k\)? What must be done if parameters of the transformation are estimated?