Likelihood Comparability Across GLMs

Why Likelihood Comparability Needs a Check

A larger maximized likelihood is not automatically evidence that one GLM is preferable to another. The number only has a meaningful comparative interpretation when the candidate models describe the same observed response data under compatible probability models. If one fit uses different observations, a different response representation, or a quantity that is not a probability likelihood, a raw comparison can mislead.

This matters for likelihood-based summaries as well as for the likelihood itself. For example, the usual AIC is \(\mathrm{AIC}=-2\ell(\hat{\theta})+2k\), where \(\ell\) is the full maximized log-likelihood and \(k\) is the number of estimated parameters. This tutorial focuses on when the likelihood input can be compared; it does not derive the criterion. In particular, AIC does not repair a comparison made on different data or from quantities that are not ordinary likelihoods.

What Must Be the Same?

Definition: Candidate likelihoods are comparable for a direct likelihood-based comparison when they are evaluated for the same observed response records, with the same conditioning information and sample definition, and are genuine probability likelihoods for those observations relative to a common dominating measure. A full log-likelihood includes every term in the probability mass or density of the observed data. A log-likelihood kernel omits terms that are constant in the model parameters.

“Same observed data” means more than having the same sample size. The response values, their observational units, and the records included must agree. The candidate models should also condition on the same relevant information, such as exposures or the fixed design. If one model drops records with missing covariates while another uses them, their likelihoods describe different observed samples. Their numerical values are not a direct measure of relative fit.

The response distributions do not have to be identical. For example, a Poisson model and a negative-binomial model can each assign probabilities to the same observed count vector. Their full likelihoods can be compared if both are proper probability models for that vector. The comparison is between the models’ probabilities for the observed data, not between the names of their distribution families.

A common dominating measure makes the comparison precise. For discrete responses, this is often counting measure; for continuous responses, it is often Lebesgue measure. If each model has a density \(f_m\) relative to the same measure, then its likelihood at the observed data \(y\) is \(L_m(\theta_m)=f_m(y;\theta_m)\). The log-likelihood ratio is then the difference between two log densities evaluated at the same \(y\). The densities need not have the same functional form, but the response and its measure must be interpreted consistently.

Worked Example: Comparing Two Count Distributions

Suppose the same two observed counts are \(y=(0,2)\). Evaluate a Poisson model with respective means \((1,2)\) and a negative-binomial model with size \(r=2\) and respective means \((1,2)\). These are specified parameter values for illustrating the comparison; they are not claimed to be fitted estimates.

For a Poisson variable, \(P(Y=y)=e^{-\mu}\mu^y/y!\). The full log-likelihood is \(\ell_{\mathrm{Pois}}=-\mu_1+y_1\log\mu_1-\log(y_1!)-\mu_2+y_2\log\mu_2-\log(y_2!)\). Substitution gives \(\ell_{\mathrm{Pois}}=-1+0-0-2+2\log 2-\log 2=-3+\log 2\approx-2.3069\).

For a negative-binomial variable parameterized by size \(r\) and mean \(\mu\), let \(p=r/(r+\mu)\). Its probability mass is \(P(Y=y)={y+r-1\choose y}p^r(1-p)^y\). For the first count, \(y_1=0\), \(\mu_1=1\), and \(p_1=2/3\), so the probability is \((2/3)^2=4/9\). For the second, \(y_2=2\), \(\mu_2=2\), and \(p_2=1/2\), so the probability is \({3\choose2}(1/2)^2(1/2)^2=3/16\). Thus \(\ell_{\mathrm{NB}}=\log(4/9)+\log(3/16)=\log(1/12)\approx-2.4849\).

Both values are full log-likelihoods for the same count observations, using proper distributions on the same count sample space. Their difference is meaningful at these parameter values. A model-selection comparison would instead use each model’s maximized likelihood and would account for its parameter count in the chosen criterion.

Constants, Kernels, and Response Representation

Likelihood calculations often discard terms that do not involve the model parameters. This is safe for maximizing a particular likelihood over its parameters. It is also safe for comparing candidate models when the omitted term is identical across all candidates. It is not safe to assume that every “constant” can be dropped in a comparison: a term may be constant in the parameters of one model but differ across models because the response distribution or its representation changed.

Result: If, for every candidate \(m\), the reported log-likelihood is \(\tilde{\ell}_m(\theta_m)=\ell_m(\theta_m)-c(y)\), where the same data-only constant \(c(y)\) is subtracted from every candidate, then the candidate rankings by maximized log-likelihood are unchanged. Their AIC differences are also unchanged when computed with the same parameter-count convention.

Proof. Because \(c(y)\) does not depend on \(\theta_m\), subtracting it does not change the maximizing parameter value. Therefore \(\max_{\theta_m}\tilde{\ell}_m=\max_{\theta_m}\ell_m-c(y)\). For two candidates \(a\) and \(b\), \(\max\tilde{\ell}_a-\max\tilde{\ell}_b=(\max\ell_a-c(y))-(\max\ell_b-c(y))=\max\ell_a-\max\ell_b\). Their maximized log-likelihood ranking is preserved. With \(\mathrm{AIC}_m=-2\max\ell_m+2k_m\), the criterion based on the shortened likelihood is \(\widetilde{\mathrm{AIC}}_m=\mathrm{AIC}_m+2c(y)\). The same quantity is added to every candidate, so AIC differences and rankings are preserved. \(\square\)

This result explains why parameter-independent constants may be omitted from a comparison only after checking that the omission is common. For example, dropping \(-\log(y!)\) from each Poisson observation is harmless across Poisson candidates evaluated on the same counts. But using different omitted terms for different families, or comparing reported software quantities that omit different constants, can shift the values and invalidate a direct comparison of their levels.

Worked Example: A Gaussian Constant That Cancels

Take two observations \(y=(1,3)\) and compare normal models with known variance \(1\), using means \(\mu=2\) and \(\mu=0\). The full log-likelihood for either model is \(\ell(\mu)=-\log(2\pi)-\tfrac12\sum_{i=1}^2(y_i-\mu)^2\).

For \(\mu=2\), the squared deviations are \((1-2)^2=1\) and \((3-2)^2=1\), so the sum is \(2\) and \(\ell(2)=-\log(2\pi)-1\approx-2.8379\). For \(\mu=0\), the squared deviations are \(1^2=1\) and \(3^2=9\), so the sum is \(10\) and \(\ell(0)=-\log(2\pi)-5\approx-6.8379\).

The common term \(-\log(2\pi)\) can be omitted from both values: the resulting kernels are \(-1\) and \(-5\), still differing by \(4\). If it were omitted from only one model’s reported value, the displayed numbers would no longer provide a valid comparison. This example also shows why knowing whether a software output is a full log-likelihood or a kernel matters.

The response representation is part of the likelihood. As one illustration, a binomial count \(Y=2\) from \(n=3\) trials has probability \({3\choose2}p^2(1-p)\). The likelihood for one particular ordered sequence with two successes and one failure is \(p^2(1-p)\); the factor \({3\choose2}=3\) accounts for the three possible sequences yielding that count. The two likelihoods differ by a factor of three, or by \(\log 3\) on the log scale. This factor does not depend on \(p\), so it cancels between models that use the same grouped-count representation. It must not be ignored when interpreting absolute likelihood values or comparing analyses that do not use the same response representation and likelihood convention.

Worked Example: A Response Transformation Adds a Jacobian

Suppose \(Y\) is positive and has density \(f_Y(y;\theta)\), and define \(Z=\log Y\). The density of \(Z\) is \(f_Z(z;\theta)=f_Y(e^z;\theta)e^z\). For the observed value \(y=2\), the log-likelihood on the \(Z\) scale is \(\ell_Z(\theta)=\ell_Y(\theta)+\log 2\), because the Jacobian of \(y=e^z\) is \(e^z\), which equals \(2\) at \(z=\log 2\).

Within a set of models all fitted to \(Z=\log Y\), this added \(\log 2\) is common and cannot change their ranking. Across models fitted to different response scales, the Jacobian terms must be included consistently before comparing densities. Otherwise the numbers are densities with respect to different coordinates, not directly comparable full likelihoods.

Quasi-Likelihood Is Not an Ordinary Likelihood

Definition: A quasi-likelihood specifies a mean-variance relationship and a score-like estimating function without requiring a fully specified probability mass or density for the response. For a scalar mean \(\mu\), a common quasi-likelihood function \(Q(y;\mu)\) is defined up to an additive term independent of \(\mu\) by \(\partial Q(y;\mu)/\partial\mu=(y-\mu)/V(\mu)\), where \(V(\mu)\) is the variance function.

Because a quasi-likelihood is not generally the log of a normalized probability model, it is not a full log-likelihood. A value computed from it cannot automatically be compared with a GLM log-likelihood or inserted into ordinary AIC. Specialized criteria may be defined for quasi-likelihood settings, but they require their own assumptions and definitions; they are not obtained simply by relabeling a quasi-likelihood value as \(\ell\).

Worked Example: Quasi-Poisson Does Not Supply an Ordinary Log-Likelihood

For a quasi-Poisson mean, take \(V(\mu)=\phi\mu\), where \(\phi>0\) is a dispersion parameter. Then the quasi-likelihood derivative is \((y-\mu)/(\phi\mu)\). Integrating with respect to \(\mu\) gives \(Q(y;\mu)=\{y\log\mu-\mu\}/\phi+C(y)\), since the derivative of \(y\log\mu-\mu\) is \(y/\mu-1=(y-\mu)/\mu\).

For observations \(y=(1,3)\), means \(\mu=(2,2)\), \(\phi=1\), and the choice \(C(y)=0\), the sum is \((\log2-2)+(3\log2-2)=4\log2-4\approx-1.2274\). This calculation gives a quasi-likelihood value under the stated convention. It does not establish that the values \(e^{Q(y;\mu)}\) form a normalized probability distribution over all possible data. Therefore \(-2Q+2k\) is not ordinary AIC merely because its formula resembles AIC.

A Practical Comparability Check

Match the data.
Confirm that the response records, observational units, missing-data handling, and conditioning information are the same.

Confirm a probability likelihood.
Check that each candidate supplies a normalized probability mass or density for the observed response, rather than only an estimating equation or quasi-likelihood.

Check the response scale and measure.
Verify that the response is represented consistently and that any transformation Jacobians or grouping factors are accounted for.

Check constants and the reported quantity.
Determine whether each value is a full log-likelihood or a kernel, and whether any omitted data-only terms are common to every candidate.

Apply the intended criterion only after these checks.
For information criteria, use the appropriate maximized likelihood and parameter count; a formula cannot compensate for incomparable inputs.

Likelihood comparability is a prerequisite, not a guarantee that one criterion answers every scientific question. Two valid models may describe different distributional assumptions, and the relevant comparison still depends on the intended response and prediction task. The immediate practical safeguard is to establish that the numbers being compared refer to the same observed data and consistently defined probability quantities.

Common Mistakes

Comparing fits with different records. A larger likelihood can simply reflect a different sample. Restrict candidates to the same response records before making a direct comparison.
Assuming different families cannot be compared. They can be compared when they assign proper probabilities or densities to the same observations on a common sample space and measure.
Dropping any term called a constant. A constant in the parameters may still differ between candidates. Check whether the omitted term is identical across the comparison.
Treating a quasi-likelihood as a log-likelihood. Its estimating function does not generally define a normalized response distribution, so ordinary likelihood comparisons and AIC do not follow automatically.
Ignoring response scale. Densities change under transformations. Include the Jacobian when the scale changes, and compare quantities defined on a consistent scale.

Key takeaway: Before comparing GLM likelihoods or likelihood-based information criteria, verify the same response data and conditioning, a genuine probability model for each candidate, a consistent response representation, and compatible treatment of likelihood constants. Quasi-likelihood values are not ordinary likelihoods.

Check Your Understanding

Use the distinctions in this tutorial to assess each comparison.

Two GLMs have the same sample size, but one excludes two observations and includes two different ones. Are their maximized log-likelihoods directly comparable? Explain.
When can a parameter-independent term be omitted from every candidate without changing their likelihood ranking or AIC differences?
Why can a Poisson and a negative-binomial model be compared using likelihoods for the same count observations?
What factor distinguishes the likelihood of a grouped binomial count from that of one particular ordered sequence of trial outcomes?
Why is a quasi-Poisson quasi-likelihood value not automatically suitable for ordinary AIC?